Much of Python meaning lives in object protocols: a function is a value, class creates a new type, a generator pauses at yield, with guarantees context exit, and a decorator replaces a name with the result of another call.
Understanding: Python objects, functions, and protocols
You can read this chapter before running the experiment. Then return to the live trace and match each concept to a real event.
Functions, classes, and modules are objects. Attribute resolution follows the data model, bound methods receive the instance as self, exceptions unwind frames toward a matching handler, iter/next define iteration, and context managers implement enter/exit behavior.
Terms used in this experiment
Understand the words first, then the execution order.
Identity
The identity of one concrete object. is asks whether two references point to that same object.
Mutable
An object whose value can change without creating another object, such as list, dict, and set.
Callable
An object accepted by call syntax (). Functions and classes are callable, and custom objects may implement __call__.
Decorator
A callable that receives a function or class and binds the original name to its returned result.
Generator
An iterator that preserves a frame and resumes execution between yield expressions.
Context manager
An enter/exit protocol used by with for deterministic resource cleanup.
Exception
An error object propagated up call frames until a compatible except catches it.
dataclass
A standard-library decorator that generates common data-class methods such as __init__ and __repr__.
Closure
A function plus references to free variables from an enclosing function scope.
Module cache
sys.modules retains loaded modules, so repeated import normally returns the same module object.
What happens step by step
Each step maps to an observable runtime state.
- 01class executes its body
Python evaluates the class body in a namespace and then creates a class object.
- 02Calling the class creates an instance
__new__ creates the object and __init__ initializes the existing instance.
- 03A method binds self
instance.method creates a bound method and passes the instance as the first argument.
- 04A decorator applies at definition time
@decorator above def is equivalent to name = decorator(name).
- 05A generator starts on demand
Calling its function creates a generator; the body begins only under next or iteration.
- 06with guarantees exit
__exit__ runs for normal flow and exceptions, making cleanup predictable.
- 07An exception searches for a handler
Frames unwind to a compatible except and finally runs on every exit path.
Where the result needs context
These details explain why similar code can sometimes produce a different trace.
Defaults are evaluated once
Default objects are created when def executes, so a mutable default accumulates state.
A class variable is shared
A mutable object in the class body is shared until an instance shadows that attribute.
property can run expensive code
order.total may invoke arbitrary descriptor code and raise an exception.
finally can replace a result
return inside finally suppresses an earlier result or exception and is usually a design bug.
A generator is single-use
After StopIteration the same generator does not restart; call the generator function again.
A closure retains objects
A captured object stays reachable while the inner function lives, just as in JavaScript.
Private is mainly convention
_name marks an internal API; __name uses name mangling but is not strict privacy.
First understand which parts of Node participate in execution.
Then remove instrumentation and focus on the central mechanism.
Finally match the model to the code that produces the live trace.
A minimal model without instrumentation
from dataclasses import dataclass
@dataclass(slots=True)
class CartLine:
sku: str
price: int
quantity: int = 1
@property
def subtotal(self) -> int:
return self.price * self.quantity
def even_squares(limit):
for value in range(limit):
if value % 2 == 0:
yield value * valueThe complete code executed by the scenario
This is not an alternative example: these are the functions and files used by the Run button.
The first file is the real CPython scenario; the second safely starts the child process and turns its JSON Lines into the live trace.
#!/usr/bin/env python3
"""Fixed, input-free CPython scenarios used by Runtime Lab."""
from __future__ import annotations
import asyncio
import dis
import gc
import io
import json
import platform
import sys
import time
import weakref
from dataclasses import dataclass
from typing import Any, Iterator
def emit(lane: str, event_type: str, key: str, **data: Any) -> None:
print(
json.dumps(
{"lane": lane, "type": event_type, "key": key, "data": data},
ensure_ascii=False,
),
flush=True,
)
def version_event() -> None:
emit(
"python",
"runtime",
"python.version",
implementation=platform.python_implementation(),
version=platform.python_version(),
)
def run_syntax() -> None:
version_event()
orders = [
{"id": "A-10", "status": "paid", "price": 120, "qty": 2},
{"id": "A-11", "status": "draft", "price": 80, "qty": 1},
{"id": "A-12", "status": "paid", "price": 50, "qty": 3},
]
emit("objects", "state", "syntax.objects", count=len(orders))
paid = [order for order in orders if order["status"] == "paid"]
total = sum(order["price"] * order["qty"] for order in paid)
emit(
"comprehension",
"result",
"syntax.comprehension",
ids=", ".join(order["id"] for order in paid),
total=total,
)
first, *middle, last = [order["id"] for order in orders]
emit(
"sequence",
"result",
"syntax.unpack",
first=first,
middle=middle,
last=last,
)
labels = [f"{index}:{order['id']}" for index, order in enumerate(orders, start=1)]
emit("loop", "result", "syntax.enumerate", labels=", ".join(labels))
def format_order(order: dict[str, Any], *, currency: str = "RUB") -> str:
return f"{order['id']} · {order['price'] * order['qty']} {currency}"
emit(
"function",
"result",
"syntax.function",
rendered=format_order(orders[0], currency="₽"),
)
emit("result", "result", "syntax.result")
@dataclass(slots=True)
class CartLine:
sku: str
price: int
quantity: int = 1
@property
def subtotal(self) -> int:
return self.price * self.quantity
def append_bad(item: str, bucket: list[str] = []) -> list[str]:
bucket.append(item)
return list(bucket)
def append_safe(item: str, bucket: list[str] | None = None) -> list[str]:
target = [] if bucket is None else bucket
target.append(item)
return target
def even_squares(limit: int) -> Iterator[int]:
for value in range(limit):
if value % 2 == 0:
yield value * value
def classify_event(event: dict[str, Any]) -> str:
match event:
case {"type": "order.paid", "payload": {"id": order_id}}:
return f"paid order {order_id}"
case {"type": event_type}:
return f"other event {event_type}"
case _:
return "invalid event"
def run_semantics() -> None:
version_event()
line = CartLine("book", 450, quantity=2)
emit(
"class",
"result",
"semantics.dataclass",
rendered=repr(line),
subtotal=line.subtotal,
)
original = ["node"]
alias = original
alias.append("python")
emit("objects", "mutation", "semantics.alias", shared=original == alias == ["node", "python"])
bad_first = append_bad("api")
bad_second = append_bad("worker")
emit(
"function",
"warning",
"semantics.mutable-default",
first=bad_first,
second=bad_second,
)
safe_first = append_safe("api")
safe_second = append_safe("worker")
emit(
"function",
"result",
"semantics.safe-default",
first=safe_first,
second=safe_second,
)
emit("generator", "result", "semantics.generator", values=list(even_squares(7)))
emit(
"pattern",
"result",
"semantics.match",
label=classify_event({"type": "order.paid", "payload": {"id": "A-42"}}),
)
try:
int("not-a-number")
except ValueError as error:
emit("exception", "caught", "semantics.exception", message=str(error))
stream = io.StringIO()
with stream:
stream.write("cleanup is deterministic")
text = stream.getvalue()
emit("context", "cleanup", "semantics.context", closed=stream.closed, text=text)
emit("result", "result", "semantics.result")
def doubled_total(values: list[int]) -> int:
return sum(value * 2 for value in values)
class CycleNode:
def __init__(self) -> None:
self.peer: CycleNode | None = None
async def traced_task(name: str, delay: float, completion: list[str]) -> str:
emit("asyncio", "start", "asyncio.started", name=name)
await asyncio.sleep(delay)
completion.append(name)
emit("asyncio", "resume", "asyncio.resumed", name=name)
return name
async def measure_timer_while(blocking_call: Any) -> float:
started = time.perf_counter()
timer = asyncio.create_task(asyncio.sleep(0.01))
await blocking_call()
await timer
return max(0.0, (time.perf_counter() - started - 0.01) * 1_000)
async def run_asyncio_round() -> None:
completion: list[str] = []
first = asyncio.create_task(traced_task("A", 0.025, completion))
second = asyncio.create_task(traced_task("B", 0.005, completion))
emit("asyncio", "schedule", "asyncio.created")
await asyncio.gather(first, second)
emit("asyncio", "result", "asyncio.result", order=" → ".join(completion))
async def block_loop() -> None:
time.sleep(0.055)
async def offload_sleep() -> None:
await asyncio.to_thread(time.sleep, 0.055)
blocked_delay = await measure_timer_while(block_loop)
emit("asyncio", "blocking", "asyncio.blocking", delay=round(blocked_delay, 1))
offloaded_delay = await measure_timer_while(offload_sleep)
emit("asyncio", "offload", "asyncio.offload", delay=round(offloaded_delay, 1))
def run_runtime() -> None:
implementation = platform.python_implementation()
gil_probe = getattr(sys, "_is_gil_enabled", None)
gil_enabled = gil_probe() if gil_probe else implementation == "CPython"
emit(
"runtime",
"config",
"runtime.config",
implementation=implementation,
version=platform.python_version(),
gil=gil_enabled,
)
operations = [instruction.opname for instruction in dis.get_instructions(doubled_total)]
emit(
"bytecode",
"result",
"runtime.bytecode",
operations=" → ".join(operations[:12]),
)
frame = sys._getframe()
visible_locals = ", ".join(sorted(name for name in frame.f_locals if not name.startswith("_")))
emit(
"frame",
"state",
"runtime.frame",
functionName=frame.f_code.co_name,
locals=visible_locals,
)
left = CycleNode()
right = CycleNode()
left.peer = right
right.peer = left
left_ref = weakref.ref(left)
right_ref = weakref.ref(right)
alive_before = left_ref() is not None and right_ref() is not None
del left, right
collected = gc.collect()
alive_after = left_ref() is not None or right_ref() is not None
emit(
"gc",
"result",
"runtime.gc",
collected=collected,
aliveBefore=alive_before,
aliveAfter=alive_after,
)
asyncio.run(run_asyncio_round())
emit("result", "result", "runtime.result")
SCENARIOS = {
"syntax": run_syntax,
"semantics": run_semantics,
"runtime": run_runtime,
}
def main() -> None:
scenario = sys.argv[1] if len(sys.argv) > 1 else ""
runner = SCENARIOS.get(scenario)
if runner is None:
raise SystemExit(f"unknown scenario: {scenario}")
runner()
if __name__ == "__main__":
main()
The scenario instruments its own trace: Python calls emit(...) and prints ordered events as JSON Lines, while Node records a timestamp as each line arrives. Source and lane labels come from application code, not a CPython/asyncio profiler. The Node bridge reads output without a shell, bounds runtime and volume, and forwards events into the HTTP stream.
Practical patterns worth keeping nearby
Compare the goal, code, and caveats instead of memorizing syntax without a model.
Function as a value
Recognize a callable without invoking it.
def normalize(value: str) -> str:
return value.strip().lower()
pipeline = [normalize, str.upper]
for transform in pipeline:
value = transform(value)- A function name without () refers to the function object.
- Each callable receives the previous result.
Safe default
Avoid sharing mutable state.
def add_tag(tag: str, tags: list[str] | None = None):
result = [] if tags is None else list(tags)
result.append(tag)
return result- None is an immutable sentinel.
- list(tags) avoids mutating the caller list.
Data class
Read dataclass and property syntax.
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class Money:
amount: int
currency: str = "USD"
@property
def label(self):
return f"{self.amount} {self.currency}"- frozen is not deep immutability.
- slots changes instance layout.
Generator
Process a stream lazily.
def read_valid(lines):
for line in lines:
text = line.strip()
if text:
yield text
for item in read_valid(file):
consume(item)- The result is not stored in full.
- The outer resource still needs cleanup.
Exception chaining
Add domain meaning without losing the cause.
try:
user = repository.get(user_id)
except DatabaseError as error:
raise UserLoadError(user_id) from error
else:
return user
finally:
metrics.increment("load.attempt")- raise ... from preserves the causal chain.
- else runs only when try succeeds.
Context manager
Guarantee transaction cleanup.
with database.transaction() as transaction:
order = transaction.insert_order(payload)
transaction.insert_outbox(order)- The concrete manager defines commit and rollback behavior.
- An exception is passed into __exit__.
How a learning mistake becomes an incident
A realistic service: the original code, observable failure, corrected implementation, and why the correction works.
Tags from one HTTP request appear in another
A helper adds audit tags and runs for every request. The author expects the default list to be created again on each function call.
The default is evaluated once when def executes. A long-lived process reuses the list, mixing user metadata and gradually retaining more memory.
def attach_tag(tag: str, tags: list[str] = []) -> list[str]:
tags.append(tag)
return tags
def audit_request(request):
return attach_tag(f"user:{request.user_id}")One function object stores one reference to the default list in __defaults__. Every call without tags mutates that same object.
def attach_tag(
tag: str,
tags: list[str] | None = None,
) -> list[str]:
result = [] if tags is None else list(tags)
result.append(tag)
return result
def audit_request(request):
return attach_tag(f"user:{request.user_id}")The immutable None sentinel is safe to store as a default. Each omitted value creates a new list, and a passed list is copied before mutation.
What the unfamiliar calls from both code samples actually do.
tags=[]- The default expression runs once when the function object is created, so calls share one mutable list.
function.__defaults__- The tuple of positional default objects where the retained reference to the shared list can be observed directly.
tags is None- Uses an immutable singleton to signal that the caller did not provide its own collection.
list(tags)- Creates a shallow copy of the iterable so the helper append does not mutate the caller-owned list.
result.append(tag)- Mutates that specific result list in place and returns None, so append is not assigned back to the variable.
Common misconceptions
The myth is on the left; the accurate model is on the right.
items=[] creates a fresh list per call.
One default list is created when def executes and is reused.
self is a special keyword.
It is the conventional name of the first instance-method parameter.
yield permanently ends the function.
yield pauses its frame and the next next call resumes after it.
except Exception catches absolutely everything.
System signals such as KeyboardInterrupt derive from BaseException instead.
with only works with files.
The protocol also supports locks, transactions, tracing, and temporary state.
Import merely declares a dependency.
The first import executes module top-level code and may perform I/O or mutate state.
Explain it in your own words
If you can explain the answer without quoting documentation, your mental model is starting to take shape.
- When is a default argument evaluated?
- Why can instances share a class-level list?
- When does a generator body begin?
- Which methods does with invoke?
- Why is a property not necessarily a cheap field?