Python Generators Are Suspended Control Flow

A generator is not merely a function that returns several values. Calling a generator function creates an object that owns a suspended frame: local variables, an instruction position, and exception state that can resume later.

def batches(items, size):
    batch = []
    for item in items:
        batch.append(item)
        if len(batch) == size:
            yield batch
            batch = []
    if batch:
        yield batch

The function body begins running immediately when batches(items, 10) is called. Execution continues until the first yield, and the call returns a generator already suspended at that point. This eager start means validation before the first yield happens at construction time.

yield receives as well as produces

At suspension, yield expression sends a value to the consumer. When resumed with send(value), the yield expression itself evaluates to the sent value.

def accumulator():
    total = 0
    while True:
        amount = yield total
        total += amount

counter = accumulator()
next(counter)       # 0
counter.send(5)     # 5
counter.send(3)     # 8

A newly created generator may be started with any value using send. The first sent value becomes the result of the first yield expression, just as on later resumptions. Calling next(generator) is simply shorthand for generator.send(None).

Exceptions can enter the frame

generator.throw(error) resumes the generator by raising at its current suspension point. The generator can catch that exception, yield another value, or let it escape. generator.close() injects GeneratorExit and expects the generator to finish without yielding again.

Cleanup belongs in finally:

def rows(connection):
    cursor = connection.cursor()
    try:
        yield from cursor
    finally:
        cursor.close()

Closing is deterministic when the consumer explicitly closes or a surrounding construct manages it. Relying on garbage collection for generator finalisation makes resource timing implementation-dependent.

yield from delegates the protocol

yield from iterable forwards iteration, but for a generator delegate it also forwards send, throw, and close. It captures the delegate’s return value from StopIteration.value.

def subtotal(values):
    total = 0
    for value in values:
        total += value
        yield value
    return total

def report(values):
    total = yield from subtotal(values)
    print('total:', total)

This makes generators composable without hand-writing a forwarding loop. The return total does not become another iterated item; it becomes the value of the yield from expression.

The useful mental model is a resumable frame with an input channel, an output channel, and an exception channel. Ordinary iteration uses only part of that protocol, while delegation exposes why generators can express parsers, pipelines, and cooperative state machines cleanly.