Ch23.1: The Coroutine Machine

1. The Shape of the Problem

A normal function has one entrance and one exit. It runs until it returns. If it needs data that is not ready yet, it has three choices: block the thread, return a handle and ask the caller to poll, or split itself into callbacks. All three are visible in the source code, and all three make the program harder to read.

A coroutine is a function that can stop in the middle, give control back to its caller, and later continue from exactly where it stopped. The local variables, the current statement and the call stack of the coroutine survive the pause. The caller does not block. The operating system thread is not consumed while the coroutine waits.

Definition. A coroutine is a function whose execution can be suspended and resumed. In C++20 and later, a function is a coroutine if its body contains co_await, co_yield or co_return. The compiler rewrites that function into a state machine, allocates its state in a coroutine frame, and connects it to a user-defined promise type.

The important word is function. A coroutine is not a thread, not a task queue and not a callback. It is a function with a saved state. Something still has to resume it: a scheduler, an event loop, another coroutine, or the caller itself.

2. Three Keywords, One Machine

The three keywords are not three independent features. They are three ways to reach the same promise object. The promise is the coroutine's control block: it decides what the coroutine returns, what happens on the first suspend, what happens on the last suspend, and how an exception is handled.

#include <generator>
#include <print>

std::generator<int> iota(int n)
{
    for (int i{}; i != n; ++i)
    {
        co_yield i;
    }
}

int main()
{
    for (int value : iota(5))
    {
        std::print("{} ", value);
    }
    std::print("\n");
}

This prints 0 1 2 3 4. The function iota is not called five times. It is called once, creates one coroutine frame, and then resumes five times. Each co_yield suspends at the same point; the loop and the variable i live in the frame, not on the caller's stack.

3. What the Compiler Builds

For every coroutine, the compiler creates a frame. The frame is a hidden object that contains at least:

• the coroutine parameters, copied or moved into the frame;
• the local variables whose lifetime crosses a suspend point;
• the promise object;
• a resume point, so the next resume() knows which statement to continue;
• the current state: running, suspended, or finished.

The frame is usually allocated with operator new and freed when the coroutine frame is destroyed. The compiler is allowed to elide the allocation when it can see the whole lifetime, but you cannot rely on that in a library API. This is why a coroutine return type such as fast_io's io_async_task takes an allocator as a template parameter: the frame is allocated through the same allocator discipline as the rest of the library, not through a global default.

The promise type is the customization point. A coroutine return type task<T> must provide a nested promise_type. The compiler calls the promise at fixed moments:

A coroutine with no promise_type is a compile error. The promise is not a detail you can ignore: it is the part that turns the language's suspend/resume primitive into a generator, a task, or an async operation.

4. The Awaiter Protocol

co_await expr does not call expr directly. It asks the result for an awaiter and then calls up to three member functions:

1. await_ready() — if it returns true, the operation is already complete and the coroutine does not suspend.
2. await_suspend(handle) — called when the coroutine is about to suspend. It may return void (suspend), bool (suspend if true), or another coroutine handle (transfer control to it).
3. await_resume() — produces the value of the whole co_await expression after resumption.

#include <coroutine>

struct immediate
{
    bool await_ready() const noexcept { return true; }
    void await_suspend(std::coroutine_handle<>) const noexcept {}
    int await_resume() const noexcept { return 42; }
};

some_task<int> use()
{
    int value = co_await immediate{};
    co_return value;
}

some_task is a placeholder for a coroutine return type with a promise_type; fast_io's is io_async_task. This example never suspends, because await_ready returns true. The interesting case is false: the coroutine stores its resume point, calls await_suspend, and returns control to whoever resumed it. Later, the operation's completion calls handle.resume(), and execution continues inside await_resume.

That is the whole trick behind async I/O. The coroutine does not wait on a socket. It submits an operation, stores its own handle, and suspends. The event loop waits for the kernel to say the operation is complete, then resumes the stored handle. From inside the coroutine, the code reads like a blocking call: read, then print, then read again.

5. Stackful and Stackless

C++ coroutines are stackless. A suspended coroutine does not keep a separate call stack. Its state is the frame, and only the coroutine's own locals are preserved. You cannot suspend inside an ordinary function called by the coroutine and expect the ordinary function's stack to survive; only the coroutine body has suspend points.

This is different from a stackful coroutine or a green thread, where the whole stack is saved and restored. Stackful designs make suspension possible anywhere, but they need a stack per coroutine and a context switch. Stackless designs are cheaper and more explicit: the compiler knows exactly which state crosses a suspend point, and the programmer writes co_await at the points where suspension is allowed.

6. A Coroutine Is Not a Thread

This is the point that causes the most confusion. A coroutine provides suspension, not parallelism. Two coroutines can be alive at the same time on one thread, but only one of them runs at any instant. They interleave cooperatively at suspend points. There is no preemption and no data race unless you introduce another thread.

caller:      resume A ──► A runs ──► A suspends ──► caller continues
event loop:  wait ──► completion ──► resume A ──► A runs ──► A finishes

A thread is a scheduling unit owned by the operating system. A coroutine is a function state owned by your program. A server can handle ten thousand connections with a handful of threads because most connections are suspended waiting for I/O, not because it created ten thousand threads. The concurrency comes from the event loop and the kernel, not from the coroutine keyword.

The corollary is just as important: a coroutine that performs a blocking operation blocks the whole event loop. Calling read, sleep or a synchronous fast_io operation inside an async coroutine stops every other coroutine on that scheduler. Async code is a discipline, not a decoration.

7. Lifetime, Cost and Failure

A coroutine frame lives until it is destroyed. For a lazy task, the task object usually owns the frame and destroys it in its destructor. For a detached task, the frame destroys itself at final_suspend. Destroying a frame while an asynchronous operation still holds a pointer into it is a use-after-free; the operation's callback will resume a frame that no longer exists. This is why the async layer distinguishes owned tasks from detached tasks and why you must keep an owned task alive until it is done().

The cost has three parts: the frame allocation, the frame size, and the resume. None of them is hidden by the syntax. A coroutine that allocates a frame per request is not free just because it is a coroutine. fast_io lets the scheduler supply the allocator so the frame comes from the same pool as the rest of the I/O state, and the async task type can be parameterized on that allocator.

Failure has two channels in this book. Ordinary C++ exceptions can escape a coroutine and are delivered to promise.unhandled_exception(). fast_io's herbceptions use the throws marker and the throw throws form; an error escaping an io_async_task body is stored in the promise by unhandled_herbception and rethrown at the co_await site, or by rethrow_if_error() for a non-coroutine caller. The hook receives the raw two-word error value, spelled ::std::cxx_std_error in the current implementation (Ch19.1); the public handle used in catch throws (::std::error e) is introduced in Ch3.16. They are the same domain-pointer-plus-code pair at different layers. This chapter only needs the rule: an async operation reports failure through the same channel as its synchronous counterpart.

Key Takeaways

• A coroutine is a function that can suspend and resume; it is not a thread.
• co_await, co_yield and co_return are the only coroutine keywords; any one of them makes the function a coroutine.
• The compiler builds a frame and calls a promise. The promise decides laziness, results, yielding and error handling.
• co_await uses the awaiter protocol: ready, suspend, resume. Async I/O is the protocol plus a scheduler that resumes the handle when the kernel reports completion.
• Suspension is cooperative. Blocking inside a coroutine blocks every coroutine on the same event loop.
• The frame has a lifetime. Keep an owned task alive until it is done, or detach it and let it destroy itself.

The next page turns this language machinery into fast_io's async model: tasks, schedulers, observers, operations and the event loop.