Ch23.3: An Async TCP Echo Server

1. The Synchronous Version First

The repository's synchronous echo server is almost trivial:

fast_io::net_service service;
fast_io::native_socket_file listener{fast_io::tcp_listen(2000)};

for (;;)
{
    fast_io::iobuf_socket_file client{fast_io::tcp_accept(listener)};
    fast_io::println(client, "hello");
}

One thread accepts one connection, writes, closes, and loops. It can serve exactly one client at a time. If the client is slow to read, the server blocks in println and stops accepting. The fix is not "add a thread per client"; the fix is to suspend the connection instead of blocking the thread.

The async server keeps the same shape — listener, accept, session — but every wait becomes a co_await. The accept loop never blocks, the session never blocks, and one thread can own thousands of suspended sessions.

2. The Whole Server

This is examples/0017.network/async/tcp_echo_server.cc. Read it once without worrying about the details; the next sections take it apart.

#include <fast_io.h>
#include <fast_io_device.h>

namespace fi = ::fast_io;
namespace fop = ::fast_io::operations;

static fi::io_async_task<> echo_session(
    fi::io_async_observer sched, fi::native_socket_file sock) throws
{
    co_await fop::async_transmit_some_bytes(sched, {}, sock, {}, sock, {}, {});
}

static fi::io_async_task<> accept_loop(
    fi::io_async_observer sched, fi::native_socket_file listener) throws
{
    for (;;)
    {
        echo_session(
            sched,
            co_await fop::async_accept(
                sched, {}, listener, fi::open_mode::no_block))
            .detach();
    }
}

int main()
{
    fi::net_service service;
    fi::io_async_scheduler scheduler{fi::io_async};

    for (;;)
    try
    {
        auto acceptor{accept_loop(
            scheduler,
            fi::native_socket_file{
                fi::tcp_listen(2000, fi::open_mode::no_block)})};
        acceptor.resume();
        while (!acceptor.done())
        {
            fi::io_async_wait(scheduler);
        }
        acceptor.rethrow_if_error();
    }
    catch throws(::std::error e)
    {
        fi::perrln(e);
    }
}

There are exactly two coroutines: accept_loop and echo_session. The accept loop lives for the lifetime of the listener. Each session lives for the lifetime of one connection. main is the supervisor and the event pump.

3. The Session: One transmit, No Buffer

static fi::io_async_task<> echo_session(
    fi::io_async_observer sched, fi::native_socket_file sock) throws
{
    co_await fop::async_transmit_some_bytes(sched, {}, sock, {}, sock, {}, {});
}

The parameters of async_transmit_some_bytes are:

Argument	Value here	Meaning
Scheduler	`sched`	The event loop that will resume this coroutine.
Timeout	`{}`	No deadline.
Output stream	`sock`	Where the bytes are written.
Output offset	`{}`	No explicit file position; the socket's own position is irrelevant.
Input stream	`sock`	Where the bytes are read from. The same socket.
Input offset	`{}`	No explicit position.
Bound	`{}`	No byte limit; read until one round is complete or EOF.

async_transmit_some_bytes is a byte pump. It reads a chunk from the input, writes that chunk to the output, and returns the number of bytes moved. The transmit engine owns its scratch buffer, so this coroutine has no buffer to allocate, no buffer to size, and no partial-write loop to write. That is exactly what an echo server needs.

The upstream example does one round per connection. A real echo service should keep pumping until the peer closes:

static fi::io_async_task<> echo_session(
    fi::io_async_observer sched, fi::native_socket_file sock) throws
{
    for (;;)
    {
        std::size_t moved{
            co_await fop::async_transmit_some_bytes(
                sched, {}, sock, {}, sock, {}, {})};
        if (moved == 0)
        {
            co_return;   // EOF: the peer closed its side
        }
    }
}

The loop is still one line of I/O. async_transmit_some_bytes performs one read/write round; the loop repeats until the input returns zero bytes. There is no pread and no pwrite in the application code. The high-level operation owns the mechanics.

4. The Accept Loop: One Coroutine, Many Connections

static fi::io_async_task<> accept_loop(
    fi::io_async_observer sched, fi::native_socket_file listener) throws
{
    for (;;)
    {
        echo_session(
            sched,
            co_await fop::async_accept(
                sched, {}, listener, fi::open_mode::no_block))
            .detach();
    }
}

async_accept suspends the accept loop until a connection arrives. When it resumes, it yields an owning native socket handle. The handle is wrapped in native_socket_file and passed by value to echo_session. The session coroutine's frame owns the socket; when the session finishes, the socket closes.

detach() is the important call. It releases the task object, marks the frame as detached, and starts it. The session now runs independently: the accept loop does not await it, and the session destroys its own frame at final_suspend. This is how one accept coroutine can own an unbounded number of connections without storing a task object for each one.

The accepted socket is opened with open_mode::no_block. On Windows this is what makes the socket overlapped and therefore usable with IOCP. On POSIX it is harmless; io_uring does not need it. Always pass it when the socket will be used asynchronously, so the code is portable.

5. The Supervisor: Surviving a Dead Listener

main constructs the listener inside the coroutine call, so the listener is moved into accept_loop's frame. The frame owns it for the lifetime of the accept loop. If async_accept fails, the error escapes the coroutine body, is stored by unhandled_herbception, and the accept loop becomes done. The supervisor then calls rethrow_if_error(), catches the error, logs it, and starts a new accept loop with a fresh listener.

for (;;)
try
{
    auto acceptor{accept_loop(
        scheduler,
        fi::native_socket_file{
            fi::tcp_listen(2000, fi::open_mode::no_block)})};
    acceptor.resume();
    while (!acceptor.done())
    {
        fi::io_async_wait(scheduler);
    }
    acceptor.rethrow_if_error();
}
catch throws(::std::error e)
{
    fi::perrln(e);
}

The inner while is the event loop. io_async_wait blocks until one completion is ready, dispatches it, and returns. The loop checks whether the accept coroutine is done after each dispatch. When the listener dies, the loop exits and the supervisor restarts.

This is deliberately not a "restart forever on any error" policy. A production server would distinguish a transient accept error from a permanent one, add a delay, and maybe stop after repeated failures. The structure is the important part: the listener is a resource owned by a coroutine, and the supervisor decides what to do when that coroutine ends.

6. Build and Run

The example is a normal fast_io program. Use the compiler setup from Ch1; the throws markers require the book's herbception-enabled toolchain. A standard compiler can use ordinary exception syntax instead.

$ g++ -std=c++26 -O2 tcp_echo_server.cc -o tcp_echo_server
$ ./tcp_echo_server

On Linux, the program uses io_uring when the kernel allows it and falls back to a thread pool otherwise. On Windows, it uses IOCP. There is no conditional compilation in the example itself; the scheduler alias picks the backend.

Test it from another terminal:

$ nc 127.0.0.1 2000
hello
hello
world
world

With the one-round upstream version, the connection may close after the first echo. With the loop version above, every line is echoed until you close the connection. If you see the first line echoed and then the connection closes, you are looking at the one-round example, not a bug in the scheduler.

To watch the event loop, run several clients at once. Each client gets its own detached session. The server still uses one thread. When one client is idle, its session is suspended and consumes no CPU; the others continue.

7. Why There Is No Buffer Management

The synchronous server used iobuf_socket_file because println needs a formatted output buffer. The async echo server uses a raw native_socket_file because it does not format anything; it moves bytes. async_transmit_some_bytes owns a bounce buffer internally and reuses it for each round.

This is the right default for a byte pump. Do not allocate a per-connection std::vector<char> just to echo. Do not call read into a stack buffer and then write it. The transmit operation already knows how to stage a partial read and finish a partial write. The application code stays at the level of "move bytes from here to there".

The moment the protocol has structure — lines, headers, integers, frames — switch to async_scan for input and async_print for output. That is the next page. The boundary is simple: bytes use transmit, structured data uses scan/print.

Key Takeaways

Two coroutines: one accept loop, one session per connection.
async_accept returns an owning socket; passing it by value transfers ownership into the session frame.
detach() turns the session into a self-owning frame; the accept loop never waits for it.
async_transmit_some_bytes is the high-level byte pump; loop until it returns zero for a real echo.
open_mode::no_block marks a socket async-capable, especially on Windows.
The supervisor owns the listener's lifetime and restarts the accept loop when it dies.

The next page keeps the same architecture but changes the protocol: HTTP requests, formatted responses, body transfer, and the places where scan/print/transmit replace raw byte windows.