Ch23.4: From Echo to HTTP

1. The Boundary: Bytes vs Structure

The echo server moved bytes. An HTTP server reads structure and writes structure, with a byte body in the middle. That changes the operation you should use:

Part of the protocol	Operation	Why
Request line and headers	`async_scan`	They are formatted tokens: method, target, version, header names and values. The scan layer refills the input buffer when a token crosses the pending window.
Status line and response headers	`async_print`	They are formatted output. The print layer formats into the output buffer and submits the write.
Body with a known length	`async_transmit_all_bytes`	It is a byte range, not a formatted value. Pass the Content-Length as the bound and let the transmit engine move it in chunks.
Body with a structured format	`async_scan` / `async_print`	JSON, form fields and line protocols are parsed and produced by the formatted layer. The body is only bytes to the transport.

The rule from the previous page still holds: scan for input, print for output, transmit for byte movement. HTTP is not an exception; it is the clearest example of all three in one session.

2. The Client: Print, Scan, Transmit

This is the shape of examples/0017.network/async/http.cc, reduced to the part that matters here:

static fi::io_async_task<> fetch(
    fi::io_async_observer sched, fi::u8iobuf_socket_file sock,
    fi::u8native_file out, char const *host) throws
{
    co_await fi::async_print(
        sched, {}, sock,
        u8"GET / HTTP/1.1\r\n"
        u8"Host: ", fi::mnp::os_c_str(host), u8"\r\n"
        u8"User-Agent: fast_io\r\n"
        u8"Accept: */*\r\n"
        u8"Connection: close\r\n\r\n");

    fi::u8http_header_buffer hdr{};
    co_await fi::async_scan(sched, {}, sock, hdr);

    for (auto [key, value] : fi::line_generator(hdr))
    {
        if (::std::u8string_view(key) == u8"Content-Length")
        {
            auto length{fi::u8to<::std::uint_least64_t>(value)};
            co_await fi::operations::async_transmit_all_bytes(
                sched, {}, out, {}, sock, {}, fi::size_t_opt{length});
            co_return;
        }
    }
}

Three operations, three levels:

async_print writes the request head. The socket is a buffered u8iobuf_socket_file, so the bytes may still be in the output buffer when the call returns.
async_scan reads the response head. The input side's tie handling flushes the pending output before the first read, so the request is on the wire before the scan waits for an answer. This is the same ordering guarantee as the synchronous layer.
async_transmit_all_bytes copies exactly Content-Length bytes from the socket to the output file. The bound is the length from the header; the transmit engine stops at that bound even if the peer keeps the connection open.

The host string is formatted through mnp::os_c_str, not concatenated into the request with std::string. The formatting layer handles the encoding and the write; there is no temporary string and no manual buffer management. The real example also uses mnp::string_filters::host to reject a hostile Host value before a single byte is written.

3. The Server: One Session Per Connection

The server keeps the echo architecture: one accept loop, one detached session per connection. Only the session body changes.

static fi::io_async_task<> http_session(
    fi::io_async_observer sched, fi::u8iobuf_socket_file sock) throws
{
    fi::u8http_header_buffer hdr{};
    co_await fi::async_scan(sched, {}, sock, hdr);

    auto const target{hdr.code()};
    fi::u8string_view path{target.data(), target.size()};

    if (path == fi::u8string_view{u8"/health"})
    {
        co_await fi::async_print(
            sched, {}, sock,
            u8"HTTP/1.1 200 OK\r\n"
            u8"Content-Type: text/plain\r\n"
            u8"Content-Length: 3\r\n"
            u8"Connection: close\r\n\r\n"
            u8"ok\n");
        co_await fi::operations::async_output_stream_flush(sched, {}, sock);
        co_return;
    }

    co_await fi::async_print(
        sched, {}, sock,
        u8"HTTP/1.1 404 Not Found\r\n"
        u8"Content-Type: text/plain\r\n"
        u8"Content-Length: 10\r\n"
        u8"Connection: close\r\n\r\n"
        u8"not found\n");
    co_await fi::operations::async_output_stream_flush(sched, {}, sock);
}

The session is still a coroutine that owns its socket. The accept loop is unchanged: accept, wrap, pass by value, detach. The only difference is that the session now understands the request instead of echoing it.

async_scan(sched, {}, sock, hdr) scans a complete HTTP head. The u8http_header_buffer owns the parsed header storage, so the strings returned by hdr.code() and line_generator(hdr) point into that buffer. They stay valid until the next scan or until the buffer is destroyed. Do not store them in a detached task that outlives the session, and do not return them to the caller.

async_print formats the response head and body into the socket's output buffer. async_output_stream_flush drains the buffer before the session returns. Without the flush, the buffered bytes might still be in user space when the socket is closed; the peer would see a truncated response. The flush is the async equivalent of the synchronous flush(out).

4. Serving a File: Content-Length and Transmit

A static file response is a formatted head followed by a byte body. Do not read the file into a std::string. Ask the filesystem for its size, write the head, and transmit the file:

static fi::io_async_task<> serve_file(
    fi::io_async_observer sched, fi::u8iobuf_socket_file sock,
    fi::u8native_file file, ::std::uint_least64_t size) throws
{
    co_await fi::async_print(
        sched, {}, sock,
        u8"HTTP/1.1 200 OK\r\n"
        u8"Content-Type: application/octet-stream\r\n"
        u8"Content-Length: ", size, u8"\r\n"
        u8"Connection: close\r\n\r\n");

    co_await fi::operations::async_transmit_all_bytes(
        sched, {}, sock, {}, file, {}, fi::size_t_opt{size});
    co_await fi::operations::async_output_stream_flush(sched, {}, sock);
}

The output is the socket, the input is the file. The bound is the exact Content-Length. The transmit engine reads a chunk, writes it, and repeats until the bound is exhausted. If the client disconnects early, the write fails, the error escapes the session, and the detached frame is destroyed. Nobody needs to catch it: the connection is already gone.

The same pattern works in the other direction: a request body with Content-Length is transmitted from the socket into a file or a memory stream. If the body is structured, scan it instead. If it is only bytes, transmit it. Do not mix the two models in the same body.

5. Keep-Alive, Pipelining and the Loop

The examples above answer one request and close. HTTP/1.1 can keep the connection open for more requests. The session then becomes a loop:

static fi::io_async_task<> http_session(
    fi::io_async_observer sched, fi::u8iobuf_socket_file sock) throws
{
    for (;;)
    {
        fi::u8http_header_buffer hdr{};
        co_await fi::async_scan(sched, {}, sock, hdr);

        bool const keep_alive{/* inspect Connection: and the version */};
        co_await fi::async_print(sched, {}, sock, /* response */);
        co_await fi::operations::async_output_stream_flush(sched, {}, sock);

        if (!keep_alive)
        {
            co_return;
        }
    }
}

The loop is safe only if the scan consumes exactly one request head. The buffer must not be reused across iterations until the previous request's strings are no longer needed. Declaring hdr inside the loop gives each iteration a fresh buffer; that is the simplest correct lifetime.

Pipelining is harder: a client may send two requests before reading the first response. The scan layer reads from the socket's input buffer, so the second request is already pending when the first response is written. The loop handles that naturally as long as the protocol state is per request, not per connection. Do not assume "one read equals one request"; the scan layer already decouples those.

6. Timeouts, Limits and Backpressure

A public server needs limits. The async layer gives you the mechanism; the policy is yours.

Header timeout. Pass a deadline to async_scan so a client that sends one byte per minute cannot hold a session forever.
Body limit. Never trust Content-Length blindly. Reject a body larger than your configured maximum before allocating or transmitting.
Header count and size. u8http_header_buffer has a fixed capacity; decide what happens when it overflows and return a 431 or 400, not a crash.
Slow readers. A response body larger than the socket buffer will suspend in the write. That is correct: the session stops reading and stops allocating more output until the peer drains. Do not create an unbounded queue to "avoid blocking".
Idle connections. If you support keep-alive, arm a read timeout per request, not one global timeout for the connection.

Backpressure is the reason the transmit operation is preferable to a hand-written read-all-then-write-all loop. The transmit engine reads the next chunk only after the previous chunk has been written. A slow client slows the producer instead of growing a buffer in memory.

7. The Mistakes That Actually Happen

Mistake	Symptom	Fix
Using synchronous `print` / `scan` inside an async session	The whole event loop stalls; other connections freeze.	Use `async_print` / `async_scan`.
Forgetting `async_output_stream_flush`	The response is truncated or never arrives.	Flush before returning or before waiting for the next request.
Storing a `u8string_view` from the header buffer past the next scan	Garbage paths, corrupted responses, or a crash.	Copy the value, or keep the buffer alive for the whole use.
Passing a reference to a local socket into a detached session	Use-after-free when the caller returns.	Pass the socket by value; the frame owns it.
Trusting Content-Length without a limit	A single request exhausts memory or disk.	Check the limit before allocating or transmitting.
Doing CPU work in the session	Every other connection waits.	Offload, chunk, or use a worker pool.
Dropping a suspended task	The frame dies while the completion still points into it.	Drive it to `done()` or `detach()` it.

8. Where to Go Next

The repository has the complete versions of these patterns:

examples/0017.network/async/tcp_echo_server.cc — the two-coroutine echo server from the previous page.
examples/0017.network/async/tcp_client.cc — a client that prints to a socket and transmits the reply to stdout.
examples/0017.network/async/http.cc — an HTTP/1.1 GET client with header scanning and a Content-Length-bounded body transfer.
examples/0017.network/async/http-server.cc — a static file server with percent-decoded paths, directory listings, ETag, ranges and keep-alive.
examples/0017.network/async/ws_echo_server.cc and ws_client.cc — WebSocket framing, including the places where a binary frame needs an exact byte window instead of a formatted scan.

Read them in that order. Each one adds one idea: a connection, a request, a response body, a protocol state machine. The async machinery underneath stays the same.

The next chapter changes subject completely: reflection. Instead of asking the operating system for bytes, it asks the compiler for the shape of your program.