Serializer redesign doc

doc/design/serializer_redesign.md

@@ -0,0 +1,304 @@
+# Serializer redesign
+
+## Summary
+
+This document describes a redesign of the library's message serializer. As in the existing design, the serializer uses a single block of memory allocated at construction time and never exceeds it. That memory is reused across messages. The serializer remains a strictly sans-I/O component, while stream handling moves to a separate `message_writer`. Body octets supplied by the caller are framed by reference, so a single gathered write can cover the header, chunk framing, and caller-owned memory without copying; small pieces are coalesced in a staging buffer that the caller can also write into directly. Because the serialized header is held back until output is actually flushed, the framing can still be improved after `start`: a chunked or close-delimited body whose total size becomes known before transfer begins is sent with an explicit `Content-Length` instead. Chunked trailers can now be sent, content codings are applied through a caller-supplied encoder, and the accounting remains exact when an operation is cancelled mid-write, so an interrupted message can be resumed.
+
+The design is not speculative: it is implemented and tested in the Burl project (<https://github.com/cppalliance/burl>).
+
+## The serializer interface
+
+```cpp
+class serializer
+{
+public:
+    /// Content encoder interface.
+    struct encoder
+    {
+        struct result
+        {
+            std::size_t consumed;
+            std::size_t produced;
+            std::error_code ec;
+        };
+
+        virtual ~encoder() = default;
+
+        virtual result
+        process(
+            capy::mutable_buffer out,
+            capy::const_buffer in,
+            bool more) = 0;
+    };
+
+    /// Settings that apply for the lifetime of
+    /// the serializer.
+    struct config
+    {
+        /// The space reserved for staging
+        /// body octets.
+        std::size_t stage_buffer = 64 * 1024;
+
+        /// The free staging capacity below
+        /// which should_drain reports true.
+        std::size_t min_prepare = 4 * 1024;
+
+        /// Supplied body data at least this
+        /// large is framed by reference,
+        /// without copying.
+        std::size_t min_direct = 2 * 1024;
+
+        /// The space reserved for the encoder
+        /// input stage; unused when no encoder
+        /// is installed.
+        std::size_t enc_buffer = 8 * 1024;
+
+        /// Bodies smaller than this may skip
+        /// encoding entirely.
+        std::size_t enc_threshold = 4 * 1024;
+    };
+
+    /// Constructor; performs the single
+    /// allocation.
+    explicit
+    serializer(config const& cfg);
+
+    /// Return true if every octet of the
+    /// message, including any trailer, has
+    /// been consumed.
+    bool
+    is_done() const noexcept;
+
+    /// Return true if every octet of the
+    /// serialized header has been consumed.
+    bool
+    is_header_done() const noexcept;
+
+    /// Return true if staged octets should
+    /// be drained.
+    bool
+    should_drain() const noexcept;
+
+    /// Start a message; the framing is
+    /// selected from msg->payload(). `head`
+    /// serializes the response to a HEAD
+    /// request, emitting only the header.
+    void
+    start(
+        message_head_base* msg,
+        encoder* enc = nullptr,
+        bool head = false) noexcept;
+
+    /// Set the trailer fields, serialized
+    /// after the final chunk.
+    void
+    set_trailer(fields_base const* t) noexcept;
+
+    /// Return a buffer for writing body
+    /// octets into the staging area.
+    std::span<capy::mutable_buffer>
+    prepare(std::span<capy::mutable_buffer> dest);
+
+    /// Make octets written into the region
+    /// returned by prepare part of the body.
+    void
+    commit(std::size_t n) noexcept;
+
+    /// Assign `dest` with descriptors for the
+    /// octets to transfer next, framing the
+    /// supplied body octets.
+    template<capy::ConstBufferSequence CB>
+    std::span<capy::const_buffer const>
+    frame(
+        std::span<capy::const_buffer> dest,
+        CB const& buffers,
+        bool more,
+        std::error_code& ec);
+
+    /// Equivalent to frame with an empty
+    /// buffer sequence.
+    std::span<capy::const_buffer const>
+    frame(
+        std::span<capy::const_buffer> dest,
+        bool more,
+        std::error_code& ec);
+
+    /// Report transferred octets; return how
+    /// many supplied body octets were accepted.
+    std::size_t
+    consume(std::size_t n) noexcept;
+};
+```
+
+## Zero-copy framing of user-provided buffers
+
+The new serializer interface allows the user to provide their buffers during the framing step. Because serialization and framing happen in the same call, the serializer can use the supplied buffers directly as part of the destination buffer sequence that is expected to be written to the stream. This simple change provides a streaming interface for writing body data from an external source without copying it into the serializer's internal buffer.
+
+The following is a possible implementation of a `write_some` algorithm using this interface:
+
+```cpp
+template<
+    capy::WriteStream S,
+    capy::ConstBufferSequence CB>
+capy::io_task<std::size_t>
+write_some(
+    S& stream,
+    serializer& sr,
+    CB buffers,
+    bool more)
+{
+    capy::const_buffer_param<CB> bp(buffers);
+    capy::const_buffer dest[16];
+    for(;;)
+    {
+        std::error_code ec;
+        auto const body = bp.data();
+        auto const bufs = sr.frame(
+            dest, body, more || bp.more(), ec);
+        auto [wec, n] = co_await stream.write_some(bufs);
+        auto const k = sr.consume(n);
+        bp.consume(k);
+        if(ec)
+            co_return { ec, k };
+        if(wec)
+            co_return { wec, k };
+        if(k != 0 || !bp.more())
+            co_return { std::error_code(), k };
+    }
+}
+```
+
+## Optimization of I/O-layer write operations
+
+There are two new configuration parameters that allow optimization of the number of write operations at the I/O layer:
+
+- `config::min_direct` sets a threshold for buffer sizes that the serializer frames directly in the destination buffers for writing. Buffers smaller than this threshold are copied into the serializer's internal buffer, where they wait for more data or for the end of the message.
+
+- `config::min_prepare` determines the minimum amount of internal buffer space that the serializer provides to the user in calls to `prepare`. As long as the serializer can satisfy that requirement, it does not hint to the I/O layer that it should flush, thereby reducing the number of write operations.
+
+## Late framing decisions
+
+Because the serializer does not request a flush until there is a reason to do so (e.g. enough body data has accumulated, the end of the body has been declared, etc.), it may still alter the framing-related fields of the header:
+
+- When the complete body arrives before the first write, chunked or close-delimited framing is replaced by an explicit `Content-Length`, and `chunked` is removed from the `Transfer-Encoding` field.
+
+- When an encoder is installed but the complete body is smaller than `config::enc_threshold`, the serializer may skip encoding entirely, remove the `Content-Encoding` field, and serialize the body unencoded.
+
+Both rewrites require that no header octet has been consumed. Setting a trailer suppresses the `Content-Length` rewrite because trailers require chunked coding. Once transfer of the header has begun, the message head is never modified.
+
+In practice, this means that the common pattern of handing the whole body to a single `write_eof` produces a `Content-Length`-framed message even when the caller never computed the size, while true streaming falls back to chunked framing automatically. For example, in the following scenario, the proper value for the `Content-Length` field is set automatically:
+
+```CPP
+response_head res;
+res.set_chunked(true); // total size unknown up front
+
+serializer sr(cfg);
+message_writer writer(&stream, &sr);
+sr.start(&res);
+
+std::string body = make_response();
+
+// The complete body arrives before the first write,
+// so the message goes out with an explicit
+// Content-Length instead of chunked framing.
+auto [ec, n] = co_await writer.write_eof(
+    capy::make_buffer(body));
+```
+
+## Trailer fields can now be sent
+
+The existing serializer has no way to emit trailer fields. In the new design, `set_trailer` installs a caller-owned field container whose wire image is serialized after the final chunk of a chunked body; with any other framing, the trailer is ignored.
+
+## No need for an encoder service
+
+An `encoder` passed to `start` applies a content coding to the body, and the encoder's output is framed in its place. As with the parser's decoder, the higher layer supplies the instance: it decides which codings to offer or accept, so it can construct the matching encoder, and arbitrary codings are supported without building any into the serializer. Setting the `Content-Encoding` field remains the caller's responsibility. However, as described in the late-framing section above, the serializer may skip encoding and remove the `Content-Encoding` header. A user can set `config::enc_threshold` to zero to force encoding for all body sizes.
+
+## No need for a serializer configuration service
+
+The serializer reads its configuration only during construction, at which point the single allocation is performed and the tuning values are stored as members. Since the configuration is never consulted again, there is no need for a shared reference or a configuration service. `start` performs no allocation and merely re-points the regions according to whether an encoder is in use.
+
+## No need for storing user-provided buffer sequences
+
+The `frame` interface receives the user-provided buffer sequence and in the same call frames it into the destination buffer span. Any octets that are not accepted remain with the caller, which re-offers them on the next call, so the serializer never needs to flatten the sequence into an internal array of descriptors or keep it alive between calls.
+
+## Error handling
+
+All errors surface from `frame`. When `frame` reports an error, the message is failed and serialization cannot proceed; the only valid operations are `start` and destruction. Error codes are split according to where the contract was broken:
+
+- A body that disagrees with a size declared in the message, a `Content-Length` mismatch, or body octets supplied for a bodiless message or a HEAD response results in `error::body_size_mismatch`.
+
+- Breaking the interface contract under chunked or close-delimited framing (for example, supplying more octets after the end of the body has been declared) results in `std::errc::invalid_argument`.
+
+- An error returned by the encoder propagates as-is, and subsequent calls on the failed message return `std::errc::state_not_recoverable`.
+
+## Cancellation and resumption
+
+The serializer's accounting is designed to remain faithful to the wire when an operation is cancelled mid-write. Completion counts already reported by the stream cover exactly the consumed octets, `consume` translates them into accepted body octets, and no per-operation counters exist that could be lost with a cancelled coroutine frame. A caller resumes an interrupted message either by re-offering the unconsumed remainder of its buffers or by committing that remainder into the staging buffer and draining it; both approaches continue the message from precisely where the wire stopped.
+
+## Quality of the implementation
+
+### One gathered write covers the whole message
+
+`frame` returns a single span of descriptors containing the remaining header octets, the current chunk prefix merged with the staged octets, the supplied body by reference, the chunk epilogue, and the trailer headers. The chunk prefix is written backwards into a small margin directly ahead of the staging region, so the prefix and the staged data form one contiguous descriptor instead of two. A small message therefore goes out in a single `write_some`, and a large zero-copy body adds exactly one gathered write per buffer.
+
+## The `message_writer` interface
+
+```cpp
+template<capy::WriteStream S>
+class message_writer
+{
+public:
+    /// Constructor; the stream and serializer
+    /// must outlive the writer.
+    message_writer(S* stream, serializer* sr) noexcept;
+
+    /// Return writable staging memory.
+    std::span<capy::mutable_buffer>
+    prepare(std::span<capy::mutable_buffer> dest);
+
+    /// Commit staged octets, flushing the
+    /// staging buffer when it runs low.
+    capy::io_task<>
+    commit(std::size_t n);
+
+    /// Commit final octets and end the body.
+    capy::io_task<>
+    commit_eof(std::size_t n);
+
+    /// End the body with no more octets.
+    capy::io_task<>
+    write_eof();
+
+    /// Write the header and any staged data,
+    /// without ending the body.
+    capy::io_task<>
+    write_header();
+
+    /// Write at least one octet of `buffers`;
+    /// small inputs are coalesced without I/O.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write_some(CB buffers);
+
+    /// Write until `buffers` is fully consumed.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write(CB buffers);
+
+    /// Write final octets and end the body.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write_eof(CB buffers);
+};
+```
+
+The rationale for `message_writer` is to satisfy the `capy::WriteStream`, `http::WriteSink` and `http::BufferSink` concepts, so it composes with generic stream algorithms.
+
+`message_writer::write_header` flushes pending output without ending the body. This is the building block for `Expect: 100-continue` support in higher-level libraries, where the server must receive the header before the body is generated. Once it returns, the framing and encoding are fixed, and the late-framing rewrites no longer apply.
+
+## Related links
+
+Reference documentation for the Burl implementation:
+
+- [`serializer`](https://develop.burl.cpp.al/burl/reference/boost/burl/serializer.html)
+- [`message_writer`](https://develop.burl.cpp.al/burl/reference/boost/burl/message_writer.html)