Hands out and returns BufferHandle byte buffers from one of
four size classes (1 KiB / 4 KiB / 16 KiB / 64 KiB) so the
HTTP/1.1 reactor's read- and write-side temporaries don't pay
alloc(N) / free(N) per request.
Background
flare's reactor reads from each accepted socket into a
List[UInt8] chunk buffer (ServerConfig.read_buffer_size,
default 8 KiB) per recv(2) call, and serialises every
response into a second contiguous List[UInt8] before the
send(2) (or writev(2) via :mod:flare.runtime.iovec)
syscall. Both lists
are constructed-and-destructed once per request on the keep-alive
hot path. At realistic high-throughput targets (>200K req/s on
4 workers) that's ~1.7 M alloc + free pairs per second per
buffer side — glibc's malloc is fast but the cache-line eviction
it causes for every small allocation is not free.
The Rust analogues to look at:
hyper keeps a per-connection 8 KiB read buffer that is
reused for the lifetime of the connection (no realloc per
request, no per-message-frame Vec::new). The buffer is
owned by the connection state, not pooled across connections.actix-web and axum (via hyper) inherit the same
shape; actix's framework adds a per-worker BytesMut
freelist for response builders.
BufferPool picks the more conservative shape: a per-worker pool
of fixed-size-class buffers that is independent of any
connection. The reactor borrows a buffer for one request, fills
it, drains it, and returns it to the pool; the next request on
any connection (same worker) gets the same buffer back. No
cross-worker handoff, no atomic, no mutex — same per-worker
discipline as DateCache and ResponsePool.
Size classes
Four power-of-two classes pinned at:
- 1 KiB — tiny payloads (most plaintext / health-check
responses, the TFB plaintext target itself).
- 4 KiB — typical JSON responses, typical HTTP/1.1 request
headers + small body.
- 16 KiB — moderate payloads (small file responses, a
typical compressed-body buffer).
- 64 KiB — large reads / writes (chunked-body single-chunk
serialisation upper bound, large file-served responses).
The acquire(min_capacity) API takes a minimum capacity and
rounds up to the smallest size class that fits. Requests larger
than 64 KiB fall through to a one-off heap allocation that bypass
the pool — the buffer is destroyed on release rather than
recycled, so giant requests don't cause the pool to grow
unbounded.
What this module provides
BufferHandle — the value moved in / out of the pool. Wraps
a List[UInt8] plus an Int recording the size class so the
matching pool bucket knows which class to push into on
release. Movable (not Copyable) for the same reason
Response is.BufferPool — per-worker bucketed pool with the four size
classes above. acquire(min_capacity) -> BufferHandle /
release(var BufferHandle) shape. Each bucket is bounded
at a small capacity (default 8 per class) so the pool's
steady-state memory is at most 8 * (1+4+16+64) KiB ≈ 680 KiB per worker.BufferPool.with_class_capacity(N) factory for tests and
custom workloads.BufferPool.size_for(min_capacity) -> Int helper exposed
publicly for callers that want to allocate without pooling
but still snap to the canonical size classes (e.g. on the
oversize fall-through path).
Storage strategy
Same as ResponsePool (B6): BufferHandle is Movable
but not Copyable, so each per-class bucket is a List[Int]
of heap addresses managed via Pool[BufferHandle]. acquire
pops an address, takes the pointee, frees the cell.
release moves the supplied handle into a fresh cell and
pushes the address. Net allocation cost: one Pool.alloc/free
pair per acquire/release; the actual win is that the underlying
List[UInt8] capacity is preserved across the move-in / out.
Wiring into the reactor's accept-and-read path is a follow-up;
this module provides the primitive + tests + re-exports.
v0.10 wiring result -- NOT WIRED. ConnHandle.read_buf /
write_buf are allocated once per connection, not per request, and
Pool[ConnHandle].free genuinely destroys the handle, so the
allocation this would remove happens once per accept. On a keep-alive
workload (64 persistent connections serving millions of requests) there
is essentially nothing to save; the target is connection churn.
So the measurement to run before wiring is the churn config, not
throughput. Wiring also needs a per-worker pool threaded through
_conn_alloc_addr and the accept paths, since the pool must outlive
the handles and must not be shared across threads.