Internals¶

How a request actually moves through the process. Useful if you are reading the source, debugging something odd, or deciding whether this design suits your workload.

Layout¶

src/            Rust crate, built as the oxbrook._core extension module
  server.rs     tokio accept loop, hyper 1, HEAD/405/413, upgrade handshake
  router.rs     matchit radix tree per method, path and query coercion
  queue.rs      bounded per-worker queue + socketpair wakeup
  worker.rs     one OS thread + one asyncio loop per worker, drain callback
  request.rs    the frozen Request handed to handlers
  responder.rs  reply channel, streaming bodies, client-disconnect signal
  websocket.rs  tokio-tungstenite bridge
python/oxbrook/  App, routing, pydantic, OpenAPI, topics, SSE, sockets, runtime

The request path¶

A tokio thread accepts the connection and hyper parses the request.
If the app has a CORS policy, a preflight is answered here. The router matches the request against a radix tree built per HTTP method, and coerces path and query parameters into owned Rust values. A request that cannot succeed — bad path parameter, missing required query parameter, wrong method, body over the limit — is answered here, and no Python worker is ever woken. The body is collected up to max_body, unless the route streams it.
The request becomes a plain Rust struct and is pushed onto the bounded queue of the least-loaded worker, counting queued plus in-flight requests. A loop held by a handler that computes cannot drain, so its count stays high and new requests go elsewhere. If that worker is at its limit it tries the next; if every worker is full the answer is 503.
If no wakeup is already in flight, one byte goes down a socketpair.
The worker's asyncio loop wakes through add_reader. A native drain callback clears the flag, pops every queued request, and schedules each handler in that one callback.
The handler returns. A dict is serialized to JSON in Rust; a pydantic model through pydantic's own serializer. The bytes travel back to the waiting tokio task over a oneshot channel.

Two properties carry the design: Python is only ever touched from the worker's own thread, and a burst of requests collapses into one wakeup.

A WebSocket upgrade takes a branch after step 2: its Origin is checked in Rust, and a refused origin is answered 403 before an authorizer or handler is ever queued.

On the way back out, CORS headers are added in Rust to every response, including the ones the server produced itself in step 2.

Invariants¶

These are load-bearing. Breaking one silently destroys performance or deadlocks.

Tokio threads never attach to the interpreter. No Python::attach, no Py::new, no refcount touch. Worth 5.4x.

Never block in native code while attached. Any blocking wait is wrapped in py.detach. Blocking while attached deadlocks free-threaded CPython at a stop-the-world point — which is exactly how this framework's first startup deadlock happened.

Wakeups coalesce. At most one wake byte in flight, and the flag is cleared before draining. Clearing it after loses a racing push.

Anything a tokio thread needs to tell Python — a request, or a notification that a client disconnected — goes through that same queue and that same byte. A tokio thread that schedules work by calling into the interpreter can block inside the event loop's self-pipe write, and on the GIL build a thread blocked while attached stops every other thread in the process.

Handlers are async def, enforced at registration.

Both Python builds work. Free-threaded 3.14t is the primary target; the GIL build runs a single worker loop.

Agent exposure is opt-in. Only tool=True routes reach /mcp.

A resource limit is never released by garbage collection. The concurrency slot is freed by an explicit call, never by Drop and never by the cyclic collector.

Anything Rust validates, Rust canonicalises, so Python's constructors cannot fail on input Rust already accepted.

Never return exception detail to a client. Tracebacks go to the log; debug=True is the only exception, and that flag is per server, never module state.

Concurrency model¶

One OS thread per worker, each running one asyncio loop, all in one process. Handlers on different loops run Python in parallel on a free-threaded build.

The Request is frozen, so it needs no locking. Topics are shared across loops and each subscription is bound to the loop that created it, which is how a producer on another worker knows where to deliver.

Streaming¶

A streaming response holds a guard that hyper drops when the connection ends. The pump races the next message against that guard, so a disconnected client is noticed without polling — including on a stream that is sitting idle on a quiet topic, which has nothing to write and therefore nothing that would fail.

Streaming request bodies¶

A route that takes a BodyStream is dispatched before its body is read. The body is pumped from hyper into a queue by the connection's own request future, so the pump cannot outlive the request: it starts only when the handler first asks for a chunk, stops reading the socket while more than about a megabyte is waiting, and is dropped when the response is sent. Chunks reach the handler through the worker queue's wakeup path, the same one sockets use, so the tokio thread never touches the interpreter.

The request timeout measures from the body's last progress. While the pump is waiting on the client, only the pump's own idle timeout applies, so a client's stall is a 408 and never races into a 504.

Startup and shutdown¶

Each worker thread runs worker_lifespan on its own loop before registering its drain callback, and again after its loop stops, having first removed the callback so nothing new starts during teardown. Workers start one at a time; if one fails, the ones already running are stopped and joined before the error surfaces. At shutdown — on SIGINT or SIGTERM, both handled in the accept loop — the server waits for every worker thread, bounded by shutdown_grace, and only then runs the process lifespan's teardown.

Testing¶

Twenty-five standalone scripts under tests/, each exiting non-zero on failure, run against a real server on a real socket.

make verify        # free-threaded
make verify-gil    # GIL build

tests/verify.py drives 5,000 concurrent requests, each carrying a unique token, and asserts every response comes back with its own. That is the check that matters for queue dispatch, where the plausible bug is a reply delivered to the wrong request.

Each case in tests/hardening.py names a defect that was demonstrated against a running server before it was fixed.

Known gaps¶

No TLS and no HTTP/2. Expects a terminating proxy in front.
Middleware does not wrap socket handlers, only their authorizer.
WebSocket origins are exact strings; there are no patterns for preview deployments.
MCP is POST/JSON only: no streaming responses, no server-to-client channel, no resource subscriptions.