Running a server¶
The target is module:attribute — here, the app in main.py — imported from
the working directory. With no attribute, app is assumed. The same command is
installed as oxb, and runs as python -m oxbrook.
From Python, the same server with the same options:
Every option can also be passed to app.build_server(...), which prepares a
server without starting it — that is what the test client
uses.
| command line | app.run |
default | what it does |
|---|---|---|---|
--host |
host |
127.0.0.1 |
interface to bind |
--port |
port |
8000 |
port to bind |
--workers |
workers |
detected | Python worker loops |
--max-concurrency |
max_concurrency |
1024 | requests per worker, queued plus in-flight |
--max-connections |
max_connections |
2048 | sockets held open |
--max-body |
max_body |
16 MiB | largest request body, in bytes |
--max-message |
max_message |
16 MiB | largest WebSocket message, in bytes |
--request-timeout |
request_timeout |
30.0 | seconds to a handler's first response, then 504 and the handler is cancelled |
--shutdown-grace |
shutdown_grace |
10.0 | seconds a stop waits for in-flight requests |
--tls-cert |
tls_cert |
none | PEM certificate chain; serve HTTPS |
--tls-key |
tls_key |
none | PEM private key for that certificate |
--no-http2 |
http2=False |
HTTP/2 on | serve HTTP/1.1 only |
The command also takes --access-log, to log every request, and
--log-level, which sets the level of Oxbrook's own loggers.
Reloading during development¶
Saving a Python file under the working directory restarts the server. Each restart is a new process, not modules re-imported in place: the Rust extension cannot be reloaded inside a running interpreter, and a fresh process runs the lifespans again, so after a reload the app is in exactly the state a deploy would leave it in.
--reload-dir DIRwatches another directory, and can be repeated.--reload-include '*.html'restarts for other files too.- Hidden directories,
__pycache__,node_modules,venv,target,site,distandbuildare never watched.
A syntax error stops the server, not the watcher: the error is printed, and the
next save starts it again. A server being replaced gets one second to finish its
requests rather than the usual ten, so an open SSE stream in a browser does not
hold every restart; --shutdown-grace overrides that.
--reload is for development. It polls files for changes and runs a second
process to do it.
Inspecting an app¶
oxbrook routes main:app # every route, including the built-in ones
oxbrook routes main:app --json
oxbrook openapi main:app -o openapi.json
routes marks tools, WebSockets, streaming bodies, forms and router middleware,
and lists /openapi.json, /docs and /mcp as the server would serve them.
openapi writes the document without starting a server, for generating clients
in CI.
For an app built by a function, pass --factory: oxbrook run main:create_app
--factory. A target that cannot be loaded exits with status 2 and says why;
an exception raised while importing the app shows its traceback.
Worker loops¶
Each worker is one OS thread running one asyncio loop. They share the process, which is what lets a topic reach subscribers on all of them.
The default is measured, not guessed:
- On a GIL build, always one. One loop was best or tied at every handler cost in the sweep; extra loops only cost throughput.
- On a free-threaded build, one is never best. Even a handler that does nothing gains from more loops, and a handler doing 500µs of work gains 3.4x. Throughput peaks around the performance-core count and falls off once the efficiency cores are oversubscribed.
So the default is "how many cores can actually run Python in parallel", which
is not os.cpu_count(): that counts efficiency cores, and inside a container it
reports the host's cores rather than the cgroup quota — which would start
dozens of loops for a two-CPU limit. Oxbrook probes cgroup quotas, CPU affinity
and performance-core counts, takes the most constrained answer, and caps it at
8.
The cap is a guard against an absurd probe result, not a measured ceiling. Raise it explicitly on a large homogeneous machine with CPU-heavy handlers:
More loops is for CPU, not for I/O
Awaiting I/O does not need more loops. An await yields the loop, so one
loop holds thousands of them. What needs more loops is CPU time spent
inside the handler.
Backpressure¶
Each worker accepts at most max_concurrency requests at once, counting both
those queued and those already running. Past that the request tries another
worker, and if every worker is full the server answers 503 with
Retry-After: 1.
Counting in-flight requests is the part that matters. The drain callback empties the queue into asyncio tasks immediately, so a handler that awaits I/O leaves the queue near empty while thousands of requests pile up inside the loop. Bounding the queue alone would look like backpressure and protect nothing.
Lower it for slow handlers, where a deep backlog only adds latency before an inevitable client timeout. Raise it to absorb larger bursts of fast requests.
max_connections is a separate limit, because an idle keep-alive connection
costs a file descriptor without ever reaching a worker. An open WebSocket
counts as the connection it was upgraded from, until it closes. At that limit the
server stops accepting rather than refusing, so the wait lands in the OS
backlog where a client's own connect timeout governs it.
HTTPS and HTTP/2¶
oxbrook run main:app --host 0.0.0.0 --port 443 \
--tls-cert /etc/certs/fullchain.pem --tls-key /etc/certs/privkey.pem
app.run(host="0.0.0.0", port=443,
tls_cert="/etc/certs/fullchain.pem", tls_key="/etc/certs/privkey.pem")
The certificate file holds the chain, the server's own certificate first, as
Let's Encrypt's fullchain.pem does; the key is PKCS#8, PKCS#1 or SEC1 PEM.
Both are read once, when the server starts, and checked against each other: a
missing file, an empty one or a key that belongs to another certificate stops
the server with a message naming the file, before it binds. TLS 1.2 and 1.3
are served; nothing older.
With TLS, the port speaks HTTPS only. A plain http:// request to it fails,
so redirecting HTTP to HTTPS takes a second listener or a proxy.
HTTP/2 is on by default, alongside HTTP/1.1, and needs nothing from the
app: routes, middleware, streaming bodies, SSE and static files behave the
same. Over TLS a client asks for it during the handshake (ALPN), which is what
every browser does. On a plain port the server recognises HTTP/2 from a
connection's first bytes, but only clients configured for HTTP/2 without TLS,
such as curl --http2-prior-knowledge or httpx.Client(http1=False,
http2=True), will use it; browsers never do. http2=False, or --no-http2,
serves HTTP/1.1 alone and stops offering HTTP/2 during the handshake.
The :authority of an HTTP/2 request is copied into the host header when the
client sent none, so a handler reads the host the same way over either
protocol.
What HTTP/2 does not change:
- WebSockets use HTTP/1.1. The server does not offer WebSockets over
HTTP/2, so a browser opens a separate HTTP/1.1 connection for a socket, as it
does for any server that does not offer them. A socket route asked over
HTTP/2 answers
426. - Capacity limits are the same. One HTTP/2 connection can carry many
requests at once, and each one still counts against
max_concurrency.
Limits specific to connections:
| limit | value | why |
|---|---|---|
| TLS handshake | 15 s | a client that connects and never finishes the handshake would hold a slot |
| request headers, HTTP/1.1 | 15 s | the same for headers sent slowly; also closes an idle keep-alive connection |
| idle HTTP/2 connection | 15–20 s | with nothing in flight, the server sends GOAWAY; a client that does not acknowledge it is dropped 15 s later |
| streams per HTTP/2 connection | 200 | concurrent requests one client may have open |
| handlers per HTTP/2 connection | 200 | including handlers whose stream the client reset; beyond it, new requests get 503 |
The last row exists because a client can reset a stream the moment it opens
it (CVE-2023-44487, "rapid reset"). The reset cancels the
handler, but only once its worker gets
to it, and a handler with cancel_on_disconnect=False is not cancelled at all;
counting handlers until they actually end keeps one connection from starting
more of them than its share.
Not supported: reloading a renewed certificate without a restart, client
certificates, HTTP/3, and the Upgrade: h2c handshake from HTTP/1.1. A
certificate renewed on disk takes effect at the next start; with --reload in
development, --reload-include '*.pem' does that.
In front of it¶
A terminating proxy in front — nginx, Caddy, a cloud load balancer — is still the usual arrangement in production: it renews certificates, redirects plain HTTP, and holds slow clients. Oxbrook's own TLS suits a service with no proxy at all, an internal service that must still be encrypted, and development against a browser feature that requires HTTPS.
WebSocket upgrades are accepted from the app's own origin, which is recognised
through Host or X-Forwarded-Host. A proxy that rewrites Host without
setting X-Forwarded-Host makes every browser socket look cross-origin and
refused with 403; forward the original host, or list the public origin in
websocket_origins.
A proxy or CDN can serve static files itself, with its own cache, and pass everything else through; for heavy static traffic that is still the better arrangement.
If the proxy also adds CORS headers, configure CORS in one place only: two
Access-Control-Allow-Origin headers on one response make a browser reject it.
A proxy that buffers request bodies defeats a streaming
upload; turn request buffering off for those
routes (in nginx, proxy_request_buffering off).
Containers¶
The repository ships a multi-stage Dockerfile that installs free-threaded
3.14 with uv, builds the wheel, and installs it into a slim runtime image.
The official Python images carry no free-threaded interpreter, so installing it
explicitly keeps the container and the development machine on the same
interpreter.
Bind to 0.0.0.0 inside a container. Anything listening only on loopback is
unreachable from outside it.
Use the exec form, as above, so the server is the process that receives the container's stop signal. Wrapped in a shell, the shell receives it instead.
Benchmark on the Docker network, not through a published port
Traffic from macOS into Docker Desktop's VM through -p measured 3.3x
slower than native, and the cost is the port forwarding, not the
container: with the load generator in a second container on the same
network, the same image measured faster than native. See
performance.
Shutdown¶
Ctrl-C (SIGINT) and SIGTERM both stop the server gracefully: it stops
accepting, waits up to shutdown_grace for in-flight requests, runs the
lifespan teardown, then exits with status 0. Streams and sockets are closed.
On Windows, which has no SIGTERM, the same drain runs on Ctrl-C, on
Ctrl-Break — what oxbrook run --reload sends its server — and when the
console window closes, though Windows allows only a few seconds for the last
of these before it ends the process regardless.
SIGTERM is what docker stop, Kubernetes and systemd send, so a deploy drains
requests rather than cutting them off. Set the orchestrator's own grace period —
terminationGracePeriodSeconds, docker stop --time — longer than
shutdown_grace, or it kills the process before the drain ends.