QUIC transport

Active contributors: Roman Arutyunyan, Sergey Kandaurov, Vladimir Homutov

Purpose

src/event/quic/ is nginx's hand-written QUIC implementation. It implements the transport defined by RFC 9000 (QUIC), RFC 9001 (TLS for QUIC), RFC 9002 (loss detection), and the relevant parts of RFC 9221 (datagram). The goal is to provide a stream-multiplexed, encrypted, congestion-controlled UDP transport that the HTTP/3 layer can build on. There is no third-party QUIC library — every byte of frame parsing, packet protection, ACK handling, congestion control, and stream state is in this directory.

QUIC code first appeared in mainline in Mar 2020; it was reorganized into its current home src/event/quic/ on 2020-12-25.

Directory layout

src/event/quic/
├── ngx_event_quic.{c,h}             # the per-connection state machine entry points
├── ngx_event_quic_connection.h      # the big ngx_quic_connection_t struct
├── ngx_event_quic_transport.{c,h}   # frame + packet wire format (parse/encode)
├── ngx_event_quic_protection.{c,h}  # packet number protection, header protection
├── ngx_event_quic_ssl.{c,h}         # OpenSSL TLS 1.3 driver for the QUIC handshake
├── ngx_event_quic_openssl_compat.{c,h}  # shim for OpenSSLs that lack QUIC keying APIs
├── ngx_event_quic_ack.{c,h}         # ACK frame handling, RTT estimation, loss detection
├── ngx_event_quic_output.{c,h}      # outbound packet builder, MTU, pacing
├── ngx_event_quic_streams.{c,h}     # streams, flow control, blocked frames
├── ngx_event_quic_frames.{c,h}      # per-frame helpers (CRYPTO, STREAM, MAX_DATA, ...)
├── ngx_event_quic_connid.{c,h}      # connection ID rotation + lookup
├── ngx_event_quic_migration.{c,h}   # path validation + connection migration
├── ngx_event_quic_socket.{c,h}      # UDP socket binding
├── ngx_event_quic_tokens.{c,h}      # retry tokens, address validation tokens
├── ngx_event_quic_udp.c             # UDP packet receive dispatch
├── ngx_event_quic_bpf.c             # eBPF-based reuseport for QUIC (Linux)
├── ngx_event_quic_bpf_code.c        # the precompiled eBPF program
└── bpf/                             # the C source for the eBPF program

That's 30 C+H files, ~14 K lines.

Key abstractions

Type / function	Role
ngx_quic_connection_t	The big struct: keys, congestion state, streams, ACK ranges, paths
ngx_quic_packet_t	A parsed inbound packet
ngx_quic_frame_t	A parsed inbound frame
ngx_quic_stream_t	Per-stream state — embedded in ngx_connection_t for the stream
ngx_quic_run()	Entry point — start a QUIC connection on an inbound packet
ngx_quic_input()	Process an inbound UDP packet
ngx_quic_output()	Build outbound packets from pending frames
ngx_quic_open_stream()	Open a server-initiated stream (for HTTP/3 server push, mostly unused)

Packet flow

sequenceDiagram
    participant U as UDP socket (ngx_event_quic_udp.c)
    participant D as Connection ID dispatch
    participant Q as ngx_quic_input
    participant F as Frame handler
    participant TLS as ngx_quic_ssl (OpenSSL)
    participant H3 as HTTP/3 layer

    U->>D: datagram with DCID
    D->>D: lookup or create connection
    D->>Q: ngx_quic_input(packet)
    Q->>Q: header protection unprotect
    Q->>Q: packet number space, decryption
    Q->>F: dispatch frames
    F->>TLS: CRYPTO frame -> SSL_provide_quic_data
    F->>F: STREAM, ACK, MAX_DATA, ...
    F->>H3: STREAM frame: stream data ready
    Note over Q: ngx_quic_output produces<br/>response packets and arms the timer

Crypto layers

QUIC has four packet-number-space-distinct levels of protection:

1. Initial — keys derived from the connection ID (no real secrecy, just integrity)
2. Handshake — keys from the early TLS 1.3 handshake state
3. 0-RTT — keys from a prior session, for early data
4. 1-RTT — full handshake-derived keys, used for the rest of the connection

ngx_event_quic_protection.c implements packet protection (AEAD) and header protection (mask) for each level. Each level has its own keys, IV, and packet number space, and ACKs only ack within the same space.

Streams

QUIC streams come in four types based on two bits in the stream ID (initiator + uni-vs-bidi). Server-side, nginx mainly cares about:

• Bidi client-initiated — these become HTTP/3 request streams.
• Uni client-initiated — HTTP/3 control / encoder / decoder streams.
• Uni server-initiated — nginx opens these for HTTP/3 control / encoder / decoder.

Each stream is wrapped in an ngx_connection_t so that higher layers (HTTP/3) can use the same ngx_recv / ngx_send / event interface they'd use on a TCP socket. This is the abstraction that lets the HTTP request engine stay protocol-agnostic.

Code: src/event/quic/ngx_event_quic_streams.c. Flow-control accounting (MAX_DATA, MAX_STREAM_DATA, STREAMS_BLOCKED, etc.) is here; output packet construction is in ngx_event_quic_output.c.

ACK + loss detection

ngx_event_quic_ack.c implements RFC 9002:

• Track sent packet numbers per packet number space, with their content (which frames, what bytes).
• On received ACK, compute newly-acked sets; for each, drop the bookkeeping and update RTT estimators.
• Schedule loss-detection timer.
• On firing, decide whether unacked packets are lost (3-packet threshold or time-based) and re-queue their frames.

Congestion control is the spec's NewReno-inspired default. There's no pluggable BBR / CUBIC; the implementation keeps congestion_window, bytes_in_flight, ssthresh, recovery_start_time like the spec.

Connection migration

Per RFC 9000, a client can move addresses (e.g., Wi-Fi to LTE) and keep the same QUIC connection. ngx_event_quic_migration.c implements path validation: when a packet arrives from a new 4-tuple, the server starts a PATH_CHALLENGE exchange before committing to the new address. Migration is enabled by default but can be disabled via quic_active_connection_id_limit.

ngx_event_quic_connid.c manages the rotating set of connection IDs the server has issued — at any time, multiple CIDs may be valid and a packet can arrive on any of them.

eBPF reuseport (Linux)

Linux's SO_REUSEPORT distributes UDP packets among multiple sockets, but plain hashing on the 4-tuple breaks QUIC connection migration: when a client's address changes, the new packets hash to a different worker. nginx's ngx_event_quic_bpf.c loads an eBPF program (compiled into ngx_event_quic_bpf_code.c) that hashes on the QUIC connection ID instead, so that migration keeps a connection on the same worker.

The eBPF program lives as C source in src/event/quic/bpf/. It's compiled at build time (when --with-quic_bpf is configured) into a byte array embedded in ngx_event_quic_bpf_code.c.

OpenSSL compat

QUIC needs to drive TLS 1.3 with manual access to keying material (per packet number space). OpenSSL 3.x and BoringSSL expose this via SSL_set_quic_method / SSL_provide_quic_data etc. When the linked OpenSSL doesn't expose those APIs, ngx_event_quic_openssl_compat.c shims:

1. Sets up custom BIOs that intercept TLS handshake bytes.
2. Manually drives the OpenSSL state machine.
3. Extracts keys at the right transitions.

This compat layer is one of the more delicate pieces of the codebase. It exists because mainstream OpenSSL only added native QUIC support relatively recently.

Configuration shape

http {
    server {
        listen 443 quic reuseport;
        listen 443 ssl;

        ssl_certificate     /etc/nginx/cert.pem;
        ssl_certificate_key /etc/nginx/key.pem;

        ssl_protocols       TLSv1.3;
        http3               on;
        quic_active_connection_id_limit 2;

        add_header Alt-Svc 'h3=":443"; ma=86400';
    }
}

Alt-Svc is what advertises HTTP/3 to clients arriving over HTTP/1.1 or HTTP/2.

Integration points

• OpenSSL bindings — for TLS 1.3 inside the QUIC handshake.
• HTTP/3 — src/http/v3/ consumes QUIC streams as if they were TCP connections.
• Event loop — UDP packets arrive via ngx_event_quic_udp.c and dispatch by connection ID.
• Configuration — most directives belong to ngx_quic_module (a hidden module within ngx_event_quic.c); a few quic_* directives are exposed at the HTTP server scope.

Entry points for modification

This is the most actively-changing part of the codebase. Recent commits (Apr 2026) have been concentrated on ngx_event_quic_ack.c, ngx_event_quic_streams.c, and the OpenSSL compat layer. Bug fixes go to the per-frame handler files; performance work goes to ngx_event_quic_output.c (packet coalescing, MTU); spec-compliance items (e.g., new transport parameters) touch ngx_event_quic_transport.c.