Upstream

Active contributors: Roman Arutyunyan, Sergey Kandaurov, Maxim Dounin, Roman Semenov

Purpose

The "upstream" subsystem is what handles outbound connections to backends. Every directive that fans out to another server — proxy_pass, fastcgi_pass, uwsgi_pass, scgi_pass, grpc_pass, memcached_pass, mirror, auth_request — runs through it. Upstream provides connection establishment, request body forwarding, response header parsing, response body buffering or streaming, retries across multiple peers, load balancing, the file cache integration, and the keepalive pool to backends.

This is the largest single file in the repository: src/http/ngx_http_upstream.c is ~7,300 lines.

Directory layout

src/http/
├── ngx_http_upstream.{c,h}                 # core upstream framework
├── ngx_http_upstream_round_robin.{c,h}     # default load balancer
├── modules/
│   ├── ngx_http_proxy_module.c             # proxy_pass to HTTP
│   ├── ngx_http_fastcgi_module.c           # FastCGI
│   ├── ngx_http_uwsgi_module.c             # uwsgi
│   ├── ngx_http_scgi_module.c              # SCGI
│   ├── ngx_http_grpc_module.c              # gRPC over HTTP/2
│   ├── ngx_http_memcached_module.c         # memcached protocol
│   ├── ngx_http_mirror_module.c            # request mirroring
│   ├── ngx_http_proxy_v2_module.c          # PROXY-protocol-aware proxy variant
│   ├── ngx_http_upstream_hash_module.c     # consistent hashing LB
│   ├── ngx_http_upstream_ip_hash_module.c  # IP hash LB
│   ├── ngx_http_upstream_keepalive_module.c # connection pool to backends
│   ├── ngx_http_upstream_least_conn_module.c # least-conn LB
│   ├── ngx_http_upstream_random_module.c   # random LB
│   ├── ngx_http_upstream_sticky_module.c   # sticky sessions
│   └── ngx_http_upstream_zone_module.c     # shared-memory upstream state (zone)

Key abstractions

Type / function	File	Role
ngx_http_upstream_t	src/http/ngx_http_upstream.h	Per-request upstream state (peer, buffers, state)
ngx_http_upstream_srv_conf_t	same	Per-upstream { } block config + peer init function pointer
ngx_http_upstream_peer_t	same	A peer entry (server addr:port + weight + max_conns + ...)
ngx_http_upstream_init()	ngx_http_upstream.c	Entry point — called by proxy_pass/fastcgi_pass/etc.
ngx_http_upstream_connect()	same	Open a TCP connection to the next peer
ngx_http_upstream_send_request()	same	Forward the request body
ngx_http_upstream_process_header()	same	Parse the response headers (with module-supplied parser)
ngx_http_upstream_send_response()	same	Stream/buffer/cache the response body
ngx_http_upstream_finalize_request()	same	Tear down (success or failure)
ngx_http_upstream_round_robin_module	ngx_http_upstream_round_robin.c	Default LB; weighted, with backup pool
ngx_http_upstream_keepalive_module	modules/ngx_http_upstream_keepalive_module.c	Per-worker keepalive pool to upstreams

How a request reaches upstream

sequenceDiagram
    participant R as Request (CONTENT phase)
    participant P as ngx_http_proxy_module
    participant U as ngx_http_upstream framework
    participant LB as Load balancer (round_robin/hash/...)
    participant K as Kernel (connect/recv/send)
    participant B as Backend

    R->>P: r->content_handler = ngx_http_proxy_handler
    P->>P: build subrequest body, headers
    P->>U: ngx_http_upstream_init(r)
    U->>LB: get_peer(r)
    LB-->>U: peer addr
    U->>K: connect(peer)
    K-->>U: writable
    U->>B: send request line + headers + body
    B-->>U: response headers
    U->>U: parse via module's process_header
    U->>R: ngx_http_send_header (reverse)
    B-->>U: response body
    U->>R: feed body through filter chain
    U->>U: ngx_http_upstream_finalize_request

The protocol module (proxy, fastcgi, etc.) plugs callbacks into ngx_http_upstream_t:

• create_request — produce the request bytes to send to the backend
• reinit_request — called on retry
• process_header — parse the response status + headers
• abort_request, finalize_request — cleanup
• rewrite_redirect, rewrite_cookie — optional response transforms

The framework handles connect, retry, body streaming, response buffering, and finalization.

Load balancing

ngx_http_upstream_srv_conf_t.peer.init is set by the active load-balancer module to a function that decides peer order per request. Modules:

Module	Algorithm
ngx_http_upstream_round_robin_module	Weighted round robin, with backup peers (default)
ngx_http_upstream_ip_hash_module	Hash on $remote_addr's first three octets
ngx_http_upstream_hash_module	Consistent hash on a configurable key (hash $request_uri;)
ngx_http_upstream_least_conn_module	Pick the peer with the fewest active connections
ngx_http_upstream_random_module	Random with weights, optional two least_conn variant
ngx_http_upstream_sticky_module	Cookie-based sticky sessions; pluggable strategy
ngx_http_upstream_zone_module	Move peer state into a shared zone so it survives across workers

Retries

proxy_next_upstream, fastcgi_next_upstream, etc., are bitmasks of conditions that trigger a retry against the next peer. Defaults: error, timeout. Optionally: invalid_header, http_500, http_502, http_503, http_504, http_403, http_404, http_429, non_idempotent.

ngx_http_upstream_t.peer.tries counts remaining attempts. When the framework decides to retry, it calls ngx_http_upstream_next(r, u, ft_type) which puts the current peer in a "down" state in the LB's view, gets the next peer, and reconnects. The retry budget is bounded by proxy_upstream_tries (or the equivalent for other protocols).

Buffering modes

Two main modes for responding:

• Buffered (proxy_buffering on;, default): the framework reads the response body into proxy_buffers, spilling to a temp file when buffers fill. The downstream client drains independently. Good for slow clients.
• Streaming (proxy_buffering off;): each chunk read from the upstream is immediately forwarded to the client. The framework still buffers headers but not body. Good for SSE / long polling.

The event_pipe (src/event/ngx_event_pipe.c) is the back-pressured pipeline that connects the upstream socket to the downstream filter chain. It manages the file/memory buffers and the read/write triggers.

Cache integration

When proxy_cache zone; is configured, before contacting the backend the framework consults the file cache (src/http/ngx_http_file_cache.c) for a hit. On miss, it adds a "we're fetching" placeholder so concurrent identical requests can collapse onto one backend fetch. After the backend responds with cacheable headers, the response body is split: it goes both to the client and to a cache file. See file-cache.

Keepalive to backends

ngx_http_upstream_keepalive_module keeps a per-worker LRU pool of idle connections to upstream peers. After a response completes, instead of closing, the connection goes back into the pool. The next request that picks the same peer pulls a connection out and skips connect+TLS handshake. Configured via keepalive 32; inside an upstream { } block.

Health checks

The open-source version of nginx does not have active health checks for upstreams. It only marks peers down passively after max_fails failures within fail_timeout. Active probing is an NGINX Plus feature.

Integration points

• HTTP request engine — content handler that calls ngx_http_upstream_init(r). After this, the engine is dormant until finalize_request.
• File cache — read on cache lookup, written to during streaming response, finalized at finalize_request.
• Variables — all the $upstream_* variables come from here ($upstream_addr, $upstream_status, $upstream_response_time, $upstream_cache_status, etc.).
• Filter chain — response body is fed through the standard output filters; the framework just calls ngx_http_output_filter with chains.

Entry points for modification

Adding a new protocol "*_pass" module: copy the structure of ngx_http_memcached_module.c (the smallest one) and provide the create_request / process_header callbacks. Adding a new load balancer: register a peer.init_upstream and a get/free peer pair. Don't touch ngx_http_upstream.c itself unless you're changing core framework behavior — most use cases extend it from a module.