Process model

Active contributors: Sergey Kandaurov, Maxim Dounin

Purpose

NGINX runs as a small group of processes, not a single multi-threaded one. A privileged master process forks unprivileged workers, each of which has its own event loop and connection table. Workers don't share heap memory; they share state through mmap'd zones (created by ngx_shared_memory_add()) and coordinate via signals to the master and a per-worker socketpair "channel."

Directory layout

src/os/unix/
├── ngx_process.{c,h}        # fork(), set up signal handlers
├── ngx_process_cycle.{c,h}  # the master/worker state machines
├── ngx_channel.{c,h}        # the master <-> worker socketpair protocol
├── ngx_setproctitle.{c,h}   # rename "nginx: master process" / "nginx: worker process"
├── ngx_setaffinity.{c,h}    # worker_cpu_affinity
└── ngx_daemon.c             # detach to background
src/os/win32/
└── ngx_process_cycle.{c,h}  # Windows variant (no master)

Key abstractions

Symbol	File	Role
ngx_master_process_cycle()	src/os/unix/ngx_process_cycle.c	The master's main loop
ngx_worker_process_cycle()	same	Each worker's main loop
ngx_cache_manager_process_cycle()	same	The cache manager process
ngx_start_worker_processes()	same	Fork workers
ngx_signal_worker_processes()	same	Send a signal to all workers
ngx_reap_children()	same	wait() on dead workers, optionally respawn
ngx_processes[]	src/os/unix/ngx_process.h	Table of children (pid, status, channel fd)
ngx_channel_t	src/os/unix/ngx_channel.h	Message format on the master/worker socketpair
ngx_quit, ngx_terminate, ngx_reconfigure, ngx_reopen, ngx_change_binary, ngx_noaccept	src/os/unix/ngx_process_cycle.c	sig_atomic_t flags set by signal handlers, polled by the loops

State machines

stateDiagram-v2
    [*] --> StartMaster
    StartMaster --> Running: fork workers + cache manager
    Running --> Reconfigure: SIGHUP
    Reconfigure --> Running: build new cycle, fork new workers, signal old
    Running --> Reopen: SIGUSR1
    Reopen --> Running: workers reopen log files
    Running --> NewBinary: SIGUSR2
    NewBinary --> Running: exec new master, new workers
    NewBinary --> Rollback: SIGHUP to old + SIGTERM to new
    Running --> ShuttingDown: SIGQUIT (graceful) / SIGTERM (immediate)
    ShuttingDown --> [*]

The master keeps these sig_atomic_t globals in sync with incoming signals (handlers in ngx_process.c):

Flag	Triggered by	Effect on master loop
ngx_reap	SIGCHLD	wait() on each dead child, possibly respawn
ngx_terminate	SIGTERM/SIGINT	Kill workers immediately, exit
ngx_quit	SIGQUIT	Tell workers to stop accepting and finish in-flight, then exit
ngx_reconfigure	SIGHUP	Re-read config, build a new cycle, fork new workers, SIGQUIT old workers
ngx_reopen	SIGUSR1	Tell workers to reopen log files (log rotation)
ngx_change_binary	SIGUSR2	exec a new nginx binary; new master inherits the listen sockets
ngx_noaccept	SIGWINCH	Tell workers to stop accepting (used during graceful binary upgrade)

A worker has a smaller set: ngx_quit and ngx_terminate (passed via the channel from the master), plus its own ngx_reopen. Workers don't see SIGHUP directly — the master kills + forks them on a reconfigure.

The channel

Master and each worker are connected by a socketpair opened before the fork. The master sends typed messages over it (NGX_CMD_OPEN_CHANNEL, NGX_CMD_CLOSE_CHANNEL, NGX_CMD_QUIT, NGX_CMD_TERMINATE, NGX_CMD_REOPEN) using ngx_write_channel(). Workers read them in ngx_channel_handler() (src/os/unix/ngx_process_cycle.c), which is registered as a regular event handler on the channel fd. The fd-passing capability of sendmsg()'s ancillary data is how the master tells worker N about worker M's channel: the workers can talk to each other this way (mostly for cache loader coordination).

Channel messages are tiny and synchronous from the worker's side; nginx never tries to push large payloads over the channel.

Reconfigure

SIGHUP triggers ngx_init_cycle(old_cycle) (src/core/ngx_cycle.c) which:

1. Allocates a new ngx_cycle_t with a fresh pool.
2. Re-runs the configuration parser on the (possibly changed) nginx.conf.
3. Re-uses listening sockets from old_cycle->listening where the address+port match.
4. Calls init_module on each module against the new cycle.
5. Returns the new cycle.

The master then forks new workers against the new cycle, sends NGX_CMD_QUIT to the old workers through their channels, and the old ones drain their connections and exit. Listening sockets stay open the whole time — connections in flight are not affected.

If anything fails during the new cycle build, ngx_init_cycle() returns NULL, the old cycle remains active, and the master logs an error. This is the basis of nginx's reload-without-downtime guarantee.

Binary upgrade

SIGUSR2 triggers ngx_change_binary. The master:

1. Calls ngx_exec_new_binary() (src/core/ngx_cycle.c), which fork()s a child that execv()s the new nginx binary.
2. Passes the listening socket fds to the new master via the NGINX environment variable (ngx_add_inherited_sockets() in src/core/nginx.c).
3. The new master forks new workers, which start accepting on the same sockets.
4. The PID files are now: new master at nginx.pid, old master at nginx.pid.oldbin.

The operator then sends SIGWINCH to the old master to stop its workers from accepting; remaining old connections drain. SIGQUIT to the old master ends the cycle.

If the new binary doesn't work, SIGHUP to the old master + SIGQUIT to the new master rolls back. This is one of the few ways to live-replace a server binary without dropping connections.

Cache manager and cache loader

If any proxy_cache_path, fastcgi_cache_path, etc., is configured, the master also forks:

• A cache manager that periodically expires cached entries.
• A cache loader that scans the cache directory once on startup to populate the in-memory rbtree.

Both run a stripped-down event loop (ngx_cache_manager_process_cycle) that's just timers — they don't accept connections. They share the cache state with workers via shared zones.

Permissions

The master starts as root (or whoever started nginx). After binding listen sockets, ngx_worker_process_init() calls setuid() to the configured user (default nobody). Workers run unprivileged. The master keeps root but does no I/O to the network.

Integration points

• Configuration system — ngx_init_cycle() is called from the master on startup and SIGHUP. Modules' init_module callbacks fire here.
• Event loop — each worker's main loop is ngx_worker_process_cycle() calling ngx_process_events_and_timers() until ngx_quit or ngx_terminate is set.
• Logging — the master's error_log is reopened on SIGUSR1; workers' too. Channel messages can carry log re-open requests.

Entry points for modification

Most changes here are around adding a new signal flag or supporting a new platform. The Linux/BSD/Solaris/macOS code paths converge in src/os/unix/ngx_process_cycle.c; Windows has its own simpler variant under src/os/win32/ngx_process_cycle.c (no separate master process — Windows runs as a single service process). When changing master/worker behavior, exercise reload + binary upgrade scenarios in nginx-tests.