OS abstraction

Active contributors: Sergey Kandaurov, Maxim Dounin, David Carlier

Purpose

src/os/ isolates per-OS code so the rest of nginx can pretend it's running on an idealized POSIX-with-extras platform. Each supported OS has its own subdirectory with implementations of file I/O, sockets, processes, signals, atomics, sendfile, AIO, and the small handful of OS-specific bootstrap (ngx_*_init.c).

Directory layout

src/os/
├── unix/        # Linux, FreeBSD, macOS, Solaris, AIX, HP-UX
└── win32/       # Windows

Inside unix/, OS-specific files are picked by auto/configure based on uname:

src/os/unix/
├── ngx_linux*.c           # Linux-specific: aio, sendfile, init
├── ngx_freebsd*.c         # FreeBSD-specific
├── ngx_darwin*.c          # macOS / Darwin
├── ngx_solaris*.c         # Solaris
├── ngx_posix_*.c          # generic POSIX fallbacks
├── ngx_files.{c,h}        # cross-Unix file ops
├── ngx_socket.{c,h}       # cross-Unix socket helpers
├── ngx_process.{c,h}      # fork(), signal()
├── ngx_process_cycle.{c,h} # master/worker cycles
├── ngx_setaffinity.{c,h}  # worker_cpu_affinity
├── ngx_setproctitle.{c,h} # process renaming
├── ngx_thread*.c          # pthread wrappers
├── ngx_atomic.h           # atomic ops dispatcher
├── ngx_gcc_atomic_*.h     # GCC __sync / __atomic builtins per arch
├── ngx_sunpro_*.h, *.il   # Sun Studio inline assembly
├── ngx_recv.c, ngx_send.c # plain recv/send
├── ngx_readv_chain.c      # readv-based chain reader
├── ngx_writev_chain.c     # writev-based chain writer
├── ngx_*_sendfile_chain.c # per-OS sendfile wrappers
├── ngx_udp_*.c            # UDP helpers (incl. udp_sendmsg_chain for QUIC)
├── ngx_user.{c,h}         # crypt() etc. for auth_basic
├── ngx_shmem.{c,h}        # mmap/MAP_SHARED for shared zones
├── ngx_dlopen.{c,h}       # dlopen/dlsym for dynamic modules
└── ngx_channel.{c,h}      # the master/worker socketpair

Per-OS sendfile

Sendfile on Linux, FreeBSD, macOS, and Solaris all have different APIs and different limits. Each gets its own implementation:

File	Targets
src/os/unix/ngx_linux_sendfile_chain.c	Linux sendfile(2)
src/os/unix/ngx_freebsd_sendfile_chain.c	FreeBSD sendfile(2)
src/os/unix/ngx_darwin_sendfile_chain.c	macOS sendfile(2)
src/os/unix/ngx_solaris_sendfilev_chain.c	Solaris sendfilev(3SOCKET)
src/os/unix/ngx_writev_chain.c	Generic POSIX writev(2) fallback

ngx_io.send_chain is a function pointer that the OS init code (ngx_linux_init.c, etc.) sets to the appropriate one. HTTP / Stream / Mail just call c->send_chain and don't care which it is.

Atomics

src/os/unix/ngx_atomic.h switches on compiler + architecture:

Selector	Backend
GCC + x86 / amd64	ngx_gcc_atomic_x86.h / _amd64.h
GCC + ppc	ngx_gcc_atomic_ppc.h
GCC + sparc64	ngx_gcc_atomic_sparc64.h
Sun Studio + sparc64 / amd64 / x86	ngx_sunpro_atomic_*.h + .il files
(default)	Mutex-based fallback

Operations: ngx_atomic_cmp_set, ngx_atomic_fetch_add, ngx_memory_barrier. Used by the slab allocator, the shared mutex (ngx_shmtx), and the connection counter.

auto/configure probes for __atomic_* builtins; modern builds typically use those.

File I/O

ngx_open_file, ngx_read_file, ngx_write_file, etc., are the cross-Unix wrappers in ngx_files.c. They handle:

• open() with the right flags (O_RDONLY, O_NONBLOCK).
• The openat() family on Linux for disable_symlinks.
• File-descriptor caching via ngx_open_file_cache (which is in src/core/, not src/os/).

AIO has two implementations:

• Linux AIO (ngx_linux_aio_read.c) via io_setup + eventfd integration.
• POSIX AIO (ngx_file_aio_read.c) via aio_read + signals — used on FreeBSD.

The aio threads mode uses the thread pool (src/core/ngx_thread_pool.c) instead, on any Unix.

Process management

fork(), execve(), signal masking, and child reaping are wrapped in ngx_process.c. The master/worker cycles live in ngx_process_cycle.c. Channels (master ↔ worker socketpair) are in ngx_channel.c. See process-model for the lifecycle.

setproctitle is implemented by overwriting argv[] and the environment on Linux/BSD; macOS uses setproctitle() directly. The "nginx: master process /sbin/nginx" string seen in ps output comes from this.

worker_cpu_affinity calls sched_setaffinity on Linux, cpuset_setaffinity on FreeBSD, processor_bind on Solaris (src/os/unix/ngx_setaffinity.c).

Shared memory

ngx_shm_alloc / ngx_shm_free (src/os/unix/ngx_shmem.c) wrap mmap(MAP_SHARED|MAP_ANON). On Linux they fall back to /dev/zero if MAP_ANON isn't honored. The slab allocator and shared-zone caches all sit on top.

Windows port

src/os/win32/ is a port of the same abstractions to Windows APIs:

• WSAOVERLAPPED + IOCP for I/O readiness (instead of epoll/kqueue).
• AcceptEx / ConnectEx for asynchronous accept/connect.
• CreateProcess + named pipe channel (instead of fork + socketpair).
• MapViewOfFile for shared memory (instead of mmap).

The Windows port is officially "proof of concept" — it builds and runs but is not intended for production. The README explicitly calls this out. Most Windows-side maintenance is keeping the build going through OpenSSL changes.

Auto-detection

auto/os/conf picks the OS-specific source files. Probes:

auto/os/...	What it detects
linux	sendfile, eventfd, io_setup, EPOLLEXCLUSIVE, SO_REUSEPORT, eBPF
freebsd	sendfile, kqueue, jail-related quirks
darwin	sendfile, kqueue, macOS API differences
solaris	event ports, /dev/poll, sendfilev
win32	IOCP, AcceptEx, etc.
hpux	a few ancient quirks (still here for the buildbot)

Integration points

• Event loop — ngx_io and c->send_chain are the function pointers that bind the cross-Unix code to the OS-specific implementations.
• Configuration — worker_rlimit_nofile, worker_cpu_affinity, working_directory, user, all dispatch through OS abstractions.
• Process model — fork, signal, channel.

Entry points for modification

Adding a new OS is rare and a multi-week project. Adding a new architecture's atomics is straightforward — mirror the existing ngx_gcc_atomic_*.h files. Tweaking sendfile chunking or AIO behavior on Linux happens in the corresponding ngx_linux_*.c files; the rest of the code base shouldn't need to change.