Variables and scripts

Active contributors: Maxim Dounin, Sergey Kandaurov

Purpose

$variable placeholders in nginx.conf and the precompiled "scripts" that evaluate them are how nginx handles dynamic configuration values. Variables are also how modules expose request-derived data to the access log, to upstream rewrites, to if conditions, and to each other.

Files

File	Role
src/http/ngx_http_variables.{c,h}	The variable hash, lookup, register helpers
src/http/ngx_http_script.{c,h}	The script bytecode engine
src/http/modules/ngx_http_rewrite_module.c	Compiles set, if, rewrite, etc. into scripts
src/http/modules/ngx_http_map_module.c	map $key $value { ... }
src/http/modules/ngx_http_geo_module.c	geo $address $value { ... } — radix-tree backed
src/http/modules/ngx_http_split_clients_module.c	split_clients — percentile-based

Variables

A variable is ngx_http_variable_t:

typedef ngx_int_t (*ngx_http_get_variable_pt)(ngx_http_request_t *r,
    ngx_http_variable_value_t *v, uintptr_t data);

struct ngx_http_variable_s {
    ngx_str_t                     name;
    ngx_http_set_variable_pt      set_handler;
    ngx_http_get_variable_pt      get_handler;
    uintptr_t                     data;
    ngx_uint_t                    flags;
    ngx_uint_t                    index;
};

There are roughly 100 built-in variables across HTTP, upstream, SSL, geoip, and stream. Examples:

Variable	Source
$uri	r->uri (after rewrite)
$args	r->args (query string)
$request_method	r->method_name
$remote_addr	r->connection->sockaddr formatted
$http_<name>	A request header (created lazily on lookup)
$sent_http_<name>	A response header
$cookie_<name>	A cookie value
$arg_<name>	A query parameter
$upstream_status	The Status: line from the upstream response
$ssl_session_id	TLS session ID
$request_time	Per-request elapsed time
$body_bytes_sent	Bytes written to client (excluding headers)

The $http_* / $sent_http_* / $cookie_* / $arg_* families are prefix variables: they share one get_handler that parses the suffix at lookup time.

Registration

Modules register variables in their preconfiguration hook:

static ngx_http_variable_t  ngx_http_my_vars[] = {
    { ngx_string("my_var"), NULL, ngx_http_my_get_var, 0, 0, 0 },
    ngx_http_null_variable
};

static ngx_int_t
ngx_http_my_add_variables(ngx_conf_t *cf)
{
    ngx_http_variable_t  *var, *v;

    for (v = ngx_http_my_vars; v->name.len; v++) {
        var = ngx_http_add_variable(cf, &v->name, v->flags);
        if (var == NULL) return NGX_ERROR;
        var->get_handler = v->get_handler;
        var->data = v->data;
    }
    return NGX_OK;
}

After all variables are registered, ngx_http_variables_init_vars() walks the hash, assigns indices, and returns. From then on, lookups are O(1) by index.

Reading at request time

Two flavors:

• Indexed (fast): when the variable is referenced from a script, the compiler stores the index. At runtime: ngx_http_get_indexed_variable(r, idx).
• By name (slower): ngx_http_get_variable(r, &name, hash_key) does a hash lookup; used for prefix variables and ad-hoc lookups.

Cached values: r->variables[idx].valid = 1 after the first lookup; subsequent reads in the same request hit the cache.

Complex values

A "complex value" is a directive argument that can mix literals and variables: proxy_pass http://$backend:$port/api, log_format ... '$status $bytes_sent', return 301 /$arg_q. The compile/evaluate split:

ngx_http_complex_value_t   cv;
ngx_http_compile_complex_value_t  ccv;

ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
ccv.cf = cf;
ccv.value = &arg;          /* the raw directive argument */
ccv.complex_value = &cv;   /* output */

if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
    return NGX_CONF_ERROR;
}

/* later, at request time */
ngx_str_t  result;
if (ngx_http_complex_value(r, &cv, &result) != NGX_OK) {
    return NGX_ERROR;
}

ngx_http_compile_complex_value walks the input string, identifies $var references, and emits a small bytecode sequence (ngx_http_script_engine_t operations) that, when evaluated, concatenates the parts into a final string. The bytecode lives in cv.values and cv.lengths.

Script bytecode

src/http/ngx_http_script.c defines the bytecode engine. Operations are function pointers stored in the cv.values and cv.lengths arrays; the engine runs them sequentially against an ngx_http_script_engine_t that holds the output buffer pointer and state.

Op categories:

• Literal copy (ngx_http_script_copy_code) — append a constant string
• Variable read (ngx_http_script_copy_var_code) — append the variable's value
• Capture copy (ngx_http_script_copy_capture_code) — append a regex capture from a previous rewrite or if
• Length / value pair — first pass measures, second pass writes (so the buffer can be sized exactly)
• Conditional (ngx_http_script_if_code, ngx_http_script_complex_value_code) — used by the rewrite module's if
• Return / break / redirect (ngx_http_script_return_code, ngx_http_script_break_code)

The two-pass nature (compute length, then build) is why most scripts allocate exactly once at runtime — no copy-and-grow.

Rewrite scripts

The rewrite module's set, if, rewrite, break, return directives all compile into the same bytecode. The phase handler (registered at REWRITE) runs the compiled script for the current location, which can mutate r->variables[], set r->uri, redirect, or short-circuit.

The "if is evil" caveat is because if opens a new mini-location with its own config and any directive that has side effects at config time (like proxy_pass) is not safe inside it.

map and geo

map is essentially a hash table from one variable's value to another:

map $http_user_agent $type {
    default       desktop;
    ~*Mobile      mobile;
    ~*Bot         bot;
}

It registers $type as a new variable; when read, the get_handler evaluates $http_user_agent, looks it up in the map, and returns the result.

geo is the same idea keyed on a CIDR-matched address (using the radix tree from src/core/ngx_radix_tree.c).

Integration points

• Configuration — every directive that wants to take a variable-bearing argument calls ngx_http_compile_complex_value at config time.
• Logging — log_format strings compile to scripts. The access log module evaluates them per request.
• Rewrite engine — set, if, rewrite, break, return are entirely script bytecode.
• Upstream — proxy_pass arguments, proxy_set_header, proxy_cache_key, all use complex values.

Entry points for modification

Adding a new variable: register it in your module's preconfiguration and provide a get_handler. Adding a new directive that takes interpolated arguments: use ngx_http_compile_complex_value and ngx_http_complex_value. Touching the script engine itself is rare; the bytecode op set has been stable since the early 1.x line.