Logging

Every Rust binary in Neon uses the tracing crate with a JSON-formatting subscriber from libs/tracing-utils/. Logs go to stderr by default. The format and field conventions are intentionally uniform across services so a single log shipper / aggregator can ingest everything.

Format

By default the formatter is JSON. Each line is a JSON object with at minimum:

{
  "timestamp": "2026-04-30T12:34:56.789Z",
  "level": "INFO",
  "fields": {
    "message": "...",
    "tenant_id": "...",
    "timeline_id": "...",
    "request_id": "..."
  },
  "target": "pageserver::page_service",
  "spans": [
    { "name": "tenant", "tenant_id": "..." },
    { "name": "get_page_at_lsn", "key": "...", "lsn": "..." }
  ]
}

The exact field names come from tracing-subscriber's JSON formatter. Set LOGFMT=text to switch to a single-line human-readable format — useful when running a binary by hand:

LOGFMT=text RUST_LOG=info cargo run -p pageserver

RUST_LOG

The environment variable controls the per-target verbosity. Examples:

RUST_LOG=info                                  # everything at info
RUST_LOG=info,pageserver=debug                 # pageserver at debug, rest at info
RUST_LOG=warn,pageserver::page_service=trace   # very narrow trace

tracing_subscriber::EnvFilter is the parser; full syntax in its docs.

Span conventions

Every per-request task wraps its work in a top-level span. Conventions (illustrative; details vary by service):

• Tenant level. tenant{id=X, shard=Y} covers all work for one tenant. Inside a single task at most one tenant span is active.
• Timeline level. timeline{id=Z} nested inside the tenant span.
• Request level. get_page_at_lsn{key=..., lsn=...} (pageserver), append{lsn=...} (safekeeper), proxy_pass{conn_id=...} (proxy).
• Background task level. compaction{level=L0} for compaction tasks, walreceiver_loop for WAL receivers.

Span fields propagate: a log line emitted inside get_page_at_lsn automatically carries tenant_id, timeline_id, key, and lsn.

request_id propagation

For HTTP requests, the tower-http RequestIdLayer adds a request_id header on the way in (or generates one if absent) and copies it onto the request span. Subsequent log lines for that request include the id. The same header propagates to outbound HTTP calls so you can trace one request across multiple services in a log aggregator.

For libpq replication-stream connections, the protocol carries an explicit request_id field in newer message versions.

Log destinations

In the local development setup with cargo neon, each service writes to a file under .neon/:

.neon/pageserver.log
.neon/safekeepers/sk1/safekeeper.log
.neon/storage_broker.log
.neon/endpoints/<name>/compute_ctl-NNN.log
.neon/endpoints/<name>/postgres.log

In production, services write to stderr and typical setups ship the stream to Loki / Cloud Logging / OpenSearch / etc.

What appears at each level

Level	When to use
error!	A failure that an operator should care about. Auto-restart loops emit error! when the loop iteration fails, then keep going. Panics become error! log lines via the panic hook.
warn!	Something unexpected but not failed (slow operation, retried request, deprecated config).
info!	Lifecycle events (service start / stop, tenant attach / detach, compaction round). One line per request at the top level for HTTP/replication endpoints.
debug!	Per-step detail useful when investigating.
trace!	Very high volume; gated off in production.

Internal helper functions usually log at debug! or below. The info! level is reserved for things an operator would expect to read.

Sensitive data

JWT tokens, Postgres passwords, and SCRAM secrets must never appear in logs. The general rule is to log identifiers (tenant_id, timeline_id, role names) but not credentials. The tracing field machinery makes this easy — passing a &str reference of a token would log it; instead, code logs a placeholder like <redacted>.

syslog forwarding (compute_ctl)

compute_tools/src/rsyslog.rs integrates compute_ctl with rsyslog so per-compute logs can be forwarded to a centralized syslog endpoint. See the file for the supported destinations.

Related

• Metrics — quantitative telemetry.
• Tracing — OTLP traces (different from the tracing-crate spans-as-logs concept).
• Debugging — how to use logs in practice.