How to monitor

The Temporal server exposes structured logs, metrics, and OpenTelemetry traces. This section is the operator-oriented entry point; the implementation details are on Systems → Telemetry.

Sub-page	Covers
Logging	Where logs go, log level configuration, structured fields, log levels per code path.
Metrics	What is tracked, key SLIs, dashboards, how to add a metric.
Tracing	OpenTelemetry setup, span naming, how to trace a request end-to-end.

A short overview

• Logs — Uber zap, structured fields, configured under the log: block of YAML config.
• Metrics — defined centrally in common/metrics/metric_defs.go, exported via Prometheus / OpenTelemetry / statsd.
• Traces — OpenTelemetry SDK + OTLP exporters, configured under global.tracerProvider.
• Health checks — every service exposes the standard gRPC health-check service plus a deeper internal check.

Health endpoints

Each service runs a gRPC health-check endpoint registered by common/rpc/interceptor/health.go. The History service additionally exposes a deep health check at service/history/deep_health_check.go that walks shard ownership, persistence, and membership.

For load-balancer probes the standard health-check is sufficient. For more discriminating health (e.g. mark a host unhealthy when it lost shard ownership but the gRPC server is still up) the deep health check is what you want.

Useful inputs to monitor

Signal	Why it matters
Per-RPC latency p99 / error rate	Standard SLI for any gRPC service.
Shard ownership transitions per second	A spike means hosts are flapping; investigate Ringpop and persistence.
Queue depth per History queue family	Unbounded growth signals a stuck executor or downstream dependency.
Replication lag	"How far behind is the standby?" — exposed as replication_* metrics.
Persistence operation latency	Bottlenecks here propagate everywhere.
Matching task forwarder rate	High forwarder rates indicate idle child partitions; consider re-shard.
DLQ message count	Anything above zero is operator attention.

Related pages

• Systems → Telemetry — implementation reference.
• How to contribute → debugging — engineer-side troubleshooting.
• Reference → configuration