Tracing

Distributed tracing for GitLab's web, worker, and external-service calls.

Source

Concern	Location
Tracing facade	lib/gitlab/tracing/, lib/gitlab/tracing.rb
Labkit (Ruby)	gitlab-labkit gem
Workhorse tracing	workhorse/internal/log/, OpenTelemetry SDK
Sidekiq tracing	lib/gitlab/sidekiq_middleware/tracing/

GitLab uses OpenTelemetry as the wire format and supports any OTLP-compatible backend (Jaeger, Tempo, GCP Cloud Trace, Datadog, Honeycomb).

What's traced

• HTTP requests (Puma + Workhorse) — root spans.
• Sidekiq jobs — root spans linked to the request that enqueued them.
• ActiveRecord queries — child spans with SQL fingerprints.
• Redis commands — child spans.
• Gitaly RPCs — child spans (server-side and client-side).
• HTTP outbound (gitlab-http) — child spans.
• Sidekiq cron — root spans.
• AI Gateway calls — child spans.
• Action Cable channels — root spans on subscribe.

Correlation

The correlation_id and metadata flow into traces as span attributes. Logs include the trace ID, so a single correlation ID can pivot across logs, metrics, and traces.

Sampling

Production sampling rate is small (sub-percent) for high-volume endpoints; specific endpoints with known issues use 100% sampling for debugging.

Configuration

Self-managed installs configure tracing via:

• OTEL_* environment variables.
• config/initializers/tracing.rb.
• The Helm chart's global.tracing.* values.

GDK has tracing off by default; flip it on with gdk config set tracing.enabled true and the local Jaeger container appears at http://localhost:16686.

Adding a custom span

Gitlab::Tracing.in_span('my.operation', attributes: { foo: 'bar' }) do
  # work
end

Or via Labkit:

Labkit::Tracing::ActionView.with_span('render') do
  # render
end

Related

• Logging — correlation_id propagation.
• Metrics — same labels.