RPC

pkg/rpc/ owns CockroachDB's gRPC and DRPC connection plane: dialing, mutual TLS, heartbeat-based liveness, clock-offset tracking, peer pooling, and the nodedialer that the rest of the cluster uses.

Purpose

CockroachDB nodes talk to each other over gRPC for most RPCs and DRPC (drpc.io) for streaming low-overhead paths. Every cross-node call goes through this package's connection layer rather than dialing gRPC directly. The package also runs continuous heartbeats that double as liveness pings and clock-offset measurements.

Directory layout

pkg/rpc/
├── context.go              // ~90 KB: rpc.Context, the cluster-wide RPC config
├── peer.go / peer_map.go   // per-peer connection state machine
├── connection.go           // a connection's outbound state
├── stream_pool.go          // pool of per-target streams (DRPC)
├── tls.go                  // TLS configuration helpers
├── auth.go / auth_tenant.go// per-call authorization (host vs tenant)
├── client.go               // small client builders
├── codec.go                // protobuf codec selection
├── drpc.go                 // DRPC adapter
├── grpc.go                 // gRPC adapter
├── heartbeat.go            // heartbeat service (with .proto)
├── clock_offset.go         // observe and bound clock offsets
├── keepalive.go            // gRPC keepalive
├── snappy.go               // snappy compression
├── settings.go             // RPC-specific cluster settings
├── nodedialer/             // nodedialer.Dialer (consumer-facing)
├── rpcbase/                // shared interfaces
└── rpcpb/                  // protobuf definitions

Key types

Type	File	Description
Context	pkg/rpc/context.go	Cluster-wide RPC config: TLS, clocks, keepalive, settings.
Peer	pkg/rpc/peer.go	One outbound conn per (target, class). Owns heartbeat.
Connection	pkg/rpc/connection.go	A single dial attempt.
RemoteOffsetMonitor	pkg/rpc/clock_offset.go	Tracks and bounds clock skew across the cluster.
Dialer	pkg/rpc/nodedialer/nodedialer.go	Higher-level dialer used by SQL, KV, and gossip.

Connection state machine

A Peer cycles through a few states:

stateDiagram-v2
  [*] --> Initializing
  Initializing --> Healthy: dial + heartbeat ok
  Healthy --> Unhealthy: heartbeat failure
  Unhealthy --> Healthy: heartbeat ok
  Unhealthy --> Closed: drained / removed
  Healthy --> Closed: drained / removed
  Closed --> [*]

A peer holds at most one alive connection per RPC class (see pkg/rpc/rpcbase/). Classes separate latency-sensitive and bulk traffic so head-of-line blocking on bulk transfers can't slow down user-visible RPCs.

Heartbeat

pkg/rpc/heartbeat.go defines the HeartbeatService. Every connection runs a periodic heartbeat:

1. Caller sends a Ping with its current cluster ID and HLC.
2. Server validates the cluster ID and tenant ID.
3. Reply carries the server's HLC.
4. Caller measures roundtrip time and updates RemoteOffsetMonitor.

If clock offsets exceed the maximum allowed (pkg/server/clock_monotonicity.go enforces it), the node terminates to preserve linearizability invariants.

TLS and auth

pkg/rpc/tls.go builds *tls.Configs using the certificates loaded by pkg/security/. Mutual TLS is on by default. The server's auth.go validates client certificates and maps them to a SQL user. For multi-tenant deployments, auth_tenant.go enforces that a tenant cannot impersonate another tenant ID by inspecting the OU=Tenants/CN=<tenantID> certificate field.

DRPC

DRPC (drpc.io) is a smaller, allocation-friendlier wire protocol than gRPC. CockroachDB uses it for streaming raft and changefeed paths where allocation pressure matters. pkg/rpc/drpc.go builds a DRPC stack on top of the same TLS connections, and pkg/rpc/stream_pool.go reuses streams across calls.

NodeDialer

pkg/rpc/nodedialer/nodedialer.Dialer is the consumer-facing API. It accepts a node ID + RPC class, looks up the node's address in gossip, dials through the Context, and returns a connection ready for use. The KV layer's DistSender, gossip's outgoing connections, and rangefeed clients all use it.

Settings

A few representative cluster settings live here (pkg/rpc/settings.go):

• rpc.connection_timeout
• rpc.heartbeat_interval
• rpc.heartbeat_timeout
• rpc.clock_uncertainty_limit_check.enabled

Related pages

• Server — owns the listener side of the RPC stack.
• Security — produces the TLS material the RPC stack consumes.
• KV client — the largest user of nodedialer.
• Gossip — also routes through nodedialer for peer discovery.