gRPC proxy

Source: server/proxy/grpcproxy/, plus the entry point server/etcdmain/grpc_proxy.go.

Purpose

The gRPC proxy is a stateful caching front-end for an etcd cluster. A single proxy can serve thousands of client connections by:

• Coalescing watches — many clients watching the same key/range share one upstream watch, with the proxy fanning out events.
• Caching reads — recent get/range responses are served from memory.
• Multiplexing connections — clients only need to know the proxy address, not every cluster member.

It is not a transparent load balancer (use etcd gateway start for that). Some advanced features (notably RequestProgress) are not forwarded by the proxy, so the new cache/ module is recommended for very large fan-outs.

Directory layout

server/proxy/grpcproxy/
├── kv.go               # KV proxy (range cache + delete fan-in)
├── watch.go            # Watch proxy
├── watch_broadcast.go, watch_broadcasts.go, watch_ranges.go
├── watcher.go
├── lease.go            # Lease proxy
├── auth.go             # Auth proxy
├── cluster.go          # Cluster proxy
├── maintenance.go      # Maintenance proxy
├── leader.go           # Leader-election helper for the proxy itself
├── lock.go             # Distributed-lock fan-in
├── election.go         # Election service forwarding
├── health.go           # gRPC health server
├── register.go         # Member registration with the upstream cluster
├── metrics.go
├── adapter/            # In-process clientv3 -> grpc.ServerStream adapter
└── cache/              # Local read cache

Key abstractions

Symbol	File	Description
kvProxy	server/proxy/grpcproxy/kv.go	Wraps pb.KVServer, intercepts Range for caching, forwards everything else
watchProxy	server/proxy/grpcproxy/watch.go	Per-prefix watchBroadcast consolidating client watchers
watchBroadcast	server/proxy/grpcproxy/watch_broadcast.go	Single upstream watch + N local subscribers
watchRanges	server/proxy/grpcproxy/watch_ranges.go	Indexes broadcasts by key range for matching
cache.Cache	server/proxy/grpcproxy/cache/cache.go	LRU-ish read cache used by kvProxy
clusterProxy	server/proxy/grpcproxy/cluster.go	Maintains the proxy's own membership advertisement under /etcd-grpcproxy/... keys

How it works

graph TD
    c1[Client A]
    c2[Client B]
    c3[Client C]
    proxy[grpc-proxy]
    upstream[etcd cluster]

    c1 -->|Range/Watch| proxy
    c2 -->|Watch same prefix| proxy
    c3 -->|Put| proxy

    proxy -->|cached or forward| upstream
    proxy -->|single Watch fan-in| upstream
    proxy --> c1
    proxy --> c2
    upstream --> proxy

1. Each client connects to the proxy normally; the proxy presents the same gRPC API as etcd itself.
2. For Watch, the proxy hashes the requested key range to a watchBroadcast. If one already exists, the new client just subscribes; otherwise the proxy opens an upstream watch.
3. For Range, the proxy consults its in-memory cache; on miss, it forwards and caches.
4. For mutation RPCs (Put, DeleteRange, Txn, lease ops), the proxy forwards directly and invalidates affected cache entries.

Configuration knobs

Highlights from server/etcdmain/grpc_proxy.go:

• --endpoints — comma-separated upstream cluster endpoints.
• --listen-addr / --advertise-client-url — the proxy's bind / advertise URLs.
• --namespace — adds a key-prefix scope so multiple unrelated workloads can share a cluster.
• --max-send-bytes / --max-recv-bytes — gRPC message-size limits.
• --cache-size — number of cached read entries.
• --cert, --key, --cacert, --insecure-skip-tls-verify — TLS toward upstream.
• --election-prefix, --lock-prefix — keyspace where the proxy's own primary/replica election lives.

Integration points

• client/v3 — the proxy uses a real clientv3.Client to talk to upstream.
• adapter/ — adapts clientv3 interfaces to the gRPC server interfaces (KVServer, WatchServer, ...) so the proxy can implement etcd's API without re-implementing each handler.
• leader-election — the proxy itself supports an HA mode by running clientv3/concurrency/Election against --election-prefix.

When to use it vs alternatives

Need	Pick
Single endpoint, no caching, no smart fan-out	etcd gateway (TCP)
Lots of clients watching overlapping prefixes, talking via gRPC	etcd grpc-proxy
Per-process watch cache colocated with the consumer	The cache/ module (page)
Simple Kubernetes-style watch fan-out colocated with the API	API-server-side compaction +cache/

Entry points for modification

• New cached call → kv.go plus the cache package.
• New broadcast routing → watch_ranges.go.
• New TLS / auth wiring → register.go + clusterProxy.