grid and rest

Two RPC building blocks. internal/rest is a lightweight HTTP-based RPC base used by the storage REST and peer REST layers; internal/grid is a binary multiplex protocol used for low-latency control RPCs.

internal/rest

A thin wrapper around http.Client that:

• Pools connections per peer.
• Adds the cluster signature header on every request.
• Handles structured error responses without reflection-heavy generics.
• Times out aggressively to avoid wedging an erasure write behind a stuck peer.

The package exposes a single Client type that the storage REST and peer REST clients embed (cmd/storage-rest-client.go and cmd/peer-rest-client.go). Tests are in the file alongside the implementation.

internal/grid

internal/grid/README.md describes the protocol in detail. The core ideas:

• One TCP connection per peer pair. Streams are multiplexed.
• Length-prefixed binary frames. No HTTP framing overhead.
• Bidirectional streams. Either side can initiate a request.
• Backpressure. Frames have a flow-control bit per stream.
• Optional TLS. Same cert config as the HTTP server.

Used today for:

• The lock-rest path in distributed mode (when the grid is enabled).
• Some admin telemetry aggregations.
• Cluster health probes.

The runtime upgrades a normal HTTP connection to a grid connection with a small handshake handler mounted in cmd/api-router.go.

Choosing between rest and grid

Trait	rest	grid
Wire protocol	HTTP/1.1, JSON or msgp body	Custom binary multiplex over TCP
Latency for tiny messages	OK	Lower
Streaming large payloads	Best (chunked transfer)	OK (segmented)
Debuggability	High (curl, mc admin trace)	Lower (binary frames)
Use today	Storage REST, peer REST, lock-rest fallback	Lock-rest fast path, control RPC

New code paths default to internal/rest unless the latency profile justifies the binary path.

Integration points

• internal/rest — used by cmd/storage-rest-client.go, cmd/peer-rest-client.go, cmd/lock-rest-client.go, and a handful of admin clients.
• internal/grid — used by selected control plane code paths; full upgrade handshake in cmd/api-router.go.

Entry points for modification

• New rest endpoint. Define request/response types (msgp), add a server route, add a client method. Mirror the convention used in cmd/storage-rest-server.go.
• New grid endpoint. Read internal/grid/README.md first, then implement the handler and client. Don't bypass the flow-control hooks.