Transport and HTTP

Purpose

The transport system is the binary RPC layer that connects nodes inside a cluster (and to remote clusters). The HTTP system serves the REST API. Both run on Netty by default but are pluggable via NetworkPlugin.

Directory layout

server/src/main/java/org/elasticsearch/transport/
├── Transport.java                Abstract transport (send/receive, connections)
├── TransportService.java         Public façade; named-action dispatch
├── TransportRequest.java, TransportResponse.java
├── ConnectionManager.java        Pool of open connections to a node
├── RemoteClusterService.java     Connections to peer clusters (CCS / CCR)
├── netty4/                       Default Netty4-based implementation (in modules/transport-netty4)
└── ...

server/src/main/java/org/elasticsearch/http/
├── HttpServerTransport.java      Abstract HTTP server
├── AbstractHttpServerTransport.java
├── netty4/                       Default implementation (modules/transport-netty4)
└── ...

modules/transport-netty4/         Netty4-based transport + HTTP

Key abstractions

Type	File	Role
TransportService	server/src/main/java/org/elasticsearch/transport/TransportService.java	Send requests, register handlers, manage connections
Transport	server/src/main/java/org/elasticsearch/transport/Transport.java	Pluggable transport SPI
RemoteClusterService	server/src/main/java/org/elasticsearch/transport/RemoteClusterService.java	Cross-cluster connection registry
HttpServerTransport	server/src/main/java/org/elasticsearch/http/HttpServerTransport.java	HTTP SPI
Netty4Transport / Netty4HttpServerTransport	modules/transport-netty4/...	Default impls
RestController	server/src/main/java/org/elasticsearch/rest/RestController.java	HTTP route table

The binary transport protocol

Each message has a small framed header (length prefix + flags + request ID + action name + transport version) followed by a Writeable-encoded payload. The receiving node looks up the handler by action name. Compression and TLS are negotiated per connection.

TransportVersion is the single source of truth for wire-format compatibility. Every breaking change to a transport message introduces a new version constant; serializers gate new fields with out.getTransportVersion().onOrAfter(...).

Threading

Netty event loops handle the bytes; the message is then dispatched to a named thread pool selected by the handler's executor argument. Handlers must never block the event loop or the dispatcher pool — they should re-submit to a worker pool if they have work to do.

HTTP server

Netty4HttpServerTransport accepts HTTP/1.1 (and HTTP/2 in 9.x), parses requests into RestRequest, and hands off to RestController. The controller does:

1. Tracing (Tracer).
2. Authentication / authorization (X-Pack security via RestInterceptor).
3. Deprecation handling (DeprecationRestHandler).
4. Route lookup (with API-version filtering).
5. Dispatch to the RestHandler.

Streaming responses are supported via ChunkedRestResponseBody and friends — used by _search, _mget, snapshot status, etc.

Connection management

Per peer, the ConnectionManager maintains a small pool of connections (one per "channel type": recovery, bulk, reg, state, ping). Each channel type has its own QoS settings; recovery traffic, for instance, is throttled so it does not starve query traffic.

Cross-cluster connections

RemoteClusterService lets a node act as a coordinator to other clusters for CCS, CCR, and CCS via a sniffed seed list. Connections can be proxy mode (for clusters reachable only through a single hostname) or sniff mode (the local node discovers data nodes from a seed and connects directly).

Network plugins

Plugins implementing NetworkPlugin can:

• Replace the transport (getTransports).
• Replace the HTTP server (getHttpTransports).
• Add TransportInterceptors — useful for security audit and tracing.
• Add HttpServerTransport.Dispatcher filters.

Tests

• MockTransportService — pure-in-memory transport for unit tests.
• MockNioTransport — a real but in-process transport for integration tests.
• TransportInterceptor — used by tests to drop, delay, or corrupt messages and exercise resilience paths.

Entry points for modification

• New transport message? Subclass TransportRequest/TransportResponse, declare a TransportVersion if the format is new.
• New HTTP route? Implement RestHandler.routes() and register via ActionPlugin.
• New deprecation path? Wrap your handler with DeprecationRestHandler.
• New cross-cluster behavior? Look at RemoteClusterService and the credentials registry.