Internal libraries (libs/)

libs/ is the home of internal libraries reused by server/, modules, plugins, and X-Pack. These are not published as a public API surface (with one exception, plugin-api, which is intentionally stable for plugin authors). They exist to share code that doesn't belong inside the server JAR.

Catalog

Library	Purpose
arrow	Apache Arrow integration for ESQL output
cli	CLI parsing scaffolding shared by bin/elasticsearch-* tools
cli-terminal	Cross-platform terminal helpers (TTY detection, color, prompts)
core	Core utilities used everywhere (IOUtils, Booleans, Releasable, RefCounted, ...)
dissect	Dissect pattern parser used by the dissect ingest processor
entitlement	Custom security framework gating JDK calls per-plugin (replacing SecurityManager)
exponential-histogram	Exponential histogram implementation for OTLP-compatible metrics
geo	Geometry types and helpers (extends Lucene's geo)
gpu-codec	Optional GPU vector codec
grok	Grok pattern parser used by the grok ingest processor
h3	Uber's H3 geospatial indexing system
log4j	Log4j shim for libraries that need it
logging	The project's own Logger interface decoupled from log4j
logstash-bridge	API surface for Logstash to call into the server
lz4	LZ4 compression bindings
native	Native helpers (mlockall, madvise, file flush) via FFI
parquet-rs	Rust-based Parquet reader (used by ESQL Parquet datasource)
plugin-analysis-api	Public analysis SPI for plugin authors
plugin-api	Public plugin SPI — the only libs/* published as ABI-stable
plugin-scanner	Compile-time scanner that produces plugin descriptors from annotations
server-launcher-common	Shared launcher logic for the server entry point
simdvec	SIMD-accelerated vector dot product / Manhattan / Hamming kernels
ssl-config	TLS configuration parsing reused by HTTP / transport / repositories
swisshash	Swiss-table-style hash maps tuned for the project's value-types
tdigest	t-digest implementation for percentile aggregations
user-agent-api	API used by the user_agent ingest processor
web-utils	Lightweight HTTP helpers used by repositories and inference
x-content	Polymorphic content layer (JSON, YAML, CBOR, Smile) on top of Jackson

Highlights

core

A near-universal dependency. Contains:

• IOUtils.close(...) — null-safe close for any number of Closeables, accumulating exceptions.
• Releasable and RefCounted — the project's lifetime types.
• Strings.format(...) — log-message formatting.
• Predicates, CheckedSupplier, CheckedFunction — functional helpers compatible with the project's exception style.
• PathUtils — Path constructors that respect forbidden-apis.
• Booleans — the unique parsers for "true" / "yes" / "on" the project uses.

x-content

The project does not call Jackson directly. Instead it uses XContent, an abstraction that supports JSON, YAML, CBOR, and Smile through a single XContentBuilder / XContentParser API. Every wire-facing type implements ToXContent for serialization and provides a static parser using XContentParser for deserialization.

plugin-api

The carefully-curated public SPI that plugin authors compile against. ABI-stable across minor versions. If you need to use a server class from a plugin, check whether it's in plugin-api; if not, propose adding it before depending on the server module directly.

entitlement

The newer security framework replacing the SecurityManager. It uses bytecode instrumentation to gate JDK calls (file, network, process, reflection) per plugin. Plugins declare their needed entitlements in plugin-descriptor.properties; missing entitlements cause an EntitlementException at runtime. See libs/entitlement/.

simdvec

SIMD-accelerated vector kernels with two implementations: a Java Vector API path (preferred on JDK 21+) and a native fallback. The native binaries are precompiled per architecture; libs/simdvec/native/publish_vec_binaries.sh is one of the most-edited files in the repo. See Vector search.

parquet-rs

A Rust-based Parquet reader pulled in via JNI. Used by the ESQL Parquet datasource (x-pack/plugin/esql-datasource-parquet) for fast columnar reads of object-storage-backed Parquet files.

tdigest

In-tree implementation of t-digest used by the percentile aggregations. Forked from the upstream library to add allocation-friendly variants and to expose internal types needed by the aggregation framework.

native

JNA + JDK 22+ Foreign Function API bindings for OS-level operations: mlockall (lock heap pages in RAM), madvise (advise the kernel about memory access patterns), file flush (fsync plus platform variants), CPU info, and a few networking helpers. Required because the JDK does not expose these directly and Elasticsearch needs them for predictable production behavior.

Build view

Each libs/<name>/ is its own Gradle subproject. Tests run via ./gradlew :libs:<name>:test. Most are JPMS-modular and are intentionally small dependencies.

Server packages

The server itself (server/) is by far the largest single package. It exposes its public API through org.elasticsearch.* packages, but most consumers should reach for libs/plugin-api instead. The server's structure is documented across the systems and features lenses.