Open-Source Wikis

/

Apache Kafka

/

Reference

/

Dependencies

apache/kafka

Dependencies

Kafka aims to keep its runtime dependency surface small. This page summarizes the libraries the project depends on and where they are declared. The authoritative source is gradle/dependencies.gradle, which lists every dependency version in one place; per-module build.gradle snippets pull from that map.

Where dependencies are declared

File Role
gradle/dependencies.gradle Single map of versions[...] and libs[...] shared by every module
build.gradle Root build script; per-module configurations and shared logic
<module>/build.gradle Per-module dependency lists (referencing libs.*)
LICENSE-binary Mirrors the runtime dependencies for redistribution; must be updated whenever a runtime dependency changes (see KAFKA-12622)

Runtime libraries (most relevant)

The wiki references these often; the table below is a curated subset of gradle/dependencies.gradle.

Library Used for
org.scala-lang:scala-library (2.13) Scala runtime for core/
com.fasterxml.jackson.* JSON parsing — admin tools, Connect, telemetry
org.apache.logging.log4j:log4j-* Broker / Connect / tools logging (Log4j 2)
org.slf4j:slf4j-api Common logging facade used everywhere
com.yammer.metrics:metrics-core Yammer metrics (still used in core/ and bridged to JMX)
org.eclipse.jetty:jetty-server, jersey-* Connect REST server and tools
net.sf.jopt-simple:jopt-simple CLI option parsing for legacy tools
net.sourceforge.argparse4j:argparse4j CLI option parsing for newer tools
org.rocksdb:rocksdbjni Default Streams state-store backend
com.github.luben:zstd-jni ZSTD compression
at.yawk.lz4:lz4-java LZ4 compression
org.xerial.snappy:snappy-java Snappy compression
com.google.re2j:re2j Linear-time regex (KIP-848 server-side regex assignor)
org.bitbucket.b_c:jose4j JWT validation for SASL/OAUTHBEARER
com.github.ben-manes.caffeine:caffeine In-memory cache used by remote-log index cache and a few others
com.dynatrace.hash4j:hash4j Specialized hashing utilities
org.pcollections:pcollections Persistent collection types in metadata/ / image/
io.opentelemetry.proto:opentelemetry-proto Wire format for KIP-714 client metrics
com.google.protobuf:protobuf-java Used to encode the OTLP messages above
org.apache.maven:maven-artifact Used by Connect plugin loader for version comparisons

The full list lives in gradle/dependencies.gradle; runtime additions / removals must be reflected in LICENSE-binary (KAFKA-12622).

Test-only libraries

Used in tests but not shipped in runtime artifacts:

  • org.junit.jupiter:junit-jupiter:* — JUnit 5
  • org.mockito:mockito-core:*, mockito-junit-jupiter — Mocking
  • net.jqwik:jqwik:* — Property-based testing
  • org.hamcrest:hamcrest:* — Assertion matchers
  • org.testcontainers:* — Containerized integration tests (Connect, security)
  • com.github.dasniko:testcontainers-keycloak:* — OIDC integration tests
  • no.nav.security:mock-oauth2-server:* — In-process OAuth2 server for SASL/OAUTHBEARER tests
  • org.openjdk.jmh:* — Benchmark harness
  • org.eclipse.jgit:org.eclipse.jgit:* — Used by release tooling
  • org.jline:jline:* — Used by the metadata shell
  • org.jfree:jfreechart:* — Test reports / charts

Build-only libraries

These run during the build but never ship:

  • com.gradle.develocity — Build scans
  • com.github.spotbugs — Static analysis
  • checkstyle — Style checks
  • scoverage, jacoco — Coverage instrumentation
  • org.jetbrains.kotlin — Indirectly, for some Gradle plugins
  • biz.aQute.bnd — OSGi metadata generation

How upgrades happen

./gradlew dependencyUpdates --no-parallel

This emits a report of dependencies with newer versions available (Ben Manes Gradle plugin). For runtime libraries:

  1. Update the version in gradle/dependencies.gradle.
  2. Update LICENSE-binary to match (KAFKA-12622).
  3. Run ./gradlew check and any relevant integration tests.
  4. Open a PR with a MINOR: prefix or, for security fixes, a regular KAFKA- ticket.

Minimum runtime versions

  • JDK — 17 minimum at build time. Clients (clients, streams) compile to Java 11 bytecode and run on JDK 11+; the broker requires JDK 17 at runtime.
  • Scala — 2.13 only.
  • Java target levelsclients, streams: release=11; streams-scala: scalac release=11; everything else: release=17.

Cross-version Streams artifacts

streams/upgrade-system-tests-*/ declare dependencies on older kafka-streams releases (0.11 through 4.x). They aren't user-facing dependencies — they're used by ducktape upgrade tests to load older Streams binaries inside a current cluster.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Dependencies – Apache Kafka wiki | Factory