Getting started

This page walks through the minimum needed to clone Kafka, build it, run a broker locally, run the tests, and find your way around the source tree. The authoritative source is README.md; this page summarizes it and adds pointers into the codebase.

Prerequisites

• JDK 17 or 25. The project compiles clients and streams for Java 11 and the rest for Java 17. Develop with JDK 17.
• Scala 2.13 (the only supported Scala). Pulled in automatically by Gradle, no separate install needed.
• Bash + Python 3 if you intend to run the system tests under tests/ (which use the ducktape framework).
• Docker (optional) for running container images defined in docker/.

The Gradle wrapper (./gradlew) is checked in, so no global Gradle install is required.

Clone and first build

git clone https://github.com/apache/kafka.git
cd kafka
./gradlew jar         # build all module JARs

The first build downloads the Gradle distribution and all dependencies; subsequent builds are incremental. Build artifacts land under <module>/build/libs/.

Generate auto-generated message classes

Many request/response classes are generated from JSON schemas under clients/src/main/resources/common/message/. If you switch branches or pull new changes, run:

./gradlew processMessages processTestMessages

The generator itself is in generator/; see clients/src/main/resources/common/message/README.md for the schema syntax.

Run a single-node broker (KRaft)

Kafka 4.x runs in KRaft mode by default — no ZooKeeper. The shell scripts in bin/ are the supported entry points.

# 1. Make a cluster ID (any UUID works; this helper produces one)
KAFKA_CLUSTER_ID="$(./bin/kafka-storage.sh random-uuid)"

# 2. Format storage for a combined controller+broker node
./bin/kafka-storage.sh format --standalone -t "$KAFKA_CLUSTER_ID" -c config/server.properties

# 3. Start the broker on the foreground
./bin/kafka-server-start.sh config/server.properties

config/server.properties configures a single-node combined controller+broker on port 9092. For multi-broker setups, copy and edit config/broker.properties and config/controller.properties. The broker entry point is kafka.Kafka.main in core/src/main/scala/kafka/Kafka.scala.

To verify the broker is running, in another terminal:

./bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic test --partitions 1
./bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test
./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

Run the test suite

./gradlew test                 # all unit + integration tests
./gradlew unitTest             # unit tests only
./gradlew integrationTest      # integration tests only
./gradlew clients:test         # one module
./gradlew clients:test --tests org.apache.kafka.clients.MetadataTest.testTimeToNextUpdate

Useful flags:

• -Pkafka.test.run.flaky=true — also run tests tagged @Flaky
• -PmaxTestRetries=1 -PmaxTestRetryFailures=3 — retry failing tests
• -PenableTestCoverage=true plus ./gradlew reportCoverage — produce coverage reports under <module>/build/reports/...

System tests live in tests/ and are run via Trogdor / ducktape (see tests/README.md); they are heavier and require Docker or Vagrant.

Code quality checks

./gradlew checkstyleMain checkstyleTest spotlessCheck
./gradlew spotbugsMain spotbugsTest -x test
./gradlew spotlessApply        # auto-fix import order

Reports land under <module>/build/reports/checkstyle/ and .../spotbugs/.

IDE setup

IntelliJ imports the Gradle project natively — open the repository root and select the Gradle build script. Make sure JDK 17 is selected as the project SDK. For Eclipse, run ./gradlew eclipse first.

Building a release tarball

./gradlew clean releaseTarGz

Output: core/build/distributions/kafka_<scala>-<version>.tgz.

Where to go next

• How to contribute — KIP process, branches, PRs.
• Development workflow — day-to-day commands.
• Testing — JUnit conventions, Trogdor, ducktape.
• Modules — what every Gradle module is for.