Install Kafka on Windows

This post is a step-by-step guide to install and run Apache Kafka on Windows.

Prerequisite

The only prerequisite for this setup is JRE.

Install Java (Skip if you already have it)

1. Download Java 8 from here. (Java 8 is recommended by Apache Kafka)
2. Run the installer and follow the instructions on Installation wizard.
3. Please note/copy the Destination Folder location.
4. Go to Control Panel -> System -> Advanced system settings -> Environment Variables.
5. Create a new user variable named JAVA_HOME and paste the path copied from step 3 to the variable value and click OK.
6. Now edit PATH variable in User variables and add “%JAVA_HOME%\bin” at the end of the variable value. If it’s an older windows version, then add “;%JAVA_HOME%\bin;” at the end of the text. If PATH variable doesn’t exist create it with the value “%JAVA_HOME%\bin”.
7. Open command prompt and type “java -version” to validate the installation.
8. If you get the following output in your command prompt you’re good to go:

java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)

Steps to install Kafka on windows:

1) Download and extract Apache Kafka from here.

Note: At the time of writing this post the current stable version is 2.4.0. If you’re using Scala for development then make sure to select the Kafka version corresponding to your Scala version.

2) Go to the folder where you’ve extracted Kafka, open the file server.properties in the config folder and then edit the line log.dirs=/tmp/kafka-logs to log.dirs=C:\path_where_kafka_is_extracted\kafka-logs.

Note: If you won’t change log.dirs value, you’ll keep getting the following error:

java.util.NoSuchElementException: key not found: /tmp/kafka-logs

3) Start Zookeeper

Kafka uses ZooKeeper so before starting Kafka you have to make sure that ZooKeeper is up and running. We’ll be running the single node ZooKeeper instance packaged with Kafka. 

Go to your Kafka installation directory and open command prompt and start Zookeeper using the following command:

> bin\windows\zookeeper-server-start.bat config\zookeeper.properties

Once zookeeper is started you should see this in the command prompt:

INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)

4) Start Kafka Server

You need to open another command prompt in the Kafka installation directory and run the following command:

> bin\windows\kafka-server-start.bat \config\server.properties

Once the Kafka server is started you should see something like this in the command prompt:

INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

5) Validate the Kafka setup

Create a Kafka topic:

Open a command prompt in the Kafka installation directory and run the following command:

> bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic

The above command will create a topic “testTopic” with 1 partition and 1 replication factor.

Produce messages to the created topic:

In the same command prompt start a kafka-console-producer and produce some messages:

> bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic testTopic
>Hello
>How are you?

Consume the messages produced by the kafka-console-producer:

Open another command prompt in the Kafka installation directory and start a kafka-console-consumer which subscribes to testTopic:

> bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic testTopic --from-beginning
Hello
How are you?

The messages which are produced by the kafka-console-producer will be visible on the kafka-console-consumer console.

Congratulations on completing a single broker Kafka setup on windows!