Aim

We will install Kafka Manager using docker compose. In this post we will learn to install three components using docker compose

  • Kafka
  • Zookeeper
  • Kafka Manager

Create a YAML file

touch kafka-docker-compose.yml
  • Put the below contents in that file
version: "3"
services:
  zookeeper:
    image: zookeeper
    restart: always
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
    - 9092:9092
    environment:
      KAFKA_ADVERTISED_HOST_NAME: [machine-ip-address]
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  kafka_manager:
    image: hlebalbau/kafka-manager:stable
    container_name: kakfa-manager
    restart: always
    ports:
      - "9000:9000"
    environment:
      ZK_HOSTS: "zookeeper:2181"
      APPLICATION_SECRET: "random-secret"
    command: -Dpidfile.path=/dev/null

In the above docker compose file we are creating three services

  1. Zookeeper
  2. Kafka
  3. Kafka Manager

Configuration Needed

  • In Kafka service you will see the [machine-ip-address]
  • You need to set this to your machine IP address

Starting the service

  • You can start the services in foreground. When you run the services in foreground mode then the logs are visible on the screen.
docker-compose -f kafka-docker-compose.yml up
  • If you want to run the service in background mode then you can use the following command
docker-compose -f kafka-docker-compose.yml up -d

Service 1 : Kafka

  • The above compose file creates a Kafka service

External client can connect to this Kafka at 9092 Port which you can see KAFKA_ADVERTISED_HOST_NAME environment variable

Service 2 : Zookeeper

Zookeeper service is running at 2181 port

Service 3 : Kafka Manager

Kafka Manger is running at 9000 port

Happy Coding

5 3 votes
Article Rating
Subscribe
Notify of
guest
2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Ramesh Hanumanthappa
Ramesh Hanumanthappa
3 months ago

Hi
I followed your tutorials on Youtube to set up kafka cluster using docker-compose file.. But offsets are not recognized in the kafka manager.. Any help

Last edited 3 months ago by Ramesh Hanumanthappa
2
0
Would love your thoughts, please comment.x
()
x