Kafka 브로커를 모니터링하거나, 어드민으로 관리하기 위한 다양한 툴이 존재한다. AKHQ, Kowl, Kafdrop, UI for Apache Kafka 등이 있다. 여기서는 현재 많이 사용되는 AKHQ를 띄워본다.

AKHQ

AKHQ는 기본적으로 8080 포트를 사용한다.

version: '3'
services:
  zookeeper-1:
    hostname: zookeeper1
    image: confluentinc/cp-zookeeper:6.2.0
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 12181
      ZOOKEEPER_DATA_DIR: /zookeeper/data
      ZOOKEEPER_SERVERS: zookeeper1:22888:23888;zookeeper2:32888:33888;zookeeper3:42888:43888
    ports:
      - 12181:12181
      - 22888:22888
      - 23888:23888
    volumes:
      - ./zookeeper/data/1:/zookeeper/data

  zookeeper-2:
    hostname: zookeeper2
    image: confluentinc/cp-zookeeper:6.2.0
    environment:
      ZOOKEEPER_SERVER_ID: 2
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_DATA_DIR: /zookeeper/data
      ZOOKEEPER_SERVERS: zookeeper1:22888:23888;zookeeper2:32888:33888;zookeeper3:42888:43888
    ports:
      - 22181:22181
      - 32888:32888
      - 33888:33888
    volumes:
      - ./zookeeper/data/2:/zookeeper/data

  zookeeper-3:
    hostname: zookeeper3
    image: confluentinc/cp-zookeeper:6.2.0
    environment:
      ZOOKEEPER_SERVER_ID: 3
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_DATA_DIR: /zookeeper/data
      ZOOKEEPER_SERVERS: zookeeper1:22888:23888;zookeeper2:32888:33888;zookeeper3:42888:43888
    ports:
      - 32181:32181
      - 42888:42888
      - 43888:43888
    volumes:
      - ./zookeeper/data/3:/zookeeper/data

  kafka-1:
    image: confluentinc/cp-kafka:6.2.0
    hostname: kafka1
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper1:12181,zookeeper2:22181,zookeeper3:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:19092
      KAFKA_LOG_DIRS: /kafka
    ports:
      - 19092:19092
    volumes:
      - ./kafka/logs/1:/kafka

  kafka-2:
    image: confluentinc/cp-kafka:6.2.0
    hostname: kafka2
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper1:12181,zookeeper2:22181,zookeeper3:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29092
      KAFKA_LOG_DIRS: /kafka
    ports:
      - 29092:29092
    volumes:
      - ./kafka/logs/2:/kafka

  kafka-3:
    image: confluentinc/cp-kafka:6.2.0
    hostname: kafka3
    depends_on:
      - zookeeper-1
      - zookeeper-2
      - zookeeper-3
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper1:12181,zookeeper2:22181,zookeeper3:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:39092
      KAFKA_LOG_DIRS: /kafka
    ports:
      - 39092:39092
    volumes:
      - ./kafka/logs/3:/kafka

  akhq:
    image: tchiotludo/akhq:latest
    hostname: akhq
    depends_on:
      - kafka-1
      - kafka-2
      - kafka-3
    environment:
      AKHQ_CONFIGURATION: |
        akhq:
          connections:
            kafka:
              properties:
                bootstrap.servers: kafka1:19092,kafka2:29092,kafka3:39092
    ports:
      - 8080:8080

image

image

여기서 value를 바꾼후에 업데이트도 가능하다.

image

topic 만들기도 가능하다.

image

image

Live Tail 은 어떤 메세지가 실시간으로 들어오는지 모니터링 할 때 좋다.

Kafka Burrow

https://github.com/linkedin/Burrow/wiki

운영에서 consumer lag을 관리하는것은 중요하다. consumer lag은 producer가 커밋한 offset과 consumer가 커밋한 offset의 차이이다. 즉, consumer가 아직 처리하지 못한 producer의 메세지 개수이다. consumer lag을 주기적으로 체크하고 알람을 받기 위해서 kafka burrow라는 툴을 사용한다. burrow는 GO 를 사용한다.

$ git clone https://github.com/linkedin/Burrow.git burrow
$ sudo apt install golang -y
$ go mod tidy
$ go install 
$ $GOPATH/bin/Burrow --config-dir </path/containing/config>

config를 포함하고 있는 toml 파일은 아래와 같다. https://github.com/linkedin/Burrow/wiki/Configuration 에서 config를 작성하는 법을 볼 수 있다.

[general]
pidfile="burrow.pid"
stdout-logfile="burrow.out"

[logging]
filename="logs/burrow.log"
level="info"
maxsize=10
maxbackups=3
use-compression=true

[zookeeper]
servers=["localhost:12181","localhost:22181","localhost:32181"]

[cluster.mycluster]
class-name="kafka"
servers=["localhost:19092","localhost:29092","localhost:39092"]
topic-refresh=60
offset-refresh=10

[consumer.myconsumers]
class-name="kafka"
cluster="mycluster"
servers=["localhost:19092","localhost:29092","localhost:39092"]
group-denylist=""
group-allowlist=""

[httpserver.listener]
address=":8000"
timeout=300

[storage.inmemory]
class-name="inmemory"
min-distance=1
workers=20
intervals=10
expire-group=604800

burrow에서 어떤 것을 보여주는지 테스트 해보자.

$ bin/kafka-topics --bootstrap-server localhos:19092 --create --topic burrowtest --partitions 3 --replication-factor 3
$ bin/kafka-console-consumer --bootstrap-server localhos:19092 --topic burrowtest --gruop burrowgroup
$ bin/kafka-console-consumer --bootstrap-server localhos:19092 --topic burrowtest --gruop burrowgroup

두개의 consumer를 띄우면 나눠서 실행하는 것을 볼 수 있다.

image

더 많은 https://github.com/linkedin/Burrow/wiki/HTTP-Endpoint 와 https://github.com/linkedin/Burrow/wiki/http-request-consumer-group-status 를 참고하면 된다.