Kafka topic creation is failing.

Kafka topic creation is failing in below case:
Node in kafka cluster: 4
Replication factor: 4
Number of nodes up and running in cluster: 3

Below is the error:

./kafka-topics.sh --zookeeper :2181 --create --topic test_1 --partitions 1 --replication-factor 4
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : Replication factor: 4 larger than available brokers: 3.

Tagged:

Comments

  • When a new topic is created in Kafka, it is replicated N=replication-factor times across your brokers. Since you have 3 brokers up and running and replication-factor set to 4 the topic cannot be replicated 4 times and thus you get an error.

    When creating a new topic you either need to ensure that all of your 4 brokers are up and running or set the replication factor to a smaller value in order to avoid failure on topic creation when one of your brokers is down.

    In case you want to create topic with replication factor set to 4 while one broker is down, you can initially create the topic with replication-factor=3 and once your 4th broker is up and running you can modify the configuration of that topic and increase its replication factor by following the steps below (assuming you have a topic example with 4 partitions):

Sign In or Register to comment.