更改Kafka同步副本

我们正在使用Avro运行开源Kafka Confluent 5.2.1,以对消息进行编码/解码。当我们构建一个新集群并向其发布模式时,我们的__consumer_offsets主题具有以下配置:

$shell> kafka-topics --zookeeper localhost:2181/apps/kafka_cluster --describe --topic __consumer_offsets
Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:1     Configs:compression.type=producer,cleanup.policy=compact,segment.bytes=104857600
        Topic: __consumer_offsets       Partition: 0    Leader: 101     Replicas: 101   Isr: 101
        Topic: __consumer_offsets       Partition: 1    Leader: 102     Replicas: 102   Isr: 102
        Topic: __consumer_offsets       Partition: 2    Leader: 101     Replicas: 101   Isr: 101
        Topic: __consumer_offsets       Partition: 3    Leader: 102     Replicas: 102   Isr: 102
...
        Topic: __consumer_offsets       Partition: 48   Leader: 101     Replicas: 101   Isr: 101
        Topic: __consumer_offsets       Partition: 49   Leader: 102     Replicas: 102   Isr: 102

然后我使用以下JSON文件重新分配分区:

{"version":1,"partitions":[
  {"topic":"__consumer_offsets","partition":0,"replicas":[101,102,103]},{"topic":"__consumer_offsets","partition":1,"replicas":[102,103,101]},"partition":2,"replicas":[103,101,102]},...
  {"topic":"__consumer_offsets","partition":45,"partition":46,"partition":47,"partition":48,"partition":49,101]}
]}

最终结果是副本已更改,但是同步副本有时会更改,而有时却没有:

$shell> kafka-topics --zookeeper bigdevmq02c:2181/apps/kafka_cluster --describe --topic __consumer_offsets                Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:3     Configs:compression.type=producer,segment.bytes=104857600
        Topic: __consumer_offsets       Partition: 0    Leader: 101     Replicas: 101,103   Isr: 101
        Topic: __consumer_offsets       Partition: 1    Leader: 102     Replicas: 102,101   Isr: 102,101
        Topic: __consumer_offsets       Partition: 2    Leader: 101     Replicas: 103,102   Isr: 101
        Topic: __consumer_offsets       Partition: 3    Leader: 102     Replicas: 101,103   Isr: 102,101
...
        Topic: __consumer_offsets       Partition: 48   Leader: 101     Replicas: 101,103   Isr: 101
        Topic: __consumer_offsets       Partition: 49   Leader: 102     Replicas: 102,101

我希望同步副本与副本匹配,然后根据副本的第一个成员运行领导者选举,如下所示:

$shell> kafka-preferred-replica-election --bootstrap-server localhost:9092

,但是当前失败。我做错了什么以及如何纠正?

非常感谢您的帮助。

更新

我运行了一个验证,在2.5小时后的一个空群集上,它仍然显示为不完整:

$shell> kafka-reassign-partitions --zookeeper localhost:2181/apps/kafka_cluster --reassignment-json-file dev.json --verify
Status of partition reassignment:
Reassignment of partition __consumer_offsets-22 is still in progress
Reassignment of partition __consumer_offsets-30 is still in progress
Reassignment of partition __consumer_offsets-8 is still in progress
Reassignment of partition __consumer_offsets-21 completed successfully
z6206519 回答:更改Kafka同步副本

环顾了一会后,我们注意到集群中有一个节点正在运行5.2.1版的Kafka,其余节点正在运行5.3.1版。将在该节点上运行的版本更改为5.3.1可以解决此问题。

本文链接:https://www.f2er.com/3084467.html

大家都在问