当前位置:
首页
文章
后端
详情

【kafka】kafka 在线增加分区副本数

  1. 创建 increase-replication-factor.json 文件

$ cat increase-replication-factor.json

{"version":1, "partitions":[
{"topic":"testTopic","partition":0,"replicas":[0,1,2]},
{"topic":"testTopic","partition":1,"replicas":[0,1,2]},
{"topic":"testTopic","partition":2,"replicas":[0,1,2]},
{"topic":"testTopic","partition":3,"replicas":[0,1,2]},
{"topic":"testTopic","partition":4,"replicas":[0,1,2]},
{"topic":"testTopic","partition":5,"replicas":[0,1,2]},
{"topic":"testTopic","partition":6,"replicas":[0,1,2]},
{"topic":"testTopic","partition":7,"replicas":[0,1,2]},
{"topic":"testTopic","partition":8,"replicas":[0,1,2]},
{"topic":"testTopic","partition":9,"replicas":[0,1,2]},
{"topic":"testTopic","partition":10,"replicas":[0,1,2]},
{"topic":"testTopic","partition":11,"replicas":[0,1,2]},
{"topic":"testTopic","partition":12,"replicas":[0,1,2]},
{"topic":"testTopic","partition":13,"replicas":[0,1,2]},
{"topic":"testTopic","partition":14,"replicas":[0,1,2]},
{"topic":"testTopic","partition":15,"replicas":[0,1,2]}]
}
  1. 执行增加副本操作
$  bin/kafka-reassign-partitions.sh --zookeeper xx.xx.xx.xx:2181  --reassignment-json-file increase-replication-factor.json  --execute
  1. 查看执行进度
$ /bin/kafka-reassign-partitions.sh --zookeeper xx.xx.xx.xx:2181  --reassignment-json-file  increase-replication-factor.json  --verify

Status of partition reassignment:
Reassignment of partition testTopic-15 completed successfully
Reassignment of partition testTopic-2 completed successfully
Reassignment of partition testTopic-7 completed successfully
Reassignment of partition testTopic-12 completed successfully
Reassignment of partition testTopic-4 completed successfully
Reassignment of partition testTopic-11 completed successfully
Reassignment of partition testTopic-10 completed successfully
Reassignment of partition testTopic-8 completed successfully
Reassignment of partition testTopic-5 completed successfully
Reassignment of partition testTopic-13 completed successfully
Reassignment of partition testTopic-1 completed successfully
Reassignment of partition testTopic-14 completed successfully
Reassignment of partition testTopic-9 completed successfully
Reassignment of partition testTopic-6 completed successfully
Reassignment of partition testTopic-0 completed successfully
Reassignment of partition testTopic-3 completed successfully

要熟练使用 Kafka 自带的 kafka-reassign-partitions.sh 脚本工具来完成对 topic 的分区分配、分区副本增加等操作。
该脚本有三个参数:

--generate:配合着 --topics-to-move-json-file 可以生成分区分配策略,该参数适用于分区多的情况
--execute: 配合着 --reassignment-json-file   可以执行分区分配策略
--verify:  配合着 --reassignment-json-file   可以检查分区分配进度

通过以上命令,既可以分配分区,也可以增加分区副本数,非常方便。

参考

kafka 迁移分区创建规则json
https://www.orchome.com/454#item-8

kafka命令大全
https://www.orchome.com/454#item-8

kafka集群中有多个offsets.topic.replication.factor?
https://cloud.tencent.com/developer/ask/sof/1540657/answer/2102855

kafka高可用失败问题 3broker,单杀一个broker就不能消费的问题探讨
https://www.orchome.com/805

免责申明:本站发布的内容(图片、视频和文字)以转载和分享为主,文章观点不代表本站立场,如涉及侵权请联系站长邮箱:xbc-online@qq.com进行反馈,一经查实,将立刻删除涉嫌侵权内容。

同类热门文章

深入了解C++中的new操作符:使用具体实例学习

C++中的new操作符是动态分配内存的主要手段之一。在程序运行时,我们可能需要动态地创建和销毁对象,而new就是为此提供了便利。但是,使用new也常常会引发一些问题,如内存泄漏、空指针等等。因此,本文将通过具体的示例,深入介绍C++中的new操作符,帮助读者更好地掌握其使用。


深入了解C++中的new操作符:使用具体实例学习

怎么用Java反射获取包下所有类? 详细代码实例操作

Java的反射机制就是在运行状态下,对于任何一个类,它能知道这个类的所有属性和方法;对于任何一个对象,都能调用这个对象的任意一个方法。本篇文章将通过具体的代码示例,展示如何通过Java反射来获取包下的所有类。


怎么用Java反射获取包下所有类? 详细代码实例操作

员工线上学习考试系统

有点播,直播,在线支付,三级分销等功能,可以对学员学习情况的监督监控,有源码,可二次开发。支持外网和局域网私有化部署,经过测试源码完整可用!1、视频点播:视频播放,图文资料,课件下载,章节试学,限时免

员工线上学习考试系统

了解Java中的volati关键字的作用 以及具体使用方法

本篇文章将和大家分享一下Java当中的volatile关键字,下面将为各位小伙伴讲述volatile关键字的作用以及它的具体使用方法。


了解Java中的volati关键字的作用 以及具体使用方法

Java Map 所有的值转为String类型

可以使用 Java 8 中的 Map.replaceAll() 方法将所有的值转为 String 类型: 上面的代码会将 map 中所有的值都转为 String 类型。 HashMap 是 Java

Java Map 所有的值转为String类型