码迷,mamicode.com
首页 > 其他好文 > 详细

如何彻底删除Kafka中的topic (marked for deletion)

时间:2019-06-20 17:21:30      阅读:161      评论:0      收藏:0      [点我收藏+]

标签:客户端   参数   相关   sudo su   c中   清理   添加   pos   follow   

工作中因为各种原因, 例如topic中消息堆积的太多,或者kafka所在磁盘空间满了等等,可能需要彻底清理一下kafka topic,那么如何彻底删除topic?
方法一(配置delete.topic.enable=true)
  修改kafaka配置文件server.properties, 添加delete.topic.enable=true,重启kafka,之后通过kafka命令行就可以直接删除topic
  通过命令行删除topic:

    ./bin/kafka-topics.sh --delete --zookeeper {zookeeper server} --topic {topic name}

方法二(没有配置delete.topic.enable=true)

1、通过命令行删除topic:

./bin/kafka-topics.sh --delete --zookeeper {zookeeper server} --topic {topic name}

  因为kafaka配置文件中server.properties没有配置delete.topic.enable=true,此时的删除并不是真正的删除,只是把topic标记为:marked for deletion
  你可以通过命令:./bin/kafka-topics --zookeeper {zookeeper server} --list 来查看所有topic
2、删除kafka存储目录(server.properties文件log.dirs配置,默认为"/tmp/kafka-logs")相关topic目录


3, 若想真正删除它,需要登录zookeeper客户端:

  命令:./bin/zkCli.sh

  找到topic所在的目录:ls /brokers/topics

   执行命令:rmr /brokers/topics/{topic name}即可,此时topic被彻底删除。

  另外被标记为marked for deletion的topic你可以在zookeeper客户端中通过命令获得:ls /admin/delete_topics/{topic name},如果你删除了此处的topic,那么marked for deletion 标记消失

zookeeper 的config中也有有关topic的信息: ls /config/topics/{topic name}暂时不知道有什么用

总结

彻底删除topic:
1、确保kafka的配置文件server.proeprties中设置delete.topic.enable=true,如果没有, 确保cluster的所有kafka配置文件设置该参数并重启,然后直接通过命令删除,如果命令删除不了,直接通过zookeeper命令行删除掉broker下的topic。

2、删除kafka存储目录(server.properties文件log.dirs配置,默认为"/tmp/kafka-logs")相关topic目录

参考文档:
1, https://stackoverflow.com/questions/17730905/is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev

stop zookeeper & Kafka server,
then go to ‘kafka-logs’ folder , there you will see list of kafka topic folders, delete folder with topic name
go to ‘zookeeper-data’ folder , delete data inside that.
start zookeeper & kafka server again.
2, https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Isitpossibletodeleteatopic?
Deleting a topic is supported since 0.8.2.x. You will need to enable topic deletion (setting delete.topic.enable to true) on all brokers first.

3, https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default

4, https://github.com/darrenfu/bigdata/issues/6

*** The only way to delete a topic permanently is as follows: ***

stop the brokers
sudo supervisorctl stop kafka-broker
remove the directories on disk
sudo rm -rf <kafka_data_dir>/<topic_name>*
remove the topic from zookeeper:
bin/zkCli.sh


rmr /config/topics/<topic_name>
rmr /brokers/topics/<topic_name>
rmr /admin/delete_topics/<topic_name>
 

verify the topic directory is deleted in zookeeper:
start the brokers back up
sudo supervisorctl start kafka-broker
re-create the kafka topic if needed

转自https://blog.csdn.net/russle/article/details/82881297

如何彻底删除Kafka中的topic (marked for deletion)

标签:客户端   参数   相关   sudo su   c中   清理   添加   pos   follow   

原文地址:https://www.cnblogs.com/zhtiy/p/11059596.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!