组件 | 版本 |
---|---|
centos | 6.5 64x |
zookeeper | 3.4.5 |
kafka | 2.10-0.8.1.1 |
tar zxvf kafka_2.10-0.8.1.1.tar.gz
cd kafka_2.10-0.8.1.1/
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties
./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
#列出 topic
./bin/kafka-topics.sh --list --zookeeper localhost:2181
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
cp config/server.properties config/server-1.properties
主要修改内容
broker.id=0
log.dirs=/home/Hadoop/development/src/kafka_2.10-0.8.1.1/logs
zookeeper.connect=canbot130:2181,canbot131:2181,canbot132:2181
scp -r ./kafka_2.10-0.8.1.1/ hadoop@canbot131:/home/hadoop/development/src/
scp -r ./kafka_2.10-0.8.1.1/ hadoop@canbot132:/home/hadoop/development/src/
copy完以后需要修改再修改 server-1.properties 中的 broker.id
broker.id=0 192.169.2.130
broker.id=1 192.169.2.131
broker.id=2 192.169.2.132
分别在canbot130/1/2三个节点都启动
./kafka_2.10-0.8.1.1/bin/kafka-server-start.sh ./kafka_2.10-0.8.1.1/config/server-1.properties &
[hadoop@canbot130 kafka_2.10-0.8.1.1]$./bin/kafka-topics.sh --create --zookeeper canbot130:2181 --replication-factor 3 --partitions 1 --topic test
提示以下内容表示创建Topic 成功
Created topic "test".
[hadoop@canbot130 kafka_2.10-0.8.1.1]$ ./bin/kafka-topics.sh --list --zookeeper canbot130:2181
test
[hadoop@canbot130 kafka_2.10-0.8.1.1]$
./bin/kafka-console-consumer.sh --zookeeper canbot130:2181 --topic test
使用该命令创建生产者,然后将在canbot132节点上创建 消费者,查看消息是否被消费
在 canbot132 节点上执行
./bin/kafka-console-consumer.sh --zookeeper canbot130:2181 --topic test
在canbot130节点上的生产者
[hadoop@canbot130 kafka_2.10-0.8.1.1]$ ./bin/kafka-console-producer.sh --broker-list canbot130:9092 --topic test
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
"holl"
[2016-05-31 04:27:30,770] INFO Closing socket connection to /192.168.2.130. (kafka.network.Processor)
"hao xiang shi tong bu l haha"
"test kafka"
在canbot132节点上的消费者所产生的信息
[hadoop@canbot132 kafka_2.10-0.8.1.1]$ ./bin/kafka-console-consumer.sh --zookeeper canbot130:2181 --topic test
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2016-05-31 04:27:22,328] INFO Closing socket connection to /192.168.2.132. (kafka.network.Processor)
holl
hao xiang shi tong bu l haha
test kafka
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
at kafka.utils.ZkUtils$.registerBrokerInZk(ZkUtils.scala:205)
at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:57)
at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:44)
at kafka.server.KafkaServer.startup(KafkaServer.scala:103)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
解决方法:该错误是由于server.properties 中的broker.id 重复
当前题目:CentOS6.5下Kafka安装部署
本文地址:http://www.shufengxianlan.com/qtweb/news33/459433.html
网站建设、网络推广公司-创新互联,是专注品牌与效果的网站制作,网络营销seo公司;服务项目有等
声明:本网站发布的内容(图片、视频和文字)以用户投稿、用户转载内容为主,如果涉及侵权请尽快告知,我们将会在第一时间删除。文章观点不代表本网站立场,如需处理请联系客服。电话:028-86922220;邮箱:631063699@qq.com。内容未经允许不得转载,或转载时需注明来源: 创新互联