10分钟搭建单机伪分布式Kafka集群

创建一个kafka目录

mkdir kafka

cd kafka

wget https://archive.apache.org/dist/kafka/1.1.0/kafka_2.11-1.1.0.tgz

拷贝3份kafka配置文件

[root@xinyeshuaiqi config]# cp server.properties server-0.properties              
[root@xinyeshuaiqi config]# cp server.properties server-1.properties
[root@xinyeshuaiqi config]# cp server.properties server-2.properties
[root@xinyeshuaiqi config]# ls
connect-console-sink.properties    connect-distributed.properties  connect-file-source.properties  connect-standalone.properties  log4j.properties     server-0.properties  server-2.properties  tools-log4j.properties
connect-console-source.properties  connect-file-sink.properties    connect-log4j.properties        consumer.properties            producer.properties  server-1.properties  server.properties    zookeeper.properties
[root@xinyeshuaiqi config]# pwd
/root/kafka/kafka_2.11-1.1.0/config

需要修改的配置

server-0.properties

broker.id=0
listeners=PLAINTEXT://127.0.0.1:9092
log.dirs=/root/kafka/logs/log_0
num.partitions=2
zookeeper.connect=localhost:2181

server-1.properties

broker.id=1
listeners=PLAINTEXT://127.0.0.1:9093
log.dirs=/root/kafka/logs/log_1
num.partitions=2
zookeeper.connect=localhost:2181

server-2.properties

broker.id=2
listeners=PLAINTEXT://127.0.0.1:9094
log.dirs=/root/kafka/logs/log_2
num.partitions=2
zookeeper.connect=localhost:2181

启动一波，直接报错

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c0000000, 1073741824, 0) failed; error=’Cannot allocate memory’ (errno=12)

修改下kafka启动脚本参数 kafka-server-start.sh

export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"

启动kafka集群

bin/kafka-server-start.sh -daemon config/server-0.properties  
bin/kafka-server-start.sh -daemon config/server-1.properties   
bin/kafka-server-start.sh -daemon config/server-2.properties

[root@xinyeshuaiqi kafka_2.11-1.1.0]# jps
2899 QuorumPeerMain
2856 QuorumPeerMain
13000 Kafka
12376 Kafka
2827 QuorumPeerMain
13341 Kafka
13678 Jps

看下zk上的节点信息

创建一个topic

./kafka-topics.sh --create --zookeeper 127.0.0.1:2181 
 --partitions 2 --replication-factor 2 --topic kafka-test

查看topic 列表

sh kafka-topics.sh --zookeeper 127.0.0.1:2181 --list

生产消息

sh kafka-console-producer.sh --broker-list 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic kafka-test

消费消息

sh kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic kafka-test

转载请注明：汪明鑫的个人博客 » 10分钟搭建单机伪分布式Kafka集群

说点什么