Kafka：部署Kafka

文件下载

• Java 8
  
• ZooKeeper 3.6.3
  
• Kafka 3.0.0
  

下载到本地。

使用Xftp或其他工具上传到虚拟机。

Java

解压：

[[email protected] ~]# cd /usr/local/
[[email protected] local]# tar -zxf jdk-8u202-linux-x64.tar.gz

配置环境：

[[email protected] local]# ll
总用量 286092
-rw-r--r--. 1 root root  12516362 12月 20 14:56 apache-zookeeper-3.6.3-bin.tar.gz
drwxr-xr-x. 2 root root         6 4月  11 2018 bin
drwxr-xr-x. 2 root root         6 4月  11 2018 etc
drwxr-xr-x. 2 root root         6 4月  11 2018 games
drwxr-xr-x. 2 root root         6 4月  11 2018 include
drwxr-xr-x. 7   10  143       245 12月 16 2018 jdk1.8.0_202
-rw-r--r--. 1 root root 194042837 12月 20 14:56 jdk-8u202-linux-x64.tar.gz
-rw-r--r--. 1 root root  86396520 12月 20 14:56 kafka_2.13-3.0.0.tgz
drwxr-xr-x. 2 root root         6 4月  11 2018 lib
drwxr-xr-x. 2 root root         6 4月  11 2018 lib64
drwxr-xr-x. 2 root root         6 4月  11 2018 libexec
drwxr-xr-x. 2 root root         6 4月  11 2018 sbin
drwxr-xr-x. 5 root root        49 11月 10 16:17 share
drwxr-xr-x. 2 root root        40 11月 11 10:33 src
[[email protected] local]# cd jdk1.8.0_202/
[[email protected] jdk1.8.0_202]# pwd
/usr/local/jdk1.8.0_202
[[email protected] jdk1.8.0_202]# yum install -y vim
[[email protected] jdk1.8.0_202]# vim /etc/profile

将以下Java配置信息填入/etc/profile配置文件：

export JAVA_HOME=/usr/local/jdk1.8.0_202
export CLASSPATH=.:$JAVA_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH

使/etc/profile配置文件修改生效：

[[email protected] jdk1.8.0_202]# source /etc/profile

可以查到Java版本就说明安装成功了：

[[email protected] jdk1.8.0_202]# java -version
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

ZooKeeper

解压：

[[email protected] jdk1.8.0_202]# cd ..
[[email protected] local]# tar -zxf apache-zookeeper-3.6.3-bin.tar.gz 

copy配置文件：

[[email protected] local]# ll
总用量 286092
drwxr-xr-x. 6 root root       133 12月 20 15:08 apache-zookeeper-3.6.3-bin
-rw-r--r--. 1 root root  12516362 12月 20 14:56 apache-zookeeper-3.6.3-bin.tar.gz
drwxr-xr-x. 2 root root         6 4月  11 2018 bin
drwxr-xr-x. 2 root root         6 4月  11 2018 etc
drwxr-xr-x. 2 root root         6 4月  11 2018 games
drwxr-xr-x. 2 root root         6 4月  11 2018 include
drwxr-xr-x. 7   10  143       245 12月 16 2018 jdk1.8.0_202
-rw-r--r--. 1 root root 194042837 12月 20 14:56 jdk-8u202-linux-x64.tar.gz
-rw-r--r--. 1 root root  86396520 12月 20 14:56 kafka_2.13-3.0.0.tgz
drwxr-xr-x. 2 root root         6 4月  11 2018 lib
drwxr-xr-x. 2 root root         6 4月  11 2018 lib64
drwxr-xr-x. 2 root root         6 4月  11 2018 libexec
drwxr-xr-x. 2 root root         6 4月  11 2018 sbin
drwxr-xr-x. 5 root root        49 11月 10 16:17 share
drwxr-xr-x. 2 root root        40 11月 11 10:33 src
[[email protected] local]# cd apache-zookeeper-3.6.3-bin/conf
[[email protected] conf]# ll
总用量 12
-rw-r--r--. 1 1000 1000  535 4月   9 2021 configuration.xsl
-rw-r--r--. 1 1000 1000 3435 4月   9 2021 log4j.properties
-rw-r--r--. 1 1000 1000 1148 4月   9 2021 zoo_sample.cfg
[[email protected] conf]# cp zoo_sample.cfg zoo.cfg

启动ZooKeeper服务：

[[email protected] conf]# cd ../bin
[[email protected] bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[[email protected] bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone

出现Mode: standalone，说明启动成功了。

Kafka

解压：

[[email protected] bin]# cd ../..
[[email protected] local]# tar -zxf kafka_2.13-3.0.0.tgz 

修改配置：

[[email protected] local]# cd kafka_2.13-3.0.0/config
[[email protected] config]# ll
总用量 72
-rw-r--r--. 1 root root  906 9月   9 05:21 connect-console-sink.properties
-rw-r--r--. 1 root root  909 9月   9 05:21 connect-console-source.properties
-rw-r--r--. 1 root root 5475 9月   9 05:21 connect-distributed.properties
-rw-r--r--. 1 root root  883 9月   9 05:21 connect-file-sink.properties
-rw-r--r--. 1 root root  881 9月   9 05:21 connect-file-source.properties
-rw-r--r--. 1 root root 2103 9月   9 05:21 connect-log4j.properties
-rw-r--r--. 1 root root 2540 9月   9 05:21 connect-mirror-maker.properties
-rw-r--r--. 1 root root 2262 9月   9 05:21 connect-standalone.properties
-rw-r--r--. 1 root root 1221 9月   9 05:21 consumer.properties
drwxr-xr-x. 2 root root  102 9月   9 05:21 kraft
-rw-r--r--. 1 root root 4674 9月   9 05:21 log4j.properties
-rw-r--r--. 1 root root 1925 9月   9 05:21 producer.properties
-rw-r--r--. 1 root root 6862 12月 20 15:19 server.properties
-rw-r--r--. 1 root root 1032 9月   9 05:21 tools-log4j.properties
-rw-r--r--. 1 root root 1169 9月   9 05:21 trogdor.conf
-rw-r--r--. 1 root root 1205 9月   9 05:21 zookeeper.properties
[[email protected] config]# vim server.properties

如果ZooKeeper和Kafka不在同一台主机上部署，还需要修改ZooKeeper的配置：

测试

启动Kafka：

[[email protected] config]# cd ..
[[email protected] kafka_2.13-3.0.0]# bin/kafka-server-start.sh config/server.properties &

创建一个topic：

[[email protected] kafka_2.13-3.0.0]# bin/kafka-topics.sh --create --bootstrap-server 192.168.31.240:9092 --replication-factor 1 --partitions 1 --topic kaven-topic
[2021-12-20 15:52:53,172] INFO Creating topic kaven-topic with configuration {} and initial partition assignment HashMap(0 -> ArrayBuffer(0)) (kafka.zk.AdminZkClient)
[2021-12-20 15:52:53,335] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions Set(kaven-topic-0) (kafka.server.ReplicaFetcherManager)
[2021-12-20 15:52:53,457] INFO [LogLoader partition=kaven-topic-0, dir=/tmp/kafka-logs] Loading producer state till offset 0 with message format version 2 (kafka.log.Log$)
[2021-12-20 15:52:53,485] INFO Created log for partition kaven-topic-0 in /tmp/kafka-logs/kaven-topic-0 with properties {} (kafka.log.LogManager)
[2021-12-20 15:52:53,486] INFO [Partition kaven-topic-0 broker=0] No checkpointed highwatermark is found for partition kaven-topic-0 (kafka.cluster.Partition)
[2021-12-20 15:52:53,487] INFO [Partition kaven-topic-0 broker=0] Log loaded for partition kaven-topic-0 with initial high watermark 0 (kafka.cluster.Partition)
Created topic kaven-topic.

查看已经创建的topic：

[[email protected] kafka_2.13-3.0.0]# bin/kafka-topics.sh --list --bootstrap-server 192.168.31.240:9092
kaven-topic
[[email protected] kafka_2.13-3.0.0]# bin/kafka-topics.sh --describe --bootstrap-server 192.168.31.240:9092 --topic kaven-topic
Topic: kaven-topic	TopicId: EvTOyGnBSPyOpvXb3oR2rQ	PartitionCount: 1	ReplicationFactor: 1	Configs: segment.bytes=1073741824
	Topic: kaven-topic	Partition: 0	Leader: 0	Replicas: 0	Isr: 0

在topic中写入一些信息（Ctrl+C结束）：

[[email protected] kafka_2.13-3.0.0]# bin/kafka-console-producer.sh --bootstrap-server 192.168.31.240:9092 --topic kaven-topic
>hello
>this is kafka producer
>加油，先溜了

新开一个窗口，用来读取在topic中写入的信息：

[[email protected] ~]# cd /usr/local/kafka_2.13-3.0.0
[[email protected] kafka_2.13-3.0.0]# bin/kafka-console-consumer.sh --bootstrap-server 192.168.31.240:9092  --from-beginning --topic kaven-topic
hello                                                                                                                           
this is kafka producer
加油，先溜了

部署Kafka就介绍到这里，以后博主还会介绍Kafka的相关概念、使用和实战，如果博主有说错的地方或者大家有不同的见解，欢迎大家评论补充。