kafka 简单入门系列之一
Kafka 入门教程之一: 安装
1. 官网下载软件
2. 安装单节点kafka
a) 创建用户kafka
[root@tjtestrac1 ~]# useradd kafka
[root@tjtestrac1 ~]# passwd kafka
Changing passwordforuser kafka.
New password:
BAD PASSWORD: The password is shorter than8characters
Retype new password:
passwd: all authentication tokens updated successfully.
b) 解压软件包
[kafka@tjtestrac1 ~]$ ls
kafka_2.12-2.1.0.tgz
[kafka@tjtestrac1 ~]$ tar -xvf kafka_2.12-2.1.0.tgz
c) 配置环境变量 kafka_home 和 java_home
[kafka@tjtestrac1 ~]$ vi .bash_profile
export KAFKA_HOME=/home/kafka/kafka_2.12-2.1.0
export JAVA_HOME=/u02/cassandra/java/jdk1.8.0_162
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$KAFKA_HOME/bin [kafka@tjtestrac1 ~]$ source .bash_profile
3. 启动zookeeper, kafka 组件
a) 为zookeeper 创建文件夹
[kafka@tjtestrac1 config]$ mkdir -p /home/kafka/zk
b) 修改kafka 内置的zk 配置文件
[kafka@tjtestrac1 config]$ vi zookeeper.properties
dataDir=/home/kafka/zk
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
c) 启动zk
kafka@tjtestrac1 config]$ zookeeper-server-start.sh$KAFKA_HOME/config/zookeeper.properties &
d) 查看后台进程
[kafka@tjtestrac1 ~]$ jps
30323QuorumPeerMain
30660Jps
4. 启动kafka
a) 为kafka 创建文件夹
[kafka@tjtestrac1 ~]$ mkdir -p /home/kafka/kf
b) 修改kafka 的配置文件
[kafka@tjtestrac1 config]$ vi server.properties
log.dirs=/home/kafka/kf
c) 启动kafka 进程
[kafka@tjtestrac1 config]$ kafka-server-start.sh$KAFKA_HOME/config/server.properties &
d) 查看后台进程
[kafka@tjtestrac1 config]$ jps
30323QuorumPeerMain
4739Kafka
5414Jps
5. 创建一个主题topic
[kafka@tjtestrac1 config]$ kafka-topics.sh --create --zookeeper localhost:2181--replication-factor1--partitions1--topic test
[2018-11-2217:27:47,517] INFO Accepted socket connection from /127.0.0.1:58228(org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-11-2217:27:47,519] INFO Client attempting to establish new session at /127.0.0.1:58228(org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-2217:27:47,521] INFO Established session0x10052ce304e0001 with negotiated timeout30000forclient /127.0.0.1:58228(org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-2217:27:47,802] INFO Got user-level KeeperException when processing sessionid:0x10052ce304e0001 type:setData cxid:0x4 zxid:0x1f txntype:-1reqpath:n/a Error Path:/config/topics/test Error:KeeperErrorCode = NoNodefor/config/topics/test (org.apache.zookeeper.server.PrepRequestProcessor)
Created topic"test".
列出主题:
[kafka@tjtestrac1 config]$ kafka-topics.sh --list --zookeeper localhost:2181
[2018-11-2217:29:31,251] INFO Accepted socket connection from /0:0:0:0:0:0:0:1:64976(org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-11-2217:29:31,253] INFO Client attempting to establish new session at /0:0:0:0:0:0:0:1:64976(org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-2217:29:31,255] INFO Established session0x10052ce304e0002 with negotiated timeout30000forclient /0:0:0:0:0:0:0:1:64976(org.apache.zookeeper.server.ZooKeeperServer)
test
[2018-11-2217:29:31,302] INFO Processed session terminationforsessionid:0x10052ce304e0002 (org.apache.zookeeper.server.PrepRequestProcessor)
[2018-11-2217:29:31,303] INFO Closed socket connectionforclient /0:0:0:0:0:0:0:1:64976which had sessionid0x10052ce304e0002 (org.apache.zookeeper.server.NIOServerCnxn)
6. 启动生产者来发送数据
[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:9092--topic test
>jason
>comm on
>test message
7. 启动消费者来接收数据
[kafka@tjtestrac1 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:9092--topic test --from-beginning
[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:9092--topic test
>jason
>comm on
>test message
>next message
