Big Data

clickhouse集群搭建

2019-05-21  本文已影响0人  盗梦者_56f2

安装jdk

安装zk

实现高可用

安装ck

  1. 添加源
    /etc/apt/sources.list中添加仓库:
    deb http://repo.yandex.ru/clickhouse/deb/stable/ main/
  2. 下载ck:
sudo apt-get install dirmngr
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4
sudo apt-get update
sudo apt-get install clickhouse-client clickhouse-server
  1. 配置
sudo vim /etc/clickhouse-server/config.xml
#生效如下两个配置,其他不变
<interserver_http_host>ck_server_01</interserver_http_host> <!--填命令hostname -f返回的值-->
<listen_host>0.0.0.0</listen_host> <!--监听IP-->
<timezone>Asia/Shanghai</timezone> <!--设置时区-->
<include_from>/etc/metrica.xml</include_from> <!--添加该参数-->
#保存退出
#创建文件metrika.xml
sudo vim /etc/metrika.xml
#添加如下配置
<yandex>
<!--ck集群节点-->
<clickhouse_remote_servers>
    <test_ck_cluster>
        <!--分片1-->
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>172.16.120.100</host>
                <port>9000</port>
                <user>default</user>
                <password>UHXQQmhb</password>
               <compression>true</compression>
            </replica>
            <replica>
                <host>172.16.120.101</host>
                <port>9000</port>
                <user>default</user>
                <password>UHXQQmhb</password>
                <compression>true</compression>
            </replica>
        </shard>
        <!--分片2-->
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>172.16.120.102</host>
                <port>9000</port>
                <user>default</user>
                <password>UHXQQmhb</password>
                <compression>true</compression>
            </replica>
            <replica>
                <host>172.16.120.103</host>
                <port>9000</port>
                <user>default</user>
                <password>UHXQQmhb</password>
                <compression>true</compression>
            </replica>
        </shard>
        <!--分片3-->
    </test_ck_cluster>
</clickhouse_remote_servers>

<!--zookeeper相关配置-->
<zookeeper-servers>
    <node index="1">
        <host>172.16.120.101</host>
        <port>2181</port>
    </node>
    <node index="2">
        <host>172.16.120.102</host>
        <port>2181</port>
    </node>
    <node index="3">
        <host>172.16.120.103</host>
        <port>2181</port>
    </node>
</zookeeper-servers>

<macros>
    <replica>hostname</replica> <!--当前节点主机名-->
</macros>

<networks>
    <ip>::/0</ip>
</networks>

<!--压缩相关配置-->
<clickhouse_compression>
    <case>
        <min_part_size>10000000000</min_part_size>
        <min_part_size_ratio>0.01</min_part_size_ratio>
        <method>lz4</method> <!--压缩算法lz4压缩比zstd快, 更占磁盘-->
    </case>
</clickhouse_compression>
</yandex>
#保存退出
sudo vim /etc/clickhouse-server/users.xml
#可以更改默认用户default的密码
#其他不必修改
#保存退出

若想更改默认配置文件路径以及数据存放目录路径,可以修改config.xml文件相关配置和/etc/init.d/clickhouse-server文件里面的相关配置。

  1. 启动
sudo service clickhouse-server start
#clickhouse-client --help
#sudo service clickhouse-server status
#sudo service clickhouse-server restart
  1. 默认主要配置文件(一般再启动前更改配置中日志,数据存放目录)
启动脚本文件:/etc/init.d/clickhouse-server
全局信息配置文件: /etc/clickhouse-server/config.xml
集群信息配置文件: /etc/metrika.xml
用户信息配置文件:/etc/clickhouse-server/users.xml
客户端配置: /etc/clickhouse-client/
日志:/var/log/clickhouse-server/
data / metadata:/var/lib/clickhouse/

测试

clickhouse-client --help
#连接数据库
clickhouse-client -u default --password password
select 1
show databases
show tables
#批量导入数据方式
#1. 命令行客户端
cat file.csv | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; 
#2. http客户端
GET 'http://localhost:8123/?query=SELECT 1'
echo 'INSERT INTO t VALUES (1),(2),(3)' | POST 'http://localhost:8123/'
#3. 第三方客户端库
#
上一篇下一篇

猜你喜欢

热点阅读