我爱编程

Hive环境搭建

2017-04-10  本文已影响252人  忘净空

Hive的安装需要提前安装Hadoop和MySQL,Hadoop的安装请参考Hadoop环境搭建,下面是MySQL的安装。

MySQL安装

  1. MySQL安装请点击

  2. 启动MySQL
    service mysqld restart

  3. 创建hive用户
    create user 'hive' identified by 'hive';

  4. 创建hive数据库
    create database hive;

  5. 将hive数据库权限设置给hive用户
    grant all privileges on . to hive@'localhost' identified by 'hive';

  6. 刷新
    flush privileges;

Hive的安装

软件安装

  1. 软件下载官网地址

  2. 解压安装
    tar -zxvf apache-hive-2.1.1-bin.tar.gz ../software

环境配置

启动Hive

  1. 初始化数据库
schematool -initSchema -dbType mysql
  1. 启动hiveserver2
[root@hadoop1 bin]# hiveserver2

Hive整合Hadoop

  1. 创建本地hive_test文件(字段使用tab分割)

    zhangshan 20
    lisi 21
    wangwu 19

  2. hdfs创建hive_test文件夹

    hdfs dfs -mkdir /hive_test

  3. 将本地文件导入到hdfs中

    hdfs dfs -copyFromLocal hive_test /hive_test

  4. 创建Hive表

    create table hive_test (
    name string,
    age int
    )row format delimited fields terminated by '\t' stored as textfile;

  5. hive关联Hadoop

    load data inpath /hive_test into table hive_test;

  6. 查询测试

    zhangshan 20
    lisi 21
    wangwu 19

Hive和HBase互通

配置

修改hive-site.xml

   <property>
      <name>hbase.zookeeper.quorum</name>
      <value>master</value>
      <description>zookeeper</description>
    </property>

    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
        <description>port</description>
    </property>

数据交互

  1. HBase新建表
create 'student','baseinfo','otherinfo'
  1. 插入数据
put 'student','1','baseinfo:name','zhangsan'
put 'student','1','baseinfo:age','21'
put 'student','1','otherinfo:hobby','sleep'
put 'student','2','baseinfo:name','lisi'
  1. 查看数据
scan 'student'
  1. Hive中新建基于HBase的表
CREATE EXTERNAL TABLE student (
rowkey string,
name string,
base map<STRING,STRING>,
other map<STRING,STRING>
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,baseinfo:name,baseinfo:,otherinfo:")
TBLPROPERTIES ("hbase.table.name" = "student");
  1. 查询HBase表
select * from student;

配置Hive高可用

基于zookeeper

配置hive-site.xml,新增如下配置

<property>
    <name>hive.server2.support.dynamic.service.discovery</name>
    <value>true</value>
</property>
<property>
    <name>hive.server2.zookeeper.namespace</name>
    <value>hiveserver2_zk</value>
</property>
<property>
    <name>hive.zookeeper.quorum</name>
    <value>master,slave01,slave02</value>
</property>

基于HAProxy

大家请参考链接基于HAProxy配置Hive HA

验证配置

上一篇 下一篇

猜你喜欢

热点阅读