Hadoop 搭建Sqoop环境
2021-07-04 本文已影响0人
Yohann丶blog
1411625392886_.pic_hd.jpg
环境
-
CentOS 6.8 64位 1核 2GB
-
JDK 1.7.0_75 64 位
-
Hadoop 1.1.2
-
Sqoop 1.4.5
安装Sqoop
- 下载 sqoop 安装包
$ wget http://archive.apache.org/dist/sqoop/1.4.5/sqoop-1.4.5.bin__hadoop-1.0.0.tar.gz
- 解压 sqoop 安装包
$ tar -xzf sqoop-1.4.5.bin__hadoop-1.0.0.tar.gz
$ mv sqoop-1.4.5.bin__hadoop-1.0.0 /app/sqoop-1.4.5
解压后，将 sqoop-1.4.5.bin__hadoop-1.0.0 移动到 /app 目录下。
- 修改 /etc/profile 文件
$ sudo vi /etc/profile
- 添加 sqoop 环境变量
export SQOOP_HOME=/app/sqoop-1.4.5
export PATH=$PATH:$SQOOP_HOME/bin
- 保存生效
$ source /etc/profile
- 验证安装成功
$ sqoop version
Warning: $HADOOP_HOME is deprecated.
21/07/03 21:01:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
Sqoop 1.4.5
git commit id 5b34accaca7de251fc91161733f906af2eddbe83
Compiled by abe on Fri Aug 1 11:15:29 PDT 2014
配置Sqoop
- 将 mysql-connector-java-5.1.22-bin.jar 复制到 /app/sqoop-1.4.5/lib 目录下。
$ cp /home/yohann/mysql-connector-java-5.1.22-bin.jar /app/sqoop-1.4.5/lib/
- 进入 /app/sqoop-1.4.5/bin 目录
$ cd /app/sqoop-1.4.5/bin
- 修改 configure-sqoop 文件
#if [ ! -d "${HBASE_HOME}" ]; then
# echo "Warning: $HBASE_HOME does not exist! HBase imports will fail."
# echo 'Please set $HBASE_HOME to the root of your HBase installation.'
#fi
## Moved to be a runtime check in sqoop.
#if [ ! -d "${HCAT_HOME}" ]; then
# echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."
# echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'
#fi
#if [ ! -d "${ACCUMULO_HOME}" ]; then
# echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."
# echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'
#fi
#if [ ! -d "${ZOOKEEPER_HOME}" ]; then
# echo "Warning: $ZOOKEEPER_HOME does not exist! Accumulo imports will fail."
# echo 'Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.'
#fi
注释掉 HBase 和 Zookeeper 等检查。
- 进入 /app/sqoop-1.4.5/conf 目录
$ cd /app/sqoop-1.4.5/conf
- 复制 sqoop-env-template.sh 命名为 sqoop-env.sh
$ cp sqoop-env-template.sh sqoop-env.sh
- 修改 sqoop-env.sh 文件
export HADOOP_COMMON_HOME=/app/hadoop-1.1.2
export HADOOP_MAPRED_HOME=/app/hadoop-1.1.2
设置 hadoop 运行程序所在路径。
- 保存生效
$ source /etc/profile
- 验证配置完成
sqoop help
