ubuntu 18.04 安装sqoop
2019-06-18 本文已影响12人
python测试开发
前提
HADOOP 和MYSQL安装ok
下载
$ sudo cp sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz /opt/
$ sudo tar xzvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
$ sudo mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop
$ vi .bashrc
export SQOOP_HOME=/opt/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
export SQOOP_CONF_DIR=$SQOOP_HOME/conf
export SQOOP_CLASS_PATH=$SQOOP_CONF_DIR
$ source ~/.bashrc
$ cd $SQOOP_HOME/conf
$ mv sqoop-env-template.sh sqoop-env.sh
$ vi /opt/sqoop/conf/sqoop-env.sh
export HADOOP_COMMON_HOME=/opt/hadoop
export HADOOP_MAPRED_HOME=/opt/hadoop
$ tar -zxf mysql-connector-java-5.1.30.tar.gz
$ su
password:
# cd mysql-connector-java-8.0.16.jar
# mv mysql-connector-java-8.0.16.jar-bin.jar /opt/sqoop/lib
$ cd $SQOOP_HOME/bin
$ sqoop-version
...
2019-06-18 18:27:50,668 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017
坑
- Sqoop 连接Mysql org.apache.commons.lang.StringUtils
下载 http://mirrors.tuna.tsinghua.edu.cn/apache//commons/lang/binaries/commons-lang-2.6-bin.zip
上传到sqoop的 lib目录。
使用 --bindir ./ 可以搞定。比如
sqoop import --connect jdbc:mysql://localhost:3306/energydata --username hive --password 12345678_ --table average_price_by_state -m 1 --bindir .
