hadoop分布式集群搭建
1.下载hadoop安装包：
wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz
tar -xvf hadoop-2.7.3.tar.gz //解压缩
2.设置环境变量
vim /home/hadoop/.bash_profile
export JAVA_HOME=/usr/local/dev/software/develop/jdk1.8.0_112
exp_-ort JRE_HOME=$JAVA_HOME/jre
export HADOOP_HOME=/home/hadoop/hadoop-2.7.3
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH
然后执行：source /home/hadoop/.bash_profile 是环境变量生效
3.修改默认配置文件：、
3.1修改文:hadoop-env.sh
vim hadoop-env.sh
添加JAVA_HOME 变量：
# The only required environment variable is JAVA_HOME. All others are
# optional. When running a distributed configuration it is best to
# set JAVA_HOME in this file, so that it is correctly defined on
# remote nodes.
# The java implementation to use.
export JAVA_HOME=/usr/local/dev/software/develop/jdk1.8.0_112
3.2 修改hdfs-site.xml
dfs.namenode.name.dir
/home/hadoop/hadoop_dir/dfs/name
dfs.datanode.name.dir
file:// /home/hadoop/hadoop_dir/dfs/data
dfs.replication
1
3.3 修改core-site.xml 文件
fs.defaultFS
hdfs://localhost:9000
hadoop.tmp.dir
/home/hadoop/app/hadoop-2.7.3/tmp
3.4 修改文件mapred-site.xml
mapreduce.framework.name
yarn
3.4修改文件yarn-site.xml
yarn.resourcemanager.hostname
hadoop01
yarn.nodemanager.aux-services
mapreduce_shuffle
4. 构建免密登陆
ssh-keygen -t rsa //生成rsa公钥 和私钥
cp ~/.ssh/id_rsa.pub authorized_keys
ssh root@localhost//可以实现免密登陆
5.格式化和启动
hdfs namenode -format //格式化hdfs文件系统
./sbin/start-all.sh //启动hadoop集群
http://localhost:50070 （HDFS管理界面）
http://localhost:8088 （MR管理界面）
6.出现的问题：
error:JAVA_HOME is not set or could not find .
在：$HADOOP_HOME/etc/hadoop/hadoop-env.sh 添加JAVA_HOME
7.hadoop常用的管理界面
http://localhost:50070:hdfs 管理界面
http://localhost:50075:dataNode管理界面
http://localhost:50090:secondaryNameNode 管理界面
http://localhost:8042:managerResoucer 管理界面
http://localhost:34036/ ：dataode管理界面
http://www.360doc.com/content/14/0615/15/9075092_386806245.shtml
http://blog.csdn.net/u010638969/article/details/53391695
https://www.iteblog.com/archives/851.html
