2016 hadoop伪分布式安装
1、VirtualBox虚拟机安装网络配置等可参考这篇文章：http://www.cnblogs.com/vovlie/archive/2012/09/01/2666807.html
配置SSH互信
分别启动h1，h2，h3虚拟机，互相ping确认是否能通。
然后分别使用grid用户，正常登录每台机器执行生成密钥命令。
ssh-keygen –t rsa
回到h1主机上，使用跨主机的管道和重定向将h2,h3主机上的公钥id_rsa.pub添加到h1上的authorized_keys文件中
cat id_rsa.pub >authorized_keys 将h1的id_rsa.pub写入authorized_keys
ssh h2 cat ~/.ssh/id_rsa.pub >>authorized_keys 将h2的id_rsa.pub追加写入authorized_keys
……
然后分别scp复制到h2,h3机器中。
scp authorized_keys grid@h2:~/.ssh/
1)chmod 600 ~/.ssh/authorized_keys 设置权限
2)修改SSH配置文件
su - -->登陆root用户修改配置文件
vi /etc/ssh/sshd_config
3)测试连接
service sshd restart -->重启ssh服务,
exit -->退出root用户，回到普通用户
ssh h2 -->连接用户测试
下载hadoop
wgethttp://mirror.nus.edu.sg/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz
解压hadoop
[grid@h1 ~]$ tar zxvf hadoop-2.7.1.tar.gz
七个配置文件
yarn-env.sh
配置jdk
export JAVA_HOME=/usr/java/jdk1.7.0_80
hadoop-env.sh
配置jdk
export JAVA_HOME=/usr/java/jdk1.7.0_80
slaves
cat slaves
添加h1和h2
master默认不用配置
yarn-site.xml
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.nodemanager.aux-services.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.resourcemanager.address
h1:8032
yarn.resourcemanager.scheduler.address
h1:8030
yarn.resourcemanager.resource-tracker.address
h1:8031
yarn.resourcemanager.admin.address
h1:8033
yarn.resourcemanager.webapp.address
h1:8088
mapred-site.xml
mapreduce.framework.name
yarn
mapreduce.jobhistory.address
h1:10020
mapreduce.jobhistory.webapp.address
h1:19888
hdfs-site.xml
注意：name和data文件夹需事先建好
dfs.namenode,secondary.http-address
h1:9001
dfs.namenode.name.dir
file:/home/grid/hadoop-2.7.1/name
dfs.datanode.data.dir
file:/home/grid/hadoop-2.7.1/data
dfs.replication
1
dfs.webhdfs.enabled
true
core-site.xml
注意：tmp文件夹需事先建好
fs.default.name
hdfs://h1:9000
io.file.buffer.size
131072
hadoop.tmp.dir
file:home/grid/hadoop-2.7.1/tmp
Abase for other temporary directories
