Hadoop 2.x 初级集群安装
2018-02-04 本文已影响0人
壹点零
一、安装文件上传解压
二、环境变量配置
vim /etc/profile
## jdk 环境变量
export JAVA_HOME=/usr/local/java/jdk1.8
export JAVA_BIN=$JAVA_HOME/bin
export JAVA_LIB=$JAVA_HOME/lib
export CLASSPATH=.:$JAVA_LIB/tools.jar:$JAVA_LIB/dt.jar
export PATH=$JAVA_BIN:$PATH
## hadoop 环境变量
export PATH=/usr/local/hadoop-2.7.5/bin:/usr/local/hadoop-2.7.5/sbin:$PATH
三、hadoop配置文件
修改/usr/local/hadoop-2.7.5/etc/hadoop/core-site.xml添加配置
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
</configuration>
修改/usr/local/hadoop-2.7.5/etc/hadoop/slaves添加slave节点
192.168.1.11
192.168.1.12
192.168.1.13
四、启动Hadoop
1、先格式化 namenode
hdfs namenode -format
2、在master上启动namenode
hadoop-daemon.sh start namenode
3、在slave上启动datanode
hadoop-daemon.sh start datanode
4、在master上关闭namenode
hadoop-daemon.sh stop namenode
5、在slave上关闭datanode
hadoop-daemon.sh stop datanode
五、验证Hadoop
hdfs dfsadmin -report
[root@localhost hadoop]# hdfs dfsadmin -report
Configured Capacity: 55479107584 (51.67 GB)
Present Capacity: 46706745344 (43.50 GB)
DFS Remaining: 46706708480 (43.50 GB)
DFS Used: 36864 (36 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (3):
Name: 192.168.107.136:50010 (slave136)
Hostname: slave136
Decommission Status : Normal
Configured Capacity: 18238930944 (16.99 GB)
DFS Used: 12288 (12 KB)
Non DFS Used: 2737418240 (2.55 GB)
DFS Remaining: 15501500416 (14.44 GB)
DFS Used%: 0.00%
DFS Remaining%: 84.99%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Feb 01 00:25:16 PST 2018
Name: 192.168.107.133:50010 (slave133)
Hostname: slave133
Decommission Status : Normal
Configured Capacity: 18238930944 (16.99 GB)
DFS Used: 12288 (12 KB)
Non DFS Used: 2737139712 (2.55 GB)
DFS Remaining: 15501778944 (14.44 GB)
DFS Used%: 0.00%
DFS Remaining%: 84.99%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Feb 01 00:25:16 PST 2018
Name: 192.168.107.132:50010 (slave132)
Hostname: localhost
Decommission Status : Normal
Configured Capacity: 19001245696 (17.70 GB)
DFS Used: 12288 (12 KB)
Non DFS Used: 3297804288 (3.07 GB)
DFS Remaining: 15703429120 (14.62 GB)
DFS Used%: 0.00%
DFS Remaining%: 82.64%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Feb 01 00:25:16 PST 2018
也可通过master提供的web程序进行访问:
http://192.168.107.131:50070