9.hadoop安装(伪分布式安装)
三副本机制详解(三个以上的随机存储)
第一副本:如果上传节点是DN,则上传该节点;如果上传节点是NN,则随机选择DN
第二副本:放置在不同机架的DN上
第三副本:放置在与第二副本相同机架的不同DN上
副本机制的作用
极大程度上避免了宕机所造成的数据丢失(除非命不好)
可以在数据读取时进行数据校验
原文链接:https://blog.csdn.net/weixin_39216383/article/details/78841983
hadoop伪分布式安装
https://hadoop.apache.org/docs/r2.6.5/hadoop-project-dist/hadoop-common/SingleCluster.html#Fully-Distributed_Operation
![](https://img.haomeiwen.com/i13219358/d9ccc6ada1ed0be4.png)
安装jdk在第4里有,这里介绍安装ssh
没有.ssh文件就ssh localhost一次
![](https://img.haomeiwen.com/i13219358/dac45665f7dd1a12.png)
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
![](https://img.haomeiwen.com/i13219358/9821b0a46ed877cf.png)
![](https://img.haomeiwen.com/i13219358/b82a6fbf706890bd.png)
![](https://img.haomeiwen.com/i13219358/d8b581c1ce43d3cd.png)
![](https://img.haomeiwen.com/i13219358/6e151c05cb477e9d.png)
![](https://img.haomeiwen.com/i13219358/2b5ef661fdff7dd8.png)
![](https://img.haomeiwen.com/i13219358/fb8c33a60dc8fc67.png)
![](https://img.haomeiwen.com/i13219358/a94c54509092b1d4.png)
tar zxvf test.tar.gz -C test
注释:上面的命令将 test.tar.gz 这个压缩包解压到当前目录下的 test 目录下。
-C 选项的作用是:指定需要解压到的目录。
tar
-c: 建立压缩档案
-x:解压
-t:查看内容
-r:向压缩归档文件末尾追加文件
-u:更新原压缩包中的文件
这五个是独立的命令,压缩解压都要用到其中一个,可以和别的命令连用但只能用其中一个。下面的参数是根据需要在压缩或解压档案时可选的。
-z:有gzip属性的
-j:有bz2属性的
-Z:有compress属性的
-v:显示所有过程
-O:将文件解开到标准输出
下面的参数-f是必须的
-f: 使用档案名字,切记,这个参数是最后一个参数,后面只能接档案名。
# tar -cf all.tar *.jpg
这条命令是将所有.jpg的文件打成一个名为all.tar的包。-c是表示产生新的包,-f指定包的文件名。
# tar -rf all.tar *.gif
这条命令是将所有.gif的文件增加到all.tar的包里面去。-r是表示增加文件的意思。
# tar -uf all.tar logo.gif
这条命令是更新原来tar包all.tar中logo.gif文件,-u是表示更新文件的意思。
# tar -tf all.tar
这条命令是列出all.tar包中所有文件,-t是列出文件的意思
# tar -xf all.tar
这条命令是解出all.tar包中所有文件,-t是解开的意思
压缩
tar -cvf jpg.tar *.jpg //将目录里所有jpg文件打包成tar.jpg
tar -czf jpg.tar.gz *.jpg //将目录里所有jpg文件打包成jpg.tar后,并且将其用gzip压缩,生成一个gzip压缩过的包,命名为jpg.tar.gz
tar -cjf jpg.tar.bz2 *.jpg //将目录里所有jpg文件打包成jpg.tar后,并且将其用bzip2压缩,生成一个bzip2压缩过的包,命名为jpg.tar.bz2
tar -cZf jpg.tar.Z *.jpg //将目录里所有jpg文件打包成jpg.tar后,并且将其用compress压缩,生成一个umcompress压缩过的包,命名为jpg.tar.Z
rar a jpg.rar *.jpg //rar格式的压缩,需要先下载rar for linux
zip jpg.zip *.jpg //zip格式的压缩,需要先下载zip for linux
解压
tar -xvf file.tar //解压 tar包
tar -xzvf file.tar.gz //解压tar.gz
tar -xjvf file.tar.bz2 //解压 tar.bz2
tar -xZvf file.tar.Z //解压tar.Z
unrar e file.rar //解压rar
unzip file.zip //解压zip
![](https://img.haomeiwen.com/i13219358/1844989a37f0092e.png)
![](https://img.haomeiwen.com/i13219358/67ea57e171e1728b.png)
![](https://img.haomeiwen.com/i13219358/ffe7d460ba7bf868.png)
注意这里是vi + /etc/proflie 我一直打成了vi *很尴尬
一直在调试
![](https://img.haomeiwen.com/i13219358/f79be472939a48a3.png)
![](https://img.haomeiwen.com/i13219358/19e06ead1f2f23d4.png)
linux 下的 source,sh,./三者区别
https://blog.csdn.net/s740556472/article/details/78176087
![](https://img.haomeiwen.com/i13219358/fee693e16efe7d6e.png)
![](https://img.haomeiwen.com/i13219358/85c2dd1fff966c1a.png)
![](https://img.haomeiwen.com/i13219358/d5291bd70e6eba73.png)
![](https://img.haomeiwen.com/i13219358/61ca50567fb3ae74.png)
![](https://img.haomeiwen.com/i13219358/70163c37df922d07.png)
![](https://img.haomeiwen.com/i13219358/7d3fc46f8d3c91e5.png)
![](https://img.haomeiwen.com/i13219358/209fc1430ecb8ffd.png)
![](https://img.haomeiwen.com/i13219358/31eb4e932e6965ce.png)
![](https://img.haomeiwen.com/i13219358/32d193c19e89a7f6.png)
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
![](https://img.haomeiwen.com/i13219358/d7040f7a18e276ca.png)
![](https://img.haomeiwen.com/i13219358/0569fb5c5f3a138a.png)
![](https://img.haomeiwen.com/i13219358/2a4922c0ad6cfdda.png)
![](https://img.haomeiwen.com/i13219358/c859e611b65b0f8f.png)
![](https://img.haomeiwen.com/i13219358/e71c353a150cab7c.png)
![](https://img.haomeiwen.com/i13219358/ee25080362f94362.png)
![](https://img.haomeiwen.com/i13219358/0fc6262384a82edd.png)
![](https://img.haomeiwen.com/i13219358/e443e96c366715b4.png)
![](https://img.haomeiwen.com/i13219358/e8fda5217efd026c.png)
![](https://img.haomeiwen.com/i13219358/8e3c3343040188ef.png)
![](https://img.haomeiwen.com/i13219358/7f195d67754bc0bb.png)
https://hadoop.apache.org/docs/r2.6.5/hadoop-project-dist/hadoop-common/core-default.xml
![](https://img.haomeiwen.com/i13219358/ec6f403c428b061a.png)
![](https://img.haomeiwen.com/i13219358/791daa0a3a9f43ab.png)
![](https://img.haomeiwen.com/i13219358/19590b53d7504f51.png)
![](https://img.haomeiwen.com/i13219358/730f7e9990aae7c9.png)
![](https://img.haomeiwen.com/i13219358/9dff15733ebc92e1.png)
![](https://img.haomeiwen.com/i13219358/49ed576fbfd6bd3c.png)
![](https://img.haomeiwen.com/i13219358/41a67b0ae5c8e1ea.png)
![](https://img.haomeiwen.com/i13219358/a450a2e81f7d8d86.png)
![](https://img.haomeiwen.com/i13219358/c03f8855e9d1418f.png)
![](https://img.haomeiwen.com/i13219358/a98e4562195e701f.png)
![](https://img.haomeiwen.com/i13219358/e8c7133183464239.png)
![](https://img.haomeiwen.com/i13219358/dc98c4b29bffaa40.png)
![](https://img.haomeiwen.com/i13219358/b23a1b376b0c3e12.png)
https://www.cnblogs.com/ftl1012/p/ss.html
ss是Socket Statistics的缩写。顾名思义,ss命令可以用来获取socket统计信息,它可以显示和netstat类似的内容。ss的优势在于它能够显示更多更详细的有关TCP和连接状态的信息,而且比netstat更快速更高效。
![](https://img.haomeiwen.com/i13219358/63aa3c00de32ec1b.png)
![](https://img.haomeiwen.com/i13219358/00d351b742828b19.png)
![](https://img.haomeiwen.com/i13219358/18bce19b0f0a1f4d.png)
![](https://img.haomeiwen.com/i13219358/90891e78076edca1.png)
![](https://img.haomeiwen.com/i13219358/acd41f2d7d31e667.png)
![](https://img.haomeiwen.com/i13219358/dbbce015978ed01a.png)
![](https://img.haomeiwen.com/i13219358/ece9dbabfd861622.png)