漫画HDFS+翻译+简单拓展
1 写文件入HDFS
<div align=center>
1.png
从上面的图，我们获得以下信息：
- 数据存HDFS之前是要被切分的
- 切分成多大是我们定的，然后客户端去执行
- 假设我们定数据块大小为128MB，那么200MB的就会被切分成128MB和72MB。
2.png
从上面的图，我们获得以下信息：
- 客户端可以把原文件切分成数据块
- 客户端存数据之前要跟NameNode报备一下，由NameNode去调度数据节点来做具体的事情。
应该是计算一下，哪些节点比较空闲吧。另外NameNode提到了数据节点离客户端的远近。
说明客户端并不是固定的。 - 客户端传数据就给第一个数据节点传，然后下面就是流水线式的操作。
3.png
看完上面的图，我基本搞懂了HDFS写数据的流程。
但是也有一个最大的疑问：
客户端是个啥！！！
从目前看，它有以下功能：
- 拆分原文件为数据块
- 上传数据（也就是说可以执行hdfs命令）
从stackoverflow上找到了一段描述很恰当的话：
Client in Hadoop refers to the Interface used to communicate with the Hadoop Filesystem. There are different type of Clients available with Hadoop to perform different tasks.
The basic filesystem client hdfs dfs is used to connect to a Hadoop Filesystem and perform basic file related tasks. It uses the ClientProtocol to communicate with a NameNode daemon, and connects directly to DataNodes to read/write block data. To perform administrative tasks on HDFS, there is hdfs dfsadmin. For HA related tasks, hdfs haadmin. There are similar clients available for performing YARN related tasks.
These Clients can be invoked using their respective CLI commands from a node where Hadoop is installed and has the necessary configurations and libraries required to connect to a Hadoop Filesystem. Such nodes are often referred as Hadoop Clients.
For example, if I just write an hdfs command on the Terminal, is it still a "client" ?
Technically, Yes. If you are able to access the FS using the hdfs command, then the node has the configurations and libraries required to be a Hadoop Client.
PS: APIs are also available to create these Clients programmatically.
总结一下，客户端的含义两句话：
- 可以使用hdfs dfs命令的机器，一般是datanode。
- 利用API和HDFS进行交互的也可以创建客户端。（API创建客户端的时候好像也得先创建一个连接应该也是和某台datanode创建连接）
