The Google File System

Google spots different requirements for the file system in their daily applications. They design the Google file system to meet the requirements. Some radically different potins in the design space include

First, component failures are the norm rather than the exception. ... constant monitoring, error detection, fault tolerance, and automatic recovery must be integral to the system.

Second, files are hugh by traditional standards. ... design assumptions and parameters such as I/O operation and block size have to revisited.

Third, most files are mutated by appending new data rather than overwriting existing data. ... appending becomes the focus of performance optimization and atomicity guarantees, while caching data block in the client loses its appeal.

Fourth, co-designing the applications and the file system API benefits the overall system by increasing our flexibility.

The architecture of GFS is shown in Fig.1.

gfs_architecture.png