Database Reference
In-Depth Information
Throughout the streaming process, as new blocks are allocated, the
NameNode is updated. After the client has completed the write operation, it
closesouttheprocesswiththeNameNode.TheNameNodeonlyverifiesthat
theminimumsetofreplicashavebeencreatedbeforereturningsuccessfully.
NOTE
Throughout this process, any write failures are handled silently and
transparently to the client. If a node fails during the write or during the
replicate process, it will be reattempted on an alternative node
automatically.
Replication
Replication provides for fault tolerance within HDFS and occurs as part
of each write or whenever the NameNode detects a DataNode failure. Out
of the box, the default HDFS replication factor is three (controlled by the
dfs.replication
property), meaning that each block is written to three
different nodes.
Figure 4.3
elaborates on the previous write operation
example to illustrate how replication occurs.
