Database Reference
In-Depth Information
the pipeline. The
DataStreamer
streams the packets to the first datanode in the
pipeline, which stores each packet and forwards it to the second datanode in the pipeline.
Similarly, the second datanode stores the packet and forwards it to the third (and last)
datanode in the pipeline (step 4).
The
DFSOutputStream
also maintains an internal queue of packets that are waiting to
be acknowledged by datanodes, called the
ack queue
. A packet is removed from the ack
queue only when it has been acknowledged by all the datanodes in the pipeline (step 5).
If any datanode fails while data is being written to it, then the following actions are taken,
which are transparent to the client writing the data. First, the pipeline is closed, and any
packets in the ack queue are added to the front of the data queue so that datanodes that are
downstream from the failed node will not miss any packets. The current block on the good
datanodes is given a new identity, which is communicated to the namenode, so that the
partial block on the failed datanode will be deleted if the failed datanode recovers later on.
The failed datanode is removed from the pipeline, and a new pipeline is constructed from
the two good datanodes. The remainder of the block's data is written to the good datan-
odes in the pipeline. The namenode notices that the block is under-replicated, and it ar-
ranges for a further replica to be created on another node. Subsequent blocks are then
treated as normal.
It's possible, but unlikely, for multiple datanodes to fail while a block is being written. As
long as
dfs.namenode.replication.min
replicas (which defaults to 1) are writ-
ten, the write will succeed, and the block will be asynchronously replicated across the
cluster until its target replication factor is reached (
dfs.replication
, which defaults
to 3).
When the client has finished writing data, it calls
close()
on the stream (step 6). This
action flushes all the remaining packets to the datanode pipeline and waits for acknow-
ledgments before contacting the namenode to signal that the file is complete (step 7). The
namenode already knows which blocks the file is made up of (because
DataStreamer
asks for block allocations), so it only has to wait for blocks to be minimally replicated be-
fore returning successfully.
