Storage Provisioning and Networking - Deploying and Managing a Cloud Infrastructure - page 269

FIGURE 9.5 HDFS architecture

NameNode
Metadata (names, replicas, etc.):
/home/user/data, 3, . .

Metadata
operations

Block operations

Client X

This DataNode has 3
data blocks

Read operation

Replication

Network

2 DataNodes in rack 1 3 DataNodes in rack 2

Write operation

Client Y

Rack awareness outlines the process by which the NameNode is used to determine the
ID of the rack to which each DataNode belongs. One policy is to never place two replicas
on the same rack. This prevents data loss in the event of a rack failure and spreads the load
all over the racks. However, write cost is high because replicas have to be written across
multiple racks. Here, the replication factor comes directly into play and would be directly
proportional to the cluster performance. Rack awareness is described further here:
http://hadoop.apache.org/docs/r0.18.3/cluster_setup
.html#Hadoop+Rack+Awareness
A typical (default) placement policy used by HDFS is to put one replica on one node in a
local rack, another on a different node in the local rack, and the third replica on a different
node in a different rack. Figure 9.6 shows such a placement policy.
Such a policy limits the inter-rack write traffic, thus decreasing unnecessary network
traffic. A rack is composed of many nodes. It is pertinent to note that a failure of the whole
rack is far less likely than a node failure. One-third of the replica is on one node, two-thirds
on one rack, and the remaining one-third distributed evenly across the remaining racks.
The disadvantage of such a policy is that during read operations, the load is compensated
by two racks instead of three. However, the write performance is significantly increased
because the network connection within a rack is less congested (and thus faster) than the
inter-rack network connection.
The HDFS tries to serve a read request from the replica that's closest to the user/reader.
A replica on the same rack is preferred, rather than a replica on another rack. If the cluster