Database Reference
In-Depth Information
Figure 3-3. Network distance in Hadoop
Finally, it is important to realize that Hadoop cannot magically discover your network topology for you;
it needs some help (we'll cover how to configure topology in
Network Topology
)
. By default, though, it
assumes that the network is flat — a single-level hierarchy — or in other words, that all nodes are on a
single rack in a single data center. For small clusters, this may actually be the case, and no further config-
uration is required.
Anatomy of a File Write
Next we'll look at how files are written to HDFS. Although quite detailed, it is instructive
to understand the data flow because it clarifies HDFS's coherency model.
We're going to consider the case of creating a new file, writing data to it, then closing the
file. This is illustrated in
Figure 3-4
.
