Database Reference
In-Depth Information
YARN
License
Apache License, Version 2.0
Activity
Medium
Purpose
Processing
Official Page
Hadoop Integration Fully Integrated
When many folks think about Hadoop, they are really thinking about two related technolo-
gies. These two technologies are the Hadoop Distributed File System (HDFS), which houses
your data, and MapReduce, which allows you to actually do things with your data. While
MapReduce is great for certain categories of tasks, it falls short with others. This led to frac-
turing in the ecosystem and a variety of tools that live outside of your Hadoop cluster but at-
tempt to communicate with HDFS.
In May 2012, version 2.0 of Hadoop was released, and with it came an exciting change to the
way you can interact with your data. This change came with the introduction of YARN,
which stands for Yet Another Resource Negotiator.
YARN exists in the space between your data and where MapReduce now lives, and it allows
for many other tools that used to live outside your Hadoop system, such as Spark and Giraph,
to now exist natively within a Hadoop cluster. It's important to understand that Yarn does not
replace MapReduce; in fact, Yarn doesn't do anything at all on its own. What Yarn does do is
provide a convenient, uniform way for a variety of tools such as MapReduce, HBase, or any
custom utilities you might build to run on your Hadoop cluster.
