Database Reference
In-Depth Information
Yet Another Resource Negotiator (YARN)
Apache Hadoop continues to undergo further development and frequent updates.
An important change was to separate the MapReduce functionality from the
functionality that manages the running of the jobs and the associated
responsibilities in a distributed environment. This rewrite is sometimes called
MapReduce 2.0, or Yet Another Resource Negotiator (YARN). YARN separates the
resource management of the cluster from the scheduling and monitoring of jobs
running on the cluster. The YARN implementation makes it possible for paradigms
other than MapReduce to be utilized in Hadoop environments. For example, a
Bulk Synchronous Parallel (BSP) [21] model may be more appropriate for graph
processing than MapReduce [22] is. Apache Hama, which implements the BSP
model, is one of several applications being modified to utilize the power of YARN
[23].
YARN replaces the functionality previously provided by the JobTracker and
TaskTracker daemons. In earlier releases of Hadoop, a MapReduce job is
submitted to the JobTracker daemon. The JobTracker communicates with the
NameNode to determine which worker nodes store the required data blocks for
the MapReduce job. The JobTracker then assigns individual map and reduce tasks
to the TaskTracker running on worker nodes. To optimize performance, each task
is preferably assigned to a worker node that is storing an input data block. The
TaskTracker periodically communicates with the JobTracker on the status of its
executing tasks. If a task appears to have failed, the JobTracker can assign the task
to a different TaskTracker.
