Scheduling and Workflow - Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset - page 129

V2 Capacity Scheduler
You need to set up the Capacity scheduler to work with YARN. You begin by configuring YARN so that its scheduler
class is defined as Capacity, rather than as the default FIFO scheduler. At the end of the yarn-site.xml file, which is
located under /etc/hadoop/conf, you add the following property to define Capacity:
<!-- add scheduler configuration -->
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>
As you remember, Hadoop V2 can support hierarchical queues for each cluster client. This way, free resources
can be shared across a client's queues before being offered to other queues. To demonstrate how this works, I define
some queues in the Capacity scheduler's configuration file (capacity-scheduler.xml, found in /etc/hadoop/conf ).
Specifically, I define and create three new queues (client1, client2, and client3) and their subqueues, as follows:
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>client1,client2,client3</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.client1.queues</name>
<value>client1a,client1b</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.client2.queues</name>
<value>client2a,client2b,client2c</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.client3.queues</name>
<value>client3a,client3b</value>
</property>
In addition, for each queue I create, I define its properties in the capacity-scheduler.xml file. The XML that
follows is an example of that needed for one of the queues. Note that this must be repeated for each parent and child
queue; therefore, I must remember to change the name of each attribute to match the queue name—that is, yarn.
scheduler.capacity.root. client1 .capacity .
As in the V1 example, the sum of the capacity values for queues in the configuration file must be 100. Also, the
sum of the capacity values for the child queues within a parent queue must be 100. For instance, the children of
client1 are named client1.client1a. This XML sets up the attributes for queue client1 in terms of capacity, state, and
access. In this case, the * values mean “anyone,” so access has been left open:
<property>
<name>yarn.scheduler.capacity.root.client1.capacity</name> <value>100</value>
</property>