Database Reference
In-Depth Information
<?xml version="1.0"?>
<configuration>
<property>
<name>
yarn.scheduler.capacity.root.queues
</name>
<value>
prod,dev
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.dev.queues
</name>
<value>
eng,science
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.prod.capacity
</name>
<value>
40
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.dev.capacity
</name>
<value>
60
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.dev.maximum-capacity
</name>
<value>
75
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.dev.eng.capacity
</name>
<value>
50
</value>
</property>
<property>
<name>
yarn.scheduler.capacity.root.dev.science.capacity
</name>
<value>
50
</value>
</property>
</configuration>
As you can see, the
dev
queue is further divided into
eng
and
science
queues of equal
capacity. So that the
dev
queue does not use up all the cluster resources when the
prod
queue is idle, it has its maximum capacity set to 75%. In other words, the
prod
queue al-
ways has 25% of the cluster available for immediate use. Since no maximum capacities
have been set for other queues, it's possible for jobs in the
eng
or
science
queues to
use all of the
dev
queue's capacity (up to 75% of the cluster), or indeed for the
prod
queue to use the entire cluster.
Beyond configuring queue hierarchies and capacities, there are settings to control the
maximum number of resources a single user or application can be allocated, how many
applications can be running at any one time, and ACLs on queues. See the
reference page
for details.
