Database Reference
In-Depth Information
Queue placement
The way that you specify which queue an application is placed in is specific to the applic-
ation. For example, in MapReduce, you set the property mapre-
duce.job.queuename to the name of the queue you want to use. If the queue does
not exist, then you'll get an error at submission time. If no queue is specified, applications
will be placed in a queue called default .
WARNING
For the Capacity Scheduler, the queue name should be the last part of the hierarchical name since the full
hierarchical name is not recognized. So, for the preceding example configuration, prod and eng are
OK, but root.dev.eng and dev.eng do not work.
Fair Scheduler Configuration
The Fair Scheduler attempts to allocate resources so that all running applications get the
same share of resources. Figure 4-3 showed how fair sharing works for applications in the
same queue; however, fair sharing actually works between queues, too, as we'll see next.
NOTE
The terms queue and pool are used interchangeably in the context of the Fair Scheduler.
To understand how resources are shared between queues, imagine two users A and B,
each with their own queue ( Figure 4-4 ). A starts a job, and it is allocated all the resources
available since there is no demand from B. Then B starts a job while A's job is still run-
ning, and after a while each job is using half of the resources, in the way we saw earlier.
Now if B starts a second job while the other jobs are still running, it will share its re-
sources with B's other job, so each of B's jobs will have one-fourth of the resources, while
A's will continue to have half. The result is that resources are shared fairly between users.
Search WWH ::




Custom Search