thread pool size, and under certain circumstances an oversized thread pool will be quite detri-
mental to performance.
All thread pools work in essentially the same way: there is a queue to which tasks are sub-
mitted. (There can be more than one queue but the concept is the same.) Then some number
of threads picks up tasks from the queue and executes them. The result of the task can be sent
back to the client (e.g., in the case of an application server), stored in a database, stored in
some internal data structure, or whatever. But after finishing the task, the thread returns to
the task queue to retrieve another job to execute (and if there are no more tasks to perform,
the thread waits for a task).
Thread pools have a minimum and maximum number of threads. The minimum number of
threads is kept around, waiting for tasks to be assigned to them. Because creating a thread is
a fairly expensive operation, this speeds up the overall operation when a task is submitted: it
is expected that an already-existing thread can pick it up. On the other hand, threads require
some system resources—including some native memory for their stacks—and having too
many idle threads can consume resources that could be used by other processes. The maxim-
um number of threads also serves as a necessary throttle, preventing too many tasks from ex-
ecuting at once.
The ThreadPoolExecutor and related classes refer to the minimum number of threads as the
core pool size , while most application servers use some term similar to minimum (e.g.,
MinThreads ). Don't let the terminology confuse you: they are the same concept. However,
there are key differences in the way a ThreadPoolExecutor determines when to resize a
pool and the way in which most Java EE application servers determine when to resize a pool.
Those differences will be explored later in this section. For now, consider the simplest case
of the ThreadPoolExecutor , which is also the way most Java EE application servers work:
if there is a task to be executed, and all current threads are busy executing another task, a
new thread is started (until the maximum number of threads has been created).
Setting the Maximum Number of Threads
Let's address the maximum number of threads first: what is the optimal maximum number of
threads for a given workload on given hardware? There is no simple answer to this; it de-
pends on characteristics of the workload and the hardware on which it is run. In particular,
the optimal number of threads depends on how often each individual task will block.
For the sake of discussion, assume there are four CPUs that are available to the JVM. It
doesn't matter if there are only four CPUs on the system, or if there are 128 hardware threads