pool for a minimum size, it should stick around for at least a few minutes to handle any spike
in load. To the extent that you have a good model of the arrival rate, you can base the idle
time on that. Otherwise, plan on the idle time being measured in minutes, at least anywhere
from 10 to 30.
Keeping idle threads around usually has little impact on an application. Usually, the thread
object itself doesn't take a very large amount of heap space. The exception to that rule is if
the thread holds onto a large amount of thread-local storage, or if a large amount of memory
is referenced through the thread's runnable object. In either of those cases, freeing a thread
can offer significant savings in terms of the live data left in the heap (which in turn affects
the efficiency of GC).
These cases are unusual for thread pools, however. When a thread in a pool is idle, it should
not be referencing any runnable object anymore (if it is, there is a bug somewhere). Depend-
ing on the pool implementation, the thread-local variables may remain in place—but while
thread-local variables can be an effective way to promote object reuse in certain circum-
should be limited.
One important exception to this rule is for thread pools that can grow to be very large (and
hence run on a very large machine). Take an example where the task queue for a thread pool
is expected to average 20 tasks; 20 is then a good minimum size for the pool. Now say the
pool is running on a very large machine, and that it is designed to handle a spike of 2,000
tasks. Keeping 2,000 idle threads around in this pool will affect its performance when it is
running only the 20 tasks—the throughput of this pool may be as much as 50% when it con-
tains 1,980 idle threads, as opposed to when it has only the core 20 busy threads. Thread
pools don't usually encounter sizing issues like that, but when they do, that's a good time to
make sure they have a good minimum value.
Thread Pool Task Sizes
The tasks pending for a thread pool are held on some sort of queue or list; when a thread in
the pool can execute a task, it pulls a task from the queue. This can lead to an imbalance: it is
possible for the number of tasks on the queue to grow very large. If the queue is too large,
then tasks in the queue will have to wait a long time until the tasks in front of them have
completed execution. Imagine a web server that is overloaded: if a task is added to the queue
and isn't executed for 3 seconds, the user has likely moved on to another page.
As a result, thread pools typically limit the size of the queue of pending tasks. The
ThreadPoolExecutor does this in various ways depending on the data structure it is con-