Java Reference
In-Depth Information
file is retrieved instead of staying connected for minutes or hours at a time, web users
don't put nearly as much load on a server as FTP users do. However, web server per‐
formance still degrades as usage grows. The fundamental problem is that while it's easy
to write code that handles each incoming connection and each new task as a separate
process (at least on Unix), this solution doesn't scale. By the time a server is attempting
to handle a thousand or more simultaneous connections, performance slows to a crawl.
There are at least two solutions to this problem. The first is to reuse processes rather
than spawning new ones. When the server starts up, a fixed number of processes (say,
300) are spawned to handle requests. Incoming requests are placed in a queue. Each
process removes one request from the queue, services the request, then returns to the
queue to get the next request. There are still 300 separate processes running, but because
all the overhead of building up and tearing down the processes is avoided, these 300
processes can now do the work of 1,000. These numbers are rough estimates. Your exact
mileage may vary, especially if your server hasn't yet reached the volume where scala‐
bility issues come into play. Still, whatever mileage you get out of spawning new pro‐
cesses, you should be able to do much better by reusing old processes.
The second solution to this problem is to use lightweight threads instead of heavyweight
processes to handle connections. Whereas each separate process has its own block of
memory, threads are easier on resources because they share memory. Using threads
instead of processes can buy you another factor of three in server performance. By
combining this with a pool of reusable threads (as opposed to a pool of reusable pro‐
cesses), your server can run nine times faster, all on the same hardware and network
connection! The impact of running many different threads on the server hardware is
relatively
minimal because they all run within one process. Most Java virtual machines
keel over due to memory exhaustion somewhere between 4,000 and 20,000 simultane‐
ous threads. However, by using a thread pool instead of spawning new threads for each
connection, fewer than a hundred threads can handle thousands of short connections
per minute.
Alternatives to Threading
If an application needs thousands of simultaneous long-lived connections (and that's a
pretty rare application) it's time to start thinking about asynchronous I/O instead of
threads. We'll take this up in
Chapter 11
. Selectors enable one thread to query a group
of sockets to find out which ones are ready to be read from or written to, and then process
the ready sockets sequentially. In this case, the I/O has to be designed around channels
and buffers rather than streams.
Given the high performance of threads in modern virtual machines and operating sys‐
tems, as well as the relative simplicity of a building a thread-based server, a thread-based
design is usually where you should start until you can prove you're hitting a wall. If you
do hit a wall, you should seriously consider sharding the application across multiple
