Internet Addresses - Java Network Programming - page 116

ExecutorService executor = Executors . newFixedThreadPool ( NUM_THREADS );
Queue < LogEntry > results = new LinkedList < LogEntry >();
try ( BufferedReader in = new BufferedReader (
new InputStreamReader ( new FileInputStream ( args [ 0 ]), "UTF-8" ));) {
for ( String entry = in . readLine (); entry != null ; entry = in . readLine ()) {
LookupTask task = new LookupTask ( entry );
Future < String > future = executor . submit ( task );
LogEntry result = new LogEntry ( entry , future );
results . add ( result );
}
}
// Start printing the results. This blocks each time a result isn't ready.
for ( LogEntry result : results ) {
try {
System . out . println ( result . future . get ());
} catch ( InterruptedException | ExecutionException ex ) {
System . out . println ( result . original );
}
}
executor . shutdown ();
}
private static class LogEntry {
String original ;
Future < String > future ;
LogEntry ( String original , Future < String > future ) {
this . original = original ;
this . future = future ;
}
}
}
Using threads like this lets the same logfiles be processed in parallel—a huge time sav‐
ings. In my unscientific tests, the threaded version is 10 to 50 times faster than the
sequential version. The tech editor ran the same test on a different system and only saw
a factor of four improvement, but either way it's still a significant gain.
There's still one downside to this design. Although the queue of Callable tasks is much
more efficient than spawning a thread for each logfile entry, logfiles can be huge and
this program can still burn a lot of memory. To avoid this, you could put the output into
a separate thread that shared the queue with the input thread. Because early entries
could be processed and output while the input was still being parsed, the queue would
not grow so large. This does, however, introduce another problem. You'd need a separate
signal to tell you when the output was complete because an empty queue is no longer
sufficient to prove the job is complete. The easiest way is simply to count the number
of input lines and make sure it matches up to the number of output lines.