BEYOND SIMPLE REQUESTS - HTTP Programming Recipes for Java Bots - page 104

Recipe #4.2: Scan for Sites
You can also use the HttpURLConnection class to determine if there is an active
web server at a specific URL. Recipe 4.2 shows how to loop through a series of IP addresses
to find any web servers. To use this program, you must specify an IP address prefix. An ex-
ample of this would be 192.168.1 . Specifying this prefix would visit 256 IP addresses. It
would visit from 192.168.1.0 to 192.168.1.255 .
This recipe shows how to decrease the timeout for connection. Because almost all of
the IP addresses will not have web servers, it takes a while for this example to run. This is
because; by default Java will wait several minutes to connect to a web server.
Because of this the connection timeout is taken down to only a few seconds. For a more
thorough scan the timeout can be increased. Listing 4.2 shows the site scanner:
Listing 4.2: Scan for Web Sites (ScanSites.java)
package com.heatonresearch.httprecipes.ch4.recipe2;
import java.io.*;
import java.net.*;
import java.util.*;
public class ScanSites
{
// the size of a buffer
public static int BUFFER_SIZE = 8192;
/**
* This method downloads the specified URL into a Java
* String. This is a very simple method, that you can
* reused anytime you need to quickly grab all data from
* a specific URL.
*
* @param url The URL to download.
* @param timeout The number of milliseconds to wait for
* connection.
* @return The contents of the URL that was downloaded.
* @throws IOException Thrown if any sort of error occurs.
*/
public String downloadPage(URL url, int timeout)
throws IOException
{
StringBuilder result = new StringBuilder();
byte buffer[] = new byte[BUFFER_SIZE];
URLConnection http = url.openConnection();