USING A SPIDER - HTTP Programming Recipes for Java Bots - page 434

This process continues endlessly, as long as the program is allowed to run.
Opening a Database Connection
Before any SQL commands can be issued to obtain stats, a connection must be opened
to the database. This is done by the open method. The open method will also create
several PreparedStatements that will execute the two SQL statements that this pro-
gram uses to obtain its statistics. The open method begins by using the information from a
SpiderOptions object to establish a connection. The SpiderOptions object was
loaded from a spider configuration file, discussed in previous recipes in this chapter.
try {
setTitle("Heaton Research Spider");
Class.forName(this.options.dbClass).newInstance();
this.connection = DriverManager.getConnection(
this.options.dbURL, this.options.dbUID,
this.options.dbPWD);
Next, two prepared statements are created for the two SQL statements that this program
uses to obtain statistics.
this.stmtStatus = this.connection.prepareStatement(this.sqlSta-
tus);
this.stmtDepth = this.connection.prepareStatement(this.sqlDepth);
Finally, catch statements are used to trap any of the errors that can occur while estab-
lishing a database connection.
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
If any error does occur, a stack trace is printed, and the method ends.
Obtaining the Statistics
Now that the database connection has been opened, the statistics can be obtained. Call-
ing the obtainStats method does this. The obtainStats method begins by clear-
ing out all of the totals and executing the stmtStatus query. This query obtains a count
for each of the status types in the spider_workload table.
this.waiting = this.processing = this.error = this.done = 0;
ResultSet rs = this.stmtStatus.executeQuery();