Java Reference
In-Depth Information
Both of the tables can be created using a data definition language (DDL) script. The DDL
script for MySQL is shown in Listing 13.2.
Listing 13.2: Example Create Table DDL for MySQL
SET NAMES latin1;
SET FOREIGN_KEY_CHECKS = 0;
CREATE TABLE 'spider_host' (
'host_id' int(10) unsigned NOT NULL auto_increment,
'host' varchar(255) NOT NULL default '',
'status' varchar(1) NOT NULL default '',
'urls_done' int(11) NOT NULL,
'urls_error' int(11) NOT NULL,
PRIMARY KEY ('host_id')
) ENGINE=MyISAM AUTO_INCREMENT=5929 DEFAULT CHARSET=latin1;
CREATE TABLE 'spider_workload' (
'workload_id' int(10) unsigned NOT NULL auto_increment,
'host' int(10) unsigned NOT NULL,
'url' varchar(2083) NOT NULL default '',
'status' varchar(1) NOT NULL default '',
'depth' int(10) unsigned NOT NULL,
'url_hash' int(11) NOT NULL,
'source_id' int(11) NOT NULL,
PRIMARY KEY (`workload_id`),
KEY `status` (`status`),
KEY `url_hash` (`url_hash`),
KEY `host` (`host`)
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
SET FOREIGN_KEY_CHECKS = 1;
There is a status field contained in both tables. The status field contains a single char-
acter that specifies the status of either the host or workload entry. These status codes are
summarized in Table 13.4.
Table 13.4: Spider Status Codes
Status Code
Purpose
D
Processed successfully.
E
Error while processing.
P
Currently processing.
W
Waiting to be processed.
