Search Engine Performance Comparisons - Distributed Artificial Intelligence, Agent Technology, and Collaborative Applications - page 161

Table 10. Adjusted wall-clock timing for a page seen from Cal State University at Los Angeles (in milli-
seconds).

Search Engine AlltheWeb MSE Yahoo

Avg. Page 1 688 383 574

Avg. Pages 2-5 534 277 521

Avg. Overall

611 330 547

For example, for MSE, the overall average first page timing from Cal State LA is 428 milli-seconds
(Table 7), the average local access timing for a randomly generated first page is 45.4 milli-seconds, then
we consider the measured timing for MSE from Cal State LA to be 428 - 45.4 = 383 milli-seconds. The
adjusted results after removing the local access timing component are presented in Table 9 for Bucknell
University and Table 10 for California State University at Los Angeles, respectively.
By comparing the data in Table 9 with that in Table 6, and the data in Table 10 with that in Table
7, the relative relations in performance among these search engines do not change. The MSE has the
fastest response time on both campuses; the one on the west coast is visibly faster even after removing
the differences in host computers. This suggests that the MSE at the time of this measurement was
located somewhere on the west coast. Yahoo! has the longest response time on the east coast which
may suggest that the Yahoo! servers are close to the west coast, while AlltheWeb has the longest one
on the west coast.

conclusion

We reviewed the theory and practices of search engine performance analysis. A performance study of
a few popular search engines is conducted including AlltheWeb, Google, Microsoft Search Engine, and
Yahoo!. The quality of the search results is examined manually by marking relevant pages returned from
the search engines in response to a set of randomly chosen queries and computing the RankPower . The
average response time is computed by constructing a client program for each of the search engines and
collecting the actual average wall-clock time between the time when a query is sent and the time when
the search is complete with the first five pages received by the client. Other statistics such as the aver-
age time needed to generate a page and the number of relevant pages to a given query both of which are
claimed by the search engine are extracted from the returning pages sent by the search engines.
By comparing the empirical results, it is shown that the Microsoft search engine (beta) performs
well in terms of speed to deliver the results and the diversity of the results. It does not do very well in
terms of the quality of the search measured by RankPower , compared to other search engines such as
Google and Yahoo!.