Search Engine Performance Comparisons - Distributed Artificial Intelligence, Agent Technology, and Collaborative Applications - page 158

Table 4. Average number of relevant URLs from search engines' view for the 27 queries

Search Engine AlltheWeb Google MSE Yahoo

Avg. No. of Relevant Pages 47,693,444 69,380,370 66,047,923 57,423,667

Table 5. Time needed to generate a return page reprted by the search engines (seconds)

Search Engine Google MSE Yahoo

Avg. Page 1 0.20 0.25 0.13

Std. Dev. Page 1 0.11 0.09 0.07

Avg. Pages 2-5 0.16 0.22 0.10

Std. Dev. Pages 2-5 0.07 0.12 0.02

Avg. Overall 0.17 0.23 0.11

is 0.76, the smallest among the five. Because the theoretical value of RankPower can be any value that
is greater than 0.5, the direct measured values shown in the table do not indicate how sensitive these
values are, the revised RankPower , as described in Section 2, does give a direct comparison because its
values vary from 0 to 1, the larger the values are, the better the results. The table indicates the revised
RankPower values for Google, AlltheWeb, and Vivisimo are 0.70, 0.68, and 0.69, respectively, notably
better than that of MSE and Yahoo!.
The second set of results presented is the average number of relevant URLs for a given query from
search engine's point of view. Four out of the five search engines presented this piece of information
somewhere in their returned pages with the exception of Vivisimo. See Figure 1 for an example from
Yahoo! where it shows that 36,500,000 pages were relevant to the query “thread”. Table 4 lists the aver-
age of relevant URLs over the 27 testing queries reported by the search engines. One should note that
it is not necessarily true that a higher average number of results implies larger overall data sets. One
also should keep in mind that Google declared to have indexed about eight billion Web pages (Google,
2005) while MSE is said to have indexed about five billion Web pages (MSE, 2005).
The next set of statistics to be reported is the average time a search engine takes to generate a return
page for a given query. This information is reported on each of the returning pages generated by Google,
MSE, and Yahoo!. To determine if there is a difference between the response time for the first page of
a search engine and that of the rest of the pages, we collected the statistics separately for the first and
the rest of the pages. The results in Table 5 include the average time for the first returning page, its
standard deviation, the average time for the returning pages 2 through 5, and their standard deviation.
As can be seen from the table, Yahoo! has overall the shortest time to generate a response page and
the smallest standard deviation. Also the difference of average response time between the first page and
the rest pages is relatively small for Yahoo!, indicating that it has an overall smooth response time. For
both Google and MSE there is a visible difference between the response time for the first page (0.20
seconds and 0.25 seconds) and the rest pages (0.16 seconds and 0.22 seconds).