Information Technology Reference
In-Depth Information
Table 1. Average recall and precision at the first 20 returned results
Recall
0.00
0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00
sum
Avg
0.00
4
0
0
0
0
0
0
0
0
0
0
4
0.00
0.10
0
2
1
1
3
0
0
1
1
1
1
11
0.48
0.20
0
6
4
1
1
4
2
5
0
3
4
30
0.52
0.30
0
0
1
2
8
4
1
1
0
0
0
17
0.43
0.40
0
1
0
0
2
1
0
0
1
1
0
6
0.52
0.50
0
0
0
0
0
0
1
0
0
0
0
1
0.60
0.60
0
0
0
0
0
0
0
0
0
0
0
0
0.00
0.70
0
1
0
1
0
0
0
0
0
0
0
2
0.20
0.80
0
0
0
0
0
0
0
0
0
0
0
0
0.00
0.90
0
0
0
0
0
0
0
0
0
0
0
0
0.00
1.00
0
1
0
0
0
0
0
0
0
0
0
1
0.10
sum
4
11
6
5
14
9
4
7
2
5
5
72
avg
0.00
0.32 0.20 0.32 0.26 0.27 0.30 0.20 0.25 0.22 0.18
Precision
the
RankPower
measure was effective and easy to interpret. A similar approach to that was discussed
in (Korfhage 1997) was used in the study. A set of 72 randomly chosen queries are sent to the chosen
search engines (AltaVista, (AltaVista, 2005) and MARS (Chen & Meng, 2002)). The first 200 returned
documents for each query are used as the document set. Each of the 200 documents for each of the
query is examined to determine the collection of relevant document set. This process continues for all
72 queries. The average recall and precision are computed at each of the recall intervals. The results
are listed in Table 1.
Shown in Table 2 are the numerical values of the various single-value measures collected from the
same data set. Following (Cooper 1968)'s discussion, five different types of ESL measures were studied.
These five types are listed as follows:
1.
Type-1:
A user may just want the answer to a very specific factual question or a single statistics.
Only one relevant document is needed to satisfy the search request.
2.
Type-2:
A user may actually want only a fixed number, for example,
six
of relevant documents to
a query.
3.
Type-3:
A user may wish to see
all
documents relevant to the topic.
4.
Type-4:
A user may want to sample a subject area as in 2, but wish to specify the ideal size for
the sample as some proportion, say of
ne-tenth
, of the relevant documents.
5.
Type-5:
A user may wish to read
all
relevant documents in case there should be less than five, and
exactly
ive
in case there exist more than five.
Notice that various ESL measures are the number of irrelevant documents that must be examined
in order to find a fixed number of relevant documents; ASL, on the other hand, is the average position
of the relevant documents;
RankPower
is a measure of average rank divided by the number of relevant
documents with a lower bound of 0.5. In all cases, the smaller the values are, the better the performance
Search WWH ::
Custom Search