Information Technology Reference
In-Depth Information
!" "
'!" "
Fig. 4.
System Recall
number of true positive functionalities for a requirement is exactly
k
. Then, the
Mean Average Precision
at level
k
of recall is denoted as:
|F
i
|
|RF |
1
|RF|
1
|F
i
|
P
(
F
i
)
MAP
@
k
=
i
=1
k
=1
|F
i
|
where
obviously denotes the number of relevant functionalities for a given
requisite
r
i
that varies with
i
,and
P
(
F
i
) expresses the precision against the first
k
true positives for
r
i
. The results in terms of Recall with respect to increasing
number of retrieved functionalities are shown in Figure 4 for different retrieval
strategies. Moreover, Table 2 report evaluation figures for varying size of the
retrieved document lists.
Tabl e 2.
Accuracy for different IR strategies and sizes of returned functionalities
1 5 10 15 20 25 30 35 40 45 50
P 0,158 0,036 0,021 0,016 0,014 0,012 0,011 0,010 0,010 0,009 0,008
No filter R 0,092 0,093 0,102 0,107 0,122 0,124 0,128 0,142 0,147 0,152 0,160
F1 0,116 0,052 0,035 0,028 0,026 0,022 0,020 0,019 0,018 0,017 0,016
P 0,175 0,046 0,029 0,024 0,022 0,020 0,018 0,017 0,016 0,015 0,015
Type R 0,097 0,112 0,126 0,141 0,157 0,178 0,196 0,211 0,219 0,226 0,242
F1 0,125 0,065 0,047 0,042 0,038 0,036 0,033 0,032 0,030 0,029 0,028
P 0,407 0,151 0,106 0,085 0,073 0,065 0,060 0,056 0,052 0,051 0,049
Capability R 0,176 0,255 0,363 0,415 0,484 0,525 0,553 0,588 0,615 0,659 0,680
F1 0,246 0,189 0,165 0,141 0,127 0,116 0,108 0,102 0,097 0,094 0,091
P 0,488 0,167 0,113 0,089 0,073 0,066 0,060 0,055 0,050 0,047 0,044
Type+Capability R 0,196 0,296 0,430 0,497 0,538 0,553 0,561 0,585 0,661 0,677 0,683
F1 0,280 0,214 0,179 0,150 0,128 0,118 0,109 0,101 0,093 0,088 0,083
Moreover, in Table 3 the results of
MAP
are reported: rows correspond to
different strategies while columns report different
MAP
values obtained when
k
is fixed to 1, 2 and 5, respectively. When no filter is applied, results are quite
low, especially in term of Recall: when 50 functionalities are retrieved only 20%
are usually relevant for the user. The high-level filter, represented by the
Type
strategy, improves results even if the difference is not very relevant. The Capa-
bility information produces a considerable improvement: when 50 functionalities
Search WWH ::
Custom Search