EA-Miner: Towards Automation in Aspect-Oriented Requirements Engineering - Transactions on Aspect-Oriented Software Development - page 27

Table 5. Data on accuracy based on knowledge of the system

VP influenced by knowledge of the system
Light Control Library

Manual EA-Miner Manual EA-Miner

Precision
VP

6/9 = 66.7% 9/10 = 90% 7/11 = 63.6% 8/9 = 88.8%

Recall
VP

6/11 = 54.5% 9/11 = 81.8% 7/8 = 87.5% 8/8 = 100%

Precision
EA

6/8 = 75% 6/6 = 100% 7/10 = 70% 4/4 = 100%

Recall
EA

6/7 = 85.7% 6/7 = 85.7% 7/8 = 87.5% 4/8 = 50%

Therefore, when compared to a manual analysis, EA-Miner can offer superior
precision and recall of viewpoints specially when combined with the engineer's
knowledge. Moreover, analyzing the list provided by EA-Miner in the two examples,
in several cases the important viewpoints are listed first. For example, for the library
system, four out of the eight correct viewpoints were among the first six listed.
Therefore, we can conclude by both time-effectiveness and accuracy results that
EA-Miner shows promising results for addressing AORE activities in a time-efficient
manner without compromising the accuracy of the produced output. The examples
showed that EA-Miner analysis was significantly faster than a manual analysis and
that the quality of the outcome can also be higher when combined with the knowledge
of the engineer without having a significant impact on time. Moreover, the case
studies showed that the tool was not influenced by the different structures of the
requirements documents used and that the tool constantly lists the most important
concepts first which helps with tasks such as attributing priorities to requirements.

5.1.3 Case Study Discussion
This first case study has addressed the issue of answering a research question on how
well does EA-Miner perform when compared to a manual approach . In order to
answer this question we collected data about a study conducted using two different
engineers that had the same task of producing an AORE specification for two
different systems (light system and library system). We collected data related to effort
and accuracy (measured in precision and recall) of the models produced.
Sections 5.1.1 and 5.1.2 analysed the data collected showing positive results
related to the use of EA-Miner, for example, that the tool helps to save time and the
end result (the resulting model) was not worse than a model done manually.
One important point that we would like to further explain is the trade-off existing
between the number of candidates listed by the tool with respect to scalability (in
terms of time) and requirements completeness.
As the identification process considers viewpoints to be noun candidates,
depending on the size of the document this number can be very high. One can think
that because of size, the time spent on the analysis can be very high. However, the