Databases Reference
In-Depth Information
work in more detail before we share our view of the most important open challenges
in the context of reuse-oriented retrieval and conclude our chapter in Sects.
12.6
and
12.7
, respectively.
12.2 Foundations
Around the turn of the millennium, the state of the art in reuse-oriented software
retrieval could be characterized from two widely contrary viewpoints. While one
opinion was that most challenges related with reuse libraries have been solved al-
ready since repositories were supposed to be mere catalogs containing only a small
number of about 50 to perhaps 250 carefully selected components [
31
], the other
opinion claimed the exact opposite. Mili et al. realized in their well-known survey
on software retrieval approaches [
26
] that there is indeed a large variety of proto-
typical component library systems, but none of them would be able to overcome
the retrieval and usability problem mentioned in the introduction, as soon as the
amount of available components would increase considerably. The reason for this
pessimistic appraisal is simple: since the
matching criterion
(such as e.g. the appear-
ance of a keyword anywhere - i.e. even in comments - within a candidate) is rather
weak in most approaches, it is very likely that they return many irrelevant results
as soon as a critical mass of indexed material has been exceeded. In other words,
10 years ago, it was still unclear how to build
internet-scale
software search systems
(with potentially millions of entries) that would be able to deliver only those com-
ponents precisely matching an existing “gap” in an application under development.
Separating the useful from the useless results in growing collections is one of the
major challenges for all information retrieval approaches [
3
] and is usually referred
to as the
precision
of a search engine. More formally, precision is defined as the
fraction of relevant results amongst all results returned for a query. Obviously, it be-
comes tedious to determine the actual relevance of more than perhaps a few dozen
results so that evaluations of common search engines often limit themselves on in-
vestigating the precision of the first 20 results (the so-called top 20 precision [
23
]).
A similar challenge exists for the second central metric used in information retrieval
to evaluate search engines, the so-called
recall
, defined as the fraction of all relevant
elements that are returned for a given query out of all relevant elements contained
in a collection for that query. As a collection may actually contain thousands of rel-
evant results, it is often not possible to determine all of them and therefore the recall
cannot be determined as well.
The results of a systematic survey in which Mili et al. [
26
] analyzed existing
software retrieval solutions for their performance are presented and discussed in the
following. The authors were able to distinguish six seminal classes of software re-
trieval approaches that we will briefly explain after their enumeration. However, due
to a lack of access to most prototypes and the non-existence of standardized eval-
uation scenarios, Mili et al. were only able to estimate the potential performance
Search WWH ::
Custom Search