Infrastructure for Building Code Search Applications for Developers - Finding Source Code on the Web for Remix and Reuse - page 137

8.2.4 Infrastructure Requirements

SCSE, being the first and the most general application has the basic requirements.
CodeGenie and SAS have some common requirements as SCSE, along with some
new ones of their own. The overall infrastructure requirements to build the three
different code search applications emerge out as follows:
Common Requirements:

- Crawling open source forges, extracting project metadata; downloading and

checking out source code from open source forges and associating project
metadata with the checked out code

- Language aware parsing of source code to extract structural information (en-

tities and relations)

- Indexing source code entities to make them searchable

- Ability to store and retrieve source code for code entities

SCSE:

- Compute rank using structural and lexical information

CodeGenie:

- Code search service that could accept a structured code query, where the query

expresses matches on various parts of the code such as class name, method
signature, return types, and method arity (number of arguments)

- Search results at the granularity of code entities such as classes and methods

so that the IDE can show types and method signatures in the result pane

- A special code slicing service that could construct a set of synthesized code

entities that makes the retrieved code entity declaratively complete (i.e., all
dependencies must be resolved)

SAS:

- Data sources to produce an index using SSI

- For a given set of code entities in search result, ability to get a list of most

popular APIs that are used by the code entities

- For a given code entity, ability to get a list of other code entities that have

similar API usage

- For a given code entity, and a set of APIs, ability to provide details on where

and how the APIs are used

8.3 Infrastructure Architecture

Sourcerer provides the requirements posed by three code search applications
through a collection of services . These services provide programmatic access to the
underlying data ( stored contents ) that Sourcerer produces and stores. The format