Developing Software with Open Source Software Components - Finding Source Code on the Web for Remix and Reuse - page 174

Fig. 9.2 Characterization of answers regarding processes to evaluate components
9.4.1.2 Evaluation of Components
Figure 9.2 shows a summary of the results regarding evaluation processes to eval-
uate components and the respondents that fall into each category. Fourteen out 19
respondents mentioned that they did not use, nor knew of any formal procedure or
method to drive the evaluation and decisions regarding components. Instead, they
proceeded informally, often without even documenting the information on the vari-
ous components for their subsequent comparison. Two respondents recognized that
the evaluation relied on personal experience and experience from others, especially
to face time-to-market demands and to capitalize on previous knowledge from the
team. Two respondents from the same organization said that they hired external con-
sultants to drive the evaluation process. They recalled that they applied a lightweight
approach of the OSMM method [ 24 ] to drive the evaluation of components. In ad-
dition, they highlighted that they were trained on how to apply the method, but they
did not apply it on a daily basis. Instead, they just informally applied a reduced
and ad-hoc set of the evaluation criteria that the given method suggested. Another
respondent emphasized that even when they did not follow established procedures,
they had developed a spreadsheet-like tool to help them assign weights according to
some relevant criteria for ranking candidate components in the Enterprise Resource
Planning (ERP) domain, being the one they usually covered.
Figure 9.3 summarizes the findings with respect to resources used to evaluate
components. Nine respondents said that it boiled down to unit and module test-
ing of very basic component functionalities. Five respondents stated that they even
built a prototype to check if the component behaved as expected. These prototypes
ranged from straightforward ones to more formal ones that required a significant
effort to set up a suitable testing infrastructure. Respondents said that this was not a
representative practice for other projects in the company. Rather, it was mainly done
when a candidate component was critical and/or used for the first time. Finally, five