Hardware Reference
In-Depth Information
The QualOSS process evaluation framework is aimed at covering the last characteris-
tic mentioned, namely, process maturity. In what follows, we describe this framework
in more detail.
3 Related Work: OSS Assessment
In recent years, Open Source Software has often been used as the target of quantita-
tive analyses of code quality, mostly due to the fact that large code repositories are
available for analysis. Many publications exist on (semi-)automatic analysis of code,
mailing lists, bug tracking, and versioning systems. Contrary to what happens with
code and repository analysis, few publications have addressed OSS processes so far.
A paper by Michlmayr [7] is one notable exception, providing evidence of disciplined
processes in OSS projects and relating it with project success.
As a reaction to the insight that software quality is not restricted to code aspects,
assessment models for OSS projects have emerged whose aim is to support potential
OSS users in making decisions regarding the selection of OSS products. The most
prominent examples are the
Qualification and Selection of Open Source Software
(QSOS) model [9], two different models called
Open Source Maturity Model
(OSMM)—one from CapGemini [2] and one from Navica [8]—and the
Open Busi-
ness Readiness Rating
(OpenBRR) model [1]. Although these models take the OSS
product into account (i.e., code, documentation), as well as the community that pro-
duces it, they only have a rudimentary process perspective, if any. For example,
QSOS considers two process criteria: quality assurance processes (with levels none,
informal, supported by tools), and bug/feature request tools (none, standard tools,
active use of tools), which, in our opinion, are far from covering the wide variety of
quality-relevant processes typically observed in OSS development. This lack of cov-
erage for the process perspective constitutes one of our main motivations for propos-
ing the more comprehensive approach discussed here.
4 Towards a Process Maturity Model for OSS
As discussed in the introduction, the idea of assessing an OSS community in order to
determine which good practices it follows, as well as how established these practices are,
is perfectly reasonable. Still, it is true that existing process assessment models cannot
generally be applied directly to OSS, as they include too many elements that are specific
to companies and other conventional development organizations. In this section, we de-
scribe our process evaluation framework, which is directly aimed at OSS development.
This model reuses a number of the ideas present in existing maturity models, but adapts
them in order to make them more directly applicable in an OSS context.
4.1 Maturity Models as a Basis for Open Source Process Assessment
In order to create an assessment model for OSS process maturity, we started by review-
ing existing maturity models with the purpose of extracting, and, where necessary,
adapting some of their elements to the specifics of OSS. Concretely, we used the
Capa-
bility Maturity Model for Software Development
(CMMI-DEV) as a starting point. Re-
leased in 2006, the current CMMI-DEV model is the latest version in a series of
