Hardware Reference
In-Depth Information
In some cases, the number of instances of a particular task is too high for manual
inspection. For example, some projects have databases of reported issues that have
been operating for years and contain thousands of reports. So far, we have ana-
lyzed such data repositories by manually choosing a small number of instances “at
random”, but this method is clearly unsatisfactory due to the high risk of introduc-
ing biases. Ideally, we should be able to guarantee that we did a fair, random sam-
ple, and that the number of instances observed is representative. We still have to do
more research in appropriate methods for this purpose, and, potentially, provide
software tools to assist this procedure.
The importance of some of the tasks listed in the previous section may vary
depending on the size of the evaluated project. For instance, many small OSS
projects have a single maintainer who is the only person with access to the main
versioning repository. Such projects will rarely, if ever, accept new permanent con-
tributors, and thus having a defined process for this purpose would be simply un-
necessary. On the other hand, large projects with tens or even hundreds of official
developers definitely require an explicit process for accepting new members. For
this reason, we are considering the idea of giving variable importance to different
tasks depending on such characteristics of a project as its number of active con-
tributors or its code size.
Future versions of the QualOSS process evaluation framework are likely to incorpo-
rate enhancements based on the previous observations.
6 Conclusions and Future Work
The purpose of the QualOSS project is to produce a comprehensive quality model for
assessing OSS projects. In this paper, we have presented a small portion of this work,
namely, a process evaluation framework aimed at OSS. We expect OSS process
evaluation to provide a better foundation for judging a community's ability to deliver
high-quality software, as well as its long-term sustainability (“will this project exist in
10 years?”). Indeed, sustainability of suppliers is critical to many stakeholders, and is
also a problem with commercial software. For example, the European defense consor-
tium EADS decided to turn a critical piece of software into OSS in order to become
independent of specific suppliers [11].
Moreover, highly regulated industries, such as the automotive, medical, or pharma-
ceutical industries, have established standards for evaluating software, which include
assessment of the supplier [3] [5]. These industries often find it problematic to use
OSS, because there is little support for the assessments required by their quality stan-
dards. Consequently, we believe that OSS assessment models that include a process
assessment may help to increase the adoption of OSS in these industries.
As mentioned in Section 5, our experience with applying the QualOSS process as-
sessment is still very limited. The final, evaluation phase of the QualOSS project will
provide us with a valuable opportunity to introduce some initial improvements—such
as those suggested in Section 5—as well as to collect more experience with using the
process evaluation framework. We expect this experience to allow us to produce a
much more robust and reliable framework during the next few months.
Search WWH ::




Custom Search