Programming languages and software engineering - The Computing Universe: A Journey through a Revolution - page 71

Fig. 4.10. Size and scale of the program-
ming teams and code base for versions
of the Microsoft Windows operating
system.

Development
Team Size

Test Team
Size

Lines of
Code (LOC)

Ship Date Product

July 1993 Windows NT 200 140 5 million

December
1999

Windows
2000

1,400 1,700 30 million

October
2001

Windows XP 1,800 2,200 40 million

In his topic Code Complete , Steve McConnell estimates the extent of the bug
problem:

Industry average experience is about 1-25 errors per 1000 lines of code for
delivered software. The Applications Division at Microsoft experiences about
10- 20 defects per 1000 lines of code during in-house testing and 0.5 defects
per 1000 lines of code in released product. 17

The problem of deciding which bugs to fix and which are likely to generate
new errors is complex. This is an area where the new field of empirical soft-
ware engineering aims to help. The Journal of Empirical Software Engineering says:

Over the last decade, it has become clear that empirical studies are a
fundamental component of software engineering research and practice:
Software development practices and technologies must be investigated by
empirical means in order to be understood, evaluated, and deployed in proper
contexts. This stems from the observation that higher software quality and
productivity have more chances to be achieved if well-understood, tested
practices and technologies are introduced in software development. Empirical
studies usually involve the collection and analysis of data and experience
that can be used to characterize, evaluate and reveal relationships between
software development deliverables, practices, and technologies. 18

This statement has now been adopted as part of the manifesto of the
International Software Engineering Research Network.
One example of this empirical approach to software engineering is the
CRANE tool developed by researchers at Microsoft - where CRANE is an acro-
nym formed from Change Risk ANalysis and impact Estimation. The CRANE
project looked at the challenges of providing support for multiple versions of
Windows, running on a wide variety of computers, with a user base of more
than a billion. One immediate challenge is that software maintenance for a
released product is done by different teams of software engineers than those
who developed the software. The goal of the CRANE project was to use his-
torical information about the software being serviced to build risk-prediction
models using advanced statistical techniques that could guide bug fixing and
testing. For every bug in any software component, the tool provides the fol-
lowing information: what has happened to the component so far in servicing;
what exactly is being changed with the proposed fix; which fixes carry more
than average risk of causing more bugs; which tests to run after the change;
which other components to test in addition to the changed component; and