Game Development Reference
In-Depth Information
relative to its size, while “phase containment�? provides an indicator of how successful
the team is at finding defects at their source, leaving fewer to escape to your customers.
Six Sigma Software
A “sigma level�? is one way to establish a goal for the outgoing quality of your game.
For software this measure is based on defects per million lines of code, excluding com-
ments (also referred to as “non-commented source lines�? or “NCSL�?). The “lines of
code�? measure is often normalized to Assembly-equivalent lines of code (AELOC) in
order to balance the different level of abstraction across the variety of languages in use
such as C, C++, Java, Visual Basic, and so on. The level of abstraction of each language
is reflected in its multiplier. For example, each line of C code is typically regarded as
the equivalent of three to four AELOC, whereas each line of Perl code is treated as
about 15 AELOC. It's best to measure this factor based on your specific development
environment and use that factor for any estimates or projections you need to make in
the future. If you are using different languages for different parts of your game, mul-
tiply the lines of code for each portion by the corresponding language factor.
Figure 6.1 shows the defect rates required to achieve a software quality measure anywhere
between three and six sigma. Six sigma (only 3.6 defects per million lines of code) is typ-
ically regarded as an outstanding result, and getting in the 5.5 sigma range is very good.
Don't fool yourself by measuring your
sigma on the sole basis of the open defects
you know about in the product. This might
reward poor testing which did not find
many defects that still remain in the game,
but wouldn't reflect the experience your
customers will have. The defects being
counted must include both the game
defects you know about that have not been
fixed, whatever defects your customers
have already found, and your projection of
defects that remain in the software which
haven't been discovered yet. It's best to wait
anywhere from 6 to 18 months after ship-
ping to calculate your sigma. If you still
have a good result after that, continue to
operate your projects in a similar manner
by repeating what went “right�? but also fix
what went “wrong.�? If
Figure 6.1
Sigma table excerpt for
various sizes of delivered
software.
you have poor
