The Test Process - Game Testing All in One - page 162

one. It can be very frustrating, however, when a fix of one defect creates another defect
elsewhere in the game, as can often happen.
The test suite for regression testing is the list of bugs claimed to be fixed by the devel-
opment team. This list, sometimes called the knockdown list , is ideally communicated
through the bug database. When the programmer or artist fixes the defect, all they
have to do is change the value of the Developer Status field to “Fixed.�? This allows the
project manager to track the progress on a minute-to-minute basis. It also allows the
lead tester to sort the regression set (by bug author or by level, for example). At a min-
imum, the knockdown list can take the form of a list of bug numbers sent from the
development team to the lead tester.
Tip
Don't accept a build into test unless it is accompanied by a knockdown list. It is a waste of the test
team's time to regress every open bug in the database every time a new build enters test.
Each tester will take the bugs they've been assigned and perform the steps in the bug
write-up to verify that defect is indeed fixed. The fixes to many defects are easily ver-
ified (typos, missing features, and so on). Some defects, such as hard-to-reproduce
crashes, may seem fixed, but the lead tester may want to err on the side of caution
before he closes the bug. By flagging the defect as verify fix , the bug can remain in the
regression set for the next build (or two), but out of the open set that the development
team is still working on. Once the bug has been verified as fixed in two or three builds,
the lead tester can then close the bug with more confidence.
At the end of regression testing, the lead tester and project manager can get a very
good sense of how the project is progressing. A high fix rate (number of bugs closed
divided by the number of bugs claimed to have been fixed) means the developers are
working efficiently. A low fix rate is cause for concern. Are the programmers arbitrar-
ily marking bugs as fixed if they think they've implemented new code that may
address the defect, rather than troubleshooting the defect itself? Are the testers not
writing clear bugs? Is there a version control problem? Are the test systems configured
properly? While the lead tester and project manager mull over these questions, it's
time for you to move on to the next step in the testing process: performing structured
tests and reporting the test results.
Test “Around�? the Bug
The old saying in carpentry is “measure twice, cut once.�? Good testers thoroughly
investigate a defect before they write it up, anticipating any questions the development
team may have.