Game Development Reference
In-Depth Information
browsers, and file sharing programs—a host of applications that were installed on their
test computers. Some even jumped into a game of
Unreal Tournament
. Bob asked the
assistant test manager why he thought it was a good idea for all the testers to have these
extraneous programs on their test configurations. “It simulates real-world conditions,�?
he shrugged, annoyed by Bob's question.
As you may have already guessed, this lab's failure to wipe their test computers clean
before each build led to a lot of wasted time chasing false defects—symptoms testers
thought were defects in the game, but which were in fact problems brought about by,
for example, email or file sharing programs running in the background, taxing the
system's resources and network bandwidth. This wasted tester time also meant a lot of
wasted programmer time, as the development team tried to figure out what in the
game code might be causing such (false) defects.
The problem was solved by reformatting each test PC, freshly installing the operating
system and latest drivers, and then using a drive image program to create a system
restore file. From that point forward, testers merely had to reformat their hard drive
and copy the system restore file over from a CD.
Whatever protocol is established,
config prep
is crucial prior to the distribution of a
new build.
Smoke Testing
The next step after accepting a new build and preparing to test it is to certify that
the build is worthwhile to formally test. This process is sometimes called performing
a
smoke test
on the build, because it's used to determine whether a build “smokes�?
(malfunctions) when run. At a minimum, it should consist of a “load & launch,�?
that is, the lead or primary tester should launch the game, enter each module from
the main menu, and spend a minute or two playing each module. If the game
launches with no obvious performance problems and each module implemented
so far loads with no obvious problems, it is safe to certify the build, log it, and
duplicate it for the test team.
So the build is distributed. Time to test for new bugs, right? Not just yet. Before test-
ing can take a step forward, you must take a step backward and verify that the bugs
the development team claims to have fixed in this build are indeed fixed. This process
is known as
regression testing
.
Regression Testing
Fix verification can be at once very satisfying and very frustrating. It gives the test
team a good sense of accomplishment to see the defects they report disappear one by
