Game Development Reference
In-Depth Information
The potential negative effects of adding test automation to your procedures was men-
tioned earlier in this chapter. Nevertheless, basic automation of source version control,
running automated tests on daily builds, running daily regression tests, and insisting
that all new code at least have basic tests run on it before it is committed are just plain
good-sense practices. You can save many person-hours simply by keeping obviously
buggy code out of the system and by ensuring everyone is working on the same ver-
sion. Limit the basic test scripts in the automated system by testing just for obvious
crashes, omissions, and so on. This will not introduce any significant new test code to
the overall system that itself then would need constant debugging and management.
Given that it is usually easy to find stable change points in your code, you can write a
script that ensures that everyone is automatically synchronized to the latest known
stable release that your automatic test system has validated.
“What finally did help is the end-to-end autobuild, or, as we call it, “churning�?. As soon as one build
completes and is successful, the testing machine gets the next change. If there is no new changelist
yet, the testing machine idles until another change is submitted. Then it syncs to the new change,
rebuilds the data, and emails the team if there are any failures. We've accepted that people are
going to make mistakes—giving up on Steve Maguire's claim that it's possible to keep bugs out of
the master sources—what we have now is a system that catches those mistakes within minutes
of when they're made.�?
Jamie Fristrom, programmer on Spider-Man 2
Your company is going to want to undertake its own cost analysis before embarking
on the introduction of any significant automated testing. Too often managers will see
the introduction of automated testing as some kind of “silver bullet�? that should cure
all their cost overruns and late product releases. While significant success stories are
starting to appear in the game industry following various companies' deployment of
automated test systems, there are far more tales of woe where companies tried to auto-
mate and got badly burned. The reality is that there are many costs to consider before
taking the step to automate.
Setting clear expectations to upper management is crucial in presenting the concept
of test automation. Clearly the need is to keep clear in your mind where you feel test
