Game Development Reference
In-Depth Information
Lessons Learned
The TSO team learned a number of important lessons from automating the testing of
their game. See which ones you can apply to your own:
Use automated regression tests to ensure that once a feature works it should
never be allowed to stop working.
■
Get solid support from management to prevent even minor deviations from the
test procedures. This could defeat much of what you are trying to accomplish
with your automation.
■
Identify the critical path elements and protect them. Basic infrastructure tests
need to precede any specific scripted feature tests.
■
Build testability into the core design of a game. The team soon discovered that
many elements of TSO were not well suited to automated testing, but they
could have been if testing had been a concern from the genesis of the design
process.
■
Develop or acquire new tools to help organize and track the information the
original test tools generate. Creating and maintaining hundreds of automated
test cases can quickly become unmanageable.
■
Automate the noise reduction aspect of the system, filtering out data that the
system doesn't need to process. Otherwise, information overload can quickly
become a problem.
■
Run a basic low-level “sniff test�? of the code before it gets into the main release
of the game. This is immensely useful in avoiding substantial roadblocks in the
development process.
■
Automate full-scale load testing, running constant tests of thousands of active
clients. These tests significantly reduce problems that you would otherwise face
at launch.
■
Abstract test commands to maximize their re-use. The SimScript language can
be used to automate single or multiplayer modes and titles, and is independent
of which platform the game is being tested on.
■
The TSO team succeeded by not trying to automate everything, focusing the automation
where it was of best use and, where possible, using game code and existing game clients
that minimized the need to create new code that itself would need to be debugged.
