Game Development Reference
In-Depth Information
How to Be a Repro Man (or Woman)
One of the most critical bits of information in any bug report is the rate of reproduction. In a defect
tracking database, this field may be called (among other things) frequency, occurrence rate, “happens,�?
or repro rate. All these various terms are used to describe the same thing. Reproduction rate can
be defined as the rate at which, following the steps described in the bug report, anyone will be
able to reproduce a defect.
This information is generally expressed as a percentage ranging from 100% to “once,�? but this can
be misleading. Assume, for example, that you find a defect during the course of your free testing.
After a little research, you narrow down the steps to a reproducible path. You follow those steps
and get the bug to happen again. You could, reasonably, report the defect as occurring 100% of
the time—you tried the steps twice and it happened both times. However, it may be just as likely
that the bug is only reproducible 50% of the time or less, and you just got lucky, as though you
had flipped a penny and got it to land heads up twice in a row.
For this reason, many QA labs report the repro rate as the number of attempts paired with the
number of observed occurrences, (for example, “8 occurrences out of 10 attempts�?). This informa-
tion is far more useful and accurate, because it allows your test lead, the project manager, and any-
one else on the team to evaluate how thoroughly the bug has been tested. It also serves to keep
you honest about the amount of testing you've given the defect before you write your report. How
likely are you to report that a crash bug happens “once�? if you only tried to reproduce it once? If
you want to maintain your credibility as a member of the test team, you won't make a habit of this.
On the other hand, with certain defects, even a relatively novice tester can be certain that a bug
occurs 100% of the time without iterative testing. Bugs relating to fixed assets, such as a typo in
in-game text, can safely be assumed to occur 100% of the time.
The word “anyone�?is critical to the definition above, because a defect report is not very helpful if
the tester who found the bug is the only one able to re-create it. Because videogame testing is
often skill-based, it is not uncommon to encounter a defect in a game (especially a sports, fight-
ing, platform jumper, or stunt game) that can only be reproduced by one tester, but that tester can
reproduce the bug 100% of the time. In an ideal situation, that tester will collaborate closely with
other members of the team so that they can zero in on a path that will allow the others to re-create
the bug.
If this is not possible due to time or other resource constraints, be prepared to send a videotape or
.AVI clip of the defect to the development team or, in the worst cases, send the tester to the devel-
oper to do a live demonstration of the bug. This is very costly and time consuming because, in addi-
tion to any travel expenses, the project is also paying for the cost of having the tester away from
the lab (and not testing) for a period of time.
In summary, the more reproducible a bug is, the more likely it is that it will be fixed. So always
strive to be a true “repro man.�?
