Game Development Reference
In-Depth Information
other events. Metrics can be gathered from hundreds of internal testers
or millions of players in a public beta. The computers can then process all
these numbers into statistical reports and graphs that designers can use to
make design decisions.
Metrics help designers see patterns and imbalances too small to
notice in playtests. For example, in a fighting game, metrics will show that
one character wins over another 55% of the time because it can sample
thousands of matches. Playtesting can't reveal this kind of data with any
accuracy because the sample size is too small. This is why metrics are
invaluable in fine-tuning. Without the ability to see small gains, only large
swings in results are visible, and progress becomes halting and erratic. By
revealing even small changes in difficulty or pacing, metrics allow us to
hill-climb our way to glory in tiny, measured steps.
Ken Birdwell writes about his experience using metrics to fine-tune
Half-Life
:
Toward the middle of the project, once the major elements were in place
and the game could be played most of the way through, it became mostly
a matter of fine-tuning. To do this, we added basic instrumentation to the
game, automatically recording the player's position, health, weapons,
time, and any major activities such as saving the game, dying, being hurt,
solving a puzzle, fighting a monster, and so on. We then took the results
from a number of sessions and graphed them together to find any areas
where there were problems. These included areas where the player spent
too long without any encounters (boring), too long with too much health
(too easy), too long with too little health (too hard), all of which gave us
a good idea as to where they were likely to die and which positions would
be best for adding goodies.
When
Half-Life
was developed in 1998, other studios depended on
guesswork and self-testing for this kind of balancing. These methods
are useful early in the design process, but they don't reveal the kind of
fine-grained data needed for perfect difficulty balancing. Metrics brought
Valve's designers a massive advantage in the quality of their design deci-
sions. And they didn't have to be any smarter than anyone else to make a
better game, because they were working in the light while everyone else
labored in darkness.
In addition to fine-tuning, metrics also let designers find rare edge-
case situations. Your 20 internal playtesters might not find that degenerate
