Game Development Reference
In-Depth Information
all fall away until the story becomes a children's fairy tale of simple cause
and effect. In fact, I've written stories like this in this topic.
The problem is that the lessons of game design process aren't in the
clean, edited story that we tell afterward. They're in the messy red her-
rings and false predictions that we edit out of the story. Hindsight bias
prevents us from learning from our mistakes by making us think events
were more predictable than they really were. Looking back, hindsight bias
makes it always feel like deep planning should have been possible. So we
think it will be possible in the future, and we overplan again and again,
unable to learn from our mistakes.
Once you know what to look for, you'll start to see these overplanning
biases in development. And you'll be able to compensate for them.
Test Protocol
The iterative process is a cycle between planning, building, and testing.
Everyone focuses on planning and building, and testing is often ignored.
But the testing stage is critical because it is the mechanism by which we
learn lessons from the real world and secure the main benefit of iteration.
The purpose of playtesting isn't to find technical problems or gather
marketing data. It is to understand how the game design works when put
into action. It means getting real people to play the game and watching
where the design works and where it fails. Where are players confused?
Where is it too easy or too hard? Is it balanced? Are there degenerate strate-
gies? Do players understand the narrative?
Running playtests is a skill. You don't just do it—it's just as hard as
planning or building. Done well, a playtest returns the information that
designers need without much cost or effort. Done poorly, it misses critical
design flaws, wastes time, and can even actively mislead designers.
The key to getting good data is using the right
test protocol
.
A TEST PROTOCOL is a set of rules and procedures for carrying out a
playtest.
Creating a good test protocol is hard because there is no feedback
when we do it wrong. Corrupted or misleading test results often look very
reasonable. Worse, bad test protocols usually make tests go more smoothly,
not less. And a badly run test is worse than useless. Before the bad test, the
designer knew he didn't know whether the game worked. Afterward, he
