Databases Reference
In-Depth Information
Easy cases are found at the top left and bottom right. In the first, (a) we have problems that
are severe and have an easy and obvious fix, and, of course, one deals with these first. At the bottom
right, (d) these are the minor problems that are hard to fix whereas, often, one must simply accept
that they are there. An example of this might be a database configuration screen that is hard to
understand. If configuration is only done occasionally and those doing it are highly trained, then we
may simply assume they will read the documentation thoroughly on each occasion.
The cases at the top right and bottom left are where the decision on what action to take is
more difficult even as the cost-benefit is more evenly balanced. On the bottom left (c) are minor
problems that are easy to fix. Usually, it is worth dealing with these as they may mask other problems
or generally reduce users' confidence in the system. However, they will, of course, have less priority
than those at (a). Finally, at (b), there may be problems that have serious impact but are expected to
be hard to fix. These are most problematic as some action is needed, but the costs of a 'proper' fix
may be too high. In this case, the best option may be to seek workarounds that are not perfect but,
in some way, limit or mitigate the problem.
One example of this last case (b) occurred in the online booking system of a large UK hotel
chain. Web-based systems often have problems when users press 'back' in the middle of an interaction
or use history, especially when viewing the confirmation page after completing a web form. In the
case of this hotel, the effect was that the user could end up duplicating a booking. There are various
ways this can be solved, but they often require a substantial re-engineering of the code. Instead of
doing the 'proper' fix, they changed the system to simply refuse to create two identical bookings on
the same day. This meant that if you did want to topic two rooms in the same hotel, you needed to
either do it as a single transaction, or wait a day for the second booking. However, this was a rare
occurrence; the page that rejected the duplicate booking explained the cause, and it was a lot less
damaging than accidentally repeated bookings.
Of course, the picture is somewhat simplistic. Severity is a multi-dimensional concept, which
includes the likelihood of these particular problems: the number and kind of users who may en-
counter it (avoiding giving the CEO headaches) and the impact of the problem (death or minor
inconvenience). The label 'cost to fix' also assumes one can assess this cost, and as in any area, this
is a combination of experience and judgement. However, while these are not metric scales, it is still
often not hard to rank issues against both these axes and so get some sense of the best place to focus
effort.
Furthermore, this also assumes that we know what needs to be fixed. Finding problems is also
a matter of the cost-benefit trade-off.
To find errors in a system, one of the most common ways is to try a prototype out with real users.
This can be a costly business, both for the analyst doing the study and for the users who will need to
stop doing their normal jobs while taking part in the study. Some years ago
Nielsen and Landauer
(
1993
) studied a number of large software projects in order to assess the optimal number of users
to study in each iteration of a design. They found that there were diminishing returns in studying
extra users, as many problems uncovered by a new user had already been uncovered by earlier users
