DECISIONAL GUIDANCE - Human-Computer Interaction and Management Information Systems: Foundations - page 96

given class of guidance—for example, suggestive guidance? Or can we draw conclusions only about
specific guidance mechanisms? Generalizing is always a challenge, but it looks to be especially dif-
ficult in the arena of decisional guidance.
Consider research on decision support systems, which faces a similar problem. The studies of
whether or not DSS improve decision making have shown mixed results (Sharda et al., 1988;
Benbasat and Nault, 1990). A number of explanations have been offered for the conflicting find-
ings, and there may be various mediating variables, but when two different studies with two dif-
ferent decision aids reach two different conclusions, might the explanation simply be that one aid
is helpful and the other is not? Put differently, might each study be making a statement not about
computer-based decision aids generally but about the particular aid? Researching DSS at any
level of generality is a challenge because there is such a wide variety of decision-aiding technol-
ogy. Of course, two conclusions can be drawn from the conflicting studies: (1) In some cases,
DSS help. (2) In some cases, they do not.
Guidance mechanisms present the same problem, since so many different mechanisms can be
constructed. Of the nine empirical studies, only two studied essentially the same mechanism. The
other mechanisms all varied greatly, even when they belonged to the same cell of the typology.
Indeed, Montazemi et al. (1996) compared guidance for two types of tasks within a single study,
but task differences necessitated studying a different guidance mechanism for each task.
Given the diversity of guidance mechanisms, what does it mean when a study concludes that
“users of the system with decisional guidance outperformed those of the one without”? The state-
ment is not that decisional guidance always, or even often, outperforms. The statement is about the
one instance of guidance examined in the study. To what extent, then, is the conclusion about guid-
ance in general and to what extent is it only about a given guidance mechanism? Suppose we
designed a terrible guidance mechanism—for example, suggestive guidance that systematically
makes poor recommendations. How meaningful would it be to claim that non-guidance outper-
formed guidance? Indeed, claiming that decisional guidance outperformed non-guidance is a state-
ment about the possible: Constructing guidance that outperforms is possible. As such the conclusion
has value, validating that the invention of guidance mechanisms and the study of their effects are
worthwhile pursuits. But it does not provide a generalized answer to any of the questions.
What should be done? First, we should acknowledge that studying isolated guidance mecha-
nisms for their own sakes and not with the intent to generalize has value. A given guidance mech-
anism that performs well in one system may be applicable to a variety of situations. Any promising
mechanism needs to be understood well, not only so that others can implement it appropriately, but
also to improve upon it or to refine it for other environments. Researchers must describe clearly the
features of the guidance mechanisms they study—clearly enough that someone else can implement
the same features in another system. This requires sharper and more detailed descriptions than we
often find today. Moreover, comparing variants of a given guidance mechanism is valuable for
improving the mechanism. In some cases, these variants might all reside within the same cell of the
typology. In other instances, differences across cells might motivate alternative implementations of
a guidance mechanism. For instance, the same guidance might be offered automatically or on
demand. In short, inventing and studying guidance mechanisms are valuable activities as they add
to the collection of mechanisms available.
But what about generalizing? Generalizing is an important research activity. Broadly applica-
ble findings can guide researchers trying to invent broadly applicable guidance mechanisms.
Broad conclusions can also benefit practitioners building guidance into their systems. But the
very nature of the typology suggests that few findings will apply to all forms of guidance. In many
ways, decisional guidance—even just the deliberate variety, as in the typology—is an umbrella