Information Technology Reference
In-Depth Information
Figure 5.5
Spatial-Visual Depiction of Electric Circuit Scenario
ten questions presented either as visibility diagrams or as expressive text, again as directed by the
experiment's design.
To minimize any ordering effect, the ten questions in each treatment combination were counter-
balanced with each question presented in each order position once. This resulted in ten different pri-
mary orderings of the ten questions in each treatment. Additionally, it was determined statistically
that each of the ten questions presented were of the same level of difficulty. Each question was dis-
played and data collected using Dell II machines with 17
monitors. The application was written in
Assymetrix's ToolBook 4.0. An example of the display used in the study is shown in Figure 5.5.
For both the expressiveness and visibility problem representations, the problems were dis-
played on the left half of the screen and the potential answers on the right half of the screen.
Figure 5.5 shows an example using the visibility diagram for the electric circuit scenario. The
subject indicated his or her choice by clicking on the diamond next to the alternative. A v was
placed in the diamond to indicate the selection. The subject used the slider just below each alter-
native to report his or her belief in the correctness of the selected alternative and other alternatives.
The corresponding numerical value was displayed in the window to the left of the slider, and the
sum of the four values was displayed in the Total window at the bottom of the screen. If the sum
was not 100
.05, the subject was instructed to adjust the inputs. To proceed to the next question,
the subject clicked on the arrow
→
at the bottom of the screen. This action also recorded the ques-
tion data. By clicking on the
arrow, the subject could return to the prior question. After answer-
ing all ten questions, the subject was debriefed, thanked, and dismissed.
←
Measuring User Calibration
To measure user calibration requires selecting a method and means for recording both decision
quality and the subject's belief in the quality of each decision, a scoring rule and procedure that
Search WWH ::
Custom Search