Database Reference
In-Depth Information
Table 1.
Type I and Type II Errors
Reality
H
0
TRUE
H
0
FALSE
H
0
TRUE
ok
Type II
Experimental
decision
H
0
FALSE
Type I
ok
Type I and Type II Errors:
If one is interested in which visualization technique
VisA or VisB helps people conduct a particular task faster one might formulate a null
hypothesis (H
0
) -
there is no difference in speed of search between VisA and VisB
.
The possible type I, false negative, and type II, false positive, errors are specified in
Table 1. The columns represent whether the null hypothesis is true or false in reality
and the rows show the decision made based on the results of the experiment. Ideally
the results of the experiment reflect reality and that if the hypothesis is false (or true)
in reality it will show as false (or true) in the experiment. However, it is possible that
the hypothesis is true in reality - VisA does support faster search than VisB - but that
this fails to be revealed by the experiment. This is a type II error. A type I error occurs
if the null hypothesis is true in reality (there is no difference) and one concludes that
there
is
a difference. Type I errors are considered more serious. That is, it is consid-
ered worse to claim that VisA improves search when it does not, than to say there was
no measurable difference.
Internal Validity: Is the relationship causal?
This concept is important when an
experiment is intended to reveal something about causal relationships. Thus, internal
validity will be important in our simple example because the study is looking at what
effect VisA and VisB have on search. The key issue here is whether the results of
one's study can properly be attributed to what happened within the experiment. That
is, that no other factors influenced or contributed to the results seen in the study. An-
other way of asking this question is: are there possible alternate causes for the results
seen in the study?
Construct Validity:
Can we generalize to the constructs (ideas) the study is based
on?
This concept considers whether the experiment has been designed and run in a
manner that answers the intended questions. This is an issue about whether the right
factors are being measured or whether the factors the experimenter intends to measure
are actually those being measured. For instance, does the experiment measure the
difference due to the techniques VisA and VisB or the difference in participant's
familiarity with VisA and VisB. For instance, if the construct is that a person will
have higher satisfaction when using VisB, does measuring error rates and completion
times provide answers for this construct? An important part of this concept of con-
struct validity is
measurement validity
. Measurement validity is concerned with
questions such as is one measuring what one intends to measure and is the method of
measurement reliable and consistent. That is, will the same measurement process
provide the same results when repeated?
Search WWH ::
Custom Search