Database Reference
In-Depth Information
of the population can lead to considerable generalizability. However, while choosing
the population carefully is extremely important, often it is difficult to control. For
example in a web-based survey, all returned answers are from those types of people
who are willing to take the time, fill out the questionnaire, etc. This is a type of bias
and thus reduces generalizability. Also, responses are hard to calibrate. For instance, a
particular paper reviewer may never give high scores and the meta-reviewer may
know this and calibrate accordingly or may not know this. Despite these difficulties,
much useful information can be gathered this way. We as a community must simply
be aware of the caveats involved.
Formal Theory:
Formal theory is not a separate experimental methodology but an
important aspect of all empirical research that can easily be overlooked. As such, it
does not involve the gathering of new empirical evidence and as a result is low in
both precision and realism. Here, existing empirical evidence is examined to con-
sider the theoretical implications. For example, the results of several studies can be
considered as a whole to provide a higher-level or meta-understanding or the results
can be considered in light of existing theories to extend, adjust or refute them. Cur-
rently this type of research is particularly difficult to publish in that there are no
new information visualizations and no new empirical results. Instead, the contribu-
tion moves towards the development of theories about the use of and practicality of
information visualizations.
Computer Simulation:
It is also possible to develop a computer simulation that has
been designed as logically complete. This method is used in battle simulation, re-
search and rescue simulation, etc. This type of strategy can be used to assess some
visualizations. For instance, a visualization of landscape vegetation that includes
models of plant growth and models of fire starts and spread can be set to simulate
passage of several hundred years. If the resulting vegetation patterns are comparable
to existing satellite imagery this provides considerable support for the usefulness of
models [22]. Since this type of research strategy does not involve participants, discus-
sion of generalizability over populations is not applicable. Also, since the models are
by definition incomplete, notions of precision in measurement are often replaced with
stochastic results. On the other hand it does provide a method of validation and offers
a parallel with which we can study realistic situations, such as explosions, turbulence
in wind tunnels, etc.
4
Focus on Quantitative Evaluation
Quantitative evaluations, most well known as laboratory experiments or studies, are
those methodologies in which precision is relatively high and in which some declara-
tion can be made about the possible generalization to a larger population. These dec-
larations can include information about the characterization of this larger population
and how likely it is that the generalization will hold. These types of experiments or
studies are part of the traditional empirical scientific experimental approach and have
evolved and been refined through the centuries of scientific research. Science does
and has depended on these methods. Slowly, through careful and rigorous application
of the experimental process, knowledge has been built up, usually one piece at a time.
Search WWH ::
Custom Search