Information Technology Reference
In-Depth Information
the practical inquiry model. As the coders gained experience, the conceptual bound-
aries of the practical inquiry model became better understood by the coders. Well
documented in the literature on content analysis is the need to train coders for this
reason.
However, it is also important to acknowledge that an alternative explanation is
also possible. Krippendorf (2004) (see also Rourke & Anderson, 2004) observed
that when coders participate in the conceptual development of the boundaries for
categories, it becomes difficult to determine whether they have gained greater
proficiency at coding or, instead, have developed a new, group-specific unwritten
consensus concerning what is expected of them. Hence, it is important to acknowl-
edge that during the content analysis process boundaries may have been shifted
by the coders until their meanings could accommodate what they were able to
code with reliability and ease. Cognizant of this potential problem, an attempt to
reduce the effects of boundary shifting between coders was achieved by having the
coders code independently of each other. However, the coders did come together and
debrief with the researcher once a week to discuss issues and problems during the
analysis process and, as such, the alternative explanation for the satisfactory inter-
rater reliability with the ensuing instructional methods could have been the result of
conceptual boundary shifting, rather than greater coding proficiency by the coders.
Ecological Validity of the Coder's
Potter and Levine-Donnerstein (1999) maintain that when conducting a content
analysis it is important to also demonstrate ecological validity. To achieve ecolog-
ical validity, coders should come from the same social community as the eventual
consumers of the findings. Acquiring ecological validity is important because the
value of a study that uses content analysis (or any study for that matter) lies in how
well the results resonate with the research consumers' experiences. In particular,
even though the inter-rater reliability can provide credibility to content analyses, it
does not tell the readers whether the patterns and observations found by the coders
are likely to be the same ones that the readers would find. As such the coders for this
study were selected because of their ecological validity: they were mature students
in higher education and educational technology disciplines and had participated in
online courses using text-based Internet communication tools. These characteristics
are also characteristics of the consumers of this research and, hence, one can be
reasonably certain that the coders' had ecological validity.
Discussion
This chapter explored the relationship between instructional methods and patterns
of cognitive presence. The data revealed that students engaged in a WebQuest pro-
duced more messages that reflected the highest levels of cognitive presence than
Search WWH ::




Custom Search