Information Technology Reference
In-Depth Information
between human learners and computer tutors (Graesser et al., 2000; McNamara
et al., 2007). LSA is used within iSTART to calculate the similarity between
the student's explanation and a collection of words representing three different
parts of the passage being explained: the target sentence, the title, and the prior
content.
Evaluations were conducted to gauge the success of the assessment algorithms
by computing linear equations based on a discriminate analysis of one data set
and assessing its ability to predict ratings for a variety of data sets (McNamara
et al., 2004, 2007; Millis et al., 2004). The
d
-primes in these studies have ranged
from 1.0 to 3.0. Thus, across a number of evaluations, the iSTART algorithms have
corresponded well to human ratings.
Evaluations of iSTART
Empirical studies on the effectiveness of iSTART have been positive. Studies at
both the college (McNamara, 2004a; O'Reilly et al., 2004b) and the high school
levels (O'Reilly et al., 2004; 2004a; O'Reilly et al., 2006; Taylor, O'Reilly, Sinclair,
& McNamara, 2006) indicate that iSTART improves text comprehension and strat-
egy use over control groups. Several studies have further confirmed that iSTART
produces learning gains equivalent to those from the human-based SERT program
(Magliano, Todaro, Millis, Wiemer-Hastings, Kim, &McNamara, 2005; O'Reilly
et al., 2004b; 2004). It was also found that students who completed training
with iSTART are significantly more effective at using self-explanation strategies
(McNamara et al., 2006).
The effects of iSTART also depend on individual differences. One study investi-
gated the effect of iSTART on adolescent students' comprehension and strategy use
(McNamara et al., 2006; O'Reilly et al., 2004a). This study also examined whether
the students' prior knowledge of reading strategies interacted with the benefits of
strategy training (McNamara et al., 2006). Half of the students were provided with
iSTART while the students in the control condition were given a brief demonstra-
tion on how to self-explain text. All of the students then self-explained a text about
heart disease and answered text-based and bridging-inference questions. Results
indicated that both iSTART training and prior knowledge of reading strategies sig-
nificantly improved the quality of self-explanations. In turn, the quality of students'
explanations of particular sentences was directly related to improved comprehension
scores on questions tapping those sentences. In addition, it was found that benefits
of reading strategy instruction depended on prior reading strategy knowledge. For
low-strategy knowledge participants, the effects of iSTART were more pronounced
at the more literal text-based level. Conversely, for high-strategy knowledge stu-
dents, the effects of iSTART were evident on more difficult and integrative bridging
inference questions.
Magliano et al. (2005; Experiment 2) found a similar pattern of results when they
investigated whether and how the benefits of iSTART depended on the students'
