Information Technology Reference
In-Depth Information
6
Evaluating Self-Explanations in iSTART:
Word Matching, Latent Semantic Analysis,
and Topic Models
Chutima Boonthum, Irwin B. Levinstein, and Danielle S. McNamara
6.1 Introduction
iSTART (Interactive Strategy Trainer for Active Reading and Thinking) is a web-
based, automated tutor designed to help students become better readers via multi-
media technologies. It provides young adolescent to college-aged students with a pro-
gram of self-explanation and reading strategy training [19] called Self-Explanation
Reading Training, or SERT [17, 21, 24, 25]. The reading strategies include (a) com-
prehension monitoring, being aware of one's understanding of the text; (b) para-
phrasing, or restating the text in different words; (c) elaboration, using prior knowl-
edge or experiences to understand the text (i.e., domain-specific knowledge-based
inferences) or common sense, using logic to understand the text (i.e., domain-general
knowledge based inferences); (d) predictions, predicting what the text will say next;
and (e) bridging, understanding the relation between separate sentences of the text.
The overall process is called “self-explanation” because the reader is encouraged to
explain di cult text to him- or herself. iSTART consists of three modules: Intro-
duction, Demonstration, and Practice. In the last module, students practice using
reading strategies by typing self-explanations of sentences. The system evaluates
each self-explanation and then provides appropriate feedback to the student. If the
explanation is irrelevant or too short, the student is required to add more informa-
tion. Otherwise, the feedback is based on the level of overall quality.
The computational challenge here is to provide appropriate feedback to the stu-
dents concerning their self-explanations. To do so requires capturing some sense of
both the meaning and quality of the self-explanation. Interpreting text is critical for
intelligent tutoring systems, such as iSTART, that are designed to interact meaning-
fully with, and adapt to, the users' input. iSTART was initially proposed as using
Latent Semantic Analysis (LSA; [13]) to capture the meanings of texts and to assess
the students' self-explanation; however, while the LSA algorithms were being built,
iSTART used simple word matching algorithms. In the course of integrating the
LSA algorithms, we found that a combination of word-matching and LSA provided
better results than either separately [18].
Our goal in evaluating the adequacy of the algorithms has been to imitate ex-
perts' judgments of the quality of the self-explanations. The current evaluation sys-
tem predicts the score that a human gives on a 4-point scale, where 0 represents an
Search WWH ::




Custom Search