Information Technology Reference
In-Depth Information
6.4 Discussion
The purpose of this chapter has been to investigate the ability of topic model algo-
rithms to identify the quality of explanations as well as specific reading strategies
in comparison to word-based and LSA-based algorithms. We found in Experiment
1 that TM systems performed comparably to the combined systems, though not
quite as well. In Experiment 2, we found that the TM models performed nearly
as well as the combined system in identifying specific strategies. These results thus
broaden the scope of NLP models that can be applied to problems such as ours —
providing real-time feedback in a tutoring environment. Indeed, the performance of
both systems in Experiment 2 was highly encouraging. These results indicate that
future versions of iSTART will be able to provide specific feedback about reading
comprehension strategy use with relatively high confidence.
Our future work with the TM systems will be to attempt to combine the TM
algorithms with the LSA and word-based algorithms. To venture toward that goal,
we need to first identify the strengths of the TM algorithms so that the combined
algorithm capitalizes on the strengths of the TM — much as we did when we created
the combined word-based and LSA-based system. This will require that we analyze
a greater variety of protocols, including self-explanations from a greater variety of
texts and text genres. We are in the process of completing that work.
These NLP theories and their effectiveness have played important roles in the
development of iSTART. For iSTART to effectively teach reading strategies, it must
be able to deliver valid feedback on the quality of the self-explanations that a student
types during practice. In order to deliver feedback, the system must understand,
at least to some extent, what a student is saying in his or her self-explanation. Of
course, automating natural language understanding has been extremely challenging,
especially for non-restrictive content domains like self-explaining a text in which a
student might say one of any number of things. Algorithms such as LSA opened up
a horizon of possibilities to systems such as iSTART — in essence LSA provided a
'simple' algorithm that allowed tutoring systems to provide appropriate feedback to
students (see [14]). The results presented in this chapter show that the topic model
similarly offers a wealth of possibilities in natural language processing.
6.5 Acknowledgments
This project was supported by NSF (IERI Award number: 0241144) and its contin-
uation funded by IES (IES Award number: R305G020018). Any opinions, findings
and conclusions or recommendations expressed in this material are those of the
authors and do not necessarily reflect the views of NSF and IES.
References
1. Birtwisle,
M.
(2002)
The
Soundex
Algorithm.
Retrieved
from:
http://www.comp.leeds.ac.uk/matthewb/ar32/basic soundex.htm
2. Bransford, J., Brown, A., & Cocking, R., Eds. (2000). How people learn: Brain,
mind, experience, and school. Washington, D.C.: National Academy Press. On-
line at: http://www.nap.edu/html/howpeople1/
Search WWH ::
Custom Search