Information Technology Reference
In-Depth Information
determines any choice candidate as either acceptable or unacceptable. In the final
evaluation, we conduct an experiment with the baseline system and show the effect of
our approach on quality and on difficulty.
The remainder of this paper is organized as follows. Section 2 introduces closely
related QG work and explains how our work differs. The generation and ranking of
choice candidates are illustrated in Section 3. We do not reveal much implementation
detail in this paper due to the page limit, yet any interested reader is referred to [21].
Section 4 gives the setup and the results of the experiments that evaluate our output
statements. Finally, in Section 5, we conclude this paper and list possible future work.
2
Related Work
Question Generation (QG) is the task of automatically generating questions from
some form of input [20]. When it comes to language learning assessment, automated
question generation research are more on grammar and vocabulary. Little work have
claimed themselves as aiming at reading comprehension assessment. Mostow and
Jang [16] introduced DQGen, a system that automatically generates multiple-choice
cloze questions to assess children's reading comprehension. They proposed to diag-
nose three types of comprehension failures by different types of distractors-
grammatical, nonsensical and plausible distractors. In our work, we avoid generating
choices that are ungrammatical or do not make sense because, to higher-level learn-
ers, they would appear to be obviously wrong choices even without the need to take a
look at the article. Heilman [6] proposed a syntactic-based approach to generate fac-
tual questions, or wh-questions, from which teachers could select and revise useful
ones for future use. In these years, many work (such as [17]) take advantage of do-
main ontology to create assessments. The generated questions, however, are not based
on any input text and are more suitable to test domain-specific knowledge, like the
quizzes in science classes.
Generating choices are, partially, equivalent to generating distractors. There is no an-
swer generation in the past because words/phrases in the source sentences of the ques-
tions are directly used as answers. Existing distractor generators, as noted by Moser, Gütl
and Liu [15], mainly consider single-word choices, or they generate multi-word or phras-
al distractors by applying simple algorithms. Mitkov and Ha [14] select multi-word con-
cepts that are similar to the answer from WordNet [13] as distractors and if this fails,
phrases with the same head as the answer are selected from a corpus as substitutes. Mos-
er et al. [15] extract key-phrases that are semantically close to the answer as distractors,
using LSA for their similarity calculation. Afzal and Mitkov [1] generate distractors for
biomedical domain based on distributional similarity. The similarity score is calculated
between the answer named entity, which are possibly multi-word, and each candidate
from a set of biomedical named entities. The higher scoring ones are more desirable dis-
tractors. Different from these approaches, we focus on generating sentential choices.
While a small part of our generating approaches is similar to the secondary approach in
[14], our approach to generate both answers and distractors via recombination of dis-
course segments and relations is novel.
Search WWH ::
Custom Search