Generating Comprehension Questions Using Paraphrase - Technologies and Applications of Artificial Intelligence - page 311

Several research have noted the problem caused by the same wording between the
generated questions and their source counterparts. Afzal and Mitkov [1] brought up
the concern that generating approaches which concentrate on sentence-to-question
transformation, are likely to result in questions that could only evaluate test takers'
superficial memorization. They solve this problem by generating questions based on
semantic relations which are extracted using information extraction methodologies.
Bernhard, De Viron, Moriceau and Tannier [3] approached the problem by using two
of the many paraphrase skills. They specify the question words and nominalize the
verbs. E.g., from “ Where has the locomotive been invented? ” to “ In which country
has the locomotive been invented? ” and “ When was Elizabeth II crowned? ” to “ When
was the coronation of Elizabeth II? ”. On the other hand, Heilman and Smith [7] have
developed sentence simplification for question generation based on syntactic rules.
Although their work is intended to generate more concise questions, their simplifica-
tion technique is also contributing to making surface difference. Our work is similar
in intentions with most of these work, but paraphrase generation have never been
systematically incorporated into these QG systems.
The Penn Discourse Treebank (PDTB) [18] is a large scale corpus based on some
early work of discourse structure and is annotated with related information of dis-
course semantics. A discourse relation captures two pieces of information and the
logical relationship between them. Prasad and Joshi [19] evaluated the feasibility of
using discourse relations in the content selection of why -questions. They showed that
the source of 71% of the questions in an independent why question answering data set
could be found in the same PDTB subset with a marked causal discourse relation.
Agarwal, Shah and Mannem [2] followed the proved idea and used discourse cues
(e.g., because , as a result ) as an indicator of question type to generate why -questions
and other question types based on temporal, contrast, concession and instantiation
relations. These work suggest the usefulness of discourse relations in QG. While they
use discourse relations in satisfying the form of certain question types, our work take
advantage of discourse relations in the generation of comprehension questions and the
development of distractors.

3 Approach

In this section, we introduce our approach to generate more challenging choices, or
statements, for multiple-choice reading comprehension questions. To generate super-
ficially different statements, our intuition is to rewrite with the four basic actions: to
rephrase, to reorder, to simplify and to combine. Most paraphrase generation systems,
in practice, are inclined to rephrase more often than to simplify or to combine because
they do not paraphrase recursively. We improve this by incorporating the structural
paraphrases into the design of choice generation rules.
The overall system consists of two sub-systems and a ranker, as shown in Fig. 2.
The arrows represent the flows of the generating process and ideally, all these flows
should work to satisfy different demand of test choices. In the experiment of this
work, only the flow that visits the three components in the order of left to right, from