Background: Corpora and Evaluation Methods - Methods for Mining and Summarizing Text Conversations - page 40

Figure 2.7: Types of summarization evaluation.

2.4 CONCLUSION

In this chapter, we reviewed several, although certainly not all, of the conversation corpora used
by researchers. We also described and compared the annotation schemes applied to such corpora.
Likewise, we presented some of the most popular evaluation metrics for summarization and text
mining of conversations, but it is not uncommon for researchers to devise their own in-house
evaluation metrics—a trend that can make it difficult to exactly compare contrasting approaches in
the literature. When presenting case studies in the following chapters, we will often report evaluation
metrics based simply on what was published in the original work.

2.5 IMPORTANT POINTS

￿ Large collections of documents, often annotated, are called corpora .