Automatically Evaluating the Quality of Contents Created by Open Collaborative Knowledge Building - Collaborative Technologies and Applications for Interactive Information Design - page 140

Figure 2. The main page of the experimental wiki. It was the only page when the pilot study started. All
other pages and categories were added and modified by students.

Desilets 2005, Kohl 2008). Collaborative knowl-
edge building using wiki always comes with the
problematic dimension of quality.
Information and data quality is not a new
topic. The term “Quality” is defined by Interna-
tional Organization of Standards (1986) as “the
totality of characteristics of an entity that bear
on its ability to satisfy stated and implied need”
(Standard 8402, 3.1). There has been considerable
research on information quality (e.g., see Wang
and Strong 1996, Eppler & Wittig 2000, Helfert
2001, and Naumann 2002). We can also implement
automatic mechanisms to estimate the quality of a
document created in a static and non-collaborative
environment (e.g., see Ng et al 2003, Tang et al
2003 and Ng et al 2006). The advance of open
collaborative knowledge building systems posts
new questions in this area: what affects the qual-
ity of an information product when it is produced
collaboratively in a distributed environment? Can
we also estimate the quality of content built in this
kind of collaborative writing environment using
automatic means? In this paper, we report a pilot
study that was designed to answer these questions,

discuss and analyze its very promising results.

exPerimentAl setuP And
method oF inquiry

The appearance of Wiki provides new oppor-
tunities for learning more of the collaborative
knowledge building process (Cress & Kimmerle
2008). In this pilot study, a MediaWiki clone was
installed in a Linux server. Students in five master
level courses would use this wiki as a learning tool
to foster their understanding of key concepts and
topics covered in the class.
In the beginning, there was only one page in
the wiki (i.e. the main entrance page, see Figure
2). After introducing to students about the wiki
and how to contribute content, students were en-
couraged to create, edit, modify, categorize and
link wiki pages collaboratively.
All contributed pages must be about topics
covered in the classes. Only registered member
could contribute contents, but users could register
under usernames and participate anonymously.