Automatically Evaluating the Quality of Contents Created by Open Collaborative Knowledge Building - Collaborative Technologies and Applications for Interactive Information Design - page 141

Table 1. Basic descriptive statistic of the wiki

Number of registered members who contributed content to the wiki at least once 59

Number of wiki pages created 143

Average number of contribution per page 3.7

Median of the number of contribution 4

Maximum number of contribution to a wiki page 18

Minimum number of contribution to a wiki page 1

Length of the shortest page (in number of words) 38

Length of the longest page (in number of words) 431

There was no way to tell whether a student has
registered and contributed contents to the wiki or
not. Participation was voluntary and would have
no effect on students' grades. Students could also
use the wiki without participation (i.e., as a non-
member who can view and search and read, but
cannot change the content.)
The pilot project was ended after five months.
About one third of the students (59 out of 153)
contributed contents to the wiki. When the project
ended, there were 143 wiki pages. All pages were
created by students (except the main entrance
page). Most of them were create collaboratively
in the sensed that they have been added and edited
by multiple members across time. Table 1 sum-
marizes the statistics of this wiki.
After the semester, students who have received
A+ grade were invited to judge the quality of the
wiki pages. Eight students participated. Each
wiki page was examined by two students who

have received A+ in the corresponding course
(corresponding to the subject area of the page).
All wiki log information and statistics, including
username, were not given to the judges. The judges
only had the contents of the pages and nothing
else to make the judgment. The judgments were
based on seven quality criteria whenever appli-
cable (see table 2, which is a modification based
on Ng et al 2006).
If a page was judged as a good quality page, it
would receive a quality score 1, otherwise, it would
receive a quality score 0. It is understandable that
sometimes students would not agree with each
other in their quality judgment (Lee et al , 2006).
If the judgments of the two judges were not the
same, the instructor of the class would decide the
final score. Table 3 summarizes the results of the
quality judgments.
With the quality judgments available, we
would investigate whether a statistical relation-

Table 2. Criteria of quality of wiki pages created by collaborative knowledge building.

Quality Aspect Definition

Accuracy The extent to which information is precise and free from known errors.

Objectivity The extent to which information is free from personal biases or personal preference.

Depth The extent to which the coverage and analysis of information is detailed.

Reference Authority The extent to which information is based on authoritative references.

Readability The extent to which information is presented with clarity and is easily understood.

Conciseness The extent to which information is well-structured and compactly represented.

Grammatical Correctness The extent to which the text is free from syntactic problems.