Trust in Online Collaborative IS - Community-Built Databases: Research and Development - page 358

Table 14.4 Precision and
recall provided by low
reputation users for short-
lived text

Models Precision Recall

Ignoring anonymous users [ 37 ] 0.058 0.378

Considering anonymous users [ 37 ] 0.190 0.904

Model 3 0.404 0.975

shows the results obtained using Model 3, the most similar of our models to their
model, to estimate reputations in English Wikipedia up to 2007 and measure
precision and recall on the same data. As the table shows, the model by Adler
et al. [ 37 ] performs better when a reputation is assigned to anonymous users, albeit
statically. Model 3 significantly outperforms the other two approaches because of
dynamic assignment of reputation to anonymous users, better token ownership
assignments, and also effective removal of side effects of reverts.

14.4 Measuring Quality Evolution of Wikipedia Articles

Since Wikipedia is a dynamic system, the articles can change very frequently.
Therefore, the quality of articles is a time-dependent function and a single content
may contain high- and low-quality content in different periods of its lifetime. The
goal of our study is to analyze the evolution of content in articles over time and
estimate the fraction of time that articles are in high-quality state.
In our analysis of the evolution of the content quality in Wikipedia articles, we
separate revisions into low- and high-quality revisions. On the basis of this assump-
tion, an article can be in low-quality ( q 1) states. In order

to assess the quality q of a revision, we take into account two factors: the reputation
of the author and whether this revision has been reverted in one of the subsequent
revisions or not. The reputation of a contributor is a value between 0 and 1 and can
be viewed as the probability that he/she produces a contribution of high quality.
This probability is computed based on the stability of the past contributions of the
user using the methods developed in [ 36 ]. The heuristic behind this reputation
assessment is that high-quality contributions tend to survive longer in the articles as
compared to low-quality contributions. This heuristic is also supported by other
work [ 37 , 53 ].
As Fig. 14.5 suggests, submission of a new revision can keep the state of the
article or move it to the other state. If the revision is reverted later in the article
history, we consider the new state of the article to be q

¼ 0) or high-quality ( q ¼

0. Otherwise, if the
reputation of the author of that revision is r , then with probability of r the new
revision will be q

¼

0.
With all these elements in place, we define Q ( T ) as the ratio of high-quality
revisions submitted for the article up to time T :

¼ 1 and with probability of 1 r the new revision will be q ¼

X

n

Q ð T Þ¼ q ð t i Þ= n (14.5)

i¼ 1