The Semantics of Tagging - Social Semantics: The Search for Meaning on the Web - page 122

5.3.2 Experimental Design

What unifies all of these models is that they assume that imitation, usually assumed
to be tag suggestions from the tagging system, has a major impact on the emergence
of a power-law distribution. With concern to the modified Yule-Simon model and the
more highly parametrized model that takes into account 'background knowledge,'
different claims are made of where the imitated tags come from. Cattuto (2006)
propose that they come from a random uniform distribution of tags while Dellschaft
and Staab propose a more topic-related distribution that itself has a power-law
distribution (Dellschaft and Staab 2008). However, just because a simple model
based on imitation of tag suggestions can lead to a power-law distribution does
not necessarily mean that tag suggestions are actually the mechanism that causes
the power-law distribution to arise in tagging systems. The research question posed
then is whether or not the tag suggestion mechanism is the main force behind the
observed power-law distributions in tagging systems.
In order to measure the effects of tag suggestions on the tag behavior of users we
developed a Web-based experiment in which users were asked to tag 11 websites,
with two varying conditions: the 'tag suggestion' condition (Condition A) in which
7 tag suggestions were presented to the user, and a 'no tag suggestion' condition
(Condition B) in which no tag suggestions were presented to the user. In this
experiment we focus on del.icio.us again, as del.icio.us was the first to introduce a
tag based collaborative bookmark system. The user interface used in our experiment
presented the tag suggestions in a similar way to del.icio.us to avoid confusion.
The 11 websites used in the experiment were selected according to two criteria.
First, the topics of the web-pages needed to appeal to the general public. Second,
the website needed to have over 200 tagging instances. The appeal to the general
public was operationalized by randomly choosing sites that were tagged with the
tag 'lifestyle' on del.icio.us. The tag “lifestyle” is a popular tag with 72,889 tagged
web-pages as of October 2008. This was chosen in order to not bias our study to
one particular specialized subject matter, and so exclude web-pages on del.icio.us
that have a highly technical content. Specialized content may not lead to normal
tagging behavior from users in the experiment who might not be familiar with the
specialist subject matter. The second criteria, of using only web-pages with over
200 tagging instances, was chosen since it has been shown that stable power-law tag
distributions emerge around the 100-150th tagging (Golder and Huberman 2006).
We did not want the tag suggestions to be from non-stable tag distributions, as it has
been shown that the variance between the top popular tags could vary widely before
100-150th tag. 11 web-pages were selected for this experiment, with the popular
tags provided from del.icio.us and the number tags. Note that while the number
of URIs in the experiment (11) may appear to be small, it is larger than previous
experiments over tag suggestions (Suchanek et al. 2008) and will be shown to be
sufficient to provide statistically significant results. It was far more critical for this
experiment to get enough subjects in order for power-law distributions to be given
the chance to arise without tag suggestions, and this would require at least 100
experimental subjects tagging each URI.