Databases Reference
In-Depth Information
#
Pearson Corr
Std Dev
1
tagcloud
0.506
1.570
2
0.504
1.661
perstagcloud
3
confidence
0.243
1.904
4
0.239
2.002
piechart
5
0.231
1.867
neighborsrating
6
average
0.226
1.772
7
0.193
1.903
barchart
8
clusteredbarchart
0.192
2.071
9
rated
4+
-0.049
1.960
10
-0.053
2.182
neighborscount
Table 6.4: Pearson correlation values between explanation ratings and actual ratings and standard devi-
ation values of the mean differences.
[Tintarev and Masthoff, 2012]. In their work, which was developed in parallel to our own work, the
authors observed in three experiments in two domains that their method of personalization hindered
effectiveness, but increased satisfaction with explanations; this also holds for the tag cloud interfaces
evaluated in our study. We will discuss the results for satisfaction later on in Section 6.4.4. We believe
that one reason for the difference regarding effectiveness might be the fact that we relied on estimated
tag preferences in this study. In future work, alternative heuristics to estimate the tag preferences can
be used as to further increase the effectiveness of the personalized interface.
Regarding the aspect of persuasiveness, explanation interfaces which lead to mean values above 0 in
Figure 6.6 cause the user to overestimate the actual rating (and real value of an item). Such interfaces
could be used in situations where an item should be promoted. The
rated4+
interface, for example, causes
the user to overestimate the actual rating by 0
.
89 on average, which we consider to be comparably high
given our 7-point rating scale. Analogously, the explanations below 0 cause the user to underestimate
the quality or value of an item.
In some domains such as finance and tourism, using interfaces that lead to an overestimate of an item's
quality could be risky. In the long term, a recommender system that tries to continuously persuade the
customer toward certain items may leave the user sooner or later with the impression that the system is
cheating because the system overstates the advantages or value of the items. Thus, we assume it to be
a comparably “safe” strategy, when we are able to keep the persuasiveness level within a certain range
or slightly on the negative side. Tintarev and Masthoff, for instance, found that users perceived overes-
timation to be less helpful than underestimation [Tintarev and Masthoff, 2008b]. Overall, we argue that
it is important to choose the direction of the persuasiveness depending on the current recommendation
scenario and goals as well as depending on the general business strategy.
Our study shows that the newly proposed tag cloud interfaces are among the most effective explanation
interfaces which at the same time have a very small tendency to cause the user to over- or underestimate
the real value of a recommended item.
The tag cloud interfaces are the only interfaces in our study which make use of domain specific content
information. We consider this as being an indication that users are able to evaluate explanations based
on content information more precisely. Note that among the best performing methods in the study
by [Herlocker et al., 2000] was the “Favorite actor or actress” interface, which also presents content
information to the user. By providing content data users are able to take advantage of their personal
experience and knowledge in the considered domain. Thus, they are able to estimate the quality, value,
or relevance of an item more precisely. Overall, our design suggestion is therefore:
Guideline 1.
Use domain specific content data to boost effectiveness.
Since previous studies [Herlocker et al., 2000; Bilgic and Mooney, 2005] did not differentiate between
explanations with and without content data, content-based explanations were not specifically chosen in
the selection process in Section 6.2 either. We see the systematic analysis of the effects of different
amounts and types of content data in explanations as one of the next steps in future work.
