Personalized Information Access Using Semantic Knowledge - Smart Information Systems: Computational Intelligence for Real-Life Applications - page 199

7.5.3 Selection of 'New Users' and Evaluation Algorithms

The selection of users who represent the new users joining a recommendation service
was done by creating a subset with all users who have rated exactly 15 items. All
test users must have the same number of items in the profile, to be able to compare
the results of the different evaluation runs in the different scenarios. The number
15 was chosen because it gives a large enough user profile for evaluation (train and
test split) and also enough users with 15 items to have statistically enough test data.
From these 15 items we use a set of 10 items as our test set and for training we
arbitrarily choose 1-5 of a user's remaining items for the initial user profile (training
set) to simulate the cold start problem. The process is visualized in Fig. 7.16 .We
conduct several test runs, starting with a user profile containing only one item, and
then iteratively increase the number of items up to five. The training set is enriched
with an additional five-nine items, depending on the initial size of the training set, so
that it always contains ten items. Results are averaged over 200 evaluation runs for
each user profile size (one-five items) using the following forms of CF algorithms:

• CF with standard profiles : The Baseline. A standard CF algorithm using the

Tanimoto coefficient [ 21 ] to compute user similarity.

• CF with enriched profiles : The standard CF algorithm using the enriched user

profiles instead of the standard profiles.

• Most Popular Recommender : A simple algorithm recommending the top n items

of the dataset.

• +

CF enriched profiles : A combined method of the first two CF methods. If the

standard CF does not find a recommendation, CF with the enriched profiles is
used. This approach avoids the recommendation depending mostly on the items
used to enrich the profile.

• CF Most Popular recommendations : An approach using most popular rec-

ommendations if the standard CF find no results.

+

• Random Recommender : Recommending randomly chosen items.

Fig. 7.16 Example of a user profile with 15 items. First, a training test split is done with ten test
items and five items in the training set. Then we enrich the user profile with five additional items.
This enriched set is then used for CF