Unformatted text preview: stically-signi cant di erence in performance from 40 examples onward. For the SF
results on average rating of the top 3, there is a statisticallysigni cant di erence at 10, 100, 150, 200, and 450 examples.
The results shown are some of the most consistent di erences for each of these metrics; however, all of the datasets
demonstrate some signi cant advantage of using collaborative content according to one or more metrics. Therefore, information obtained from collaborative methods can be used
to improve content-based recommending, even when the actual user data underlying the collaborative method is unavailable due to privacy or proprietary concerns. 90 20 0 Figure 3: 100 % Precision Top 10 3
LIBRA-NR 0.1 SF 4 350 400 450 Precision at Top 10 after 450 examples. 3.3 Results on the Role of Collaborative Content
Since collaborative and content-based approaches to recommending have somewhat complementary strengths and
weaknesses, an interesting question that has already attracted
some initial attention 3, 4 is whether they can be combined to produce even better results. Since Libra exploits
content about related authors and titles that Amazon produces using collaborative methods, an interesting question is
whether this collaborative content actually helps its performance. To examine this issue, we conducted an ablation"
study in which the slots for related authors and related titles
were removed from Libra's representation of book content.
The resulting system, called Libra-NR, was compared to
the original one using the same 10-fold training and test
sets. The statistical signi cance of any di erences in performance between the two systems was evaluated using a 1tailed paired t-test requiring a signi cance level of p 0:05.
Overall, the results indicate that the use of collaborative content has a signi cant positive e ect. Figures 1, 2,
and 3, show sample learning curves for di erent important
metrics for a few data sets. For the Lit1 rank-correlation 4 FUTURE WORK
We are currently developing a web-based interface so that
Libra can be experimentally evaluated in practical use with
a larger body of users. We plan to conduct a study in which
each user selects their own training examples, obtains recommendations, and provides nal informed ratings after reading one or more selected books.
Another planned experiment is comparing Libra's contentbased approach to a standard collaborative method. Given
the constrained interfaces provided by existing on-line recommenders, and the inaccessibility of the underlying proprietary user data, conducting a controlled experiment using
the exact same training examples and book databases is difcult. However, users could be allowed to use both systems
and evaluate and compare their nal recommendations.2
Since many users are reluctant to rate large number of
training examples, various machine-learning techniques for
maximizing the utility of small training sets should be utilized. One approach is to use unsuperv...
View Full Document
- Fall '13
- Machine Learning, Spearman's rank correlation coefficient, Recommender system, training examples