Transcript Content-based Recommendations

Recommender System
Wenkai Mo
The Recommender Problem
Estimate a utility function
to predict how a user will
like an item.
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Collaborative Filtering
• The task of predicting (filtering) user preferences on new items by
collecting taste information from many users (collaborative).
• Challenges:
•
•
•
•
•
Many items to choose from
Very few recommendations to propose
Few data per user
No data for new users
Very large dataset
Collaborative Filtering – Memory-based
• User-based Similarity
• More dynamic
• Precomputing user neighbourhood can lead to
poor predictions.
• Item-based Similarity
• Static (No Personality)
• We can precompute item neighbourhood. Online
computation of the predicted ratings.
Collaborative Filtering – data sparse
• Large products set but few users rating, so the user rating matrix is
very sparse.
• Model-based Collaborative Filtering
Model-based CF
• Clustering
• Association Rules
• Matrix Factorization
• Restricted Boltzmann Machine
• Recurrent Neural Networks
Model-based CF
• Clustering
• Association Rules
• Matrix Factorization
• Restricted Boltzmann Machine
• Recurrent Neural Networks
Clustering - CF
• Cluster customers into categories based on preferences and past
purchases
• Computer recommendations at the cluster level: all customers within
a cluster receive the same recommendations.
• + Easy and faster
• - Less personalized
Association rules – CF
• Past purchases used to find relationships of common purchases.
Association rules – CF
• + Fast to implement
• + Fast to execute
• + Not much storage space required
• + Very successful in broad applications for large populations, such as
shelf layout in retail stores
• - Not suitable if preferences change rapidly
• - Rules can be used only when enough data validates them. False
associations can arise
Matrix Factorization – CF
Matrix Factorization – CF
• Frequentists
• SVD
Loss Function
Regularization
• SVD++ (Implicit Feedback)
Koren Y. Factorization meets the neighborhood: a multifaceted collaborative
filtering model[C]//Proceedings of the 14th ACM SIGKDD international conference
on Knowledge discovery and data mining. ACM, 2008: 426-434.
Matrix Factorization – CF
• Frequentists - Sparsity
Ning X, Karypis G. Slim: Sparse linear methods for top-n recommender systems[C]//Data Mining
(ICDM), 2011 IEEE 11th International Conference on. IEEE, 2011: 497-506.
Matrix Factorization – CF
• Frequentists - Neighbour
Kabbur S, Ning X, Karypis G. Fism: factored item similarity models for top-n recommender systems[C]//Proceedings of
the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2013: 659-667.
Matrix Factorization – CF
• Bayesians
Salakhutdinov R, Mnih A. Probabilistic matrix factorization[C]. NIPS, 2011.
Restricted Boltzmann Machine – CF
Contract
Divergence
Train
Predict
Can update to Deep Belief Networks
Restricted Boltzmann Machine – CF
• Salakhutdinov R, Mnih A, Hinton G. Restricted Boltzmann machines
for collaborative filtering[C]//Proceedings of the 24th international
conference on Machine learning. ACM, 2007: 791-798.
• Georgiev K, Nakov P. A non-iid framework for collaborative filtering
with restricted boltzmann machines[C]//Proceedings of the 30th
International Conference on Machine Learning (ICML-13). 2013:
1148-1156.
Recurrent Neural Network – CF
• Model Sequences. (music recommendation, film recommendation,
etc.)
linear
softmax
linear
Tensor Regression - CF
Park S T, Chu W. Pairwise preference regression for coldstart recommendation[C]//Proceedings of the third ACM
conference on Recommender systems. ACM, 2009: 21-28.
Limitations of CF
• Cold Start
• It needs to have enough users in the system.
• New items need to get enough ratings.
• Sparsity:
• it is hard to find users who rated the same items.
• Popularity Bias:
• Cannot recommend items to users with unique tastes.
• Tends to recommend popular items.
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Content-based Recommendation
• Recommendations are based on the information on the content of
items rather than on other users’ opinions.
• Use a machine learning algorithm to model the users' preferences
from examples based on a description of the content.
Content-based Recommendation
• Collaborative Topic Regression
Wang C, Blei D M. Collaborative topic modeling for recommending scientific articles[C]//Proceedings of the
17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2011: 448-456.
Content-based Recommendation
• Deep Learning – Consider Noise
Wang H, Wang N, Yeung D Y. Collaborative deep learning
for recommender systems[C]//Proceedings of the 21th
ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining. ACM, 2015: 1235-1244.
Content-based Recommendation
• Content + CF -> Hybrid -> Cold Start
Saveski M, Mantrach A. Item cold-start recommendations: learning
local collective embeddings[C]//Proceedings of the 8th ACM
Conference on Recommender Systems. ACM, 2014: 89-96.
Content-based Recommendation
• + No need for data on other users.
• + No cold-start or sparsity problems.
• + Can recommend to users with unique tastes.
• + Can recommend new and unpopular items
• + Can provide explanations of recommended items by listing contentfeatures that caused an item to be recommended.
Content-based Recommendation
• - Only for content that can be encoded as meaningful features.
• - Some types of items (e.g. movies, music)are not amenable to easy
feature extraction methods
• - Even for texts, IR techniques cannot consider multimedia
information, aesthetic qualities, download time: a positive rating
could be not related to the presence of certain keywords
• - Users’ tastes must be represented as a learnable function of these
content features.
• - Hard to exploit quality judgements of other users.
• - Difficult to implement serendipity
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Context-based Recommendations
• Context is a dynamic set of factors describing the state of the user at
the moment of the user's experience.
• Context factors can rapidly change and affect how the user perceives
an item. For example, temporal, spatial, social, etc.
Pre-filtering
Post-filtering
Tensor Factorization
Karatzoglou A, Amatriain
X, Baltrunas L, et al.
Multiverse
recommendation: ndimensional tensor
factorization for contextaware collaborative
filtering[C]//Proceedings
of the fourth ACM
conference on
Recommender systems.
ACM, 2010: 79-86.
Factorization Machines
Rendle S, Gantner Z, Freudenthaler
C, et al. Fast context-aware
recommendations with factorization
machines[C]//Proceedings of the
34th international ACM SIGIR
conference on Research and
development in Information Retrieval.
ACM, 2011: 635-644.
Rendle S. Factorization
machines[C]//Data Mining (ICDM),
2010 IEEE 10th International
Conference on. IEEE, 2010: 9951000.
Time-awareness
Koren Y. Collaborative filtering with temporal dynamics[J].
Communications of the ACM, 2010, 53(4): 89-97.
Context-based Recommendations
• Zhao Z, Cheng Z, Hong L, et al. Improving User Topic Interest Profiles by Behavior
Factorization[C]//Proceedings of the 24th International Conference on World
Wide Web. International World Wide Web Conferences Steering Committee,
2015: 1406-1416.
• Yang S H, Long B, Smola A J, et al. Collaborative competitive filtering: learning
recommender using context of user choice[C]//Proceedings of the 34th
international ACM SIGIR conference on Research and development in Information
Retrieval. ACM, 2011: 295-304.
• Nguyen T V, Karatzoglou A, Baltrunas L. Gaussian process factorization machines
for context-aware recommendations[C]//Proceedings of the 37th international
ACM SIGIR conference on Research & development in information retrieval. ACM,
2014: 63-72.
• Zhang Y, Zhang M, Zhang Y, et al. Daily-Aware Personalized Recommendation
based on Feature-Level Time Series Analysis[C]//Proceedings of the 24th
International Conference on World Wide Web. International World Wide Web
Conferences Steering Committee, 2015: 1373-1383.
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Learning to Rank
• Pointwise
• Ranking function minimizes loss function defined on individual
relevance judgment e.g.
• Ranking score based on regression or classification
• Ordinal regression, Logistic regression, SVM
Learning to Rank
• Pairwise
• Loss function is defined on pair-wise preferences
• Goal: minimize number of inversions in ranking
BPR
Rendle S, Freudenthaler C, Gantner Z, et al. BPR: Bayesian personalized
ranking from implicit feedback[C]//Proceedings of the twenty-fifth conference
on uncertainty in artificial intelligence. AUAI Press, 2009: 452-461.
Pairwise Tensor Regression
Park S T, Chu W. Pairwise preference regression for coldstart recommendation[C]//Proceedings of the third ACM
conference on Recommender systems. ACM, 2009: 21-28.
Pairwise Learning to Rank
• RankBoost
• RankNet
• Frank
• EigenRank
• pLPA
• CR
•……
Learning to Rank
• List-wise
• Direct optimization of ranking metrics
• List-wise loss minimization for CF a.k.a Collaborative Ranking
Listwise Learning to Rank
• CoFiRank: optimizes an upper bound of
• NDCG (Smooth version)
• CLiMF : optimizes a smooth version of MRR
• TFMAP: optimizes a smooth version of MAP
• AdaRank: uses boosting to optimize NDCG
• ListRank
• WLT
• GAPfm
• xCLiMF
•……
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Diversity
• Recommendations from a music on-line retailer:
• Problem: No diversity: pop albums from female singers.
• Some are redundant.
Diversity
• Re-ranking
Sub-profile
Diversity and Novelty
Novelty Model
Popularity-based Item Novelty
Distance-based Item Novelty
Browsing Model
• A chosen item must obviously be seen, and relevant items are more
likely to be chosen than irrelevant ones.
Vargas S, Castells P. Rank and relevance in novelty and diversity
metrics for recommender systems[C]//Proceedings of the fifth ACM
conference on Recommender systems. ACM, 2011: 109-116.
Reference
• Smyth B, McClave P. Similarity vs. diversity[M]//Case-Based Reasoning
Research and Development. Springer Berlin Heidelberg, 2001: 347361.
• Lathia N, Hailes S, Capra L, et al. Temporal diversity in recommender
systems[C]//Proceedings of the 33rd international ACM SIGIR
conference on Research and development in information retrieval.
ACM, 2010: 210-217.
Outline
• Collaborative Filtering
• Content-based Recommendations
• Context-based Recommendations
• Ranking
• Diversity
• Socialization
Socialization
• 1. Trust-based Social Recommendation
Edge: Trust Value (user sim., SNS, etc.)
Vertex: Item Rating
Jamali M, Ester M. Trustwalker: a random walk model for combining trust-based and
item-based recommendation[C] //Proceedings of the 15th ACM SIGKDD international
conference on Knowledge discovery and data mining. ACM, 2009: 397-406.
Socialization
• 1. Trust-based Social Recommendation
• Forsati R, Mahdavi M, Shamsfard M, et al. Matrix factorization with explicit
trust and distrust side information for improved social recommendation[J].
ACM Transactions on Information Systems (TOIS), 2014, 32(4): 17.
• Jamali M, Ester M. A matrix factorization technique with trust propagation for
recommendation in social networks[C]//Proceedings of the fourth ACM
conference on Recommender systems. ACM, 2010: 135-142.
• J. Golbeck. Computing and Applying Trust in Web-based Social Networks. PhD
thesis, University of Maryland College Park, 2005
• P. Massa and P. Avesani. Trust-aware recommender systems. In RecSys 2007,
USA.
• Levien and Aiken. Advogato’s trust metric. online at
http://advogato.org/trust-metric.html, 2002
Socialization
• 2. MF-based Social Recommendation
User himself
Friends
Delporte J, Karatzoglou A, Matuszczyk T, et al. Socially enabled preference
learning from implicit feedback data[M]//Machine Learning and Knowledge
Discovery in Databases. Springer Berlin Heidelberg, 2013: 145-160.
Socialization
• 2. MF-based Social Recommendation
• Jamali M, Ester M. A matrix factorization technique with trust propagation for
recommendation in social networks[C]//Proceedings of the fourth ACM
conference on Recommender systems. ACM, 2010: 135-142.
• Noel J, Sanner S, Tran K N, et al. New objective functions for social
collaborative filtering[C]//Proceedings of the 21st international conference on
World Wide Web. ACM, 2012: 859-868.
• Yang X, Steck H, Guo Y, et al. On top-k recommendation using social
networks[C]//Proceedings of the sixth ACM conference on Recommender
systems. ACM, 2012: 67-74.
• H. Ma, I. King, and M. R. Lyu. Learning to recommend with social trust
ensemble. In SIGIR 2009, pages 203–210.
• H. Ma, H. Yang, M. R. Lyu, and I. King. Sorec: social recommendation using
probabilistic matrix factorization. In CIKM 2008, pages 931–940. ACM, 2008.
Socialization
• 3. Finding Experts
• X. Yang, H. Steck, Y. Guo, and Y. Liu. On top-k recommendation using social
networks. In Proc. of RecSys’12, 2012.