Yang_conference-intro
Download
Report
Transcript Yang_conference-intro
Recsys12 && KDD 12 Brief Summary
Xiwang Yang
1
the 6th ACM International Conference on
Recommender Systems (RecSys 2012)
RecSys 2012
2
Conference Introduction
Evaluation Metrics
Learning to Rank
Social RS
Context-aware and location-based recommendations
HCI, User-centric, interfaces & explanations
System Design
RecSys12 Introduction
Premier global forum for discussing the state of the
art in recommender System
Long paper acceptance rate: 24/119 = 20%
Single Track
> 270 attendence
1/3 from industry
3
Evaluation Metrics
Workshop:Recommendation Utility Evaluation: Beyond RMSE
Organizer: Xavier Amatriain (Netflix), Harald Steck (Netflix),
Pablo Castells (UAM), Arjen de Vries, and Christian Posse (LikedIn)
Specific questions that the workshop aims to address
include the following:
4
What are the unmet needs and challenges for evaluation in
the RS field? What changes would we like to see? How could
we speed up progress?
What relevant recommendation utility and quality dimensions
should be cared for? How can they be captured and
measured?
How can metrics be more clearly and/or formally related to
the task, contexts and goals for which a recommender
application is deployed?
How should IR metrics be applied to recommendation tasks?
What aspects require adjustment or further clarification?
What further methodologies should we draw from other
disciplines (HCI, Machine Learning, etc.)?
Evaluation Metrics
Workshop:Recommendation Utility Evaluation: Beyond RMSE
Can we predict the success of a recommendation algorithm
with our offline experiments? What offline metrics
correlate better and under which conditions?
What are the outreach and limitations of offline evaluation?
How can online and offline experiments complement each
other?
What type of public datasets and benchmarks would we want
to have available, and how can they be built?
How can the recommendation effect be traced on business
outcomes?
How should the academic evaluation methodologies improve
their relevance and usefulness for industrial settings?
How do we envision the evaluation of recommender systems
in the future?
5
Evaluation Metrics
Industry Keynote: Ron Kohavi (Microsoft): Online Controlled
Experiments: Introduction, Learnings, and Humbling Statistics
Ron Kohavi, General Manager Experimentation Platform, Microsoft
Controlled experiments at Microsoft Bing, very good work, 2012
kdd paper;
6
7
Controlled Experiments in One Slide
Concept is trivial
Randomly split traffic between
two (or more) versions
• A (Control)
• B (Treatment)
Collect metrics of interest
Analyze
Must run statistical tests to confirm differences are not
due to chance
Best scientific way to prove causality, i.e., the changes in
metrics are caused by changes introduced in the
treatment(s)
Evaluation Metrics
Session: Multi-Objective Recommendation and Human Factors
Multiple Objective Optimization in Recommendation Systems
Mario Rodriguez and others explain how they design LinkedIn
recommendations by optimizing to several objectives at once
(e.g. candidate that is good for the job + who is open to new
opportunities). They report results from an AB Test run on
LinkedIn
8
Evaluation Metrics
Session: Multi-Objective Recommendation and Human
Factors
Pareto-Efficient Hybridization for Multi-Objective
Recommender Systems
• Marco Tulio Ribeiro-Universidade Federal de Minas Gerais &
Zunnit Technologies
• The problem of combining recommendation algorithms grows
significantly harder when multiple objectives are considered
simultaneously.
• take the multi-objective a step further. In their case, they
optimize the system to not only be accurate, but also present
novel or diverse items.
9
Outline
Learning to Rank
• Session: Top-N Recommendation
• Social top-k RS:
• Industry invited talk: Ralf Herbrich (Facebook):
Distributed, Real-Time Bayesian Learning in Online
Services
10
Learning to Rank
Focus more on ranking than rating prediction NOW!
Session: Top-N Recommendation
CLiMF: Learning to Maximize Reciprocal Rank with
Collaborative Less-is-More Filtering
• Best paper, Yue Shi, Delft, intern at Telefonica
• Optimize Mean Reciprocal Rank (MRR) directly.
• MRR is a well-known information retrieval metric for
measuring the performance of top-k recommendations
Similar work: "TFMAP: Optimizing MAP for top-n contextaware recommendation
• SIGIR 2012.
• Optimize Top-N, Mean Average Precision
• Uses tensor factorization to model implicit feedback data
(e.g.,purchases, clicks) with contextual information
• fast learning algorithm
11
Learning to Rank
Focus more on ranking than rating prediction NOW!
Session: Top-N Recommendation
"Ranking with Non-Random Missing Ratings: Influence of
Popularity and Positivity on Evaluation Metrics
• An interesting study on the very important issue of
negative sampling, and popularity bias in learning to rank.
The paper discusses these effects on the AUC (Area
Under the Curve) measure.
12
Learning to Rank
Session: Top-N Recommendation
"Sparse Linear Methods with Side Information for
Top-N Recommendations“
• University of Minnesota in the Twin Cities
• multidimensional context-aware learning to rank
Alternating Least Squares for Personalized Ranking
• Gravity R&D
• Dense math
• invited anyone not interested in Mathematics to leave the room
• proposed a computationally efficient ranking based
method RankALS that optimizes the original objective
function, without sampling.
"On Top-k Recommendation Using Social Networks”
13
Learning to Rank
"Real-Time Top-N Recommendation in Social Streams
University of Hannover
Focus on analyzing social streams(twitter) in real-time
for personalized topic recommendation and discovery.
Industry invited talk: Ralf Herbrich (Facebook): Distributed,
Real-Time Bayesian Learning in Online Services
Bayesian Factor Models for large-scale distributed ranking
The same author and others from MSR named it as
"Matchbox“, is now used in different settings
Poster "The Xbox Recommendation System“
Apply matchbox to recommending movies and games for the Xbox
Poster “Collaborative Learning of Preference Rankings”
Erasmus School of Economics & MSR
apply it to sushi recommendation
14
Social RS
Session: Social Recommendation
"Spotting Trends: The Wisdom of the Few”
Wisdom of the Few, using a reduced set of experts for
recommendations
Popular != Trending
iCoolhunt users are encouraged to take pictures of
objects that they think ‘cool’, upload them and share
them with friends online.
15
Social RS
Workshop on Recommender Systems and the Social Web
Extending FolkRank with Content Data
Leveraging Publication Metadata and Social Data into
FolkRank for Scientific Publication Recommendation
Context Determines Content - An Approach to
Resource Recommendation in Folksonomies
FReSET - An Evaluation Framework for Folksonomybased Recommender Systems
Aggregating Content and Network Information to
Curate Twitter User Lists
Online Dating Recommender Systems: The SplitComplex Number Approach
Social Media-Driven News Personalization
Trust-Based Local and Social Recommendation
16
Context-aware and location-based
recommendations
Workshop: Personalizing the Local Mobile Experience
Workshop on Context-Aware Recommender Systems
Session: Contextual and Semantically Aware
Recommendation
Context-Aware Music Recommendation Based on Latent
Topic Sequential Patterns",
• playlist generation
"Ads and the City: Considering Geographic Distance
Goes a Long Way”
• location-aware recommendations.
17
HCI, User-centric, interfaces & explanations
Tutorial: Conducting User Experiments in Recommender
Systems
• Bart Knijnenburg, UCI
• overview of how to conduct user studies for recommender
systems
Paper: TasteWeights: A Visual Interactive Hybrid
Recommender System
• USSB
Paper: Inspectability and Control in Social
Recommenders
• Bart Knijnenburg, UCI
• Analyze the effect of giving more information and control
to users in the context of social recommendations.
Workshop on Interfaces for Recommender System.
18
System Design
Tutorial: Building Industrial-scale Real-world
Recommender Systems
• Xavier Amatriain, Netflix
Mendeley Suggest: Engineering a Personalized Article
Recommendation System
• Kris Jack from Mendeley
• he explained how they make use of AWS and Mahout in a
system that can generate personalized recommendations
for about $60 a month
From a toolkit of recommendation algorithms into a real
business
• Domonkos Tikk from Gravity R&D
• evolved from being a team in the Netflix Prize to a realworld company with very interesting projects
19
RecSys in China- Booming
RecSys China
Technical Community, ~5000 members
RecSys 13 in HongKong
Baidu
• RecSys team built two years ago, size: ~100
Taobao
• RecSys team: > 100
Weibo
• RecSys team: ~20
Tencent
Huawei
20
• Noah's Ark Research Lab in Hong Kong
• Built July, 2012
• long term ~100 researchers
How user evaluate each other in social media
Keynote-Jure Leskovec-Stanford University
21
22
23
KDD2012
August12-16, Beijing, China
First held in Asian
Attendance: 1232
Long Paper Acceptance Rate: 133/755 = 17.6%
Three Research Tracks about Recommendation
Research Session : Personalization and
Recommendation
Research Session: Ads and Video Recommendation
Research Session: Recommendation
Research Session : Matrices and Tensors
24
Mining Heterogeneous Information
Networks- Jiawei Han - UIUC
25
26
27
28
29
30
31
32
33
Big Data Panel Discussion
debate on the following questions:
What is the nature of Big Data? What are the Big Data
problems that you have encountered? Is this a longterm challenge or a short-term fad?
What opportunities and challenges does data mining
face on Big Data?
What are effective Big Data solutions? What
platforms, sampling solutions, and applications are most
effective for handling Big Data?
34
Some of the Opinions
Christos Faloutsos – CMU
Large data size: > 100 machines, > 1 Tera Bytes
Jiawei Han
Large Complexity
Michael I. Jordan
? Sampling everything
Some other voice
Big data rate in HFT
35