Houssam Nassif houssamnamazoncom Recently published Amazon papers Diversifying Music Recommendations Nassif H Cansizlar KO Goodman M and Vishwanathan SVN International Conference on Machine Learning ICML16 Workshops ID: 644262
Download Presentation The PPT/PDF document "Diversifying Amazon Recommendations" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Diversifying Amazon Recommendations
Houssam Nassifhoussamn@amazon.comSlide2
Recently published Amazon papers
Diversifying Music RecommendationsNassif H, Cansizlar KO, Goodman M, and Vishwanathan
SVN
International Conference on Machine Learning (ICML'16) Workshops
, New York, 2016.Adaptive, Personalized Diversity for Visual Discovery (Best Paper Award)Teo CH, Nassif H, Hill D, Srinavasan S, Goodman M, Mohan V, and Vishwanathan SVNACM Conference on Recommender Systems (RecSys'16), Boston, pp. 35-38, 2016.
2Slide3
Recently published Amazon papers
Diversifying Music RecommendationsNassif H, Cansizlar KO, Goodman M, and Vishwanathan
SVN
International Conference on Machine Learning (ICML'16) Workshops
, New York, 2016.Adaptive, Personalized Diversity for Visual Discovery (Best Paper Award)Teo CH, Nassif H, Hill D,
Srinavasan
S, Goodman M, Mohan V, and
Vishwanathan SVNACM Conference on Recommender Systems (RecSys'16), Boston, pp. 35-38, 2016.
3Slide4
Outline
MotivationJaccard Swap diversity method
Submodular diversity method
Experiment
4Slide5
About Amazon Prime Music
Free benefit for prime membersMillions of songs
Thousands of expert-programmed playlists
Upload your own music
Create personal playlists5Slide6
Amazon Prime Music mobile app
Access your music from anywhereList-form recommendationDevices with limited interaction capability
6Slide7
Music considerations
Explicit clusters: album, artistSame album: same meta-data (album cover graphic, title)User behavior: play album songs back-to-back
Similar scores to same-album songs
7Slide8
Why diversify music stream?
8Slide9
Outline
MotivationJaccard Swap diversity method
Submodular diversity method
Experiment
9Slide10
Explanation-based diversity
C. Yu, L. Lakshmanan, S. Amer-Yahia. It takes
variety
to make a world: Diversification in recommender systems.
EDBT 2009. : user, : item
: similarity measure between two items
: Set of items user
interacted
with
10Slide11
Jaccard diversity distance
if explanation sets completely separate
if explanation sets
are identical
11Slide12
Algorithm Swap
12
Recommender score
Explanatory set
7
6.2
6
5
1
2/3
1
1/2
1.17
1.66
1.5
1.5
If diversity increases
And if
Slide13
Outline
MotivationJaccard
Swap
diversity method
Submodular diversity methodExperiment13Slide14
D
iminishing returns
14
Cumulative number of explanation items in set
+11
+5
+3
Incremental utility
t
apers off
Set
u
tilitySlide15
Submodular diversity mix
15
Diversified list:
+11
+4
+5
+3
+2Slide16
Submodular diversity
: category,
: item,
: diversified set, score(
): ’s recommender scoreCategory utility:
Maximize sum of all category utilities:
Greedy near-optimal solution:
16Slide17
Outline
MotivationJaccard
Swap
diversity method
Submodular diversity methodExperiment17Slide18
Experimental setup
Baseline: Rank by recommender scoreItem-to-item collaborative filtering recommender provides item score and explanation setArtist
and album as
Jaccard explanation set features and submodular categories ( , , , …)18Slide19
Results
Treatment
comparison
Increase in minutes streamedSubmodularity vs Baseline0.64% (p=0.03)
Jaccard
Swap vs Baseline
0.40% (p=0.18)Submodularity
vs
Jaccard
Swap
0.24%
(p=0.41)
19
Diversity affects recommendation quality
Submodularity
method
improvement is significantSlide20
Baseline vs Submodular
20Slide21
Submodular approach benefits
Smoothness:Submodularity produces uniformly diverse set. All contiguous subsets are also diverse.
Jaccard
Swap doesn’t.
Relevance:Swap may not retain most relevant content. Submodularity ensures most relevant item is first, followed by mix of most relevant items within each category. 21Slide22
Recently published Amazon papers
Diversifying Music Recommendations
Nassif H,
Cansizlar
KO, Goodman M, and Vishwanathan SVNInternational Conference on Machine Learning (ICML'16) Workshops, New York, 2016.
Adaptive, Personalized Diversity for Visual Discovery
(Best Paper Award)
Teo CH, Nassif H, Hill D, Srinavasan S, Goodman M, Mohan V, and Vishwanathan SVNACM Conference on Recommender Systems (RecSys'16), Boston, pp. 35-38, 2016.
22Slide23
Amazon Stream
:
www.amazon.com
/streamSlide24
Challenges
Need for product scoring functionExplore new products and exploit popular ones
Need for
diverse
streamHedge against the uncertainty in customer intentNeed for season-aware stream
Customers can engage with relevant products all year round
Need for
personalized streamCustomers can focus more on products that they likeSlide25
25
Explore-exploit in product scoring
Product ID
Brand
Price
Category
100352463
Jacket
Calvin Klein
$98
Bayesian Linear
Probit
Regression
Click
or
No-click
Product attributes,
x
Attribute weight distributions, w
Product Score =
F
(
Σ
i
w
i
x
i
)
Thompson Sampling
Bandit
C
ustomer
actions
W
i
~ N(0,1)Slide26
Ranking by Product Scores Slide27
More Variety of Products for DiscoverySlide28
Challenges
Need for product scoring function
Explore new products and exploit popular
ones
Need for diverse streamHedge against the uncertainty in customer intentNeed for season-aware stream Customers can engage with relevant products all year round
Need for
personalized
streamCustomers can focus more on products that they likeSlide29
Diversity
Diverse stream consists of a large variety of categoriesStream Category: (
Department
,
Product Type, Price Range)
womens
-
dress
-
highSlide30
Submodular Utility
Define utility of a product set D:
“
weighted
sum of per-category product counts and scores” utility(D) = Σk
w
k
log[1 + c(k, D) + s(k,D
)
]
Number of products in category k
Weight for category k
Total score
of products in category kSlide31
Multinomial Diversified Ranking
Assign a category, k ~ Multinomial(w), to each position
Assign products from each category to their corresponding positions in decreasing order of their scoresSlide32
Results: Diversified Ranking
Submodular vs multinomial diversitySubmodular
ranker put more relevant products at the top
Metric
Result
Time spent
+0.05%
product view count-1.32%
Product
clicks
+9.82%Slide33
Challenges
Need for item scoring function
Explore new products and exploit popular ones
Need for
diverse streamHedge against the uncertainty in customer intentNeed for season-aware stream Customers can engage with relevant products all year round
Need for
personalized
streamCustomers can focus more on products that they likeSlide34
Seasonality
Fashion trends are seasonal Popularity of categories change with seasonsFall colors are different from spring colors
Learn about seasonal trends from customer behaviorSlide35
Adaptive Category Weights
Estimate weight for a category by its smoothed click-through-rate over a rolling-windoww
k
= (
clicksk + αk) / (viewsk + αk + βk) Slide36
Results: Seasonality
Adaptive vs static global weights
Showing seasonally
relevant
products increases customer engagementMetric
Result
Time spent
+5.39%Item view count+1.08%
Product
clicks
+8.29%Slide37
Challenges
Need for product scoring function
Explore new products and exploit popular
ones
Need for diverse streamHedge against the uncertainty in customer intentNeed for season-aware
stream
Customers can engage with relevant products all year round
Need for personalized streamCustomers can focus more on products that they likeSlide38
Recall
Define utility of a product set D:
“
weighted
sum of per-category product counts and scores” utility(D) = Σk
w
k
log[1 + c(k, D) + s(k,D
)
]
Number of products in category k
Weight for category k
Total score
of products in category kSlide39
Personalization
Category weights are personalized using past actionsAgnostic of the diversificationEasy to incorporate: searches
,
purchases
, or returnsModeling:clicks,
c
~ Multinomial(w)weights,
w
~
Dirichlet
(
d
)
where
d
are default weight
s
Personalized weights
, E
p(
w
|
c
,
d
)
[
w
] = (
c + d) / |c + d|1Slide40
Personalized Category Weights
Distribution over categories might be sparse (i.e., filter bubble)Slide41
Category-category Correlations
Diffuse preference for category A to category B based on the
correlation
between A and B
“clicked A then clicked B”
Customers who interacted with
any
women’s categories tend to also interact with women’s accessory, low-price dress and low-price handbag.Slide42
×
Category-category correlations
Customer preference vector
Smoothed
customer preference vectorSlide43
Results: Personalized Diversifier
Personalized vs global weights
Personalizing weights:
Decreases scroll through
Increases customer interactions
Metric
Result
Time spent+1.10%
Item view count
-4.95%
Product
clicks
+12.58%Slide44
Takeaways
Explore/exploit strategy is important for product discoveryModeling diminishing returns is essential for diversity
Seasonal and fashion
trends
can be learned from customer behaviorPersonalization enhances customer experienceSlide45
Why Amazon?
Great data and problems – seriously.If you can convince the powers that be that it is an important project, resources will be provided.
I
am rarely the smartest person in the room.
Even our worst employees are competent.45Slide46
Data @ Amazon
46
Product search/browse/purchase
Customer contact
Product catalog
Seller marketplace and offers
Product reviews & seller feedback
Pricing, inventory, fulfillment, shipping, and demand
Click-stream
Digital content (books, movies, music, etc.)
Local offers data (Amazon Local)
Advertisements
AWS performance and resource utilization
High quality
product imagesSlide47
Research @ Amazon
Opportunity to work on exciting new amazon initiativesCustomer facing experience, direct
impact on
customers
Significant business impact, huge potential for growthA culture of experimentation and ownershipAlready established research and applied scientist career
tracks
47Slide48
Global Research Groups
48
Seattle
Machine Learning
ForecastingAmazon PrimeData ScienceOperations ResearchComputer VisionNatural LanguageInformation SecurityPrime AirAWSBay Area
Search
Computer Vision
AWSBostonSpeech RecognitionComputer VisionBerlin, GermanyMachine LearningComputer VisionNatural Language Aachen, GermanySpeech Recognition
Bangalore, India
Machine Learning
Cambridge, UK
Speech Recognition
Prime Air Slide49
Thank you!
Questions?
houssamn@amazon.com
49