Joint Inference of Multiple Label Types in Large Networks - PowerPoint Presentation

1. Joint Inference of Multiple Label Types in Large NetworksDeepayan Chakrabarti (deepay@fb.com)Stanislav Funiak(sfuniak@fb.com)Jonathan Chang(jonchang@fb.com)Sofus A. Macskassy (sofmac@fb.com)1

2. Profile InferenceProfile:Hometown: Palo AltoHigh School: GunnCollege: StanfordEmployer: FacebookCurrent city: SunnyvaleHobbies, Politics, Music, …A complete profile is a boon:People are easily searchableTailored news recommendationsGroup recommendationsAd targeting (especially local)How can we fill in missing profile fields????2

3. Profile InferenceUse the social networkand the assumption of homophilyFriendships form between “similar” people Infer missing labels to maximize similarityuv1v2v3v4v5H = Palo AltoE = MicrosoftH = MPKE = FBH = AtlantaE = GoogleH = Palo AltoE = ?H = ?E = ?H = ?E = ?3

4. Previous WorkRandom walks [Talukdar+/09, Baluja+/08]Statistical Relational Learning [Lu+/03, Macskassy+/07]Relational Dependency Networks [Neville+/07]Latent models [Palla+/12]Either:too generic; require too much labeled data;do not handle multiple label types;are outperformed by label propagation [Macskassy+/07]4

5. Previous WorkLabel Propagation[Zhu+/02, Macskassy+/07]“Propagate” labels through the networkProbability (I have hometown H)= fraction of my friends whose hometown is HIterate until convergenceRepeat for current city, college, and all other label typesuv1v2v3v4v5H = Palo AltoH = MPKH = AtlantaH = Palo AltoH = ?H = Palo Alto (0.5) MPK (0.25) Atlanta (0.25)H = Palo Alto (…) MPK (…) Atlanta (…)5

6. ProblemuH = CalcuttaH = CalcuttaCC = BangaloreCC = BerkeleyH = ?CC = ?H = CalcuttaCC = BangaloreInteractions between label types are not considered6

7. The EdgeExplain ModelInstead of taking friendships as given,explain friendships using labelsA friendship u∼v is explained if:u and v share the same hometown OR current city OR high school OR college OR employer7

8. The EdgeExplain ModeluH = CalcuttaH = CalcuttaCC = BangaloreCC = BerkeleyH = ?CC = ?H = CalcuttaCC = BerkeleyHometown friendsCurrent City friendsWe set H and CC so as to jointly explain all friendships8

9. Find f to maximize ∏explained (fu, fv)The EdgeExplain Modelu∼v“Soft” ORover label typesProbability distribution for each label typeExplain all friendships9

10. Find f to maximize ∏explained (fu, fv)The EdgeExplain Modelu∼v“Soft” ORover label typesexplained (fu, fv) = softmax( is_reasont (fut, fvt) )t∊Τis_reasont (fut, fvt) = ∑ futℓ . fvtℓℓ∊L(t)softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c) t∊Τt∊ΤIs u∼v explained by label type t?Chances of sharing a label of type tSigmoid for softmax10

11. H = ?CC = ?The EdgeExplain Model∑t is_reasontuH = CalcuttaCC = BerkeleyH = CalcuttaCC = Bangalore∑t is_reasont11softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)

12. The EdgeExplain ModelH = CalcuttaCC = ?∑t is_reasontuH = CalcuttaCC = BerkeleyH = CalcuttaCC = Bangalore∑t is_reasontH=CalH=Cal12softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)

13. H = CalcuttaCC = BangaloreThe EdgeExplain Model∑t is_reasontH=CalMarginal gain with CC = BangaloreuH = CalcuttaCC = BerkeleyH = CalcuttaCC = Bangalore∑t is_reasontH=CalH=CalCC=B’loreH=CalCC=B’lore13softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)

14. H=CalCC=BerkeleyH = CalcuttaCC = BerkeleyThe EdgeExplain Model∑t is_reasontH=CaluH = CalcuttaCC = BerkeleyH = CalcuttaCC = Bangalore∑t is_reasontH=CalH=CalCC=BerkeleyMore gain with CC = Berkeley14softmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c)

15. α controls the slopehigh α  steep  one reason per edge is enoughlow α  linear  consider multiple reasons per edgeH=CalCC=BerkeleyThe EdgeExplain Modelsoftmax( is_reasont (fut, fvt) ) = σ (α . ∑ is_reasont (fut, fvt) + c) ∑t is_reasontH=Cal∑t is_reasontH=CalH=CalCC=Berkeley15

16. Experiments1.1B users of the Facebook social networkO(10M) labels5-fold cross-validationMeasure recallDid we get the correct label in our top prediction? Top-3?Inference:proximal gradient descentimplemented via message-passing in Apache Giraph [Ching/13]Sparsify graph by considering K closest friends by age16

17. Results (varying closest friends K)K=100 or K=200 closest friends is bestK=400 hurts; these friendships are probably due to other factorsRecall@1Recall@317HometownCurrent cityHigh schoolCollegeEmployerLift of EdgeExplain over K=20Lift of EdgeExplain over K=20HometownCurrent cityHigh schoolCollegeEmployer

18. Results (versus Label Propagation)Joint modeling helps most for employerSignificant gains for high school and college as well18HometownCurrent cityHigh schoolCollegeEmployerLift of EdgeExplain over Label PropagationHometownCurrent cityHigh schoolCollegeEmployerLift of EdgeExplain over Label PropagationRecall@1Recall@3

19. ConclusionsAssumption: each friendship has one reasonModel: explain friendships via user attributes Results: up to 120% lift for recall@1 and 60% for recall@319

20. Result (effect of α)20High α is best  one reason per friendship is enoughLift of EdgeExplain over α=0.1HometownCurrent cityHigh schoolCollegeEmployer

21. Results (varying closest friends K)K=100 or K=200 closest friends is bestK=400 hurts; these friendships are probably due to other factorsRecall@1Recall@321

22. Results (versus Label Propagation)Joint modeling helps most for employerSignificant gains for high-school and college as wellRecall@1Recall@322

23. Problem23Hometown friendsFriends in my current cityHometown friends, now in Bangalore