areuserspecifclatentmatrixanditemspecifclatentmatrixForthesocialtrustgraphbasedontheintuitionthatusersalwaysprefertheitemsrecommendedbythefriendstheytrustweinferandformulatetherecommendationpr ID: 411602
Download Pdf The PPT/PDF document "LearningtoRecommendwithSocialTrustEnsemb..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
LearningtoRecommendwithSocialTrustEnsembleHaoMa,IrwinKingandMichaelR.LyuDepartmentofComputerScienceandEngineeringTheChineseUniversityofHongKongShatin,N.T.,HongKong{hma,king,lyu}@cse.cuhk.edu.hkABSTRACTAsanindispensabletechniqueinthe\feldofInformationFilteringRecommenderSystemhasbeenwellstudiedanddevelopedbothinacademiaandinindustryrecently.How-ever,mostofcurrentrecommendersystemssu erthefol-lowingproblems:(1)Thelarge-scaleandsparsedataoftheuser-itemmatrixseriouslya ecttherecommendationqual-ity.Asaresult,mostoftherecommendersystemscan-noteasilydealwithuserswhohavemadeveryfewratings.(2)Thetraditionalrecommendersystemsassumethatalltheusersareindependentandidenticallydistributed;thiassumptionignorestheconnectionsamongusers,whichisnotconsistentwiththerealworldrecommendations.Aim-ingatmodelingrecommendersystemsmoreaccuratelyandrealistically,weproposeanovelprobabilisticfactoranalysisframework,whichnaturallyfusestheusers'tastesandtheitrustedfriends'favorstogether.Inthisframework,wecoithetermSocialTrustEnsembletorepresenttheformulationofthesocialtrustrestrictionsontherecommendersystemsThecomplexityanalysisindicatesthatourapproachcanbeappliedtoverylargedatasetssinceitscaleslinearlywiththenumberofobservations,whiletheexperimentalresultsshowthatourmethodperformsbetterthanthestate-of-the-artapproaches.CategoriesandSubjectDescriptors:H.3.3[Informa-tionSearchandRetrieval]InformationFiltering;J.4[ComputerApplications]SocialandBehavioralSciencesGeneralTerms:Algorithm,ExperimentationKeywords:RecommenderSystems,SocialNetwork,SocialTrustEnsemble,MatrixFactorization1.INTRODUCTIONAstheexponentialgrowthofinformationgeneratedontheWorldWideWeb,theInformationFilteringtechniqueslikeRecommenderSystemshavebecomemoreandmoreim-portantandpopular.Recommendersystemsformaspeci\fctypeofinformation\flteringtechniquethatattemptstosugPermissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributedforprotorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitationontherstpage.Tocopyotherwise,torepublish,topostonserversortoredistributetolists,requirespriorspecicpermissionand/orafee.SIGIR'09,July19–23,2009,Boston,Massachusetts,USA.Copyright2009ACM978-1-60558-483-6/09/07...$10.00.gestinformationitems(movies,books,music,news,Webpages,images,etc.)thatarelikelytointeresttheusers.Typically,recommendersystemsarebasedonCollaborativeFiltering,whichisatechniquethatautomaticallypredictstheinterestofanactiveuserbycollectingratinginformationfromothersimilarusersoritems.Althoughrecommendersystemshavebeenwidelystud-iedintheacademiaanddeployedintheindustry,suchasAmazonandEbay,mostofthesetechniquessu erseveralinherentweaknesses.The\frstwellknownchallengeisthedatasparsityproblem.Asreportedin[20],thedensityoftheavailableratingsincommercialrecommendersystemsisoftenlessthan1%.Manycollaborative\flteringalgorithmsareimpededbythesparsityproblem,hencecannothan-dleuserswhohaveratedfewitems.Secondly,traditionalrecommendersystemsignorethesocialconnectionsortrustrelationsamongusers.Butthefactis,intherealworld,wealwaysturntofriendswetrustforbook,music,orrestau-rantrecommendations,andourfavorscaneasilybea ectedbythefriendswetrust.Therefore,traditionalrecommendesystems,whichpurelyminetheuser-itemratingmatrixforrecommendations,donotproviderealisticoutput.Recently,trust-awarerecommendersystemshavedrawnlotsofatten-tion[14,15],butmostofthesemethodsarebasedonsomeadhocheuristics,andtheystillhavethedatasparsityandscalabilityproblems.Moreover,therelationshipbetweentheuser-itemmatrixandtheusers'trustnetworkarenotfullyunderstood.Inthispaper,aimingatsolvingtheaboveproblemsandmodelingtherecommendersystemsmoreaccuratelyandre-alistically,wemakethreeassumptionsbasedonourobser-vationsontherealworldrecommendationprocesses.Usershavetheirowncharacteristics,andtheyhavedif-ferenttastesondi erentitems,suchasmovies,books,music,articles,food,etc.Userscanbeeasilyin\ruencedbythefriendstheytrust,andprefertheirfriends'recommendations.Oneuser's\fnaldecisionisthebalancebetweenhis/herowntasteandhis/hertrustedfriends'favors.Basedontheaboveintuitions,weendowanovelunder-standingtoalltheratingsintheuser-itemmatrix.Weinterprettheratingijintheuser-itemmatrixastherep-resentationmixedbyboththeuser'stasteandhis/hertrustedfriendstastesontheitem.Thisassumptionnatu-rallyemploysboththeuser-itemmatrixandtheusers'sociatrustnetworkfortherecommendations.Intermsoftheusers'owntastes,wefactorizetheuser-itemmatrixandlearntwolow-dimensionalmatrices,which areuser-speci\fclatentmatrixanditem-speci\fclatentma-trix.Forthesocialtrustgraph,basedontheintuitionthatusersalwaysprefertheitemsrecommendedbythefriendstheytrust,weinferandformulatetherecommendationprob-lempurelybasedontheirtrustedfriends'favors.Then,byemployingaprobabilisticframework,wefusetheusersandtheirtrustedfriends'tastestogetherbyanensembleparameter.Finally,byperformingasimplegradientdescentontheobjectivefunction,welearnthelatentlow-dimensionauser-speci\fcanditem-speci\fcmatricesforthepredictionofusers'favorsondi erentitems.TheexperimentalresultsoalargeEpinionsdatasetshowthatourmethodoutperformsthestate-of-the-artcollaborative\flteringandsocialtrust-basedrecommendationalgorithms,especiallywhentheuserhaveveryfewratings.Moreover,thecomplexityanalysisindicatesthatourapproachcanbeappliedtoverylargedatasets,sinceitscaleslinearlywiththenumberofobser-vations.Theremainderofthispaperisorganizedasfollows.InSection2,weprovideanoverviewofseveralmajorapproacheforrecommendersystemsandotherrelatedwork.Section3presentsourworkonrecommendersystemwithsocialtrustensemble.TheresultsofanempiricalanalysisarepresenteinSection4,followedbytheconclusionsandfutureworkinSection5.2.RELATEDWORKInthissection,wereviewseveralmajorapproachesforrec-ommendersystems,including(1)traditionalrecommendersystemswhicharemainlybasedoncollaborative\flteringtechniques,and(2)socialtrust-basedrecommendersystemwhichhavedrawnlotsofattentionrecently.Traditionalcollaborative\flteringalgorithmsmainlyfo-cusontheuser-itemmatrix.Amongallofthesemethods,thememory-basedapproachesarethemostpopularmeth-odsandtheyarewidelyadoptedincommercialcollaborative\flteringsystems[10,17].Thesemethodsemploydi erentstrategiesto\fndsimilarusersanditemsformakingthepredictions,whichareknownasuser-basedapproaches[3,6,9,12]anditem-basedapproaches[5,10,20],respectively.Topredictaratingijofagivenitemforanactiveuser,user-basedmethodssearchforotheruserssimilartotheuserandutilizetheirratingstotheitemforprediction,whileitem-basedmethodsleveragetheratingsofotheritemssimilartotheitemfromtheuserin-stead.Inordertotakeadvantagesofthesetwotypesofmethods,Wangetal.in[23]andMaetal.in[12]proposedtwofusionmodelstocombineuser-basedmethodwithitem-basedmethod.Inadditiontothememory-basedmethods,model-basedapproaches,whichemploystatisticalandma-chinelearningtechniquestolearnmodelsfromthedata,alsoplayanimportantroleinthecollaborative\flteringresearch.Examplesofmodel-basedapproachesincludeaspectmodels[7,8,21],thelatentfactormodel[4],theBayesianhi-erarchicalmodel[24]andtherankingmodel[11].Recently,severalmatrixfactorizationmethods[16,18,19,22]havebeenproposedforcollaborative\fltering.Thesemethodsfocusonfactorizingtheuser-itemratingmatrixusinglow-rankrepresentations,andthenutilizethemtomakefurtherpredictions.Themotivationbehindalow-dimensionalfac-torizationmodelisthatthereisonlyasmallnumberof http://www.epinions.comfactorsthatareimportant,andauser'spreferencevectorideterminedbyhoweachfactorappliestothatuser.Recallthatalltheabovemethodsforrecommendersys-temsarebasedontheassumptionthatusersareindependentandidenticallydistributed,andignoresthesocialtrustre-lationshipsbetweenusers,whichisnotconsistentwiththerealitythatwenormallyasktrustedfriendsforrecommen-dations.Basedonthisintuition,manyresearchershaverecentlystartedtoanalyzetrust-basedrecommendersys-tems[1,2,13,14,15].Andersenetal.in[1]developedasetof\fvenaturalax-iomsthatatrust-basedrecommendationsystemmightbeexpectedtosatisfy,andthenprovedthatnosystemcansi-multaneouslysatisfyalltheaxioms.Apparently,thisworkisoutofthescopeofthispapersincewefocusonhowtoemploybothsocialtrustnetworkanduser-itemmatrixtoprovidemoreaccurateandrealisticrecommendations.In[14,15],MassaandAvesanistudiedthetrust-awarerec-ommendersystems.Theirworkreplacesthesimilarity\fnd-ingprocesswiththeuseofatrustmetric,whichisabletopropagatetrustoverthetrustnetworkandtoestimateatrustweight.Theexperimentsonalargerealdatasetshowsthatthisworkincreasesthecoverage(numberofratingsthaarepredictable)whilenotreducingtheaccuracy(theerrorofpredictions).Bedietal.in[2]proposedatrust-basedrec-ommendersystemfortheSemanticWeb;thissystemrunsonaserverwiththeknowledgedistributedoverthenet-workintheformofontologies,andusestheWeboftrusttogeneratetherecommendations.Thetrust-basedmeth-odshavebecomeapopularresearchtopicrecently,however,thereareseveralproblemswithpreviousmethods.Firstly,theseapproachesonlyemploysomeheuristicstogeneraterecommendationswhiletherelationshipbetweenthetrustnetworkandtheuser-itemmatrixhasnotbeenstudiedsys-tematically.Moreover,thesemethodsarenotscalabletoverylargedatasets,sincetheymayneedtocalculatethepairwiseusersimilaritiesandpairwiseusertrustscores.Inrecentworkproposedin[13],Maetal.developedafactoranalysismethodbasedontheprobabilisticgraphicamodelwhichfusestheuser-itemmatrixwiththeusers'sociatrustnetworksbysharingacommonlatentlow-dimensionaluserfeaturematrix.Theexperimentalanalysisshowsthatthismethodgeneratesbetterrecommendationsthanthenon-socialcollaborative\flteringalgorithms.However,thedisad-vantageofthisworkisthatalthoughtheusers'socialtrustnetworkisintegratedintotherecommendersystemsbyfac-torizingthesocialtrustgraph,therealworldrecommenda-tionprocessesarenotre\rectedinthemodel.Thisdrawbacknotonlycauseslackofinterpretabilityinthemodel,butalsoa ectstherecommendationqualities.Amorenovelandrealisticapproachisneededtomodelthetrust-awarerecommendationproblem.3.RECOMMENDATIONWITHSOCIALTRUSTENSEMBLETraditionalrecommendersystemtechniques,likecollabo-rative\fltering,onlyutilizetheinformationoftheuser-itemratingmatrixforrecommendationswhileignorethesocialtrustrelationsamongusers.Astheexponentialgrowthofonlinesocialnetworks,incorporatingsocialtrustinforma-tionintorecommendersystemsisbecomingmoreandmoreimportant.Inthissection,we\frstdescribethetrust-awar (a)SocialTrustGraph (b)User-ItemRatingMatrixFigure1:ExampleforTrustbasedRecommendationrecommendationprobleminSection3.1,andthenprovidethesolutioninSections3.2,3.3and3.4.3.1ProblemDescriptionIntherealworld,theprocessofrecommendationscenarioincludestwocentralelements:thetrustnetworkandthefavorsofthesefriends,whichcanessentiallybemodeledbytheexamplesofthetrustgraphinFig.1(a)andtheuser-itemratingmatrixinFig.1(b),respectively.InthetrustgraphillustratedinFig.1(a),totally,5users(nodes,froto)areconnectedwith9relations(edges)betweenusers,andeachrelationisassociatedwithaweightijintherange(01]tospecifyhowmuchuserknowsortrustsuser.Normally,thetrustrelationsintheonlinetrustnetworkareexplicitlystatedbyonlineusers.AsillustratedinFig.1(b),eachuseralsoratedsomeitems(fromtoona5-pointintegerscaletoexpresstheextentofthefavorofeachitem(normally,1,2,3,4and5represent\hate",\don'tlike",\neutral",\like"and\love",respectively).Theproblemwestudyinthispaperishowtopredictthemissingvaluesfortheuserse ectivelyandecientlybyemployingthetrustgraphandtheuser-itemratingmatrix.3.2UserFeaturesLearningInordertolearnthecharacteristicsorfeaturesoftheusers,weemploymatrixfactorizationtofactorizetheuseritemmatrix.Theideaofuser-itemmatrixfactorizationistoderiveahigh-quality-dimensionalfeaturerepresentationofusersandofitemsbasedonanalyzingtheuser-itemmatrix.Supposeinauser-itemratingmatrix,wehaveusers,items,andratingvalueswithintherange[01].Actually,mostrecommendersystemsuseintegerratingval-uesfrom1tomaxtorepresenttheusers'judgementsonitems.Inthispaper,withoutlossofgenerality,wemaptheratings1;:::;Rmaxtotheinterval[01]usingthefunction)=x=Rmax.Letijrepresenttheratingofuserforitem,andandbelatentuseranditemfeaturematrices,withcolumnvectorsandrepre-sentingthe-dimensionaluser-speci\fcanditem-speci\fcla-tentfeaturevectorsofuseranditem,respectively.Notethatthesolutionsofandarenotunique.In[19],theconditionaldistributionovertheobservedratingsisde\fnedas:U;V;)==1=1ij;iij(1)where;)istheprobabilitydensityfunctionoftheGaussiandistributionwithmeanandvariance,andijistheindicatorfunctionthatisequalto1ifuserrateditemandequalto0otherwise.Thefunction)isthelogisticfunction)=1(1+exp()),whichmakesitpossibletoboundtherangeofwithintherange[01].Thezero-meansphericalGaussianpriorsarealsoplacedonuseranditemfeaturevectors:)==1;;p)==1;(2)Hence,throughaBayesianinference,wehaveU;VR;;;U;V;=1=1ij;iij=1;=1;(3)ThegraphicalmodelofEq.(3)isshowninFig.2(a).Thisequationrepresentsthemethodonhowtoderivetheusers'latentfeaturespaceorusers'characteristicspurelybasedontheuser-itemratingmatrixwithoutconsideringthefavorsofusers'trustedfriends.Inthenextsection,wewillsysteaticallyillustratehowtorecommendbasedonthetastesoftrustedfriends.3.3RecommendationsbyTrustedFriendsInthissection,weanalyzehowoursocialtrustnetworksa ectourdecisionsorbehaviors,andproposeamethodtorecommendonlybyusingthetastesoftrustedfriends.Supposewehaveadirectedsocialtrustgraph=(),wherethevertexset=1representsalltheusersinasocialtrustnetworkandtheedgesetrepresentsthetrustrelationsbetweenusers.Letijdenotethematrixof,whichisalsocalledthesocialtrustmatrixinthispaper.Forapairofvertices,and,letij(01]denotetheweightassociatedwithanedgefromto,andij=0,otherwise.Thephysicalmeaningoftheweightijcanbeinterpretedashowmuchausertrustsorknowsuserinasocialnetwork.Notethatsocialtrustmatrixisanasymmetricmatrix,sinceinatrust-basedsocialnetworkusertrustingdoesnotnecessaryindicateusertrustsAsanalyzedinSection1,wealwaysturntoourfriendsforrecommendationssincewetrustourfriends.Wealsobelievethatmostprobablywewillliketheitems(books,music,movies,etc.)thatourtrustedfriendsrecommend.Eveniftherecommendeditemsarenotthetypeswelike,westillhaveahighprobabilitytobein\ruencedbyourtrustedfriends.Intherealworld,supposeauserwantstoseethemovie\TheDarkKnight"(supposeitistheiteminFig.1(b)),whichisnowplayingatthetheaters,buthe/sheknowsnothingaboutthemovie,likeuserinFig.1(b).Whatthisusernormallydoistotakeintoaccounthis/hertrustedfriends'recommendations.Amongallofhis/hertrustedfriendsinFig.1(a),andratedthismovieas4and5,andtrusts(weight1.0)morethan(weight0.6).Basedontheinformation,thereisaveryhighprobabilitythatwilldrawtheconclusionthat\TheDarkKnight"isaverygoodmovieworthofwatching.Fromtheaboveanalysis,wecangeneralizetheaboveso-cialprocessas (a)FactorizationofUser-ItemMatrix (b)RecommendationsbyTrustedFriends (c)RecommendationswithSocialTrustEnsembleFigure2:GraphicalModelsik2Tjkij jT(4)whereikisthepredictionoftheratingthatuserwouldgiveitemjkisthescorethatusergaveitemisthefriendssetthatusertrustsandjTisthenumberoftrustedfriendsofuserintheset).jTcanbemergedintoijsinceitisthenormalizationtermoftrustscores.Hence,Eq.(4)canbesimpli\fedasik2Tjkij(5)Thenthepredictionoftheratingsthatusergivestoalltheitemscanbeinferredas:::in1121:::R1222:::R:::::::::::::::Rmn:::im(6)WecantheninferthatforalltheuserstoobtainSR;(7)whereSRcanbeinterpretedastherecommendationspurelybasedonthetrustedfriends'tastes.Fromthesocialtrustnetworkaspect,wede\fnethecon-ditionaldistributionovertheobservedratingsasS;U;V;)==1=1ij2Tik;ij(8)whereikisnormalizedbyjT,whichisthenumberoftrustedfriendsofuserintheset).ijistheindicatorfunctionthatisequalto1ifuserrateditemandequalto0otherwise.Hence,similartoEq.(3),throughaBayesianinference,wehaveU;VR;S;;;S;U;V;S;S;(9)InEq.(9),wecanassumethatisindependentwiththelow-dimensionalmatricesand,thenthisequationcanbechangedtoU;VR;S;;;S;U;V;=1=1ij2Tik;ij=1;=1;(10)where)and)arezero-meansphericalGaus-sianpriorsonuseranditemfeaturevectors.Thisequationspeci\festhemethodtorecommendpurelybasedonusers'trustedfriends'tastes.ThegraphicalmodelisshowninFig.2(b).3.4SocialTrustEnsembleInSection3.2,giventheuser-itemratingmatrix,theob-servedratingijisinterpretedbytheuser'sfavoronitem,whileinSection3.3,giventheuser-itemratingmatrixandusers'socialtrustnetwork,theobservedratingijisrealizedasthefavorsonitemofuser'strustedfriends.Actually,bothoftheaboveassumptionsarepartiallyrightsinceintherealworldsituation,everyuserhashis/herowntasteandatthesametime,everyusermaybein\ruencedbyhis/herfriendshe/shetrusts.Hence,inordertode\fnethemodelmorerealistically,everyobservedratingintheuseritemmatrixshouldre\rectbothofthesetwofactors.Basedonthismotivation,wemodeltheconditionaldistributionovertheobservedratingsas:U;VR;S;;;=1=1ij U+(12Tik;ij=1;=1;(11)InEq.(11),theusers'favorsandthetrustedfriends'favoraresmoothedbytheparameter,whichnaturallyfusesappropriateamountofrealworldrecommendationprocessesintotherecommendersystems.Theparametercontrolshowmuchdouserstrustthemselvesortheirtrustedfriends.ItisalsothereasonwecallourapproachRecommendationwithSocialTrustEnsemble(RSTE).ThegraphicalmodelofRSTEisshowninFig.2(c). Thelogoftheposteriordistributionfortherecommenda-tionsisgivenbylnU;VR;S;;;)= =1=1ijij U+(12Tik)) =1 =1 =1=1ij)ln mllnnlln)+(12)whereisaconstantthatdoesnotdependontheparame-ters.Maximizingthelog-posteriorovertwolatentfeaturewithhyperparameters(i.e.,theobservationnoisevariancandpriorvariances)kept\fxedisequivalenttominimizingthefollowingsum-of-squared-errorsobjectivefunctionswithquadraticregularizationterms:R;S;U;V =1=1ijij U+(12Tik)) 2kUk2F+V (13)where==,andkkdenotestheFrobeniusnorm.AlocalminimumoftheobjectivefunctiongivenbyEq.(13)canbefoundbyperforminggradientdescentin @U=1ij U+(12Tik U+(12Tikij+(12B=1 U+(12T U+(12T @V=1ij U+(12Tik U+(12Tikij U+(12Tik)+(14)where)isthederivativeoflogisticfunction)=exp((1+exp())and)isthesetthatincludesalltheuserswhotrustuser.Inordertoreducethemodelcomplexity,inalloftheexperimentsweconductinSection4weset3.5ComplexityAnalysisThemaincomputationofgradientmethodsisevaluatingtheobjectfunctionanditsgradientsagainstvariables.Becauseofthesparsityofmatricesand,thecompu-tationalcomplexityofevaluatingtheobjectfunctionis kl),whereisthenumberofnonzeroentriesinthematrix,and istheaveragenumberoffriendsthatTable1:StatisticsofUser-ItemRatingMatrixofEpinions Statistics User Item Max.Num.ofRatings 1960 7082 Avg.Num.ofRatings 12.21 7.56 Table2:StatisticsofSocialTrustNetworkofEpinions Statistics TrustperUser BeTrustedperUser Max.Num. 1763 2443 Avg.Num. 9.91 9.91 ausertrusts.Sincealmostalloftheonlinesocialnetworks\ftthepower-lawdistribution,alargelongtailofusersonlhavefewtrustedfriends.Thisindicatesthatthevalueof isrelativelysmall.Thecomputationalcomplexitiesforthgradients @Uand @VinEq.(14)are pl p kl)and kl),respectively,where istheaveragenumberoffriendswhotrustauser,whichisalsoasmallvalue.Ac-tually,inasocialtrustgraph,thevalueof isalwaysequaltothevalueof ,whichis991inthedatasetweemployintheSection4.Therefore,thetotalcomputationalcomplex-ityinoneiterationis pl p kl),whichindicatesthattheoretically,thecomputationaltimeofourmethodislinearwithrespecttothenumberofobservationsintheuser-itemmatrix.Thiscomplexityanalysisshowsthatourproposedapproachisveryecientandcanscaletoverylargedatasets.4.EMPIRICALANALYSISInthissection,weconductseveralexperimentstocom-paretherecommendationqualitiesofourRSTEapproachwithotherstate-of-the-artcollaborative\flteringandtrust-awarerecommendationmethods.Ourexperimentsarein-tendedtoaddressthefollowingquestions:(1)Howdoesourapproachcomparewiththepublishedstate-of-the-artcollaborative\flteringandtrust-awarerecommendationalgorithms?(2)Howdoesthemodelparametera ecttheaccuracyofprediction?(3)Whatistheperformancecom-parisononuserswithdi erentobservedratings?(4)Canouralgorithmachievegoodperformanceevenifusershavefewobservedratingrecords?(5)Isouralgorithmecientwhentrainingthemodel?4.1DatasetDescriptionWechooseEpinionsasthedatasourceforourexperi-mentsonrecommendationwithsocialtrustensemble.Epin-ions.comisawellknownknowledgesharingsiteandreviewsite,whichwasestablishedin1999.Inordertoaddreviews,users(contributors)needtoregisterforfreeandbeginsubmittingtheirownpersonalopinionsontopicssuchasprod-ucts,companies,movies,orreviewsissuedbyotherusers.Userscanalsoassignproductsorreviewsintegerratingsfrom1to5.Theseratingsandreviewswillin\ruencefuturecustomerswhentheyareabouttodecidewhetheraprod-uctisworthbuyingoramovieisworthwatching.EverymemberofEpinionsmaintainsa\trust"listwhichpresentsasocialnetworkoftrustrelationshipsbetweenusers.Epinionsisthusanidealsourceforexperimentsonsocialtrustrecommendation.Thedatasetusedinourexperimentsiscollectedbycrawl-ingtheEpinions.comsiteonJan2009.Itconsistsof51,670userswhohaveratedatotalof83,509di erentitems.Thetotalnumberofratingsis631,064.Thedensityoftheuser-itemratingmatrixislessthan0015%.Wecanobservethat Table3:PerformanceComparisons(ASmallerMAEorRMSEValueMeansaBetterPerformance) TrainingData Metrics Dimensionality=5 Dimensionality=10 Trust PMF SoRec RSTE Trust PMF SoRec RSTE 90% MAE 0.9054 0.8676 0.8442 0.8377 0.9039 0.8651 0.8404 0.8367 RMSE 1.1959 1.1575 1.1333 1.1109 1.1917 1.1544 1.1293 1.1094 80% MAE 0.9221 0.8951 0.8638 0.8594 0.9215 0.8886 0.8580 0.8537 RMSE 1.2140 1.1826 1.1530 1.1346 1.2132 1.1760 1.1492 1.1256 1-10 11-20 21-40 41-80 81-160 �160 0 0.5 1 1.5 2x 104 Number of Observed RatingsNumber of Test Ratings (a)DistributionofTestingData(90%asTrainingData) 1-10 11-20 21-40 41-80 81-160 �160 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 Number of Observed RatingsMAEDimensionality = 10 Trust PMF SoRec RSTE (b)MAEComparisononDi erentUserRatingScales(90%asTrainingData) 1-10 11-20 21-40 41-80 81-160 �160 1 1.05 1.1 1.15 1.2 1.25 1.3 1.35 1.4 Number of Observed RatingsRMSEDimensionality = 10 Trust PMF SoRec RSTE (c)RMSEComparisononDi erentUserRatingScales(90%asTrainingData)Figure3:PerformanceComparisononDi erentUserstheuser-itemratingmatrixofEpinionsisverysparse,sincthedensitiesforthetwomostfamouscollaborative\flterindatasetsMovielens(6,040users,3,900moviesand1,000,20ratings)andEachmovie(74,424users,1,648moviesand2,811,983ratings)are425%and229%,respectively.More-over,animportantfactorthatwechoosetheEpinionsdataseisthatusersocialtrustnetworkinformationisnotincludeintheMovielensandEachmoviedatasets.ThestatisticsoftheEpinionsuser-itemratingmatrixissummarizedinTa-ble1.Astotheusersocialtrustnetwork,thetotalnumberofissuedtruststatementsis511,799.ThestatisticsofthidatasourceissummarizedinTable2.4.2MetricsWeusetwometrics,theMeanAbsoluteError(MAE)andtheRootMeanSquareError(RMSE),tomeasurethepre-dictionqualityofourproposedapproachincomparisonwithothercollaborative\flteringandtrust-awarerecommenda-tionmethods.ThemetricsMAEisde\fnedas:MAEi;ji;ji;j (15)wherei;jdenotestheratingusergavetoitemi;jde-notestheratingusergavetoitemaspredictedbyamethod,anddenotesthenumberoftestedratings.ThemetricsRMSEisde\fnedas:RMSE i;ji;ji;j (16)4.3ComparisonInthissection,inordertoshowtheperformanceimprove-mentofourRSTEapproach,wecompareourmethodwiththefollowingapproaches.1.PMF:thismethodisproposedbySalakhutdinovandMinhin[19].Itonlyusesuser-itemmatrixfortherec-ommendations,anditisbasedonprobabilisticmatrixfactorization.2.Trust:thisisthemethodpurelyusestrustedfriends'tastesmakingrecommendations.ItisproposedinSec-tion3.3inthispaper.ItisalsoaspecialcaseofRSTEwhen=0.3.SoRec:thisisthemethodproposedin[13].Itisasocialtrust-awarerecommendationmethodthatfac-torizestheuser-itemratingmatrixandusers'socialtrustnetworkbysharingthesameuserlatentspace.Weusedi erentamountsoftrainingdata(90%,80%)totestthealgorithms.Trainingdata90%,forexample,meanswerandomlyselect90%oftheratingsfromEpinionsdatasetasthetrainingdatatopredicttheremaining10%ofratings.Therandomselectionwascarriedout5timesindependently.Theexperimentalresultsusing5and10dimensionstorep-resentthelatentfeaturesareshowninTable3.Theparametersettingsofourapproachare=04forboth90%trainingdataand80%trainingdata,001,andinalltheexperimentsconductedinthefollowingsections,wesetalloftheparametersequalto0001.FromTable3,wecanobservethatourapproachRSTEout-performstheothermethods.Ingeneral,twosocialtrustrecommendationapproachesSoRecandRSTEallperformbetterthanthePMFmethod(onlyusestheuser-itemma-trixforrecommendations).However,theTrustmethodper-formsworsethanthePMFmethod,whichindicatespurelyutilizingtrustedfriends'tastestorecommendisnotap-plicable.Amongthesethreetrust-awarerecommendationmethods,ourRSTEmethodgenerallyachievesbetterper-formancethantheSoRecandTrustmethodsonbothMAEandRMSE.Thisdemonstratesthatourinterpretationontheformationoftheratingsisrealisticandreasonable.4.4PerformanceonDifferentUsersOnechallengeoftherecommendersystemsisthatitisdiculttorecommenditemstouserswhohaveveryfew 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.82 0.84 0.86 0.88 0.9 0.92 0.94 Values of aMAE90% as Training Data (a) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.12 1.14 1.16 1.18 1.2 1.22 1.24 Values of aRMSE90% as Training Data (b) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.84 0.86 0.88 0.9 0.92 0.94 Values of aMAE80% as Training Data (c) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.12 1.14 1.16 1.18 1.2 1.22 1.24 Values of aRMSE80% as Training Data (d)Figure4:ImpactofParameter(Dimensionality=10) 0 50 100 150 200 250 300 350 400 0.8 0.85 0.9 0.95 1 1.05 1.1 1.15 1.2 1.25 1.3 IterationsMAEDimensionality = 10 RSTE a = 0 RSTE a = 0.4 RSTE a = 0.7 RSTE a = 1 (a) 0 50 100 150 200 250 300 350 400 1.1 1.15 1.2 1.25 1.3 1.35 1.4 1.45 1.5 1.55 IterationsRMSEDimensionality = 10 RSTE a = 0 RSTE a = 0.4 RSTE a = 0.7 RSTE a = 1 (b)Figure5:EciencyAnalysis(90%asTrainingData)ratings.Hence,inordertocompareourapproachwiththeothermethodsthoroughly,we\frstgroupalltheusersbasedonthenumberofobservedratingsinthetrainingdata,andthenevaluatepredictionaccuraciesofdi erentusergroups.TheexperimentalresultsareshowninFig.3.Usersaregroupedinto6classes:\110",\1120",\2140",\4180",\81160"and\160",denotinghowmanyratingsusershaverated.Fig.3(a)summarizesthedistributionsoftestingdataac-cordingtogroupsinthetrainingdata(90%astrainingdata).Forexample,thereareatotal3,360user-itempairstbepredictedinthetestingdatasetinwhichtherelateduserinthetrainingdatasethaveratingnumbersfrom1to10.InFig.3(b)andFig.3(c),weobservethatourRSTEalgorithmconsistentlyperformsbetterthanothermethods,especiallywhenfewuserratingsaregiven.Whenusers'ratingrecordsarerangingfrom1to80,ourRSTEmethodperformsmuchbetterthantheTrust,PMFandSoRecapproaches.4.5ImpactofParameterInourmethodproposedinthispaper,theparameterbalancestheinformationfromtheusers'owncharacteristicsandtheirfriends'favors.Itcontrolshowmuchourmethodshouldtrustusersthemselvesandtheirfriends.If=1,weonlyminetheuser-itemratingmatrixformatrixfactoriza-tion,andsimplyemployusers'owntastesinmakingrecom-mendations.If=0,weonlyextractinformationfromthesocialtrustgraphtopredictusers'preferencespurelyfrothefriendstheytrust.Inothercases,wefuseinformationfromtheuser-itemratingmatrixandtheusersocialtrustnetworkforprobabilisticmatrixfactorizationand,further-more,topredictratingsfortheusers.Fig.4showstheimpactsofonMAEandRMSE.Weobservethatthevalueofimpactstherecommendationre-sultssigni\fcantly,whichdemonstratesthatfusingtheusers'owntasteswiththeirfriends'favorsgreatlyimprovestherec-ommendationaccuracy.Nomatterusing90%trainingdataor80%trainingdata,asincreases,theMAEandRMSEdecrease(predictionaccuracyincreases)at\frst,butwhensurpassesacertainthreshold,theMAEandRMSEin-crease(predictionaccuracydecreases)withfurtherincreaseofthevalueof.Thisphenomenoncon\frmswiththeintu-itionthatpurelyusingtheuser-itemratingmatrixorpurelusingtheusers'socialtrustnetworkforrecommendationscannotgeneratebetterperformancethanfusingthesetwofavorstogether.FromFig.4(a)andFig.4(b),whenusing90%ratingsastrainingdata,weobservethat,ourRSTEmethodachievesthebestperformancewhenisaround0.4,whilesmallervalueslike=01orlargervalueslike=07canpoten-tiallydegradethemodelperformance.Thisindicatesthatweneedtotrustmoreaboutthetastesofusers'trustedfriendsthantheirowntastes,sincethetrainingdataofuser-itemmatrixisverysparse,whichcanhardlylearntheaccu-ratecharacteristicsofusers.InFig.4(c)andFig.4(d),whenusing80%ratingsastrainingdata,theoptimalvalueofisalsoaround0.4.However,lessratingsforuserswillleadtoanoveralldegradationoftherecommendationresults.4.6TrainingEfciencyAnalysisThecomplexityanalysisinSection3.5statesthatthecomputationalcomplexityofourapproachislinearwithre-specttothenumberofratings,whichshowsthatourap-proachisscalabletoverylargedatasets.Actually,ourap-proachisveryecientevenwhenusingaverysimplegra-dientdescentmethod.Intheexperimentsusing90%ofthedataastrainingdata,ourmethodonlyneedslessthan400iterationsfortraining,andeachiterationonlyrequireslessthan20seconds.AlltheexperimentsareconductedonanormalpersonalcomputercontaininganIntelPentiumDCPU(3.0GHzDualCore)and1Gmemory.Fig.5(a)andFig.5(b)showtheperformance(MAEandRMSE)changeswiththeiterations.Weobservethatwhenusingalargevalueof,suchas=1or=07,attheendofthetraining,themodelbeginstoover\ft(especiallyfortheRMSE),whilearelativelysmaller,suchas=0or=04,doesnothavetheover\fttingproblem.Theseexperimentsclearlydemonstratethatinthisdataset,anapproachignoringthesocialtrustinformationcancausetheover\fttingproblem,andthatthepredictiveaccuracycanbeimprovedbyincorporatingappropriateamountofsocialtrustinformation.5.CONCLUSIONSANDFUTUREWORKThispaperismotivatedbythefactthatauser'strustedfriendsontheWebwilla ectthisuser'sonlinebehavior.Basedontheintuitionthateveryuser'sdecisionsontheWeb shouldincludeboththeuser'scharacteristicsandtheuser'strustedfriends'recommendations,weproposeanovel,e ectiveandecientprobabilisticmatrixfactorizationframeworkfortherecommendersystems.Experimentalanaly-sisontheEpinionsdatasetshowsthepromisingfutureofourproposedmethod.Moreover,themethodintroducedinthispaperbyusingprobabilisticmatrixfactorizationisnotonlyworkingintrust-awarerecommendersystems,butalsoapplicabletootherpopularresearchtopics,suchassocialsearch,collaborativeinformationretrieval,andsocialdatamining.Inthispaper,althoughweemploythetrustedfriends'opinionsinthesocialtrustnetworktomakerecommenda-tionsfortheusers,wedonotconsiderthepossibledi usionoftrustsbetweenvarioususers.Underthecircumstancethaboththeuser-itemratingmatrixandthetrustrelationsofasocialnetworkareverysparse,thedi usionsoftrustre-lationsbecomeinevitablesincethisconsiderationwillhelptoalleviatethedatasparsityproblemandwillpotentiallyincreasethepredictionaccuracy.Weplantoemploythedi usionprocessesinourfuturework.InmanypopularapplicationsontheWeb,usersnotonlycankeepalistoftrustrelationships,butalsohavetherightstoestablishalistofdistrustorblockrelationships.Ifauserisinthedistrustlistofauser,mostprobably,itisbe-causetheuserthinkstheuser'stasteistotallydi erentfromhim/her.Actually,thisinformationisveryusefulontherecommendersystems.Unfortunately,tothebestofourknowledge,nopreviousworkcanemploythisinformationwellintorecommendersystems.Theunderstandingofdis-trustrelationsisstilluncleartotheresearchers:Wecannotusedi usionmethodstomodelitduetothereasonthatoneperson'senemy'senemyisnotnecessarilytheenemyofthisperson.Inthefuture,weplantostudytheformationandnatureofthedistrustrelations,andexplicitlymodelthemintherecommendersystems.Astheexponentialgrowthofonlinesocialnetworksitescontinues,theresearchofsocialsearchisbecomingmoreanmoreimportant.Wealsoplantodevelopsimilartechniquestoallowusers'trustedfriendstoin\ruencetheusers'searcresultsorquerysuggestions.Theintuitionbehindthisisthatifalargenumberofourfriendsaresearchingforsome-thing,it'slikelythatwemaybeinterestedinthattopictooThiswouldbeaninterestingsearchphenomenontoexploreinsocialnetworks.6.ACKNOWLEDGMENTSWethankDr.HaixuanYang,Mr.ShikuiTuandMr.TomChaoZhouformanyvaluablediscussionsonthistopic.TheworkdescribedinthispaperissupportedbygrantsfromtheResearchGrantCounciloftheHongKongSpecialAd-ministrativeRegion,China(ProjectNo.:CUHK4128/08EandCUHK4158/08E).ThisworkisalsoaliatedwiththeMicrosoft-CUHKJointLaboratoryforHuman-centricCom-putingandInterfaceTechnologies.7.REFERENCES[1]R.Andersen,C.Borgs,J.Chayes,U.Feige,A.Flaxman,A.Kalai,V.Mirrokni,andM.Tennenholtz.Trust-basedrecommendationsystems:anaxiomaticapproach.InProc.ofWWW'08,pages199{208,NewYork,NY,USA,2008.ACM.[2]P.Bedi,H.Kaur,andS.Marwaha.Trustbasedrecommendersystemforsemanticweb.InProc.ofIJCAI'07,pages2677{2682,2007.[3]J.S.Breese,D.Heckerman,andC.Kadie.Empiricalanalysisofpredictivealgorithmsforcollaborative\fltering.InProc.ofUAI'98,1998.[4]J.Canny.Collaborative\flteringwithprivacyviafactoanalysis.InProc.ofSIGIR'02,pages238{245,NewYork,NY,USA,2002.ACM.[5]M.DeshpandeandG.Karypis.Item-basedtop-nrecommendation.ACMTransactionsonInformationSystems,22(1):143{177,2004.[6]J.L.Herlocker,J.A.Konstan,A.Borchers,andJ.Riedl.Analgorithmicframeworkforperformingcollaborative\fltering.InProc.ofSIGIR'99,pages230{237,NewYork,NY,USA,1999.ACM.[7]T.Hofmann.Collaborative\flteringviagaussianprobabilisticlatentsemanticanalysis.InProc.ofSIGIR'03,pages259{266,NewYork,NY,USA,2003.ACM.[8]T.Hofmann.Latentsemanticmodelsforcollaborative\fltering.ACMTransactionsonInformationSystems22(1):89{115,2004.[9]R.Jin,J.Y.Chai,andL.Si.Anautomaticweightingschemeforcollaborative\fltering.InProc.ofSIGIR'04pages337{344,NewYork,NY,USA,2004.ACM.[10]G.Linden,B.Smith,andJ.York.Amazon.comrecommendations:Item-to-itemcollaborative\fltering.IEEEInternetComputing,pages76{80,Jan/Feb2003.[11]N.N.LiuandQ.Yang.Eigenrank:aranking-orientedapproachtocollaborative\fltering.InProc.ofSIGIR'08pages83{90,NewYork,NY,USA,2008.ACM.[12]H.Ma,I.King,andM.R.Lyu.E ectivemissingdatapredictionforcollaborative\fltering.InProc.ofSIGIR'07pages39{46,NewYork,NY,USA,2007.ACM.[13]H.Ma,H.Yang,M.R.Lyu,andI.King.SoRec:SocialrecommendationusingprobabilisticmatrixfactorizationInProc.ofCIKM'08,pages931{940,NewYork,NY,USA,2008.ACM.[14]P.MassaandP.Avesani.Trust-awarecollaborative\flteringforrecommendersystems.InProc.ofCoopIS/DOA/ODBASE,pages492{508,2004.[15]P.MassaandP.Avesani.Trust-awarerecommendersystems.InProc.ofRecSys,pages17{24,NewYork,NY,USA,2007.ACM.[16]J.D.M.RennieandN.Srebro.Fastmaximummarginmatrixfactorizationforcollaborativeprediction.InProc.ofICML'05,2005.[17]P.Resnick,N.Iacovou,M.Suchak,P.Bergstrom,andJ.Riedl.Grouplens:Anopenarchitectureforcollaborativ\flteringofnetnews.InProc.ofCSCW'94,1994.[18]R.SalakhutdinovandA.Mnih.Bayesianprobabilisticmatrixfactorizationusingmarkovchainmontecarlo.InProc.ofICML'08,2008.[19]R.SalakhutdinovandA.Mnih.Probabilisticmatrixfactorization.InAdvancesinNeuralInformationProcessingSystems,volume20,2008.[20]B.Sarwar,G.Karypis,J.Konstan,andJ.Reidl.Item-basedcollaborative\flteringrecommendationalgorithms.InProc.ofWWW'01,pages285{295,NewYork,NY,USA,2001.ACM.[21]L.SiandR.Jin.Flexiblemixturemodelforcollaborative\fltering.InProc.ofICML'03,2003.[22]N.SrebroandT.Jaakkola.Weightedlow-rankapproximations.InProc.ofICML'03,2003.[23]J.Wang,A.P.deVries,andM.J.T.Reinders.Unifyinguser-basedanditem-basedcollaborative\flteringapproachesbysimilarityfusion.InProc.ofSIGIR'06pages501{508,NewYork,NY,USA,2006.ACM.[24]Y.ZhangandJ.Koren.Ecientbayesianhierarchicalusermodelingforrecommendationsystem.InProc.ofSIGIR'07,pages47{54,NewYork,NY,USA,2007.ACM.