PCA facilitates dimensionality reduction for of64258ine clus tering of users and rapid computation of recommendations For a database of users standard nearestneighbor tech niques require processing time to compute recom mendations whereas Eigentaste ID: 31547
Download Pdf The PPT/PDF document "Eigentaste A Constant Time Collaborative..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Eigentaste:AConstantTimeCollaborativeFilteringAlgorithmKenGoldbergandTheresaRoederandDhruvGuptaandChrisPerkinsIEORandEECSDepartmentsUniversityofCalifornia,BerkeleyAugust2000EigentasteisacollaborativeÞlteringalgorithmthatusesuniversalqueriestoelicitreal-valueduserratingsonacom-monsetofitemsandappliesprincipalcomponentanalysis(PCA)totheresultingdensesubsetoftheratingsmatrix.PCAfacilitatesdimensionalityreductionforofßineclus-teringofusersandrapidcomputationofrecommendations.Foradatabaseof UCBElectronicsResearchLaboratoryTechnicalReportM00/41.Pdfformatavailablefromwww.ieor.berkeley.edu/goldberg/pubs/. Averybriefandpreliminaryreportonthisalgorithmappearedin[11].ApatentapplicationthatincludessomeelementsofthisalgorithmhasbeenÞledbytheUCRegents. measurement[1]anduser-interfaceadvantagesasdiscussedintheconclusion.Eigentastesplitscomputationsintoofßineandonlinephases.Ofßine,Eigentasteusesprincipalcomponentanal-ysisforoptimaldimensionalityreductionandthenclustersusersinthelowerdimensionalsubspace.Theonlinephaseuseseigenvectorstoprojectnewusersintoclustersandalookuptabletorecommendappropriateitemssothatruntimeisindependentofthenumberofusersinthedatabase.Thispaperisorganizedasfollows.Section2reviewsre-latedwork.Section3introducestheEigentastealgorithm,PCA,andourrecursiverectangularclusteringmethod.Sec-tion4describestheapplicationofEigentastetoJester,aCFsystemforrecommendingjokes,includingadescriptionofthebootstrappingprocess.Section5proposesthenormal-izedMeanAbsoluteError(NMAE)metricandcomparesperformanceofseveralalgorithmsontheJesterdatasetintermsofaccuracyandefÞciency.Section7reviewsthere-sultsanddiscussesfuturework.2RelatedWorkInthissectionwereviewonlyasmallsampleofthepapersonCollaborativeFiltering.Rich[29]isconsideredanearlyreference.ThereisalonghistoryofpatentsrelatedtoCF,rangingfrom[14]in1989to[9]in2000.In1992,D.Gold-berget.al.coinedthetermÒcollaborativeÞlteringÓinthecontextofasystemforÞlteringemailusingbinarycategoryßags[10].Excellentsurveysofresearchcanbefoundin[31,12,7].ShardanandandMaes[30]designedacollaborativeÞl-teringsystemformusic(Ringo)andexperimentedwithanumberofmeasuresofdistancebetweenusers,includingPearsoncorrelation,constrainedPearsoncorrelation,andvectorcosine.Theycomparefourdifferentrecommenda-tionalgorithmsbasedontheMeanAbsoluteErrorofpre-dictions.Alloftheirneighborhood-basedalgorithmsre-quiretimelinearinthenumberofusers.GroupLensisapioneeringandongoingeffortincol-laborativeÞltering[28,18,19,12].TheGroupLensteaminitiallyimplementedaneighborhood-basedCFsystemforratingUsenetarticles.Theyuseda1-5integerratingscaleandcomputeddistanceusingPearsoncorrelations.OneofthenewsgroupsthatGroupLensconsideredwasrec.humor,anunmoderatednewsgroupthatreceiveshun-dredsofpostsaday,mostofthemnotveryfunny(notthatrec.humor.funnyismuchbetter).Thiswasreßectedintheratings,where75%ofthejokesreceivedthelowestpossi-bleratingof1(notfunny).TheGroupLensteamreportedcorrelationvaluesfor500pairsofusers;therelativelyhighvalueofthesecorrelationswasusedtoclaimthatusersgen-erallyagreeontheratingsofjokes,incontrasttotherecipespostedonrec.food.recipes.However,thepredominanceofÒnotfunnyÓratingsinthedataskewedallcorrelationsdramaticallyupward.TheGroupLensteamdidnotetheexistenceofasubstantialnumberoflowandnegativecorrelationsinrec.humor,sug-gestingthattheremaybeindeedbesomevarianceinusertastes.ToevaluateEigentaste,wealsousehumorasado-mainbutusejokeswithamuchhighervariance.ThereareanumberofdifferencesbetweenGrouplensandEigentaste.Breeseet.al.[4]classifycollaborativeÞlteringal-gorithmsintotwoclasses:Memory-basedandModel-based.Memory-basedalgorithmsoperateovertheentireuserdatabasetomakepredictions.Themostcommonmemory-basedmodelarebasedonthenotionofnearest-neighbors,usingavarietyofdistancemeasures.Model-basedsystemsarebasedonacompactmodelinferredfromthedata.InthisframeworkEigentastewouldbeconsid-eredModel-based.Breeseet.al.compareanumberofalgorithmsincludingBayesianclusteringanddecision-treemodels.TheyshowthatBayesiannetworkandcorrelationmodelsarethebest-performingbutdonotdiscusscompu-tationalcomplexity.PennockandHorvitz[26]suggestPersonalityDiagno-sis(PD),alatentvariableapproachbasedoncomputingtheprobabilitythatanewuserisofanunderlyingÒpersonal-itytype,Óandthatuserpreferencesareamanifestationofthispersonalitytype.ThepersonalitytypeofagivenuseristakentobethevectorofÒtrueÓratingsforitemstheuserhasseen.AtrueratingdiffersfromtheactualratinggivenbyauserinGaussiannoise.GiventhepersonalitytypeofauserA,PDÞndstheprobabilitythatthegivenuserisofthesamepersonalitytypeasotherusersinthesystem,andthustheprobabilitythattheuserwilllikesomenewitem.Tocombattheproblemcommontomemory-basedmodelsofincreasedcomputationaleffortasthesetofex-istingusersgrow,PennockandHorvitzexploreaValueofInformation(VOI)computationwhichmaximizespredic-tivevaluewhileminimizingthenumberofexplicitratingsneededfromauser.ThisapproachrequiresthespeciÞca-tionofutilityfunctions.However,VOIcanalsobeusedofßinetoÒpruneÓthedatainthesysteminordertoreducetheamountofdatastoredwhilemaintainingmaximumpre-dictivepower.Inadifferentpaper,PennockandHorvitz[25]proposeanaxiomaticfoundationforcollaborativeÞltering.CFmakespreferencepredictionsbycombiningpreferencesofexistingusers.TheauthorsnotethatÒaggregationÓofpreferenceshasbeenstudiedinSocialChoicetheorysincethe1960Õs[2].Theyarguethatonlyasinglenearest-neighbormodelwillsatisfytheaxiomaticconditions.Delgado[7]takesanagent-basedapproachtoCF,de- velopingseveralalgorithmsthatcombineratingsdatawithothersourcesofinformationsuchasthegeographiclocationoftheuser.Weightedmajorityvotingisusedtocombinerecommendationsfromdifferentsources.Intheirrecentpaper,Herlockeret.al.[12]divideneighbor-basedCFalgorithmsintothreesteps:i)weight-ingpossibleneighbors,ii)selectingneighborhoodsand,iii)producingapredictionfromaweightedcombinationofneighborsratings.TheyexplorealternativemethodsforeachstepandproposeSpearman(rank-based)correlationweightingasanalternativetoPearsoncorrelationsandaÒsigniÞcanceweightingÓbasedonthenumberofitemstwousershaveratedincommon.TocomputepredictionstheyÞndthatsubtractingglobalmeansimprovesperformance,whileconversiontoZ-scoresdoesnot.Tomeasureaccu-racy,theyproposeReceiverOperatorCharacteristic(ROC)sensitivityfromdecision-supporttheory.ManyCFresearchershaverecognizedtheproblemofsparseness:manyvaluesintheratingsmatrixarenullsinceallusersdonotrateallitems.Computingdistancesbetweenusersiscomplicatedbythefactthatthenumberofitemsusershaveratedincommonisnotconstant.Analterna-tivetoinsertingglobalmeansfornullvaluesorsigniÞcanceweightingisSingularValueDecomposition(SVD),whichreducesthedimensionalityoftheratingsmatrixandidenti-Þeslatentfactorsinthedata.AnapplicationofSVDinthecontextofdocumentre-trievalhasbeenpatentedandiswidelyknownasLatentSe-manticIndexing(LSI)[21,6,8,15].InLSI,SVDisappliedtofactorthenon-squareterm-documentfrequencymatrixintoothogonalfactormatriceswithcorrespondingsingularvalues.ThelargestsingularvaluescorrespondtothemostsigniÞcantfactorweightings,whichcanbeusedtocreateadimension-reducinglinearprojectionoftheoriginaldata.BillsusandPazzani[3]andPryor[27]haveappliedSVDtoCFindifferentways.BillsusandPazzani[3]treatCFasaclassiÞcationproblemanddiscretizeratingsintoasmallnumberofclasses(eg.two:likevs.dislike).Saythereareitemsthatformthebasisforrecommendation,anditemstoconsiderrecommending.Let.TheydiscretizetheoriginalratingsmatrixintoaBooleanfeaturematrix,witharowforeachcombinationofuserandcategory.TheyapplySVDtotoreduceitsdimen-sionalityfrom.TheprincipalvectorsareusedtoprojecteachitemtoapointintheBillsusandPazzanithencreatefeedforwardneuralnet-works,oneforeachuserinthedatabase.Theyuseback-propagationtotraineachnetworkusingvectors(onevectorforeachitemratedbythatuser).Af-tertraining,eachnetworkwillmapa-dimensionalvectorrepresentinganunseenitemtoapredictedratingforthatitem.UsingtheMovielensdataset,theydemonstratethattheirmethodyieldsreasonablepredictionaccuracybutnotethatitissigniÞcantlymorecomputationallyexpensivethanothermethodsduetotheneedtotrainaneuralnetworkforeachuser.Pryor[27]recommendswebpagesbasedonBooleanvisitpatternsand7-pointdiscreteratings.ApplyingSVDtothevisitmatrixproducesasetofvectorscorrespondingtofeaturesinthematrix.HefoundthatusingonlythemostsigniÞcantfeatures(asmeasuredbytheirsingularvalues)reducesdimensionalityandprovidesaneffectivedistanceInEigentasteweaddresssparsenessusinguniversalqueries,whichinsurethatallusersrateacommonsetofgaugeitems.Sincetheresultingsubmatrixisdense,wedirectlycomputethesquaresymmetriccorrelationmatrixandthendoalinearprojectionusingPrincipleComponentAnalysis,aclosely-relatedfactoranalysistechniqueÞrstde-scribedbyPearsonin1901[24,5,20,17].LikeSVD,PCAreducesdimensionalitybyoptimallyprojectinghighlycor-relateddataalongasmallernumberoforthogonaldimen-3TheEigentasteAlgorithmInthissectionwedescribethegenericEigentastealgorithm.3.1NotationandTerminologysetofallusersindatabasesetofallitemstoberatedand/orrecommendedsetofitemsinthegaugesetnumberofusers,totalnumberofitems,numberofitemsinthegaugeset,(thusthereareitemsavailableforrecommendation)matrixofrawuserratingsnormalizedmatrixofuserratingsofitemsincorrelationmatrixoftheitemsinthegaugesetorthogonalmatrixofeigenvectorsofdiagonalmatrixofeigenvaluesofrawratingofitembyuseruserrmin;~rmax][;rijnormalizedratingofitembyuserpredictedratingofitemforuseraverageratingofitemsetofitemsratedbyusersetofusershavingrateditem3.2NormalizingRatingsmatrixofrawratingsfromusersand.Weselectedoftheseitemstoformthecommongaugeset(allvalidusersratedallitemsinthegaugeset). Wenormalizethissubsetoftoproduce,thesubmatrixofgaugesetratings.Eachratingisnormalizedbysubtractingitsmeanratingoverallusers,andthendividingbyitsstandarddeviation.Sincevalidusershaveratedallitemsinthegaugeset,therewillbenonullratings.Themeanratingoftheiteminthegaugesetis andthevarianceofthegaugesetitemis ,thenormalizedratingissetto 3.3PearsonsCorrelationMatrixIfweassumeacontinuousratingscaleandalinearrelation-shipbetweenvariables,wecandeÞnetheglobalcorrelation(=[overallusers. issymmetricandpositivedeÞnite.3.4PrincipalComponentAnalysisPrincipalComponentAnalysiswasÞrstintroducedin1901byKarlPearson[24].Hotellinggeneralizedittorandomvariablesin1933[16].Weapplyeigen-analysistosolveforsuchthatECEbealineartransformofsuchthatthetransformedpointsareuncorrelated(itscorrelationmatrixisdiagonal): ECEEachcolumnhasvariance.Aftersortingbyeigenvalue,Figure1showsthevarianceinleftunac-countedforbyeachsuccessivecolumnofinatypicalTheideaistokeeponlytheÒprincipalÓeigenvectors.Thenumberofeigenvectorstoretaindependsonthevariances(eigenvalues)butistypicallysmall.Ifeigenvectorsareretained,dataisprojectedalongtheÞrstprincipaleigen-vectors: Figure1:Screecurveofvariancesexplainedbyconsecutiveeigenvectors.ThelargestamountofvarianceisexplainedbytheÞrsteigenvector.TheÞrsttwoeigenvectorstogetheraccountforforalmost50%ofthetotalvariance.Onpopularchoiceistosetsothatdataarepro-jectedontotheÒeigen-planeÓforhumanvisualization.PCAandtheEigentastealgorithmgeneralizeeasilytohigherdi-3.5RecursiveRectangularClusteringTherearemanywaystoclustertheprojecteddata.ForthedatainSection4,afterplottingintwodimensionswedis-coveredahighconcentrationaroundtheorigin.Weimple-mentedarecursiverectangularclusteringwherecellsizede-creasesneartheorigin.Thiscanbegeneralizedtohigherdi-mensionsandavarietyofalternateclusteringmethodscanbeusedwithEigentaste.1.Startwiththeminimalrectangularcellthatenclosesallthepoints(userprojections)intheeigenplane.Thisformstheoutermostrectangularsubdivision.2.Bisectthiscellalongthexandyaxestoyield4rect-angularsub-cells.3.Foreachnewsub-cellthathastheoriginasoneofitsvertices,performtheoperationinstep2togeneratesub-cellsatthenexthierarchicallevel.4.RepeatStep3foreachleveluntiladesireddepthisFigure2isanillustrationofrecursiverectangularcluster-ingin2dimensionsafter4levelsofrecursionWetreateachcellasaclusterofneighborsintheeigenplane.Foreachcluster,wecomputethemeanforeachnon-gaugeitem,basedonthenumberofuserswhohaverated Figure2:4levelsofrecursion,formingatotalof40clus-ters.thatitem.Sortingthenon-gaugeitemsinorderofdecreas-ingmeanratingsyieldsalookuptableofrecommendationsforthatcluster.Allofthisoccursofßineandperiodically.3.6OnlineComputationofRecommendationsWhenanewuserentersthesystem,1.Collectratingsforallitemsinthegaugeset.2.Usetheprincipalcomponentstoprojectthisvectorontotheeigenplane.3.Findtherepresentativecluster.4.Lookupappropriaterecommendations,presentthemtothenewuser,andcollectratings.4ExperimentalImplementation:JesterWeusehumorasadomainforevaluatingEigentaste.Cananautomatedsystemrecommendafunnyjoke?SincethecriteriaforhumoraredifÞculttoformalize,thisisanon-trivialinformationretrievalproblem.Therehavebeenanumberofpsychologicalstudiesonthehumansenseofhu-mor.Ziv[33]attemptedtocategorizetasteinhumorbasedonsocial,emotional,andintellectualcharacteristics.Thesecharacteristicsinturndependonfactorssuchasgender,age,socialupbringing,etc.Ourapproachavoidssuchsemanticcategoriesandreliessolelyonnumericalratings:Wetreateachuserandeachjokeasablackbox.See[23]forresearchonhumor.WerefertotheimplementedCFsystembasedonEigen-tasteasJester.JesterincludesanHTMLclientinterface Figure3:TheJesterinterface:usersareshownajokeandaskedtorateitbyclickingonthecontinuousratingsbaratthebottomofthescreen.thatallowsInternetuserstoratejokes.WechoseonlyjokesthatcanÞton1-2screenstominimizeevaluationtime.Wepresentjokesandcollectreal-valuedratingsasusersclickonaratingbarimplementedusingtheimagemapcontrolprovidedinHTML.Aftertheuserrateseachjoke,anotherispresented.Afteralljokesinthegaugesetarerated,Jesterrecommendsjokestotheuserandcontinuestocollectratingsoneachrecommendedjoke.Thecommu-nicationbetweentheinterfaceandtheserverscriptstakesplacethroughaCGIscriptwritteninC.Figure3illustratestheinterface.AllcollaborativeÞlteringsystemsexperiencetheÒcoldstartÓproblem[22].Oneneedsratingstopredictratings.Toaddressthis,westartedwithasimplewebsitetocol-lectjokeratings.Wechosetheinitialsetof40jokesfromfriendsandnewsgroups,doingourbesttoavoidhighlyof-fensivejokes.Wethenasked80friendsandstudentstorateall40jokesbyvisitingthewebsite.Weselectedhalfofthesejokes(=20)forthegaugesetbasedonacom-binationoftheircorrelationsandvariances.Herlockeret.al.[13]hypothesizethatgivinghighvarianceitemsmoreinßuenceindeterminingacorrelationwillimprovepredic-tioneffectiveness.Howtochoosethegaugesetwillbeasubjectofafuturepaper.WenextimplementedJester1.0,anaiverecommendersystembasedonlyonthesinglenearestneighborinEu-clideanspace.Weadded30newjokestothesystem().Newuserswereaskedtorateeachjokeinthegaugesetand5non-gaugejokesselectedatrandomtoseedfuturerec-ommendations.Foreachuser,thesinglenearestneighborwasusedtogeneraterecommendationsfromtheremaining Seehttp://eigentaste.berkeley.edu jokes.Figure2isanillustrationoftherecursiverectangularclusteringschemein2dimensions,using4levels.WeregisteredtheJestersitewithanumberofsearchen-gines.BytheendofNovember1998,aboutamonthandahalfofthesysteminception,wehadaround150registeredusers.On2December,at9:25am,JesterwasfeaturedintheCulturesectionofWiredNews[32].OnlinenewssitelikeYahoo,Excite,andNetscapeNewswentupwiththesamestory..Thisproducedasuddeninßuxof41350pagerequests.Sincesystemprocesstimegrewlinearlywiththenumberofusers,Jester1.0wasquicklyoverwhelmed.Itcrashedandwasofßineforseveraldays.OurexperiencewiththistrafÞcoverloadmotivatedustodevelopamorescalablealgorithm,Eigentaste.Jester2.0wasreleasedonMarch1st,1999.Itsgraphicswereredesigned,weadded30newjokestothesystemsothat=100,andthegaugesetwasreducedto10jokes.5Results5.1NormalizedMeanAbsoluteError(NMAE)TheerrormetricusedmostoftenintheCFliteratureistheMeanAbsoluteError(MAE)[30,4,26].Ifisthepre-dictionforhowuserwillrateitem,theMAEforuserisdeÞnedasMAE= isthenumberofitemsuserhasrated.MAEforasetofusersistheaverageMAEoverallmembersofthatSinceournumericalratingscalegivesratingsoverthethe 10;+10],wenormalizetoexpresserrorsasper-centagesoffullscale:NormalizedMeanAbsoluteError,is:NMAE= Herlockeretal.[12]discussesavarietyofothererrormeasures.IntheAppendix,weconsiderNMAEfromatheoreticalperspective.5.2TheJesterDatasetSinceMarch1999,Jesterhascollectedapproximately2,500,000ratingsfrom57,000users.Thereare10jokesinthegaugesetand90non-gaugejokes.Theaveragenum-berofratingsperuseris46.Webasedourexperimentson18,000randomusersfromthissample.TheJesterDataset,includinganonymousratingsfromtheseusers,isavailableuponrequest.Fortheexperimentsbelow,werandomlydividetheusersintotwodisjointsets:trainingandtest.Weusedatafromthetrainingsettocomputepredictionsusingeachalgorithm(ie,toÒtrainÓthesystem).DatafromthetestsetisthenusedtoevaluateefÞciencyandaccuracy.5.3POPAlgorithmThesimplestrecommendationalgorithmistotreatallusersascomingfromthesameglobalclusterandtobaserecom-mendationsforallusersonglobalmeanratings.WeusedthisÒPOPÓalgorithmasourcontrolcase.(Note:ThenameÒPOPÓistakenfrom[4]).POPpredictsratingsforeveryjokebasedonitsglobalaverage.Weusethetrainingsettocomputetheglobalaver-ageandthetestsettoevaluatethepredictions.POPyieldsNMAEof0.203.5.4NearestNeighborAlgorithmsThenearestneighboralgorithmanditsvariantsaretheonesmostwidelyreferencedintheliterature.Theformulagener-allyusedtoÞndthepredictedratingforuseranditemi;p)(~istheaveragejokeratingforuser,andisanormalizingfactorensuringthattheabsolutevalueoftheweightssumto1.Theweightsi;pcanreßectdistances,correlations,orsimilaritiesbetweenuserandotherusersthathaveratedthesameitems.Mostcommonly,i;pthePearsoncorrelationcoefÞcientbetweenusersi;p)(~ q wherethesummationsoverincludeitemsthatbothuseranduserhaveratedincommon[4,30].Wealsoimplementedaweightednearestneighboral-gorithm.WeusedafunctionofEuclideandistancefromtouserastheweighti;p,andi;pSpeciÞcally,ifweareinterestedinnearestneighbors,i;pi;q+1)i;p.ThisensuresthatÕsclosestneighborhasthelargestweight. Torequestthisdata,pleaseemailyourcontactinformationandadescriptionofintendedresearchtotheÞrstauthor. Usingonlyonenearestneighbor(1-NN)provedtobealessaccuratepredictorthantheglobalaveragejokerating(POP).Fornearestneighborcalculations,thepredictedrat-ingforagiventestuserwastheactualratingofhis/hernear-estneighborinthetrainingset.Usingonenearestneighbor,thenormalizedMAEwas0.238,anincreaseoverPOP.InadditiontoÞndingthepredictionerrorforonenear-estneighbor,wecalculatederrorswhenrecommendationswerebasedonormorenearestneighbors(-NN).Inthesecases,thepredictedratingforauserinthetestsetwasthe(unweighted)averageoftheratingsofhisnearestneigh-bors(onecanviewthisasaspecialcaseofthesimilarity-basedweighting,wherei;k)=1foralldqborsthathaveratedtheitem).For,thereissharpimprovement:NMAE=0.224.Inaddition,theerrormono-tonicallydecreasesuntil.Afterthispoint,theer-rorincreasesagain,andasymptoticallyapproachesthePOPvalueasexpected.SeeFigures4and5. Figure4:MAEasafunctionofthenumberofnearestneigh-borsused,for Figure5:MAEasafunctionofthenumberofnearestneighborsused,for.Theerrordecreasessharply,thensteadilyincreasesforAsvisibleinFigure4,thelowestNMAEis0.187,forapproximately80nearestneighbors.5.5EigentasteFortheJesterdataset,theÞrsttwoeigenvectorsaccountedfornearly50%ofallthevariance(seeFigure1).WetooktheÞrsttwoprincipalcomponents()andprojectedthedataontotheeigenplane:Eachuserrepresentsapointinthistwo-dimensionaleigenplane.Aftercomputingpredictionsusingthetrain-ingsetandcomparingwithactualratingsinthetestset,theNMAEforEigentasteis0.187.6ComputationalComplexityRecallthattheratingsdatabasecontainsusersanditemsforratingandrecommendation.InEigentastetherekmitemsinthegaugeset.6.1POPAlgorithmThecomputationalcomplexityforthePOPalgorithmistocomputeglobalmeansand,constanttime,toproviderecommendations.6.2NearestNeighborAlgorithmsFor1-NN,theonlinetimeneededtoÞndrecommendationsforanewuseris.InanimplementationofÞndingthenearestneighbor,alltrainingsetusersarescannedintomemory,andthentraversedforeachtestsetuser.Thetimerequiredtoscanall8853trainingsetuserswas31.64sec-onds.ToÞndthenearestneighborandgeneratethepre-dictionsforonetestusertook438msec.Itisimportanttonotethattheoverridingfactorisscanninginthetrainingsetusers.Forauseronline,itwouldtakeover32secondstogethis/herrecommendationsÑassumingtherearenootherprocessesrunningonthesystem.Itmayalsobecomenec-essarytoscantrainingsetusersfromaÞleiftheirnumbergrowslargerthanisreasonabletohaveinmemory.ThiswouldsigniÞcantlyincreasethetimenecessarytoÞndpre-For-NN(where)thetimerequiredwillslightlyincreaseasgrows,butthetimetoÞndthenearestneigh-borsremains6.3EigentasteAlgorithmFortheofßinephase,Þndingthecorrelationmatrix,cal-culatingtheeigenvectors,projectingusersintotheeigenplane,andclusteringthemtakestime.Notethat issmall(inourcase=10),sothisphaseisnottoode-manding.Usingasimilarimplementationtotheoneusedforthenearestneighboralgorithmabove,ittook27.85sec-ondstoscanall8853trainingsetusers,444msectogener-atetheeigenvectors/eigenvalues,and298msectogeneratetheclustersandthepredictionsforeachcluster.Thetotalofßinetimeneededfor8853trainingsetuserswas29.59Online,however,recommendationscanbemadeincon-stanttime:.Thus,thereisnoincreaseintimerequiredtoÞndarecommendationasthenumberofusersinthesys-temincreases.ForJester,thetimetoprojecttheratingsfromthegaugesetandtolookuptherecommendationsis3.22msecperuser.TheEigentastealgorithmprovidesasigniÞcantdecreaseinonlinecomputationtime.7DiscussionInthispaperwedescribeEigentaste,anewCFalgorithmthatappliesPCAtoadensesubsetoftheratingsmatrix.Eigentasteusesuniversalqueriestoelicitreal-valueduserratingsonacommonsetofitems.PCAfacilitatesdimen-sionalityreductionforofßineclusteringofuserandrapidonlineclusterassignment.AccuracyandefÞciencyresultsontheJesterdatasetaresummarizedinthetablebelow. Accuracy Of ine Online Online Algorithm (NMAE) time peruser POP 0.203 O( O(1) - 1-NN 0.237 - O(nk) 350msec 80-NN 0.187 - O(nk) 350msec Eigentaste 0.187 O(k2n) O(k) 3.2msec TheseNormalizedMeanAbsoluteError(NMAE)valuesindicatethatpredictedratingsvalueswillbewithinroughly20%ofthetrueratingsvaluesforeachalgorithm.Soitemswithpredictedratingswellabovethemeanforanewuserwillinmanycasescorrespondtodesireableitemsforthatuser.Itisinterestingtonotethattheseaccuraciesarecompa-rablewiththosereportedforacompletelydifferentdataset(movies);thealgorithmsinHerlockeretal[12],whennor-malizedtothe4unitratingscale(1-5),yieldNMAEfrom0.192to0.207.ThePOP(globalmean)algorithmoffersausefulbaselineforaccuracy(seeAppendixonotherbaselinecomparisons).IntermsofNMAEthePOPalgorithmperformsreasonablywell,asotherresearchershavefound[12].Itiscompu-tationallyefÞcientbutcompletelyignoresdifferencesbe-tweenusers.Nearest-neighbormethodsofferimprovedaccuracy,un-lessnotenoughneighborsareconsidered(eg.1-NN),whichmakesindividualrecommendationshighlysusceptibletotonoise.Ifthe80nearestneighborsareconsidered(80-NN),noiseisreducedandaccuracyimprovesabout8%overPOP,butatthecostofconsiderableonlinecomputationalasthenumberofusersgrows.Ofcourseitmaybepossibletopre-processtheusergrouptoselectorcreateasmallnumberofrepresentativeusers(ÒmentorsÓ)tokeepForthisdatasetandimplementation,EigentasteÕsaccu-racyisasgoodas80-NNbutitsonlinecomputationisfasterbytwoordersofmagnitude.Theseresultssuggestthatspeedupcanbeachievedwithoutcompromisingpredictionaccuracy.Weareexperimentingwithanumberofvariations,suchasofßinek-meansclusteringwithcosinedistancemeasures,andhybridapproacheswithadaptiveonlineweightingtofurtherimproveaccuracywithoutalteringonlinecomputa-tiontime.Eigentasteaddressesthesparsenessproblemwithuniver-salqueriesinsteadofuser-selectedqueries;eachquerycon-tainsashortunbiaseddescription(eg,booksummaryorÞlmsynopsis)sothatuserscanformanimmediateopin-ion.Usinguniveralqueries,Eigentastepresentseachuserwiththesamegaugesetofitemstorateduringitspro-Þlingphase.Theresultingsubsetoftheratingsmatrixisdense.AsdiscussedinSection1,itcanbearguedthatuni-versalqueriesarelesseffectivethanuser-selectedqueries.Butuniversalqueriesareparticularlyappropriateforsomedomains,suchasjokes,newsarticles,images,andmusicclips,whereabriefsampleisavailabletobeevaluatedbyallusers.Althoughweapplieduniversalqueriesinthedo-mainofjokes(wherethequeryissimplythejokeitself),weareintheprocessoftestingEigentasteinotherdomainsincludingbooks.Weplantowriteanotherpaperonthede-signofuniversalqueriesandthechoiceofwhichitemstoincludeinthegaugeset.Whenproperlydesigned,universalqueriesoffertheadvantageofrapidandconsistentproÞlingandtheabilitytocollectimmediatefeebackonallrecom-mendeditems.Theeffectivenessofuniversalqueries,con-Þdencequeries,andthepotentialforhybridquerymodelsdependsonthedomainandisasubjectforfuturestudy.EigentastecapturesuserratingsonacontinuousratingscaleusingtheHTMLimagemapprotocol.Continuousrat-ingshavethreeadvantages:1)theyavoiddiscretizationef-fectsinmatrixcomputations2)theycapturetastewithÞnergranularity[1],and3)usersreportthattheyÞndthecon-tinuousratingbareasiertouse.UsersoftenreportadesiretochooseavalueÒbetweenÓtwodiscreteoptions.Theet-ymologyofthewordÒtasteÓsuggeststhedigestivesystem:auserÕsratingisliterallyaÒgutreactionÓ.Ifso,thecontin-uousratingbarmayofferamorevisceralinterface.Last,usersmayprovidemoredataiftheinterfacefeelsmorelikenavigatingavideogamethanansweringaquestionnaire.Moreresearchonthisissueisneeded. IntheAppendix,weconsidertheaccuracymetricfromatheoreticalperspective.WehavereportedNormalizedMeanAbsoluteErrorvaluesforourexperimentsandnotedthatthesevaluescomparesurprisinglywelltovaluesre-portedinotherdomainsandexperiments.HowsigniÞcantisaNMAEof?TocomparetheCFperformancetorandomguessing,weuseUniformandNormalnoisedistri-butionmodelstoderiveanalyticestimatesofNMAE.WeÞndthatifuserratingsareuniformlydistributed,randompredictionsyieldNMAE=33%.ThissuggeststhatthereisroomforimprovedaccuracyforallcurrentCFalgorithms.AAppendix:NMAEforRandomPredic-TocomparetheCFperformancetorandomguessing,weuseUniformandNormaldistributionmodelstoestimateNormalizedMeanAbsoluteError(NMAE)analytically.betheuserÕsratingandbethepredictedrating.beindependentrandomvariables.UniformDistributionrandomvariablesontheintervalal 10;10].Theprobabilitydistributionoftheerror,isatriangularfunctionovertherange.Takingtheabsolutevaluefoldsthisfunctionontothepositiveaxis.Normalizingtointegrateto1,theMAEdensityfunction,,is.TheexpectedvaluefortheMAEEMAENormalizingovertherangeofvalues,NMAE=0.333.Thatis,ifactualandpredictedvaluesareuniformlydis-tributed,weÕdexpecttherandompredictions,onaverage,tobeoffbyathirdoffullscale.ToconÞrm,weperformedaMonteCarloexperimentwheretheactualandpredictedratingsarerandomnumbersuniformlydistributedbetween-10to+10.TheNMAEwefoundwas0.320,veryclosetotheanalyticprediction.NormalDistributionAssumenowthatboththemeasuredandpredictedratingsareNormallydistributedrandomvariableswiththesame,andvariances,respectively,are,again,independent.Fromthemoment-generatingfunc-tionofrandomvariables, 221t2+1 isalsoNormallydistributed,withmean0andvarianceFortheJesterdataset,theaveragestandarddeviationis.Assumeboththeactualandpredictedratingshave,thedensityfunctionfor p TheexpectedMAE,then,is:is:MAE 5p TheexpectedNMAEis0.282.Thatis,ifactualandpre-dictedvaluesarenormallydistributed,weÕdexpectthepre-dictions,onaverage,tobeoffby28%.BAcknowledgmentsThisworkwassupportedinpartbyNSFAwardIRI-9553197.WethankMarkDigiovanniandHiroNaritafortheirhelpindevelopingtheinitialversionofJester1.0,andMatthiasRunteforearlycomparisonswithnearest-neighbormethods.Wearegratefulfortheinsightfulin-putwereceivedfromfromHalVarian,GordonRios,MingZhang,AdamJacobs,JohnCanny,MikeJordan,RichardWallace,AlperAtmturk,MartiHearst,BobFarzin,RashmiSinha,SteveBui,andIlanAdleraswedevelopedtheal-gorithm.ThankstoDavidPescovitzandMalcolmGlad-wellforcoveringthisworkinthepress.Wewerefortu-natetohavereceivedaserverfromVivekSanghiofIntelandexcellentrealworldadvicefromLeonardShlain,ReedGaither,AlanEyzaguirre,BobStein,PamAranow,andJor-danShlain.References[1]G.Albaum,R.Best,andD.Hawkins.Continuousvsdiscretesemanticdifferentialratingsscales.Psycho-logicalReports,49:90Ð97,1981.[2]K.J.Arrow.SocialChoiceandIndividualValues.YaleUniversityPress,2edition,1963.[3]DanielBillsusandMichaelPazzani.Learningcollab-orativeinformationÞlters.InAAAIWorkshoponRec-ommenderSystems,August1998.[4]Breese,Heckermen,andKadie.EmpiricalanalysisofpredictivealgorithmsforcollaborativeÞltering.crosoftResearchTechnicalReport,(MSR-TR-98-12),October1998.[5]B.V.Dasarathy.NNPatternClassiÞcationTechniquesIEEEComputerSocietyPress,CA,1991. [6]S.Deerwester,S.Dumais,G.Furnas,T.Landauer,andR.Harshman.Indexingbylatentsemanticanalysis.JournaloftheAmericanSocietyforInformationSci-,41(6):391-407,1990.[7]JoaquinA.Delgado.Agent-BasedInformationFil-teringandRecommenderSystemsontheInternetPhDthesis,NagoyaInstituteofTechnology,February[8]C.H.Q.Ding.Asimilarity-basedprobabilitymodelforlatentsemanticindexing.InProceedingsoftheSIGIRACM,August1999.[9]AlexanderChislenkoetal.Uspatent6092049:MethodandapparatusforefÞcientlyrecommend-ingitemsusingautomatedcollaborativeÞlteringandfeature-guidedautomatedcollaborativeÞltering.18July2000.[10]DavidGoldberg,DavidNichols,BrianOki,andDou-glasTerry.UsingcollaborativeÞlteringtoweaveaninformationtapestry.CommunicationsoftheACM35(12):61-70,1992.[11]D.GuptaandK.Goldberg.Jester2.0:AlineartimecollaborativeÞlteringalgorithmappliedtojokes.InProceedingsoftheSIGIR.ACM,August1999.[12]JonathanHerlocker,JosephKonstan,AlBorchers,andJohnRiedl.AnalgorithmicframeworkforperformingcollaborativeÞltering.InProceedingsoftheSIGIRACM,August1999.[13]JonathanHerlocker,JosephKonstan,AlBorchers,andJohnRiedl.AnalgorithmicframeworkforperformingcollaborativeÞltering.InProceedingsoftheSIGIRACM,August1999.[14]JohnHey.Uspatent4870579:Systemandmethodofpredictingsubjectivereactions.26September1989.[15]T.Hofmann.Probabilisticlatentsemanticindexing.ProceedingsoftheSIGIR.ACM,August1999.[16]H.Hotelling.Analysisofacomplexofstatisticalvari-ablesintoprincipalcomponents.JournalofEdua-tionalPsychology,24:417Ð441,1933.[17]J.E.Jackson.AUserGuidetoPrincipalComponents:AProblemSolvingApproach.JohnWileyandSons,NewYork,1991.[18]J.A.KonstanandK.Bharat.IntegratedpersonalandcommunityrecommendationsincollaborativeÞlter-ing.InCSCWWorkshop,1996.[19]JosephKonstan,BradleyMiller,DavidMaltz,JonathanHerlacker,LeeGordon,andJohnRiedl.Grouplens:ApplyingcollaborativeÞlteringtousenetnews.CommunicationsoftheACM,40(3):77-87,March1997.[20]P.R.KrishnaiahandL.N.Kanal.ClassiÞcation,pat-ternrecognitionandreductionindimensionality.InHandbookofStatistics2.North-HollandPublishingCompany,AmsterdamNewYorkOxford,1982.[21]T.Landauer,M.Littman,andBellCommunica-tionsResearch(Bellcore).Uspatent5301109:Com-puterizedcross-languagedocumentretrievalusingla-tentsemanticindexing.5April1994.[22]D.A.MaltzandK.Ehlrich.Pointingtheway:ActivecollaborativeÞltering.InChiÕ95ProceedingsPapers[23]DonNilsen,editor.InternationalJournalofHumorResearch.WalterdeGruyterInc.,NewYork.[24]K.Pearson.OnlinesandplanesofclosestÞttosys-temsofpointsinspace.Phil.Mag.,2:559Ð572,1901.[25]D.M.PennockandE.Horvitz.Analysisoftheax-iomaticfoundationsofcollaborativeÞltering.InAAAIWorkshoponArtiÞcialIntelligenceforElec-tronicCommerce,Orlando,Florida,July1999.Na-tionalConferenceonArtiÞcialIntelligence.[26]D.M.PennockandE.Horvitz.CollaborativeÞlter-ingbypersonalitydiagnosis:Ahybridmemory-andmodel-basedapproach.InIJCAIWorkshoponMa-chineLearningforInformationFiltering,Stockholm,Sweden,August1999.InternationalJointConferenceonArtiÞcialIntelligence.[27]MichaelPryor.Theeffectsofsingularvaluedecom-positiononcollaborativeÞltering.DartmouthCollegeCSTechnicalReport,PCS-TR98-338,1998.[28]P.Resnick,N.Iacovou,M.Suchak,P.Bergstrom,andJ.Riedl.Grouplens:Anopenarchitectureforcol-laborativeÞlteringofnetnews.InProceedingsoftheACMConferenceonComputerSupportedCoopera-tiveWork,1994.[29]ElaineRich.Usermodelingviastereotypes.Cognitive,3:335-366,1979.[30]U.ShardanandandP.Maes.SocialinformationÞl-tering:Algorithmsforautomatingwordofmouth.ACMConferenceonComputerHumanInteraction,1995. [31]HalVarianandPaulResnick.Specialissueoncfandrecommendersystems.CommunicationsoftheACM40(3),March1997.1997.www.wired.com/news/news/culture/story/ [33]A.Ziv.PersonalityandSenseofHumor.SpringerPublishingCo.,NewYork,1984.