stanfordedu cristianmpiswsorg Abstract Social media systems rely on user feedback and rating mechanisms for personalization ranking and content 64257ltering However when users evaluate content con tributed by fellow users eg by liking a post or votin ID: 23015
Download Pdf The PPT/PDF document "How Community Feedback Shapes User Behav..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
sequentposts,whilerewardedauthorsdonotimprovesig-nicantly.Inspiteofthisdetrimentaleffectoncontentquality,itisconceivablethatcommunityfeedbackstillhelpsregu-latequantitybyselectivelydiscouragingcontributionsfrompunishedauthorsandencouragingrewardedauthorstocon-tributemore.Surprisingly,wendthatnegativefeedbackac-tuallyleadstomore(andmorefrequent)futurecontributionsthanpositivefeedbackdoes.3Takentogether,ourndingssuggestthatthecontentevaluationmechanismscurrentlyimplementedinsocialmediasystemshaveeffectscontrarytotheinterestofthecommunity.Tofurtherunderstanddifferencesinsocialmechanismscausingthesebehaviorchanges,weconductedastructuralanalysisofthevoternetworkaroundpopularposts.Wedis-coverthatnotonlydoespositiveandnegativefeedbacktendtocomefromcommunitiesofusers,butthatthevotingnet-workismostpolarizedwhenvotesaresplitequallybetweenup-anddown-votes.Theseobservationsunderscoretheasymmetrybetweentheeffectsofpositiveandnegativefeedback:thedetrimen-talimpactofpunishmentsismuchmorenoticeablethanthebenecialimpactofrewards.Thisasymmetryechoesthenegativityeffectstudiedextensivelyinsocialpsychologylit-erature:negativeeventshaveagreaterimpactonindividu-alsthanpositiveeventsofthesameintensity(KanouseandHanson1972;Baumeisteretal.2001).Tosummarizeourcontributions,inthispaperwevalidatethroughacrowdsourcingexperimentthatthepro-portionofup-votesisarobustmetricformeasuringandaggregatingcommunityfeedback,introduceaframeworkbasedonpropensityscorematch-ingforquantifyingtheeffectsofcommunityfeedbackonauser'spostquality,discoverthateffectsofcommunityevaluationsaregener-allydetrimentaltothecommunity,contradictingtheintu-itionbroughtupbytheoperantconditioningtheory,andrevealanimportantasymmetrybetweenthemechanismsunderlyingnegativeandpositivefeedback.Ourresultsleadtoabetterunderstandingofhowusersre-acttopeerevaluations,andpointtowaysinwhichonlineratingmechanismscanbeimprovedtobetterserveindivid-uals,aswellasentirecommunities.FurtherRelatedWorkOurcontributionscomeinthecontextofanextensivelit-eratureexaminingsocialmediavotingsystems.Onemajorresearchdirectionisconcernedwithpredictingthehelpful-nessratingsofproductreviewsstartingfromtextualandsocialfactors(GhoseandIpeirotis2007;Liuetal.2007;Otterbacher2009;TsurandRappoport2009;Luetal.2010; quenceoftheactualtextualqualityofthepost,butisalsoaffectedbycommunitybiaseffects.Weaccountforthisthroughexperi-mentsspecicallydesignedtodisentanglethesetwofactors.3Wenotethattheseobservationscannotsimplybeattributedtoamewars,astheyspreadoveramuchlargertimescale.MudambiandSchuff2010)andunderstandingtheunderlay-ingsocialdynamics(Chen,Dhanasobhon,andSmith2008;Danescu-Niculescu-Miziletal.2009;WuandHuberman2010;Sipos,Ghosh,andJoachims2014).Themechanismsdrivinguservotingbehaviorandtherelatedcommunityef-fectshavebeenstudiedinothercontexts,suchasQ&Asites(Andersonetal.2012a),Wikipedia(BurkeandKraut2008;Leskovec,Huttenlocher,andKleinberg2010;Andersonetal.2012b),YouTube(Siersdorferetal.2010),socialnewsaggregationsites(LampeandResnick2004;LampeandJohnston2005;Muchnik,Aral,andTaylor2013)andon-linemultiplayergames(Shoresetal.2014).Ourworkaddsanimportantdimensiontothisgenerallineofresearch,byprovidingaframeworkforanalyzingtheeffectsvoteshaveontheauthoroftheevaluatedcontent.Thesettingconsideredinthispaper,thatofcommentsonnewssitesandblogs,hasalsobeenusedtostudyothersocialphenomenasuchascontroversy(ChenandBerger2013),politicalpolarization(Parketal.2011;Balasubramanyanetal.2012),andcommunityformation(G´omez,Kaltenbrun-ner,andL´opez2008;Gonzalez-Bailon,Kaltenbrunner,andBanchs2010).Newscommentingsystemshavealsobeenanalyzedfromacommunitydesignperspective(MishneandGlance2006;Gilbert,Bergstrom,andKarahalios2009;Di-akopoulosandNaaman2011),includingaparticularfocusonunderstandingwhattypesofarticlesarelikelytoattractalargevolumeofusercomments(Tsagkias,Weerkamp,anddeRijke2009;YanoandSmith2010).Incontrast,ouranal-ysisfocusesontheeffectsofvotingonthebehavioroftheauthorwhosecontentisbeingevaluated.Ourndingshererevealthatnegativefeedbackdoesnotleadtoadecreaseofundesireduserbehavior,butratherattenuatesit.Giventhedifcultyofmoderatingundesireduserbehavior,itisworthpointingoutthatanti-socialbe-haviorinsocialmediasystemsisagrowingconcern(Hey-mann,Koutrika,andGarcia-Molina2007),asemphasizedbyworkonreviewspamming(Limetal.2010;Mukher-jee,Liu,andGlance2012;Ott,Cardie,andHancock2012),trolling(ShachafandHara2010),socialdeviance(Shoresetal.2014)andonlineharassment(Yinetal.2009).MeasuringCommunityFeedbackWeaimtodevelopamethodologyforstudyingthesubtleeffectsofcommunity-providedfeedbackonthebehaviorofcontentauthorsinrealisticlarge-scalesettings.Tothisend,westartbydescribingalongitudinaldatasetwheremillionsofusersexplicitlyevaluateeachothers'content.Followingthat,wediscussacrowdsourcingexperimentthathelpses-tablisharobustaggregatemeasureofcommunityfeedback.DatasetdescriptionWeinvestigatefouronlinenewscommunities:CNN.com(generalnews),Breitbart.com(politicalnews),IGN.com(computergaming),andAllkpop.com(Koreanentertain-ment),selectedbasedondiversityandtheirlargesize.Com-montoallthesesitesisthatcommunitymemberspostcom-mentson(news)articles,whereeachcommentcanthenbeup-ordown-votedbyotherusers.Werefertoacommentasapostandtoallpostsrelatingtothesamearticleasathread. AcknowledgmentsWethankDisqusforthedatausedinourexperimentsandouranonymousreviewersfortheirhelpfulcomments.ThisworkhasbeensupportedinpartbyaStanfordGraduateFellowship,anAlfredP.SloanFellow-ship,aMicrosoftFacultyFellowshipandNSFIIS-1159679.ReferencesAnderson,A.;Huttenlocher,D.;Kleinberg,J.;andLeskovec,J.2012a.DiscoveringValuefromCommunityActivityonFocusedQuestionAnsweringSites:ACaseStudyofStackOverow.InProc.KDD.Anderson,A.;Huttenlocher,D.;Kleinberg,J.;andLeskovec,J.2012b.Effectsofusersimilarityinsocialmedia.InProc.WSDM.Balasubramanyan,R.;Cohen,W.W.;Pierce,D.;andRedlawsk,D.P.2012.Modelingpolarizingtopics:Whendodifferentpolit-icalcommunitiesresponddifferentlytothesamenews?InProc.ICWSM.Baron,A.;Perone,M.;andGalizio,M.1991.Analyzingthere-inforcementprocessatthehumanlevel:canapplicationandbe-havioristicinterpretationreplacelaboratoryresearch?BehavioralAnalysis.Baumeister,R.F.;Bratslavsky,E.;Finkenauer,C.;andVohs,K.D.2001.Badisstrongerthangood.Rev.ofGeneralPsychology.Burke,M.,andKraut,R.2008.Moppingup:modelingwikipediapromotiondecisions.InProc.CSCW.Cartwright,D.,andHarary,F.1956.Structuralbalance:ageneral-izationofheider'stheory.PsychologicalRev.Chen,Z.,andBerger,J.2013.When,Why,andHowControversyCausesConversation.J.ofConsumerResearch.Chen,P.-Y.;Dhanasobhon,S.;andSmith,M.D.2008.AllReviewsareNotCreatedEqual:TheDisaggregateImpactofReviewsandReviewersatAmazon.Com.SSRNeLibrary.Cialdini,R.B.,andTrost,M.R.1998.Socialinuence:Socialnorms,conformityandcompliance.McGraw-Hill.Danescu-Niculescu-Mizil,C.;Kossinets,G.;Kleinberg,J.;andLee,L.2009.Howopinionsarereceivedbyonlinecommunities:AcasestudyonAmazon.comhelpfulnessvotes.InProc.WWW.Diakopoulos,N.,andNaaman,M.2011.Towardsqualitydiscourseinonlinenewscomments.InProc.CSCW.Ghose,A.,andIpeirotis,P.G.2007.DesigningNovelReviewRankingSystems:PredictingtheUsefulnessandImpactofRe-views.Proc.ACMEC.Gilbert,E.;Bergstrom,T.;andKarahalios,K.2009.BlogsareEchoChambers:BlogsareEchoChambers.InProc.HICSS.G´omez,V.;Kaltenbrunner,A.;andL´opez,V.2008.StatisticalanalysisofthesocialnetworkanddiscussionthreadsinSlashdot.InProc.WWW.Gonzalez-Bailon,S.;Kaltenbrunner,A.;andBanchs,R.E.2010.Thestructureofpoliticaldiscussionnetworks:Amodelfortheanalysisofonlinedeliberation.J.ofInformationTechnology.Heymann,P.;Koutrika,G.;andGarcia-Molina,H.2007.FightingSpamonSocialWebSites:ASurveyofApproachesandFutureChallenges.IEEEInternetComputing.Kanouse,D.E.,andHanson,L.R.1972.NegativityinEvaluations.GeneralLearningPress.Kim,S.-M.;Pantel,P.;Chklovski,T.;andPennacchiotti,M.2006.Automaticallyassessingreviewhelpfulness.Proc.EMNLP.Lampe,C.,andJohnston,E.2005.Followthe(Slash)Dot:EffectsofFeedbackonNewMembersinanOnlineCommunity.InProc.ACMSIGGROUP.Lampe,C.,andResnick,P.2004.Slash(dot)andburn:Distributedmoderationinalargeonlineconversationspace.Proc.CHI.Leskovec,J.;Huttenlocher,D.;andKleinberg,J.2010.Gover-nanceinSocialMedia:AcasestudyoftheWikipediapromotionprocess.InProc.ICWSM.Lim,E.-P.;Nguyen,V.-A.;Jindal,N.;Liu,B.;andLauw,H.W.2010.Detectingproductreviewspammersusingratingbehaviors.Proc.CIKM.Liu,J.;Cao,Y.;Lin,C.-Y.;Huang,Y.;andZhou,M.2007.Low-qualityproductreviewdetectioninopinionsummarization.Proc.EMNLP-CoNLL.Lu,Y.;Tsaparas,P.;Ntoulas,A.;andPolanyi,L.2010.Exploitingsocialcontextforreviewqualityprediction.InProc.WWW.Mishne,G.,andGlance,N.2006.Leaveareply:Ananalysisofweblogcomments.InWWE.Muchnik,L.;Aral,S.;andTaylor,S.2013.SocialInuenceBias:ARandomizedExperiment.Science341.Mudambi,S.M.,andSchuff,D.2010.Whatmakesahelpfulonlinereview?AstudyofconsumerreviewsonAmazon.com.MISQuarterly.Mukherjee,A.;Liu,B.;andGlance,N.2012.SpottingFakeRe-viewerGroupsinConsumerReviews.InProc.WWW.Ott,M.;Cardie,C.;andHancock,J.2012.Estimatingthepreva-lenceofdeceptioninonlinereviewcommunities.InProc.WWW.Otterbacher,J.2009.`Helpfulness'inonlinecommunities:amea-sureofmessagequality.Proc.CHI.Park,S.;Ko,M.;Kim,J.;Liu,Y.;andSong,J.2011.Thepoliticsofcomments:predictingpoliticalorientationofnewsstorieswithcommenters'sentimentpatterns.InProc.CSCW.Parsons,T.1951.Thesocialsystem.London:Routledge.Rosenbaum,P.R.,andRubin,D.B.1983.Thecentralroleofthepropensityscoreinobservationalstudiesforcausaleffects.Biometrika.Shachaf,P.,andHara,N.2010.Beyondvandalism:Wikipediatrolls.J.ofInformationScience.Shores,K.;He,Y.;Swanenburg,K.L.;Kraut,R.;andRiedl,J.2014.TheIdenticationofDevianceanditsImpactonRetentioninaMultiplayerGame.InProc.CSCW.Siersdorfer,S.;Chelaru,S.;Nejdl,W.;andSanPedro,J.2010.Howusefulareyourcomments?:AnalyzingandpredictingYouTubecommentsandcommentratings.InProc.WWW.Sipos,R.;Ghosh,A.;andJoachims,T.2014.WasThisReviewHelpfultoYou?ItDepends!ContextandVotingPatternsinOnlineContent.InProc.WWW.Skinner,B.F.1938.Thebehavioroforganisms:anexperimentalanalysis.Appleton-Century.Tsagkias,M.;Weerkamp,W.;anddeRijke,M.2009.Predictingthevolumeofcommentsononlinenewsstories.InProc.CIKM.Tsur,O.,andRappoport,A.2009.Revrank:Afullyunsuper-visedalgorithmforselectingthemosthelpfulbookreviews.Proc.ICWSM.Wu,F.,andHuberman,B.A.2010.Opinionformationundercostlyexpression.ACMTrans.onIntelligentSystemsandTechnology.Yano,T.,andSmith,N.A.2010.What'sworthyofcomment?Contentandcommentvolumeinpoliticalblogs.InProc.ICWSM.Yin,D.;Xue,Z.;Hong,L.;Davison,B.D.;Kontostathis,A.;andEdwards,L.2009.DetectionofharassmentonWeb2.0.InProc.CAW2.0WorkshopatWWW.