Steering User Behavior with Badges Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell University Cornell University Stanford University ashtoncs - PDF document

Presentation on theme: "Steering User Behavior with Badges Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell University Cornell University Stanford University ashtoncs"— Presentation transcript:

levels,thesite'sdesignerscanattemptto“steer”auser'sactivitiestowardparticularformsofcontribution.Forexample,aquestion-answering(Q&A)sitemayhavearangeofactivitiesthatincludeaskingquestions,answeringquestions,up-votinganddown-votingquestionsandanswers,andothers.OnaQ&Asitewhereeveryonewantstoaskquestionsandfewpeoplewanttoanswerthem,intro-ducingabadgeforuserswhohavecontributedacertainnumberofanswerscansteerthecommunitytowardcontributingtothisun-derrepresentedactiontype.Badgesforvotingcansimilarlytrytosteeruserstowardprovidingenoughfeedbacktomaintainausefulqualitysignalonthecontent.Insummary,badgesprovidearichlanguageforexpressingin-centives,butwithlittleexistingframeworkforreasoningabouttheireffects.Ourworkaddressesasetofquestionsthatcanhelpprovideinsightintobadgesandtheiruse.Inparticular,anaturalrstquestionis:Dobadgeswork?Thatis,canwendconcreteev-idencethatbadgesincreasesiteparticipationorsteeruserstowardstakingactionstheymightnothavetakenotherwise?Ifbadgesdohaveaneffectonusers,howcanwemodeluserbehaviorinthepresenceofbadges?Andtotheextentthatdesignerscanindeedsteeruserbehaviorwithbadges,howshouldtheydenebadgestoachievetheoutcomestheywant?OverviewofResultsOurworkisbasedonthissetofquestions,andcomprisesthreedifferentcomponents.First,wedevelopamodelforuserbehav-iorinthepresenceofbadgesonasitewithmultipletypesofac-tivities.Second,weevaluateourmodelondatafromthepopularquestion-answeringsiteStackOverow,andshowthatthemainqualitativepredictionsofthemodelmatchwhatweobserveintheaggregateuserbehaviorontherealsite.Third,weconsiderhowasitedesignercanusesuchamodeltodenebadgeswiththegoalofachievingadesiredpatternofbehavior.TheoreticalModelofUserBehavior.Giventhediscussionofbadgesthusfar,whatarethebasicingredientsweneedinordertodeneamodelofuserbehaviorwithbadges?Intuitively,wewouldliketohaveamulti-dimensionalspacerepresentingthepos-sibletypesofactionsonthesite;usershaveapreferredmixtureofactivitiesinthisspace,andintroducingbadgescaninducethemtoshifttheirmixtureinparticularways.Ourmodelbringsthesefeaturestogetherinanaturalway.Sinceakeyaspectofbadgesisthewayinwhichdifferentbadgesrewarddifferentactivities,wemodelthesiteashavingndifferentactiontypes;asinourexampleofaQ&Asiteabove,theactiontypescouldcorrespondtoaskingaquestion,answeringaques-tion,votingonaquestionoranswer,andothers.Usersperformactions,choosingfromamongthesetypes;auser'smixofactionsoverhislifetimecanthusbethoughtofasdeningavectorinann-dimensionalspacewhoseithcoordinaterecordsthenumberofactionsoftypeithathehastaken.1Eachbadgeisdenedbyspec-ifyinghowmanyactionsofeachtypemustbeperformedinorderforittobeawarded;inthissense,eachbadgedenesa“frontier”inthespacethattheuser'svectormustcrossinorderforthebadgetobeawarded.Werefertothisfrontierasthebadgeboundary.Inordertotalkabouttheincentivesthatbadgescreate,wealsoneedamodelofauser'sutility,whichwerepresentintwocom-ponents.First,eachuserhasapreferredmixofactionsonthesite,correspondingtoadistributionoverthepossibleactiontypes;intheabsenceofbadges,hewouldsimplyperformactionsaccordingtothismixturebysamplingfromhisdistributionoverpossibleactiontypes.Samplingactiontypesfromadifferentdistributionincursa 1Throughoutthispaperweusemalepronounstorefertousersofasiteandfemalepronounstorefertositedesigners.costtotheuser,basedonhowfarthedistributionisfromhispre-ferredone.Second,abadgeconfersutilityonceitisawarded,andsoauserhasanincentivetoshifthisdistributionofactivitiessome-whatinordertoachievethebadgemorequickly,whichtradesoffagainstthecostfordeviatingfromthepreferreddistribution.Theuser'sbehavioristhusdeterminedbythesolutiontoanindividual-leveloptimizationproblem,tradingoffthecostofshift-inghisactivitydistributionagainstthevalueofthebadge.Wewillseethatwhenthisoptimizationissolved,theforceoftheincen-tivesfrombadgesclearlyemergesnaturally:theuser'sdistributionofactionsisdeectedinthedirectionofbadges,andthereisan“acceleration”effectinwhichthisdeectionbecomesstrongerastheuserapproachesabadgeboundary.Thusfarthemodelcaptureshowabadgecansteeruserstowardcertaintypesofactions.Atahigherlevel,abadgecanalsoincreasetheoveralllevelofuserparticipationonthesite.Whileitmayini-tiallyseemthatthesearetwodifferentphenomena—participationlevelversuschoiceofactionsonthesite—wecanunifythemwithinthemodelbyrepresentingtheuser'soff-siteactivitiesviaasingleadditionalactiontype,correspondingtooneextradimensioninthespaceofpossibleactions.Werefertothisadditionalactiontypeasthelife-action,sinceitcorrespondstothesumtotaloftheuser'sac-tivitiesinhislifeoffthesite;theuser'spreferreddistributionofac-tivitieswillincludeaprobabilityassociatedwiththelife-actionaswell.Withthisextendedframework,wecanaskabouttheeffectofabadgeintwodistinctrespects:theextenttowhichitdrawsproba-bilitymassawayfromthelife-action,thusincreasingparticipationonthesite,andtheextenttowhichisshiftsprobabilitymassbe-tweendifferentsiteactions,thussteeringbehaviorwithinthesite.EmpiricalEvaluation.Afterdevelopingthemodel,weinvestigatewhetheritsqualitativepredictionsmatchtheaggregatebehaviorweseeinalarge-scalesettingthatmakesstronguseofbadges.Todothis,weanalyzethebehaviorofseveralmillionusersonStackOverow,alargeandveryactivequestion-answeringsite.OnStackOverow,badgesplayaprominentrole—theyaredisplayedalong-sideauser'snamewhereverhepostsonthesite,andtheyaredis-cussedextensivelyonStackOverowforums.WendthatthemainqualitativepredictionsofthemodelmatchthebehaviorweobserveonStackOverowinfourkeyrespects.(i)Ifacertainlevelofactivityofaparticulartypeisrewardedbyabadge,userswillincreasetheiractivityofthistypeastheyapproachthelevelneededforthebadge.Thisisthenotionofsteeringtowardabadgeboundarymentionedabove.(ii)Differentbadgesproducedifferentamountsofsteering.(iii)Theextentofsteeringdependsonhowclosetheuseristothebadgeboundary—an“acceleration”effectinwhichausersteersparticularlystronglytowardabadgewhentheyareclosetoachievingit.(iv)Steeringinvolvesbothshiftsinthemixtureofactionstheuserperformsonthesite,aswellasintheiroveralllevelofparticipationonthesite.Whatwendpromisingabouttheseresultsisthateachof(i)-(iv)arisesnaturallyasapredictionofourmodelingframework,despitethefactthatwedidnot“buildin”anyofthem.Rather,themodelgeneratesthesebehaviorsbysimplypositinguserswhoacttomaximizeutility.TheBadgePlacementProblem.Finally,havingproposedamodelofuserbehaviorinthepresenceofbadgesandshowingthatitsmainpredictionsareconsistentwithempiricaldata,wereturntothemodelandexploreitfromthesitedesigner'sperspective.Supposethesitedesignerhasadesiredmixtureofactionsq—overthefulluserpopulation,shewouldliketoseetheactiontypesperformed isverygeneral,andcanbeeasilygeneralizedtoincorporatericheruserutilitiesthatmaydependextensivelyontheuser'sstate.Twoexamplesincludeauserwho:(a)wantstomaintaintheproportionsinthevectorpgloballyoverhisentirelifetime(thatis,hewantstochooseadistributionatagivenvectorathattriestobringhimbacktowardapointwheretheproportionofactiontypesperformedsofarisbalancedaccordingtothecoordinatesofp),and(b)valuesbadgesdifferentlydependingonwhichotherbadgeshehasalreadyreceived—thiscouldbeincorporatedintothevaluefunctionV.Nextweconsidertheutilityassociatedwithbadges.WeuseBtodenotethesetofallbadgesonthesite.Abadgeb2BconfersavalueVbtoauserineachstepafterhereceivesit.(Alternately,thevalueofbcouldbeconferredentirelyina“lump-sumpayment”inthestepwhentheuserreceivesb;thiswouldleadtoamodelthatisformallyequivalent,butexpressingthevalueasbeingconferredineachstepafterthebadgeisreceivedisaneasierformalismtoworkwith.)Inkeepingwiththefactthatbadgesproducegenuineincentivesonthesiteunderconsideration,wetreatthenotionofabadge'svalueasaprimitiveinthemodel.Thebroaderquestionofwhatabadge'svaluemeansisaninterestingissuethatwereturntointhediscussionattheendofthepaper.Inadditiontotheincentivetoachieveabadge,wealsowanttocapturetheideathatitisbettertoreceiveabadgesoonerratherthanlater.Wedothisbythestandardapproachofdiscountingthefuture:wesaythatauserhasaxedexogenousprobability�0thathepermanentlyleavesthesystemaftereachaction,andwrite=1�forthecorrespondingprobabilitythathesurvivestoper-formanotherstep.Theuserceasestoreceiveutilityonceheleavesthesystem.Inthisway,underaplanthatinvolvesreceivingabadgesomedistanceintothefuture,thevalueofthebadgeisdiscountedbytheprobabilitythathesurviveslongenoughinthesystemtoactuallyreachthebadge.2Weemphasizethattheparametercon-trolsanexogenousprocessbywhichtheuserleavesthesystem,inthesensethatitoperatesindependentlyoftheuser'schoices.Thereisaseparateendogenouswayfortheusertostopperformingactionsonthesite,whichissimplytochooseadistributionxoveractionsthatplacesaprobabilitymassof1onthelife-action.Intuitively,atacurrentvectora,auserthereforeneedstochooseadistributionxsoastotradeoffbetweenremainingclosetopandachievingbadgesmorequickly.Thisisthetrade-offthatleadsausertosteertowardabadgeboundary.Auser'spolicy.Letussupposethatauserhasdecided,foreachpossibleactionvectora,whichdistributionxaovernextactionshewilluseshouldhendhimselfatvectora.WecallthischoiceofdistributionsX=fxagtheuser'spolicy.Wecanwritetheutilitytheuserinstateareceivesfrompolicyxa,asfollows.U(xa)=Xb2BIb(a)Vb+n+1Xi=1xia
U(xa+ei)�g(xa;p)(1)Therstsumontheright-handsideistheuser'svaluefromallthebadgesheholdsatvectora.Thesecondsumistheexpectedvaluearisingfromthevectortheuserreachesaftersamplingaccordingtoxa:withprobabilityxiatheusermovestoa+ei,wherehewillusedistributionxa+eiandhenceobtainutilityU(xa+ei).Thissecondsumisdiscountedby,sincetheuseronlyreachesthisnextstepwithprobability.Thethirdtermisthecostfromthedifferencebetweentheactualactiondistributionxaandthepreferredp. 2Wefocusonthisbasicformofdiscountingforsimplicity;withoutmuchmodicationonecanalsoformulatethemodelusingmoresophisticatedtypesofdiscountingofthekinddevelopedinthebe-havioraleconomicsliterature[11],andtheresultsinthiscasearequalitativelyunchanged.2.2TheUser'sOptimizationProblemHavingfullyspeciedoursetting,wecannowaskhowauserwillbehaveunderthemodelinthepresenceofasetofbadgesB.Formally,wewillassumethattheuserchoosesapolicyX=fxagtomaximizehisutilityU(x0)startingfromtheorigin(i.e.,beforeperforminganyactions).Aswithanymodelofthisform,theun-derlyingoptimizationproblemisintendedtorepresentanabstrac-tionofarealuser'sdecision-making,whenhetradesoffthetermsinEquation(1),balancingdelitytohispreferreddistributionpagainstthelong-rangeprospectofobtainingbadges.Wenotethattheuser'soptimizationproblemcanbecastastheoptimumofaMarkovdecisionprocess(MDP),butthisobserva-tiondoesn'tdirectlyhelpussinceMDPscanbecomputationallyexpensivetosolveevenforinstanceswithstatesetsandactionsetsofmoderatesize,andourmodelhereproducesinstanceswithacountablyinnitestatesetandacontinuumofpossibleactions.Infactwewillbeabletodevelopanefcientalgorithm,butitrequiresmakinguseoftheinherentstructureoftheproblemasitarisesfromourmodel,ratherthaninvokingageneralclassofresults.Formostofthesection,wewillfocusonthresholdbadges,whichrewardacertainnumberofactionsofaparticulartype.Con-cretely,athresholdbadgecanbedescribedbyapair(k;i),specify-ingthatauserisawardedthebadgeassoonashehastakenatleastkactionsoftypeAi(thatis,Ib(a)=1ifandonlyifaik).Wesaythatathresholdbadgedescribedby(k;i)targetsdimensioni.Indevelopingthealgorithm,wefocusonthresholdbadgesthattargetoneortwodimensions;wewillseethatsuchbadgesillustratetherangeofprincipalbehaviorsthatwesubsequentlyobserveinthedataaswell.Attheendofthesection,wediscusstheextensiontogeneralbadges.OnetargeteddimensionFromEquation(1),weseethattheuser'sutilityatanactionvec-toradependsonhisutilityateachofthevectorsa+ei.Wethere-foreapproachtheproblemusingdynamicprogramming,withthevectorsaasthestatesofthedynamicprogram.Forsimplicity,wewillconsiderthecasen=2,sothereisoneadditionalsiteactionplusthelife-action,butthealgorithmtriviallyextendstolargern.IfwewanttosetupthedynamicprogrammingrecurrencebasedonEquation(1),thenweneedtoinitializeit.Wemustdothiscare-fully,sinceeachstatedependsonastateassociatedwithavectoroflargernorm.Wehandlethisviathefollowingtwoobservations.First,notethatwhentheusertakeshiskthactionoftypeA1heachievesthebadge,andafterthisthereisnofurtherbadgeutilitytobegained.Hence,foranystateathatispastthebadgeboundary,theutilityfromxadependsentirelyonthecostg(xa;p),andsointheoptimalpolicywehavexa=pforallsuchstates.Second,ifaand~ahavethesamecoordinateindimension1,thenasequenceofactionsstartingatacrossesthebadgeboundaryifandonlyifthesamesequencestartingat~acrossesthebadgeboundary.Thus,weeffectivelyhaveaone-dimensionalproblem,inwhichthevalueofcoordinate1inthecurrentvectoristheonlyvariablethatmatters.TheutilityfunctionU(
)canthereforebeexpressedsimplyasafunctionofa1,thenumberofA1actionstheuserhastaken.Abus-ingnotationslightlytowriteUintermsofbothvectorsandthecoordinateinthisonetargeteddimension,wehaveU(a1)=3Xj=1xja
U(xa+ej)�g(xa;p)=
[x1aU(xa+e1)+x2aU(xa)+x3aU(xa)]�g(xa;p) Figure4:Electoratebadge.Giventhatuserhastakenxquestion-votesandyanswer-votes,whatistheprobabilitythatnextactionwillbeaquestionvote.Top:Rawprobability.Bot-tom:Relativechangeinprobabilityofquestion-voting.Noticetheeffectsof“turning”towardsthebadgeboundary.Tosummarize,theaboveexperimentsdemonstratethatbadgescansteeruserbehaviorinamannerconsistentwithourmodel.First,weobserveusersiteactivityincreasesasusersapproachthebadgeandthussiteactivityincreasesattheexpenseofthelife-action.Andsecond,weobservethatusersalsoshifttheireffortonthesitetowardsactionsthatleadthemtobadges.Toourknowl-edge,thisistherstconcreteevidenceofuserschangingtheirbe-haviorineitherofthesetwodistinctwaysinresponsetobadges.4.THEBADGEPLACEMENTPROBLEMHavingseenthatourmodelmatchesupwellwithreal-worldbe-haviorforthresholdbadgesonStackOverow(badgesthatareawardedonceauserhastakenaspeciednumberofactionsofcertaintypes)wenowinvestigatehowourmodelcanhelpprovideinsightsintothedesignofbadgesforonlinecommunities.Inpartic-ular,thesitedesignerhastheabilitytodecideontheconditionsforobtainingbadges;withinourmodelwethinkofthisas“placing”thebadgeboundarieswithinthespaceofactionvectors.Wearein-terestedinaddressingthefollowingquestions:(1)Howmuchusersteeringdodifferentbadgeplacementsprovide?(2)Howmightsitedesignerplacebadgestobestachievedesireduserbehavior?(3)Whatisthespaceofuserbehaviorsxedbadgescanelicit?Wepresentourresultsorganizedaroundthesethreecentralques-tions.First,wendthattheeffectivenessofabadgeisgenerallymaximizedataninternaloptimumthatissurprisinglyhigh,andweexplorehowtheeffectivenessvarieswiththesetting.Second,theeffectivenessofmultiplebadgesworkingtogetherismaximizedwhentheyareofequalvalueandarespacedroughlyevenlyapart.Finally,weexplorethe“feasibleregion”ofuserbehaviorsthede-signercanelicitwithaxedsetofbadgesandndittobeahighlycomplexandcounterintuitivestructureworthyoffuturestudy.WearrivedattheseresultsbyrunningexperimentsonourmodeldevelopedinSection2.Byrepeatingthesimulationsusingmanydifferentparametersettings,wefoundthatthepresentedresultsallholdforawiderangeofparametervaluesandthusillustratebroaderprinciplesofthebadgeplacementproblem.Moreover,weusedthesameg(
)asbefore:g(x;p)=kx�pk22.Throughoutthissection,werefertothetotalfractionofactionsonatargetedaction(overtheuser'slifetime)thatresultsfromaparticularsetofbadgeplacementsastheyieldofthosebadges.Wealsodenegaintobethedifferencebetweentheyieldandthede-faultfractionofactionstheusertakesintheabsenceofanybadges.OptimallocationandyieldwithonebadgeWestartourinvestigationswiththecaseinwhichthesitede-signerhasasinglebadgeatherdisposalandwishestomaximizetheyieldonasingledimension,whichwetaketobeA1withoutlossofgenerality.Thissetting,whilesimple,isofbothconceptualandpracticalvalue,sincetheeffectsofplacingasinglebadgearemosteasilyseenwhenthebadgeistheonlyinuenceoveruserbe-havior,andinpracticeitisoftendesirableforthedesignertolimitherselftoasinglebadge.Placingabadgeofxedprespeciedvalueonasingledimensionrequiresstrikingtherightbalancebetweentwomutuallycompet-ingforces.First,ifthebadgeistohaveaneffectonmanyactions,itsthresholdshouldbesethighenoughsothatittakesmanystepstoachieveit.However,ifitsthresholdissettoohigh,thenevensurvivinglongenoughonthesitetoachievethebadgebecomesalow-probabilityeventandthususerswillnotbesufcientlyin-centivizedtosteerstronglytowardsit.Thesolutiontothebadgeplacementproblemisthereforeingeneralaninternaloptimumbe-tweenthesetwogoals.Figure5showshowthefractionofactionsonthetargeteddi-mensionA1(theyield)variesasafunctionofwherethebadgeofxedvalueisplaced.Weplotthisforvariouschoicesofp=(x;0:5�x;0:5),x2f0:05;:::;0:45g(whilekeepingallotherparametersxed).Noticethateachcurvehasauniqueinter-nalmaximum.WecanalsoseethattheresultingfractionofA1actionsishigherforlargervaluesofp1,whichintuitivelymakessensesincetheuserisincreasinglypredisposedtotakeA1actions.Moresubtle,however,isthattheoptimalbadgelocationalsoin-creaseswithp1:forp1=0:05itisA1=75,whereasforp1=0:45itisA1=90.Anotherinterestingobservationisthatthisoptimallocationissurprisinglyhigh:since=0:99inthisexample,p1=0:05impliesthattheuserwouldonlytake5A1actionsintheabsenceofbadges,yettheoptimalbadgelocationisfarbeyondthispoint,atA1=75.Theseresultssuggestinterestingrelationshipsbetweenthemodelparametersettingsandtheamountofsteeringinducedbythebadge.Forexample,wejustsawthattheyieldincreaseswithp1,andthatthebadgegetsplacedfurtherawayfromtheoriginasp1increases.InFigure6,weplottheyieldattheoptimalbadgelocationasafunctionofp1,Vb,andtheuser'sexpectedlifetime1=(1�).Weobservethattherelationshipfollowsadifferentfunctionalformforeachsetting:theyieldincreasesapproximatelylinearlyinp1(butthegaindecreaseslinearly),increaseswithdi-minishingreturnsinVb,andsurprisinglydecreasesintheexpectedlifetimeoftheuser.Thedecreasingrelationshipsareparticularlyinteresting;theyimplythatthedesignercansteerusersmoreontheactionstheydislike(i.e.,lowp1)thanthosetheylike,andthatbadgeshavestrongereffectswhenusers'lifetimesareshorter.Twoprinciplesforplacingmultiplebadges Figure5:Theresultingfractionofactionsonthetargeteddi-mension(hereA1)asafunctionofwherethebadgeisplaced.Thedifferentcurvesshowhowtherelationshipvariesastheuser'spreferencespchange. Figure6:Howoptimalyieldvarieswithsettingparameters.Sofarwehaveestablishedthatasinglebadgeisoptimallyplacedatanidealmiddleground(whichissurprisinglyhigh)andexploredhowtheoptimalyielddependsonthesetting.Nowweinvestigatehowoneshouldplaceusemultiplebadgesinconcertwitheachother(i.e.,targetthesamedimension).Exploringoptimalplace-mentsforpairsofbadges(ratherthanindividualbadges)uncoverstwoadditionalbasicprinciplesthatwenowdiscuss.Optimalspacing.Wendthatyieldonasingletargeteddimensionismaximizedbytwobadgeswhentheyarespacedroughlyevenlyapart.InFigure7,eachpoint(x;y)representsplacingthetwobadgesatxandyA1actions,respectively,andthecolorindicatestheresultingyield(i.e.,thecumulativeactivityonbothtargetedactions).Moreeffectivebadgeplacementsarecoloredred,whilelesseffectiveplacementsarecoloredblue.Duetosymmetrytheplotisnaturallymirroredbelowthediagonal.Thesalientqualitativeobservationisthattheoptimalsetofbadgelocationsisoffthemaindiagonal,indicatingthatitisbettertoplacethetwobadgesatdistinctlocationsthantocombinethemintoasinglelargebadgeatanysinglelocation.Infact,thisobservationisconsistentwiththesolutionoftheuser'soptimizationprobleminSection2.2inthecaseofmulti-plebadgesonthesamedimension,wherewesawthatthelastbadgeexertsinuenceovernotonlytheregionafterthesecond-lastboundary,butalsoovertheearlierregionsaswell.There,oncewesolvedtheuser'soptimaldirectionforthelastbadge,solvingthesecond-lastbadgeregionbecameequivalenttotheone-badgecasewithabadgevalueequaltothesecond-lastbadgevalueplustheexpectedutilityfrompotentiallywinningthelastbadge.Wefurthernotethatintheoptimalsetofplacements,badgesarespacedapproximatelyevenlyapart.Iftherstbadgeisata1=x,thesecondbadgeisata1=2x�forsomesmall.Thereasonfortheisthatthesecondbadgeisplacedalittleclosertotheoriginthanitwouldbeplacedinaone-badgeproblemstartingata1=xsinceitnotonlysteersbehaviorfora1xbutalsohasaslighteffectontheuserfora1x. Figure7:Yieldasafunctionofwheretwobadgesareplacedonthesametargeteddimension.Everycell(x;y)correspondstoplacingonebadgeata1=xactionsandanotherata1=yactions,andthecolorofthecellindicatestheresultingyield. Figure8:Howtheoptimalyieldachievablewithtwobadgesdependsonhowaxedamountofutilityissplitacrossthetwobadges.Themoreeventhesplit,thehighertheyield.Yieldismaximizedwithbadgesofequalvalue.Inthepreviouscase,thetwobadgestargetedthesameactionbutbothhadsomexedequalvaluez.Nowweallowthetwobadgestotakedif-ferentvalueswhiletheirtotalvalueisheldconstant.Assumetherstbadgehasvaluexandthesecond2z�x.Nowxzandthequestionis:whatistheoptimalwaytosplitthevalueof2zamongthetwobadgestogainmaximumyield?Figure8plotstheyieldasafunctionofx.Noticetheeffectivenessofthetwobadgesismaximizedwhenthebadgeshaveequalvalue(x=z).Thisresultsuggeststhatthedesignershouldcreatebadgesinsuchawaythattheyhaveaboutequalvalue.TargetingtwodimensionsWejustestablishedhowyieldononetargeteddimensionismax-imizedbyasinglebadgeandwhentwobadgesareactingtogether.Theyieldofasetofbadgeswasanaturalobjectivefunction,sinceitexpresseshowmuchofthegivenactioncanbeinducedbythebadge(s).Withbadgesthatcantargetdifferentdimensions,thereisamorecomplexsetoftrade-offs,sinceweneedtodecidehowwewanttobalanceincreasesinthevolumeofonesiteactionwithincreasesinanother,whenbotharetargetedbybadges.Moregen-erally,weask:whatisthesetofallpossiblebehaviorsthatcanbeinducedusingbadgestargetingmultipledimensions?Forsimplicityweconsiderthescenarioinwhichthedesignerhastwothresholdbadgesthatshecanplacehowevershewants.ShecanputthembothonA1,bothonA2,oroneoneachaction.Figure9plotstheallpossiblebehaviorsthatcanbeinducedusingtwothresholdbadges.Thegeneralcontouroftheplotismaintainedacrossawiderangeofmodelparametersettings.(Forexample,thecontourgrowsastheuser'sexpectedlifetimegetslonger.)Theopacityofeach(x;y)cellreectshowmanydistinctplace-mentsoftwobadgesproduceayieldofxonA1andayieldofyonA2.Thedarkoutlinesoftheblueandredregions(wherethede-signertargetsthesamedimensionwithbothbadges)indicatethatit'seasier(inthesensethattherearemoreuniqueplacements)to Figure9:Thefeasibleregionofwhatyieldsareachievablewithtwobadgesofxedvalue(hereVb1=Vb2=200andp=(0:1;0:1;0:8)).Abluecellat(x;y)indicatesthatthede-signercanhavebothbadgestargetA1actionsandelicitx%ofA1actionsandy%ofA2actions;redcellsaresimilarbutfortheA2dimension;andyellowcellsindicatewhatisachievablebyputtingonebadgeoneachdimension.Thecellopacitycor-respondstohowmanydifferentpairsofbadgelocationselicitthatparticularyield.elicityieldontheborderthaninthemiddleofthesewing-shapedregions.Ifweconsidertheblueregionasanexample,theupperoutline,nearthey=10line,isdarkbecauseitcorrespondstousingonebadgenearlyoptimallyandessentially“discarding”theotherone(byplacingitsofarfromtheoriginthatitdoesn'tshapeuserbehavior),whereasthelowerdarkoutline(smoothlyvaryingfrom(10%;10%)to(40%;2%))correspondstousingbothbadgesinconcert.Therearesimilarrecurring“veins”inthegoldregion,indicatingthatsomeparameterizedlinesthroughthefeasiblere-gionareimplementedbymorebadgeplacementsthanothers.Themaximumyieldonasingledimensionnaturallyoccurswhenthedesignerplacesbothbadgesonthatdimension,andthemid-dlegroundwhereboththeyieldishighonbothdimensionsoc-curswhensheplacesonebadgeoneachdimension.However,notethatthefeasibleregionisinterestinglynon-convexbetweenthesetwocases.Thismeansthatdesignercannotinducecertainbehav-iorsregardlessofhowsheplacesthebadges.Forexample,thedesignercanelicit(5%;70%)yieldusingtwobadgesonA2and(15%;50%)usingonebadgeoneach,butshecannotelicitthein-termediate(10%;60%)usinganycombinationoftwobadges.5.RELATEDWORKAsnotedintheintroduction,theuseofbadgesisagrowingtrendinthedesignofonlinecommunities,socialcomputingapplications,andelectroniccommerce(e.g.see[19]),andresearchershavebe-guntostudytherolethatbadgesplayinthesesites.AntinandChurchill[1]presentaconceptualorganizationfordifferenttypesofbadges,considering,amongotherthings,themotivationtheyprovidefromasocial-psychologicalviewpoint.Oktayetal.[18]usequasi-experimentaldesignstosupportcausalclaimsinthecon-textofbadges.Theuseofbadgescanbeviewedaspartofthegrow-ingphenomenonofgamication[9],inwhichelementstradition-allyassociatedwithcomputergamesareusedtomotivatepeopleinotherdomains.Ourworkcontributestothisliteraturebypropos-ingaformalframeworkforreasoningabouttheeffectsbadgeswillhave,whichcanpotentiallybeusedforthebadgedesigninbothcurrentandnewcontexts.Badgesandbadge-likerecognitionshavebeenproposedinoff-linedomainsaswell.Thereisagrowingmovementtowardus-ingsystemsofbadgesineducationalsettings;asoneexample,theBadgesforLifelongLearningCompetitionhasmadeuseoftheMozillaOpenBadgesproject[10,16]forthispurpose.Thusfarsuchinitiativesineducationhaveusedbadgesprimarilyasaformofcredentialing,butmovingbeyondthistowardsengagingandmo-tivatingstudentsisanotherkeyopportunitywithbadgesinthisset-ting.Theadvantagesofbadgesinsuchcontextsisathemethathasbeenadvocatedinearlierresearchineducationaswell[4].Rewardsforcumulativeefforthavebeenconsideredinseveraldomains.Forexample,onecaninterpretcustomerloyaltypro-gramsascontainingbadge-likeincentives,suchasthedifferentstatusmilestonesinairlinefrequentierprograms,andresearcheconomicsandmarketinghasstudiedtheeffectofsuchprogramsoncustomerbehavior[14,15].Inadifferentdirection,Zhangetal.considerplacinglimitedamountsofrewardinaMarkovdecisionprocess,asaninstanceofwhattheytermenvironmentdesign[20].Finally,ourworkisrelatedtothemoregeneralquestionofin-centivesforcontributioninsocialmedia.Thisisaverybroadarea,encompassinganumberofapproachesbeyondjusttheuseofbadgesforexpressingincentives.Asnotedintheintroduction,twomethodologiesthathavebeenbroughttobearonthisquestioninthecomputingliteratureare(i)social-psychologicalperspec-tivesonthenotionofengagementandsocialmotivators[5,6,17];and(ii)algorithmicgame-theoreticandeconomicapproaches,in-cludingincentivesforrecruitment,contributionquality,andcrowd-sourcedeffort[2,7,8,12,13].6.CONCLUSIONBadgesystemsareanincreasinglywidespreadfeatureofonlinesocialmediasites,andtheycanproducestrongincentiveeffectsontheusersinthesedomains.Ourworkhasproposedamethod-ologyforreasoningabouttheseincentives,startingwithamodelofuserswhooptimizetheirbehaviorgivenopportunitiestoreceivebadges.ThemainqualitativepredictionsofourmodelareborneoutbyananalysisofuserbehaviorinthepresenceofbadgesonStackOverow,andwehaveseenhowthemodelprovidesusefulqualitativeinsightsintotheproblemofplacingbadgestooptimizetheirincentiveeffects.Weseeanumberofimportantdirectionsinwhichfurtherworkcouldbepursued.First,thenotionofabadge'svaluehasbeentakenasaprimitivedenitioninthiswork,butasdiscussedearlier,itisinterestingtothinkaboutthemechanismsbywhichbadgesac-quire(non-monetary)value,andtheextenttowhichasitedesignercaninuencethevalueofabadge.Moregenerally,ourmodelsug-geststhatinoptimizingasystemofbadges,certainparameters—notjustthebadges'valuesbutalsothestructureofthepossibleactions,users'preferencesfortheseactions,andusers'expectedlifetimes—allcanplayapotentialroleintheprocess.Develop-ingmethodsofestimatingtheseparametersforuseinthedesignofbadgesystemsisaninterestingdirectionforfurtherresearch.Incentivizinguserstoincreasetheiractivitynaturallybringsupthequestionofhowthisaffectsthequalityoftheiractions.Forex-ample,wendonStackOverowthatusers'votesonquestionsaresignicantlymorepositivebeforetheyreceivetheElectoratebadgethanafterit.Developingprincipledwaysofincorporatingactionqualityintomodelsofuserbehaviorinthepresenceofbadgesisanexcitingdirectionforfuturework. Now,clearlyan_-badgeisamonotonebadge.Butthereisalsoanon-trivialconversetothisobservation,asfollows.(3)Abadgeismonotoneifandonlyifitisan_-badge.Sinceonedirectioniseasy,thecruxofproving(3)istoshowthatanymonotonebadgeisan_-badge.Toprovethis,letbbeamonotonebadge.SinceSbisamonotonesubsetofNm,Dickson'sLemmasaysthatithasonlynitelymanyminimalelements;wedenotealltheminimalelementsofSbbyc1;c2;:::;ck.Letb0bethe_-badgewithgeneratorsetc1;c2;:::;ck.Weclaimthatbandb0arethesamebadge,inthesensethatSb=Sb0.ClearlySb0Sb,sinceci2Sbforalli,andSbismonotone.Conversely,consideranya2Sb;theremustexistaminimalele-mentciofSbforwhichcia.Itfollowsthata2Sb0,andsinceawasarbitrary,wehaveSbSb0.ThusSb=Sb0,andthisproves(3).Now,consideramonotonebadgeb,withageneratorsetc1;c2;:::;ck.Supposethatallcoordinatesofallvectorsciareupper-boundedbythenaturalnumberw,andletwbethevectorinNmallofwhosecoordinatesarew.ThekeyconstructionistodivideupNmbythinkingofwasthe“origin,”andlookingatthe2mregionsdenedbywhethereachco-ordinateislargerorsmallerthanw.Specically,let[m]denotethesetofdimensionsf1;2;:::;mg;foraset[m],letTNmdenotethesetofvectorsa=(a1;a2;:::;am)suchthataiwifandonlyifi2.NoticethatT[m]isaniteset,andallotherTfor6=[m]areinnite.ThesetT[m]hasthepropertythatallgeneratorsofSblieinT[m].ThisleadstothefollowingcrucialpropertyofourdecompositionintothesetsT.(4)Leta;~a2Tagreeonallcoordinatesin,andletb2Nm.Thena+b2Sbifandonlyif~a+b2Sb.Supposebywayofcontradictionthatthereexista,~a,andbasinthestatementof(4)suchthata+b62Sbbut~a+b2Sb.Leta0bethecoordinatewisemaximumofaand~a;bymonotonicity,a0+b2Sb.Now,wereducea0+btoaminimalelementofSbviathepro-ceduredescribedbeforeStatement(2),usingapermutationinwhichwereduceallcoordinatesinthecomplementofbeforeanycoordinatein.Letadenotethevectorwereachatthemo-mentwhenallthecoordinatesinthecomplementhavejustbeencompleted.Itcannotbethecasethataa+b,sincea2Sbwhilea+b62Sb.Butaanda+bagreeonallcoordinatesin,andsoitfollowsthatai�(a+b)iforsomecoordinatei62.Finally,(a0+b)i=ai,sincetheremainderoftheproceduretoproduce(a0+b)fromanevermodiescoordinateiagain,andhence(a0+b)i�(a+b)i.Butsincei62,wehaveai�wandhence(a+b)i�w.Thiscontradictsthedenitionofw—allminimalelementsofSbhaveallcoordinatesboundedbyw—andhencethiscompletestheproofof(4).Using(4),wecannowdescribeourdynamicprogrammingap-proachtotheuser'soptimizationprobleminthepresenceofbadgeb,consistingofachoiceofxaforeacha2Nm.Westartbysettingxa=pforalla2Sb;sinceTSb,thisincludesalla2T.Wethendeterminethevaluesofxafora2Tbyinduc-tiononthecardinalityof:wecanstartwith=asthebasecase,usingxa=pforalla2T.Ingeneral,consideraset[m],andsupposewehavedeter-minedthevaluesofallxa2T forall thatarepropersubsetsof.Thekeypointisthatifa;~a2Tagreeonallcoordinatesin,then(4)impliesthatxa=x~a,sinceanysequenceofactionsstartingataresultsinthebadgeifandonlyifthesamesequenceofactionsstartingat~aresultsinthebadge.Giventhis,fora2T,letf(a)bethevectorinTthatagreeswithaonallcoordinatesin,andinwhichallcoordinatesnotinhavebeensetequaltow+1.LetT=ff(a):a2Tg.Wehavexf(a)=xa,andsoitisenoughtodeterminexajustforalla2T.ThesetTisnite,sinceeachcoordinateofavectora2Tisboundedbyw+1.Wedeterminethevaluesofxaforalla2Tbybackwardinductiononthesumofcoordinatesina.Whenwegettoavectorainthiscomputation,wecanevaluatetheutilityofchoosingavectorxainstateaviatherecurrenceU(xa)=Ib(a)Vb+mXi=1xia
U(xa+ei)�g(xa;p)Ofthetermsontheright-handside,therstisaconstantandthethirdhasaclosed-formexpression.Inthesumthatformsthesec-ondterm,eachsummandxia
U(xa+ei)involvesavectora+eisuchthatf(a+ei)satisesoneofthreepossibleproperties:(i)itbelongstoT forsome thatisapropersubsetof;(ii)itbelongstoTandhaslargercoordinatesumthanf(a);or(iii)f(a+ei)=f(a).Inthersttwocaseswecanllinthevalueofthesummandbyinduction;inthethirdcase,wehavexa=xa+eiintheoptimalsolution,andsowecanbringthesesummandsovertheleft-handside.Inthisway,wecansolveamaximizationprob-leminthecoordinatesofxatondtheoptimalchoiceofxa.Workingbackwardbyinductioninthisway,wecanthuscom-putetheoptimalchoiceofxaforallstatesa2Nm.B.OPTIMIZATIONPROBLEMSInSection2.2,weclaimedthattheuser'soptimizationproblemintwotargeteddimensionscanbeefcientlysolved.Whereasanaive,approximatesolutiontotheproblemrequiressolvingmanyquadraticprograms,hereweshowhowtoefcientlyndtheexactsolutionviaareductiontoaone-variableoptimizationproblem.(Thetechniqueforsolvingtheoptimizationprobleminonetargeteddimensionissimilarafterasuitablechangeofvariables.)Wewishtosolvethefollowingoptimizationproblem:maximizexPnj=1Cjxj�kx�pk22 1�
xn+1subjecttoxj0;j=1;2;:::;n+1;n+1Xj=1xj=1LetC=[C1C2:::Cn]TbethevectorofconstantsCjforj=1;2;:::;n,andlet~x=[x1x2:::xn]Tand~p=[p1p2:::pn]Tbetheprojectionsofxandpontotherstndimensionsrespectively.Ifweconsiderxingxn+10forthemoment,thisproblemcanberecastasthefollowingminimizationproblem:minimizexk~p�~xk22�
CT~xsubjectto~xj0;j=1;2;:::;n;nXj=1xj=1�xn+1Firstweaddtheequalityconstrainttotheobjectivefunctionf(~x)withaLagrangemultiplier: