Goldstein Microsoft Research NYC 102 Madison Ave 12th Floor New York NY 10016 dggmicrosoftcom R Preston McAfee Google Strategic Technologies 1600 Amphitheatre Parkway Mountain View CA 94043 prestonmcafeecc Siddharth Suri ID: 2197 Download Pdf
Tags :Download Pdf - The PPT/PDF document "The Cost of Annoying Ads Daniel G" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation on theme: "The Cost of Annoying Ads Daniel G"— Presentation transcript
etarycostofannoyingads.Therstandmaincontributionofthisworkisthatwemeasurethecompensatingwagedif-ferentialofannoyingads.Thatis,wemeasurehowmuchmoreonemustpayausertodothesameamountofworkinthepresenceofannoyingadscomparedtoinnocuousadsornoads.Thecompensatingdierentialisimportanttomeasurebecauseitcapturessomeofthenegativeeectsofadvertising,whichpublishersneedtoheedasalowerboundwhensettingthepricetorunanad.Inatwo-experimentinvestigation,wecomputethecom-pensatingdierentialforannoyingads.Intherstexper-imentusersrandomlyratedeitherananimatedadoritsstaticcounterpart.Thisdesignshowsthatanimationhasanegativeimpactonuserratings.Forthoseadsthatusersrateasannoyingweaskthemtoexplaintheirthinking.Ananalysisoftheratingsandcommentsyieldsabetterofun-derstandingofwhatusersndannoyingabouttheseads.Thisanalysiswillalsoexhibithowannoyingadsnegativelyaectuserperceptionsofadvertisers.Theseanalysesareadditionalcontributionsofthiswork.Inthesecondexperiment,weusethoseadsidentiedasmoreorlessannoying,alongwiththerecentmethodolog-icalinnovationofToomimetal.[18],toestimatethepayrateincreasenecessarytogenerateanequalnumberofpageviewsinthepresenceofannoyingads,comparedtoinnocu-ousadsornoads.Thisestimateisthecostofannoyingadsinourexperiment.Wechosecategorizingemailsasthetasktoproxyforusingapublisher'ssitebecauseusersei-therimplicitlyorexplicitlyneedtocategorizetheiremailsasspamornotspaminthepresenceofadswhenusingfreeweb-basedemailservicessuchasYahoo!Mail,GMail,andMail.com.Finally,weprovideatheoreticalmodelofhowourempiricalndingscouldaectthedisplayadvertisingindustry,whichisthethirdcontributionofthiswork.2.RELATEDWORKAsmentionedinSection1,weusethemethodologicalin-novationofToomimetal.[18]forcomputingcompensatingdierentials.Toomimetal.conductedaMechanicalTurkexperimentinwhichparticipantsrandomlyexperiencedaneasy,medium,orhardversionofataskatarandomlyas-signedpayrate.Thisallowedtheauthorstocomputehowmuchmoreonewouldhavetopayaworkertodothehardtaskoverthemediumandeasytasks.Theauthorsalsoex-hibitedthistechniqueinanexperimentinwhichparticipantswererandomlyassignedtouseeitheran\ugly"ora\pretty"interfacetodoatask.Wewillusethistechniquetoisolatetheeectoftheadqualityonuserabandonment.Next,wedescribepriorexperimentalworkwhichstudiestheimpactofadqualityonbehavior.DrezeandHussherr[4]conductedanexperimentontheef-fectivenessofdisplayadvertisementsusingeye-trackingtech-nology.Theirconclusion,thatusersrarelyfocusdirectlyonbannerads,isoftenreferredtoasbannerblindness,atermcoinedbyBenway[1].Burkeetal.[2]hadparticipantsper-formvisualsearchtasksinthepresenceofnoads,astaticdisplayad,orananimateddisplayad.Theyfoundthatadsdidreducesearchtime,however,therewasnosignicantdierencebetweenanimatedandstaticads.Perhapsevenmoresurprisingly,theydidaposthoctestwhichfoundthatanimatedadswererememberedlessfrequentlythanstaticads.YooandKim[21]askedasimilarresearchquestion.Theyconductedalarger-scalelaboratoryexperimentinwhichpar-ticipantswererandomlyexposedtowebpageswithadswithnoanimation,slow-movinganimationorfast-movinganima-tion.Theyfoundthatmoreanimationdidincreaseattentiontoads.Moreover,moderateanimationincreasedadrecogni-tionratesandbrandattitudes.Highlyanimatedads,how-ever,decreasedrecognitionratesandbrandattitudes.ThisresultcomplementstheresultsofBurkeetal.[2].YooandKim[21]concludethat,\Webadvertisersshouldbeawareofthepossibilitythatexcessiveanimationcanbackreagainsttheoriginalintentionofeectivecommunication."GoldfarbandTucker[5]conductedaeldexperimentinwhichtheyfoundthatadsthatmatchedthesite'scontentoradsthatwereintrusiveincreasedparticipant'sself-reportedintenttopurchase.However,adsthatwerebothintru-siveandmatchedthewebsite'scontentreducedintenttopurchase.Adswereconsideredintrusiveif,forexample,theyproducedapopupwindow,tookoverthewholescreen,playedmusic,orobscuredthewebpagetext.Theauthorssuggestthatthereasonforthisinteractioneectisthatusersaremoresensitivetotargetedandintrusiveadswhentheproductadvertisedisprivacysensitive.Inthecontextofsponsoredsearch,Buscheretal.[3]foundthatadsthatarerelevanttothesearchtermsreceivedmorevisualatten-tionthanadsthatwerelessrelevant.ThiscomplementstheresultsofGoldfarbandTucker[5]whichwerefoundinthedomainofdisplayadvertising.Takenasawhole,thesestudiessuggesttheremaybeben-etstoasmalldegreeofanimationorintrusivenessinad-vertising,butthattoomuchanimationorintrusivenesscanhaveadetrimentalimpactontheadeectiveness.3.RATINGTHEQUALITYOFADSWenextdescribeourexperiments,bothofwhichwereconductedonAmazon'sMechanicalTurk1,anonlinelabormarket.Sinceitwasoriginallybuiltforjobsthataredif-cultforcomputersbutareeasyforhumans(e.g.,imagerecognition),jobsonMechanicalTurkarecalledHumanIn-telligenceTasksorHITs.TherearetwotypesofpeopleonMechanicalTurk:requestersandworkers.RequesterscanpostHITsandworkerscanchoosewhichHITstodoforpay.AfteraworkersubmitsaHIT,therequestercaneitheracceptorrejecttheworkbasedonitsquality.ThefractionofHITsthataworkersubmitswhichareacceptedisthatworker'sapprovalrating.Thisfunctionsasarep-utationmechanism.TheAmazonAPIgiveseachworkeraccountaunique,anonymousidentier.BytrackingtheIDsoftheworkerswhoacceptedourHITs,wecouldenforcethatparticipantswereonlyallowedtoparticipateinoneofthetwoexperiments,andtheywereonlyallowedtodothatexperimentonetime.ThereisaburgeoningliteratureonconductingbehavioralexperimentsonMechanicalTurk[12,11,16,6,7,20,13,9,17].Inthissetting,theexperimentertakesontheroleoftherequesterandtheworkersarethepaidparticipantsoftheexperiment.MasonandSuri[10]provideahow-toguideforconductingbehavioralexperimentsonMechanicalTurk.Wenowdescribethedesignandresultsofourrstexperiment,whichservedtoidentifysetsofmoreandlessannoyingads 1http://www.mturk.com Figure1:Thetoppanelranksadsbyannoyingnessandshowsthatthe21mostannoyingadswereanimatedandthe24leastannoyingadswerestatic.Thebottompanelrankspairsofadsbytheannoyingnessoftheanimatedvariant.Thestaticvariantstendtofallbelowtheiranimatedversions,suggestingthatanimationincreasesannoyingness,evenwhentheadvertiserandproductareheldconstant.Errorbarsare1standarderror. Figure4:Screenshotoftheemailcategorizationtaskshowingthebadadscondition.Atthebottomofeachemailclassicationpage,partici-pantswereshownhowmanyemailstheyhadrated,theirpayrate,andareviewoftheinstructions.Thefooterincludedtwobuttons:oneallowingthemtosubmitandrateanotheremail,andasecondallowingthemtostopcategorizingandcollecttheirpayment.Participantswereallowedtoclassifyupto1000emails.4.2ResultsLetanimpressionbeoneparticipantviewingoneemail(anditsaccompanyingads,ifany),regardlessofwhethertheparticipantclassiestheemailorquitsbeforeclassifyingit.Sinceanemailispresentedassoonastheuseracknowledgestheinstructions,eachofthe1223participantsgeneratedatleastoneimpression.Theoveralldistributionofimpressionsperpersonisskewedwithameanof61,amedianof25andrstandthirdquartilesof6and57.Beingboundedby1frombelowandeectivelyunboundedfromabove(onlytwoparticipantsreachedtheupperlimit),theseimpressionsconstitutecountdata.Inparticular,theyareoverdispersedcountdatarelativetothePoisson(observedvariance/the-oreticalPoissondatavarianceis228.7,p.0001)andthuswellsuitedtoanegativebinomialgeneralizedlinearmodel(GLM)[19].Model1inTable2providesthecoecientsofanegativebinomialGLMofimpressionsonpayrateanddummyvariablesforthepresenceof\goodads"ornoads,relativetothebaselineof\badads".Relativetoabaseline Model1Model2 (Intercept)3:433:43(0:12)(0:12)Goodads0:17(0:10)Noads0:22(0:10)Payrate26:4726:61(4:80)(4:80)Goodadsornoads0:19(0:08) AIC12158:5712156:85BIC12184:1212177:29LogLikelihood6074:296074:43Deviance1481:001481:04Numberofobservations12231223 ***p0:01,**p0:05,*p0:1Table2:NegativebinomialGLMofimpressionsonadconditionandpayrate.Badadsleadtofewerimpressionsthangoodadsornoads.Coecientsareexpressedinlogimpressions;predictedvaluesaredisplayedinFigure5.Payrateisindollarsperveimpressions(.01,.02,.03).Standarderrorsareinparentheses.of\badads",boththe\goodads"conditionandthenoadsconditionledtosubstantiallymoreimpressions(19%and25%moreimpressions,respectively).Model2isthesameasModel1butreplacesthetwoaddummieswithonenewdummyrepresentingthe\goodads"andnoadsconditionscombinedandresultsinasimilarconclusion.Astheco-ecientsinTable2areexpressedinlogterms,theeectsoftheconditionsonrawimpressionsismosteasilyseeninFigure5,whichalsomakesclearthatthedierenceinim-pressionsbetweenthe\goodads"and\noads"conditionsisnotsignicant.ThemodelexpressedinTable2andFigure5canbeusedtoestimatethecompensatingdierentialofannoyingadsinthisexperiment.Sincethecurvesareslightlynon-linear,arangeofcompensatingdierentialscouldbecalculatedacrossthepayrateandadconditions.Togetasimple,sin-gleapproximationweusethemiddle,\goodads"conditiontoestimatetheeectofpayraises.Wetaketheaverageofthe.2to.4and.4to.6centdierences,givinganestimatedincreaseof16.58impressionsresultingfroma.2centperimpressionpayraise.Whensummarizingtheeectofadquality,weusethenumberofimpressionsatthe.4centpayrate.Movingfrom\badads"tonoads,impressionsincreaseby12.68.Thepayraiserequiredtoachievea12.68impres-sionincreaseis.153centsperimpression(=:212:68=16:58)or$1.53CPM(costperthousandimpressions).Thatis,inthisexperiment,aparticipantinthe\badads"conditionwouldneedtobepaidanadditional$1.53perthousandim-pressionstogenerateasmanyimpressionsasapersonintheconditionwithoutads.Similarly,movingfromthe\badads"conditiontothe\goodads"conditionresultedinanadditional9.52impressionsperperson.Itwouldrequireapayraiseof.115centsperimpression(=:29:52=16:58)togenerate9.52additionalimpressions,meaningthatpeoplein Figure7:PhasediagramrelatingmarketsharetouserutilityasdescribedbyTheorem1Inaddition,ifxx,thenuserutilityu-383;uandisde-creasingovertime.Ifx-383;x,uuandisincreasingovertime.TheproofofthistheoremisgiventheinAppendix.ThesolutionisillustratedinthephasediagramgiveninFigure7.Theequilibriumforanystartingmarketsharexinvolvesthepathpointingtoward(x;u).Thevalueofuadjuststoputthepublisheronthispath.Startingwithalowmarketshare,thepublishersetsahighuserutilitywhichisacombinationoflowadvertisingandhighcontentquality,andthengradu-allydegradesuserutilityandincreasesadvertisingintensity.Incontrast,apublisherwhostartswithahighmarketsharewillsetaverylowcontentqualityandhighadvertisingin-tensity,andthengraduallyimprovetheuserexperience.Anincreaseintheinterestratedecreasesx,theasymptoticmarketshare.Anincreaseinthecompetitiveleveluin-creasesxwhenislog-convexandvice-versa.Thereareseveralconclusionsonecandrawfromthismodel.First,sincetheterminalmarketsharepredictedinTheo-rem1dependson,whichdependsonAandu,themodeljustiestheratiooftherevenuetousercostasthekeymetricforadvertisingselection.Second,inacompetitiveadvertis-ingmarket,alladswillsellforaconstanttimestheusercost.Annoyingadswillrunonlywhentheirrevenueisveryhighorthepublisherisextremelywillingtosacriceuserexpe-rienceforrevenue.Third,alegacypublisher,whosemarketshareislargebecausetheyinitiallyfacedlittlecompetition,willstartwithaloweruserexperienceinvolvingbothmoreadsandworsecontentthananentrant.Thiswillresultinthelegacypublisherseeingafastdeclineinuserbase.Thelegacypublisherscontentwillgraduallyimproveuntilasta-blepointisreached.Finally,ifconsumersreactsucientlyslowlytochangesincontent(thatis,issmall),alegacypublisherwillgraduallygoextinctratherthanoerabetteruserexperience.6.CONCLUSIONTherststudyreportedhereshowedthatpeoplendani-matedadvertisementsmoreannoyingthanstaticones,hold-ingallelseconstant.Thisstudyalsoidentiedvecategoriesofcomplaintsaboutannoyingadsprovidingarstpassatidentifyingundesirablefeatures.Weusedthegoodandbadadsfromthisstudytomeasurethecompensatingwagedif-ferentialinthesecondstudy.Themainresultofthispa-peristhatannoyingadsleadtositeabandonmentandthusfewerimpressionsthangoodadsornoads.Inwhatmightbeseenasgoodnewsforpublishers,goodadsandnoadsledtoroughlyequalnumbersofimpressions.Annoyingadsimpairedpeople'sabilitytocarryoutanemailclassicationtask,suggestingthatannoyingadshavearealcosttousersbeyondmereannoyance.Finally,weprovidedatheoreticalmodelthatcomputesadynamicequilibrium,whichpermitsstudyingnotonlypropertiesofthesteadystate,butthead-justmenttothatstateaswell.Thismodelcanbeusedtounderstandthebehavioroflegacypublishers,whoinheritedalargemarketshare,inthefaceofcompetitionfromnewentrants.Wecalculatedthecompensatingwagedierentialinourexperimentofbadadstonoadstobe$1.53CPM,badadstogoodadstobe$1.15,andgoodadstonoadstobe$.38CPM.Somecaremustbetakenininterpretingthesenumbers.Whilewepickedatask|classifyingemails|thatshouldbefamiliarandcommonformostinternetusers,thistaskmaynotberepresentativeofotherinternettaskslikereadingnewsstoriesorsearchingforproductstopurchase.Abandonmentratesmaydierwithdierenttasksandtheeectsofadvertisingmayvaryaswell.Whilevirtuallyev-erywebservicefeaturescompetition,theswitchingcostsvaryfromverylowinconsumingnewstorelativelyhighinchangingemailservices.BecauseourusersonMechanicalTurkhaveanoutsideoptionofworkingonanalternativeHIT,weexpectourresultstobemostapplicabletosit-uationsinvolvinglowerswitchingcosts.Nevertheless,weexpectthatourndingthatannoyingadscosttheuseratleast$1CPMovermorepleasantadswillbeobtainedinsomeotherenvironments.Forthesereasons,wesuggestfurtherstudiesbedoneonMechanicalTurk,aseldexperiments,andinlaboratoriestomeasurethisdierentialonsimilaranddierenttasks.Ifstudiesacrossvariousdomainswithavarietyoftasksandoutsideoptionsarriveatsimilardierentials,morecredencecanbeplacedonthesenumbers.Weviewthisworkasarststepinthisdirection.Iffutureworkarrivesatsimilarestimatesacrossavarietyofpublishers,suchestimatescouldserveasausefullowerboundforwhatapublishershouldchargetoruntheseads.Moreover,itwillbevaluabletousethecompensatingdierentialsapproachtopricethevariousbadaspectsofads,includinganimationandpooraesthetics.Thisworkalsosuggestsavarietyofpolicyrecommenda-tions.Mostdirectly,the$1CPMusercostofbadadshaspracticalconsequencesforpublishers,especiallyasbadadsoftencommandlowerCPMs.Itisareasonthatpublishersshouldinsistonasubstantialpremiumforannoyingadver-tisements.Moreover,apublishercouldrandomizewhichusersseewhichadsandtrackbothtimespentonthepageandthefrequencywithwhichusersreturntothesite.Thistypeofexperimentationwouldcapturelongertermeectsofannoyingadsthanthosestudiedhere.Also,publisherscouldgiveusersanoptiontocloseorreplaceanad.Areplace-menteventwouldallowthepublishertoinferthatauserwouldpreferarandomadovertheadcurrentlyshown.Ad-vertiserswithahighclosurerateshouldbechargedmore.Furthermore,itwouldbereasonabletoassumethatmore annoyingadswouldbeclosedorreplacedfasterthanlessannoyingads.Adreplacementwouldhelptheuserbyre-movingtheannoyingadandthepublisherbymakingitpos-sibletochargefortwoimpressions.7.ACKNOWLEDGMENTSWethankRandallA.Lewis,JustinM.Rao,andDavidH.Reileyforhelpfulconversations.APPENDIXInthissectionwegivetheproofofTheorem1.Proof.Deney=logx 1x.Note,y0=x0 x+x0 1x=x0 x(1x),andx=ey 1+ey.Furthermore,1+ey=1+x 1x=1 1x.Thuswecanreformulatethepublisher'soptimizationproblemasthatofmaximizingR10ertey 1+eyu+1 y0dt.LetF(y;y0;t)=ertey 1+eyu+1 y0.TheEulerequationforthisproblemis0=@F @yd dt@F @y0=ertey (1+ey)2u+1 y01 d dtertey 1+ey0u+1 y0=x ert[(1x)(u)+r0(u)0(u)(uu)(1x)00(u)u0]Thus,00(u)u0=(1x)(u)+r0(u)(uu)(1x)0(u):Asteadystateofthesystemholdswhenx0=u0=0,oru=uand0=(1x)(u)+r0(u).Thisisequivalentto1x=r 0(u) (u):Ifr 0(u) (u)1,alloptimalpathsinvolvex!0asthereisnointernalsteadystate.Whenr 0(u) (u)1,thereisaninteriorsteadystate.Theu0=0curveoccurswhen0=(1x)(u)+r0(u)(uu)(1x)0(u):Thus,near(x;u),du dxu0=0=(u)(uu)0(u) (1x)0(u)+r00(u)(1x)0(u)(uu)(1x)000(u)=(u)(uu)0(u) +r00(u)(uu)(1x)000(u)(u) r00(u)0:WecanobtaininsightaboutthepathsnearthissolutionbyarstorderTaylorapproximation.Thestrategylookslikethis.Writex0u0= (uu)x(1x)(1x)(u) 00(u)+r0(u) 00(u)(uu)(1x)0(u) 00(u)!=g(x;u)h(x;u):x0u0 @g/@x@g/@u@h/@x@h/@u!(x;u)=(x;u)xxuu.Locallythebehaviorofthegeneralsystemisapproximatedbythebehaviorofthelinearsystem.Theonlychallengingterminthematrixis@h @uu=u;x=x=@ @u(1x)(u) 00(u)+r0(u) 00(u)(uu)(1x)0(u) 00(u)u=u;x=x=(1x)(u)000(u) 00(u)2+r10(u)000(u) 00(u)2=rThus,x0u00x(1x)(u) 00(u)rxxuuTheeigenvaluesofthelinearsystemaredeterminedbyso-lutionsto0=detx(1x)(u) 00(u)r0=2r+2x(1x)(u) 00(u)solvingforgives,=1 2 rs r242x(1x)(u) 00(u)!Because00(u)0,thereisonepositiveandonenegativeeigenvalueandbotharereal.Thus,thebehaviorofthesystemisasaddle,asillustratedinFigure7.Thereareinnitelymanypathsconsistentwithequilibriumgivenbythedierentialequations.Whichoneistherightone?Inthecasewhenr 0(u) (u)1,allpathsthatdon'tviolatetransversalityleadtox=0.Supposexisacandidatelimit.Considersettingu=u+fortunitsoftime.Thermearns Zt0ersdsx(u+)+Zt0ersds(x+x(1x)t)(u)y=1 r1ertx(u+)+1 rert(x+x(1x)t)(u)1 t@ @=0=1 t1 r1ertx0(u)+ rertx(1x)t(u)= rx(u)1ert t0(u) (u)+ert(1x)= rx(u) r0(u) (u)+(1x)Thus,itpaystoincreaseaconvergentxifandonlyifx1r 0(u) (u),implyingthatthisisonlycandidateforconvergentpathswhenr 0(u) (u)1.