The Cost of Annoying Ads Daniel G - PDF document

etarycostofannoyingads.Therstandmaincontributionofthisworkisthatwemeasurethecompensatingwagedif-ferentialofannoyingads.Thatis,wemeasurehowmuchmoreonemustpayausertodothesameamountofworkinthepresenceofannoyingadscomparedtoinnocuousadsornoads.Thecompensatingdierentialisimportanttomeasurebecauseitcapturessomeofthenegativeeectsofadvertising,whichpublishersneedtoheedasalowerboundwhensettingthepricetorunanad.Inatwo-experimentinvestigation,wecomputethecom-pensatingdierentialforannoyingads.Intherstexper-imentusersrandomlyratedeitherananimatedadoritsstaticcounterpart.Thisdesignshowsthatanimationhasanegativeimpactonuserratings.Forthoseadsthatusersrateasannoyingweaskthemtoexplaintheirthinking.Ananalysisoftheratingsandcommentsyieldsabetterofun-derstandingofwhatusersndannoyingabouttheseads.Thisanalysiswillalsoexhibithowannoyingadsnegativelyaectuserperceptionsofadvertisers.Theseanalysesareadditionalcontributionsofthiswork.Inthesecondexperiment,weusethoseadsidentiedasmoreorlessannoying,alongwiththerecentmethodolog-icalinnovationofToomimetal.[18],toestimatethepayrateincreasenecessarytogenerateanequalnumberofpageviewsinthepresenceofannoyingads,comparedtoinnocu-ousadsornoads.Thisestimateisthecostofannoyingadsinourexperiment.Wechosecategorizingemailsasthetasktoproxyforusingapublisher'ssitebecauseusersei-therimplicitlyorexplicitlyneedtocategorizetheiremailsasspamornotspaminthepresenceofadswhenusingfreeweb-basedemailservicessuchasYahoo!Mail,GMail,andMail.com.Finally,weprovideatheoreticalmodelofhowourempiricalndingscouldaectthedisplayadvertisingindustry,whichisthethirdcontributionofthiswork.2.RELATEDWORKAsmentionedinSection1,weusethemethodologicalin-novationofToomimetal.[18]forcomputingcompensatingdierentials.Toomimetal.conductedaMechanicalTurkexperimentinwhichparticipantsrandomlyexperiencedaneasy,medium,orhardversionofataskatarandomlyas-signedpayrate.Thisallowedtheauthorstocomputehowmuchmoreonewouldhavetopayaworkertodothehardtaskoverthemediumandeasytasks.Theauthorsalsoex-hibitedthistechniqueinanexperimentinwhichparticipantswererandomlyassignedtouseeitheran\ugly"ora\pretty"interfacetodoatask.Wewillusethistechniquetoisolatetheeectoftheadqualityonuserabandonment.Next,wedescribepriorexperimentalworkwhichstudiestheimpactofadqualityonbehavior.DrezeandHussherr[4]conductedanexperimentontheef-fectivenessofdisplayadvertisementsusingeye-trackingtech-nology.Theirconclusion,thatusersrarelyfocusdirectlyonbannerads,isoftenreferredtoasbannerblindness,atermcoinedbyBenway[1].Burkeetal.[2]hadparticipantsper-formvisualsearchtasksinthepresenceofnoads,astaticdisplayad,orananimateddisplayad.Theyfoundthatadsdidreducesearchtime,however,therewasnosignicantdierencebetweenanimatedandstaticads.Perhapsevenmoresurprisingly,theydidaposthoctestwhichfoundthatanimatedadswererememberedlessfrequentlythanstaticads.YooandKim[21]askedasimilarresearchquestion.Theyconductedalarger-scalelaboratoryexperimentinwhichpar-ticipantswererandomlyexposedtowebpageswithadswithnoanimation,slow-movinganimationorfast-movinganima-tion.Theyfoundthatmoreanimationdidincreaseattentiontoads.Moreover,moderateanimationincreasedadrecogni-tionratesandbrandattitudes.Highlyanimatedads,how-ever,decreasedrecognitionratesandbrandattitudes.ThisresultcomplementstheresultsofBurkeetal.[2].YooandKim[21]concludethat,\Webadvertisersshouldbeawareofthepossibilitythatexcessiveanimationcanbackreagainsttheoriginalintentionofeectivecommunication."GoldfarbandTucker[5]conductedaeldexperimentinwhichtheyfoundthatadsthatmatchedthesite'scontentoradsthatwereintrusiveincreasedparticipant'sself-reportedintenttopurchase.However,adsthatwerebothintru-siveandmatchedthewebsite'scontentreducedintenttopurchase.Adswereconsideredintrusiveif,forexample,theyproducedapopupwindow,tookoverthewholescreen,playedmusic,orobscuredthewebpagetext.Theauthorssuggestthatthereasonforthisinteractioneectisthatusersaremoresensitivetotargetedandintrusiveadswhentheproductadvertisedisprivacysensitive.Inthecontextofsponsoredsearch,Buscheretal.[3]foundthatadsthatarerelevanttothesearchtermsreceivedmorevisualatten-tionthanadsthatwerelessrelevant.ThiscomplementstheresultsofGoldfarbandTucker[5]whichwerefoundinthedomainofdisplayadvertising.Takenasawhole,thesestudiessuggesttheremaybeben-etstoasmalldegreeofanimationorintrusivenessinad-vertising,butthattoomuchanimationorintrusivenesscanhaveadetrimentalimpactontheadeectiveness.3.RATINGTHEQUALITYOFADSWenextdescribeourexperiments,bothofwhichwereconductedonAmazon'sMechanicalTurk1,anonlinelabormarket.Sinceitwasoriginallybuiltforjobsthataredif-cultforcomputersbutareeasyforhumans(e.g.,imagerecognition),jobsonMechanicalTurkarecalledHumanIn-telligenceTasksorHITs.TherearetwotypesofpeopleonMechanicalTurk:requestersandworkers.RequesterscanpostHITsandworkerscanchoosewhichHITstodoforpay.AfteraworkersubmitsaHIT,therequestercaneitheracceptorrejecttheworkbasedonitsquality.ThefractionofHITsthataworkersubmitswhichareacceptedisthatworker'sapprovalrating.Thisfunctionsasarep-utationmechanism.TheAmazonAPIgiveseachworkeraccountaunique,anonymousidentier.BytrackingtheIDsoftheworkerswhoacceptedourHITs,wecouldenforcethatparticipantswereonlyallowedtoparticipateinoneofthetwoexperiments,andtheywereonlyallowedtodothatexperimentonetime.ThereisaburgeoningliteratureonconductingbehavioralexperimentsonMechanicalTurk[12,11,16,6,7,20,13,9,17].Inthissetting,theexperimentertakesontheroleoftherequesterandtheworkersarethepaidparticipantsoftheexperiment.MasonandSuri[10]provideahow-toguideforconductingbehavioralexperimentsonMechanicalTurk.Wenowdescribethedesignandresultsofourrstexperiment,whichservedtoidentifysetsofmoreandlessannoyingads 1http://www.mturk.com Figure1:Thetoppanelranksadsbyannoyingnessandshowsthatthe21mostannoyingadswereanimatedandthe24leastannoyingadswerestatic.Thebottompanelrankspairsofadsbytheannoyingnessoftheanimatedvariant.Thestaticvariantstendtofallbelowtheiranimatedversions,suggestingthatanimationincreasesannoyingness,evenwhentheadvertiserandproductareheldconstant.Errorbarsare1standarderror. Figure4:Screenshotoftheemailcategorizationtaskshowingthebadadscondition.Atthebottomofeachemailclassicationpage,partici-pantswereshownhowmanyemailstheyhadrated,theirpayrate,andareviewoftheinstructions.Thefooterincludedtwobuttons:oneallowingthemtosubmitandrateanotheremail,andasecondallowingthemtostopcategorizingandcollecttheirpayment.Participantswereallowedtoclassifyupto1000emails.4.2ResultsLetanimpressionbeoneparticipantviewingoneemail(anditsaccompanyingads,ifany),regardlessofwhethertheparticipantclassiestheemailorquitsbeforeclassifyingit.Sinceanemailispresentedassoonastheuseracknowledgestheinstructions,eachofthe1223participantsgeneratedatleastoneimpression.Theoveralldistributionofimpressionsperpersonisskewedwithameanof61,amedianof25andrstandthirdquartilesof6and57.Beingboundedby1frombelowandeectivelyunboundedfromabove(onlytwoparticipantsreachedtheupperlimit),theseimpressionsconstitutecountdata.Inparticular,theyareoverdispersedcountdatarelativetothePoisson(observedvariance/the-oreticalPoissondatavarianceis228.7,p.0001)andthuswellsuitedtoanegativebinomialgeneralizedlinearmodel(GLM)[19].Model1inTable2providesthecoecientsofanegativebinomialGLMofimpressionsonpayrateanddummyvariablesforthepresenceof\goodads"ornoads,relativetothebaselineof\badads".Relativetoabaseline Model1Model2 (Intercept)3:433:43(0:12)(0:12)Goodads0:17(0:10)Noads0:22(0:10)Payrate26:4726:61(4:80)(4:80)Goodadsornoads0:19(0:08) AIC12158:5712156:85BIC12184:1212177:29LogLikelihood�6074:29�6074:43Deviance1481:001481:04Numberofobservations12231223 ***p0:01,**p0:05,*p0:1Table2:NegativebinomialGLMofimpressionsonadconditionandpayrate.Badadsleadtofewerimpressionsthangoodadsornoads.Coecientsareexpressedinlogimpressions;predictedvaluesaredisplayedinFigure5.Payrateisindollarsperveimpressions(.01,.02,.03).Standarderrorsareinparentheses.of\badads",boththe\goodads"conditionandthenoadsconditionledtosubstantiallymoreimpressions(19%and25%moreimpressions,respectively).Model2isthesameasModel1butreplacesthetwoaddummieswithonenewdummyrepresentingthe\goodads"andnoadsconditionscombinedandresultsinasimilarconclusion.Astheco-ecientsinTable2areexpressedinlogterms,theeectsoftheconditionsonrawimpressionsismosteasilyseeninFigure5,whichalsomakesclearthatthedierenceinim-pressionsbetweenthe\goodads"and\noads"conditionsisnotsignicant.ThemodelexpressedinTable2andFigure5canbeusedtoestimatethecompensatingdierentialofannoyingadsinthisexperiment.Sincethecurvesareslightlynon-linear,arangeofcompensatingdierentialscouldbecalculatedacrossthepayrateandadconditions.Togetasimple,sin-gleapproximationweusethemiddle,\goodads"conditiontoestimatetheeectofpayraises.Wetaketheaverageofthe.2to.4and.4to.6centdierences,givinganestimatedincreaseof16.58impressionsresultingfroma.2centperimpressionpayraise.Whensummarizingtheeectofadquality,weusethenumberofimpressionsatthe.4centpayrate.Movingfrom\badads"tonoads,impressionsincreaseby12.68.Thepayraiserequiredtoachievea12.68impres-sionincreaseis.153centsperimpression(=:212:68=16:58)or$1.53CPM(costperthousandimpressions).Thatis,inthisexperiment,aparticipantinthe\badads"conditionwouldneedtobepaidanadditional$1.53perthousandim-pressionstogenerateasmanyimpressionsasapersonintheconditionwithoutads.Similarly,movingfromthe\badads"conditiontothe\goodads"conditionresultedinanadditional9.52impressionsperperson.Itwouldrequireapayraiseof.115centsperimpression(=:29:52=16:58)togenerate9.52additionalimpressions,meaningthatpeoplein Figure7:PhasediagramrelatingmarketsharetouserutilityasdescribedbyTheorem1Inaddition,ifxx,thenuserutilityu&#x-383;uandisde-creasingovertime.Ifx&#x-383;x,uuandisincreasingovertime.TheproofofthistheoremisgiventheinAppendix.ThesolutionisillustratedinthephasediagramgiveninFigure7.Theequilibriumforanystartingmarketsharexinvolvesthepathpointingtoward(x;u).Thevalueofuadjuststoputthepublisheronthispath.Startingwithalowmarketshare,thepublishersetsahighuserutilitywhichisacombinationoflowadvertisingandhighcontentquality,andthengradu-allydegradesuserutilityandincreasesadvertisingintensity.Incontrast,apublisherwhostartswithahighmarketsharewillsetaverylowcontentqualityandhighadvertisingin-tensity,andthengraduallyimprovetheuserexperience.Anincreaseintheinterestratedecreasesx,theasymptoticmarketshare.Anincreaseinthecompetitiveleveluin-creasesxwhenislog-convexandvice-versa.Thereareseveralconclusionsonecandrawfromthismodel.First,sincetheterminalmarketsharepredictedinTheo-rem1dependson,whichdependsonAandu,themodeljustiestheratiooftherevenuetousercostasthekeymetricforadvertisingselection.Second,inacompetitiveadvertis-ingmarket,alladswillsellforaconstanttimestheusercost.Annoyingadswillrunonlywhentheirrevenueisveryhighorthepublisherisextremelywillingtosacriceuserexpe-rienceforrevenue.Third,alegacypublisher,whosemarketshareislargebecausetheyinitiallyfacedlittlecompetition,willstartwithaloweruserexperienceinvolvingbothmoreadsandworsecontentthananentrant.Thiswillresultinthelegacypublisherseeingafastdeclineinuserbase.Thelegacypublisherscontentwillgraduallyimproveuntilasta-blepointisreached.Finally,ifconsumersreactsucientlyslowlytochangesincontent(thatis,issmall),alegacypublisherwillgraduallygoextinctratherthanoerabetteruserexperience.6.CONCLUSIONTherststudyreportedhereshowedthatpeoplendani-matedadvertisementsmoreannoyingthanstaticones,hold-ingallelseconstant.Thisstudyalsoidentiedvecategoriesofcomplaintsaboutannoyingadsprovidingarstpassatidentifyingundesirablefeatures.Weusedthegoodandbadadsfromthisstudytomeasurethecompensatingwagedif-ferentialinthesecondstudy.Themainresultofthispa-peristhatannoyingadsleadtositeabandonmentandthusfewerimpressionsthangoodadsornoads.Inwhatmightbeseenasgoodnewsforpublishers,goodadsandnoadsledtoroughlyequalnumbersofimpressions.Annoyingadsimpairedpeople'sabilitytocarryoutanemailclassicationtask,suggestingthatannoyingadshavearealcosttousersbeyondmereannoyance.Finally,weprovidedatheoreticalmodelthatcomputesadynamicequilibrium,whichpermitsstudyingnotonlypropertiesofthesteadystate,butthead-justmenttothatstateaswell.Thismodelcanbeusedtounderstandthebehavioroflegacypublishers,whoinheritedalargemarketshare,inthefaceofcompetitionfromnewentrants.Wecalculatedthecompensatingwagedierentialinourexperimentofbadadstonoadstobe$1.53CPM,badadstogoodadstobe$1.15,andgoodadstonoadstobe$.38CPM.Somecaremustbetakenininterpretingthesenumbers.Whilewepickedatask|classifyingemails|thatshouldbefamiliarandcommonformostinternetusers,thistaskmaynotberepresentativeofotherinternettaskslikereadingnewsstoriesorsearchingforproductstopurchase.Abandonmentratesmaydierwithdierenttasksandtheeectsofadvertisingmayvaryaswell.Whilevirtuallyev-erywebservicefeaturescompetition,theswitchingcostsvaryfromverylowinconsumingnewstorelativelyhighinchangingemailservices.BecauseourusersonMechanicalTurkhaveanoutsideoptionofworkingonanalternativeHIT,weexpectourresultstobemostapplicabletosit-uationsinvolvinglowerswitchingcosts.Nevertheless,weexpectthatourndingthatannoyingadscosttheuseratleast$1CPMovermorepleasantadswillbeobtainedinsomeotherenvironments.Forthesereasons,wesuggestfurtherstudiesbedoneonMechanicalTurk,aseldexperiments,andinlaboratoriestomeasurethisdierentialonsimilaranddierenttasks.Ifstudiesacrossvariousdomainswithavarietyoftasksandoutsideoptionsarriveatsimilardierentials,morecredencecanbeplacedonthesenumbers.Weviewthisworkasarststepinthisdirection.Iffutureworkarrivesatsimilarestimatesacrossavarietyofpublishers,suchestimatescouldserveasausefullowerboundforwhatapublishershouldchargetoruntheseads.Moreover,itwillbevaluabletousethecompensatingdierentialsapproachtopricethevariousbadaspectsofads,includinganimationandpooraesthetics.Thisworkalsosuggestsavarietyofpolicyrecommenda-tions.Mostdirectly,the$1CPMusercostofbadadshaspracticalconsequencesforpublishers,especiallyasbadadsoftencommandlowerCPMs.Itisareasonthatpublishersshouldinsistonasubstantialpremiumforannoyingadver-tisements.Moreover,apublishercouldrandomizewhichusersseewhichadsandtrackbothtimespentonthepageandthefrequencywithwhichusersreturntothesite.Thistypeofexperimentationwouldcapturelongertermeectsofannoyingadsthanthosestudiedhere.Also,publisherscouldgiveusersanoptiontocloseorreplaceanad.Areplace-menteventwouldallowthepublishertoinferthatauserwouldpreferarandomadovertheadcurrentlyshown.Ad-vertiserswithahighclosurerateshouldbechargedmore.Furthermore,itwouldbereasonabletoassumethatmore annoyingadswouldbeclosedorreplacedfasterthanlessannoyingads.Adreplacementwouldhelptheuserbyre-movingtheannoyingadandthepublisherbymakingitpos-sibletochargefortwoimpressions.7.ACKNOWLEDGMENTSWethankRandallA.Lewis,JustinM.Rao,andDavidH.Reileyforhelpfulconversations.APPENDIXInthissectionwegivetheproofofTheorem1.Proof.Deney=logx 1�x.Note,y0=x0 x+x0 1�x=x0 x(1�x),andx=ey 1+ey.Furthermore,1+ey=1+x 1�x=1 1�x.Thuswecanreformulatethepublisher'soptimizationproblemasthatofmaximizingR10e�rtey 1+ey�u+1 y0
dt.LetF(y;y0;t)=e�rtey 1+ey�u+1 y0
.TheEulerequationforthisproblemis0=@F @y�d dt@F @y0=e�rtey (1+ey)2u+1 y0�1 d dte�rtey 1+ey0u+1 y0=x e�rt[(1�x)(u)+r0(u)�0(u)(u�u)(1�x)�00(u)u0]Thus,00(u)u0=(1�x)(u)+r0(u)�(u�u)(1�x)0(u):Asteadystateofthesystemholdswhenx0=u0=0,oru=uand0=(1�x)(u)+r0(u).Thisisequivalentto1�x=�r 0(u) (u):If�r 0(u) (u)1,alloptimalpathsinvolvex!0asthereisnointernalsteadystate.When�r 0(u) (u)1,thereisaninteriorsteadystate.Theu0=0curveoccurswhen0=(1�x)(u)+r0(u)�(u�u)(1�x)0(u):Thus,near(x;u),du dxu0=0=(u)�(u�u)0(u) (1�x)0(u)+r00(u)�(1�x)0(u)�(u�u)(1�x)000(u)=(u)�(u�u)0(u) +r00(u)�(u�u)(1�x)000(u)(u) r00(u)0:WecanobtaininsightaboutthepathsnearthissolutionbyarstorderTaylorapproximation.Thestrategylookslikethis.Writex0u0= (u�u)x(1�x)(1�x)(u) 00(u)+r0(u) 00(u)�(u�u)(1�x)0(u) 00(u)!=g(x;u)h(x;u):x0u0 @g/@x@g/@u@h/@x@h/@u!(x;u)=(x;u)x�xu�u.Locallythebehaviorofthegeneralsystemisapproximatedbythebehaviorofthelinearsystem.Theonlychallengingterminthematrixis@h @uu=u;x=x=@ @u(1�x)(u) 00(u)+r0(u) 00(u)�(u�u)(1�x)0(u) 00(u)u=u;x=x=�(1�x)(u)000(u) 00(u)2+r1�0(u)000(u) 00(u)2=rThus,x0u00x(1�x)�(u) 00(u)rx�xu�uTheeigenvaluesofthelinearsystemaredeterminedbyso-lutionsto0=det�x(1�x)�(u) 00(u)r�0=2�r+2x(1�x)(u) 00(u)solvingforgives,=1 2 rs r2�42x(1�x)(u) 00(u)!Because00(u)0,thereisonepositiveandonenegativeeigenvalueandbotharereal.Thus,thebehaviorofthesystemisasaddle,asillustratedinFigure7.Thereareinnitelymanypathsconsistentwithequilibriumgivenbythedierentialequations.Whichoneistherightone?Inthecasewhen�r 0(u) (u)1,allpathsthatdon'tviolatetransversalityleadtox=0.Supposexisacandidatelimit.Considersettingu=u+
fortunitsoftime.Thermearns Zt0e�rsdsx(u+
)+Zt0e�rsds(x+x(1�x)
t)(u)y=1 r�1�e�rt
x(u+
)+1 re�rt(x+x(1�x)
t)(u)1 t@ @

=0=1 t�1 r�1�e�rt
x0(u)+ re�rtx(1�x)t(u)
= rx(u)1�e�rt t0(u) (u)+e�rt(1�x)= rx(u) r0(u) (u)+(1�x)Thus,itpaystoincreaseaconvergentxifandonlyifx1�r 0(u) (u),implyingthatthisisonlycandidateforconvergentpathswhen�r 0(u) (u)1.