GOLDSTEIN Microsoft Research and R PRESTON MCAFEE Microsoft Corporation and SIDDHARTH SURI Microsoft Research Display advertisements vary in the extent to which they annoy users While publishers know the payment they receive to run annoying ads ID: 32033
Download Pdf The PPT/PDF document "The Cost of Annoying Ads DANIEL G" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
TheCostofAnnoyingAdsDANIELG.GOLDSTEINMicrosoftResearchandR.PRESTONMCAFEEMicrosoftCorporationandSIDDHARTHSURIMicrosoftResearch Displayadvertisementsvaryintheextenttowhichtheyannoyusers.Whilepublishersknowthepaymenttheyreceivetorunannoyingads,littleisknownaboutthecostsuchadsincurduetouserabandonment.Weconductedtwoexperimentstoanalyzeadfeaturesthatrelatetoannoyingnessandtoputamonetaryvalueonthecostofannoyingads.TherstexperimentaskeduserstorateandcommentonalargenumberofadstakenfromtheWeb.Thisallowedustoestablishsetsofannoyingandinnocuousadsforuseinthesecondexperiment,inwhichusersweregiventheopportunitytocategorizeemailsforaper-messagewageandquitatanytime.Participantswererandomlyassignedtooneofthreedi erentpayratesandalsorandomlyassignedtocategorizetheemailsinthepresenceofnoads,annoyingads,orinnocuousads.Sinceeachemailcategorizationconstitutedanimpression,thisdesign,inspiredbyToomimetal.(2011),allowedustodeterminehowmuchmoreonemustpayapersontogeneratethesamenumberofimpressionsinthepresenceofannoyingadscomparedtonoadsorinnocuousads.CategoriesandSubjectDescriptors:J.4[SocialandBehavioralSciences]:EconomicsGeneralTerms:Economics,ExperimentationAdditionalKeyWordsandPhrases:display,advertising,quality,compensatingdi erential 1.INTRODUCTIONDisplayadvertisingistheprevalentwayforpublisherstomonetizecontentontheWeb.Publishersreceivepaymentfromadvertisersforplacingadsneartheircontentorintheirapplications.Publishersaretypicallypaidbythenumberofimpressionstheycandeliver.Thus,theyhaveanincentivetoattractandretainuserswithvaluablecontent,experiences,andapplications,andhaveadisincentivetoloseusersduetoannoyances.Displayadsvaryintheextenttowhichtheyannoyusers.Annoyingadsareasourceoftensionforpublisherssincetheybothmakemoney,throughpaymentsfromadvertisers,andcostmoney,throughadecreaseinpageviewsduetousersabandoningthesite.Thistensionhasledtocon ictwithinpublishingorganizations Authors'addresses:dgg@microsoft.com,preston@mcafee.cc,andsuri@microsoft.com.Thisarticleisanadaptationofthefollowingconferenceproceedingsarticleandappearsherewithpermissionofthepublishers:Goldstein,DanielG.,R.PrestonMcAfee,andSiddharthSuri(2013),\TheCostofAnnoyingAds,"Proceedingsofthe22ndInternationalWorldWideWebConference.c 2013InternationalWorldWideWebConferenceCommittee.Thisworkwaslaterexpandedandpresentedin[Goldsteinetal.2014].ACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52 48D.G.Goldsteinetal.betweensalespeople,whohaveanincentiveintheformofcommissiontosellanyads,andmanagement,whoareconcernedwithlong-termgrowthofusersandtrac.Thecontinuedlong-termdisplayofannoyingadsmayexertnegativee ectsonthepublisher,theuser,andtheadvertiser,whichwediscussinturn.First,annoyingadscanexertnegativee ectsonpublishers.Apartfromtheuserabandonmente ectsweinvestigateinthispaper,annoyingadsmightsignalthatthewebsite,onwhichtheadisplaced,lacksstability(\WhyshouldItrustmyemailtoasitethatissodesperateforcashitacceptsadsofsuchpoorquality?"),reputability(\WhyshouldItrusttheobjectivityofasitethatissointhepocketofadvertisersitwon'trefuseanyofthem?"),orsafety(\WhywouldItrustthispublishertoprotectmefromphishingattacks,scams,malware,etc.iftheyaresoindiscriminateaboutwhotheyletadvertise?").Second,annoyingadscanexertanegativeimpactonusers.Adswithexces-siveanimationcangetinthewayoftheuserconsumingthepublisher'scontent,underminingtheveryreasonthatbroughtthemtothesite.Finally,annoyingadsmayharmtheadvertiserthatcreatedthem.Aswillbeshown,annoyingadsareoftencharacterizedbyexaggeratedattemptstocapturevisualattentionsuchasthroughfast-movinganimationorbizarreimagery.Whilethesemanipulationsdocaptureattention,theymayalsosignalthattheadvertiserisdesperateforbusinessorlowonresources,underminingtheclassicalsignalofqualitythatadvertisingistheorizedtobring[Riley2001].Furthermore,experi-mentshaveshownthattoomuchanimationcanresultinloweradrecognitionratescomparedtoadswithmoderateornoanimation[YooandKim2005;Burkeetal.2005].Intheseways,annoyingadsmayactuallylowerbrandreputationandrecall,twometricsadvertiserstypicallystrivetoincrease.Ifannoyingadsexhibitsomanynegativee ectsforpublishers,usersandadver-tisers,onemaywonderwhyapublisherwouldrunannoyingadsatall.Theanswermaybethatitishasbeenhistoricallydiculttomeasurethemonetarycostofannoyingads.Themaincontributionofthisworkisthatwemeasurethecompen-satingwagedi erentialofannoyingads.Thatis,wemeasurehowmuchmoreonemustpayausertodothesameamountofworkinthepresenceofannoyingadscomparedtoinnocuousadsornoads.Thecompensatingdi erentialisimportanttomeasurebecauseitcapturessomeofthenegativee ectsofadvertising,whichpublishersneedtoheedasalowerboundwhensettingthepricetorunanad.Usingtwoexperiments,wecomputethecompensatingdi erentialforannoyingads.Intherstexperimentusersratedasetofadsintermsofhowannoyingtheyfoundeachad.Inthesecondexperiment,weusethoseadsidentiedasmoreorlessannoying,alongwiththerecentmethodologicalinnovationofToomimetal.[2011],toestimatethepayrateincreasenecessarytogenerateanequalnumberofpageviewsinthepresenceofannoyingads,comparedtoinnocuousadsornoads.Thisestimateisthecostofannoyingadsinourexperiment.Wechosecategorizingemailsasthetasktoproxyforusingapublisher'ssitebecauseuserseitherimplicitlyorexplicitlyneedtocategorizetheiremailsasspamornotspaminthepresenceofadswhenusingfreeweb-basedemailservicessuchasYahoo!Mail,GMail,andOutlook.com.ACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52 TheCostofAnnoyingAds492.RATINGTHEQUALITYOFADSWenowdescribethedesignandresultsofourrstexperiment,whichservedtoidentifysetsofmoreandlessannoyingads(henceforth\badads"and\goodads"forbrevity)foruseinthesecondexperiment.2.1MethodThegoalofthisexperimentistorankasetofactualdisplayadsintermsofannoyingness.Afterpreviewingalltheads,userswereshowneachadindividually,inrandomorder,andaskedtorateeachadona5-pointscalewiththefollowinglevels:1)Muchlessannoyingthantheaverageadinthisexperiment,2)Abitlessannoyingthantheaverageadinthisexperiment,3)Averageforthisexperiment,4)Abitmoreannoyingthantheaverageadinthisexperiment,and5)Muchmoreannoyingthantheaverageadinthisexperiment.Ifweviewattributesofanadasresidinginamultidimensionalspace,theaverageratingsindicatehowusersprojectthatmultidimensionalspaceontoaone-dimensionalannoyingnessscale.The10mostandleastannoyingadsserveasthesetsof\bad"and\good"adsinthenextexperiment.3.MEASURINGTHECOSTOFADSWeusethemethodofToomimetal.[2011]alongwiththesetsof\bad"and\goodads"tomeasurethecostofannoyingads.3.1MethodTheparticipantswere1223MechanicalTurkworkerswhoparticipatedforabasepayof25centsandabonus.Uponacceptingthetask,participantswererandomlyassignedtooneofnineconditions:threepayconditionsandthreeadconditions.Thepayconditionso eredabonusofone,two,orthreecentsperveemailsclassied(i.e.,.2,.4,or.6centsperemail),andtheadconditionsvariedwhether\badads",\goodads",ornoadsweredisplayedinthemarginasthetaskwascompleted.Achi-squaredtestfoundnosignicantdi erenceinthenumberofparticipantsbeginningworkacrossthenineconditions.Inallconditions,thetaskconsistedofclassifyingthecontentofemailsas\spam",\personal",\work"or\e-commerce"related.Emailsweredrawnfromthepublic-domainEnronemaildataset1withoneemailpresentedperpage,alongwithaccom-panyingads,ifany.Inthe\badads"condition,twoadsrandomlydrawnfromthe10mostannoyingadsinourrstexperimentweredisplayedinthemarginsaroundtheemailbeingclassied.The\goodads"conditionwasthesame,excepttheadsweredrawnfromthe10leastannoyingads.Inbothconditions,adsweredrawnrandomlyfromtheirrespectivepoolswitheachpageload,andtheURLsfortheadsweresuchthatadblockingsoftwarewouldnotlterthemout.The\noads"conditionsimplyhadwhitespaceinthemargin.Thefooterincludedtwobuttons:oneallowingthemtosubmitandrateanotheremail,andasecondallowingthemtostopcategorizingandcollecttheirpayment. 1http://www.cs.cmu.edu/~enron/Identifyinginformationsuchasemailaddresses,phonenum-bersandthename\Enron"wereremoved.ACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52 50D.G.Goldsteinetal. Fig.1.Estimatedimpressionsperconditionbasedonthenegativebinomialmodel.Errorbarsare1standarderror.3.2ResultsLetanimpressionbeoneparticipantviewingoneemail(anditsaccompanyingads,ifany),regardlessofwhethertheparticipantclassiestheemailorquitsbeforeclas-sifyingit.Theoveralldistributionofimpressionsperpersonisskewedwithameanof61,amedianof25andrstandthirdquartilesof6and57.Beingboundedby1frombelowande ectivelyunboundedfromabove(onlytwoparticipantsreachedtheupperlimit),theseimpressionsconstitutecountdata.Inparticular,theyareoverdispersedcountdatarelativetothePoisson(observedvariance/theoreticalPoissondatavarianceis228.7,p:0001)andthuswellsuitedtoanegativebi-nomialgeneralizedlinearmodel(GLM)[VenablesandRipley2002].Thee ectsoftheconditionsonrawimpressionsaremosteasilyseeninFigure1,whichalsomakesclearthatthedi erenceinimpressionsbetweenthe\goodads"and\noads"conditionsisnotsignicant.Relativetoabaselineof\badads",boththe\goodads"conditionandthenoadsconditionledtosubstantiallymoreimpressions(19%and25%moreimpressions,respectively).ThemodelexpressedinFigure1canbeusedtoestimatethecompensatingdi erentialofannoyingadsinthisexperiment.Sincethecurvesareslightlynon-linear,arangeofcompensatingdi erentialscouldbecalculatedacrossthepayrateandadconditions.Togetasimple,singleapproximationweusethemiddle,\goodads"conditiontoestimatethee ectofpayraises.Wetaketheaverageofthe.2to.4and.4to.6centdi erences,givinganestimatedincreaseof16.58impressionsresultingfroma.2centperimpressionpayraise.Whensummarizingthee ectofadquality,weusethenumberofimpressionsatthe.4centpayrate.Movingfrom\badads"tonoads,impressionsincreaseby12.68.Thepayraiserequiredtoachievea12.68impressionincreaseis.153centsperimpression(=:212:68=16:58)or$1.53CPM(costperthousandimpressions).Thatis,inthisexperiment,aparticipantinthe\badads"conditionwouldneedtobepaidanadditional$1.53perthousandACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52 TheCostofAnnoyingAds51impressionstogenerateasmanyimpressionsasapersonintheconditionwithoutads.Similarly,movingfromthe\badads"conditiontothe\goodads"conditionresultedinanadditional9.52impressionsperperson.Itwouldrequireapayraiseof.115centsperimpression(=:29:52=16:58)togenerate9.52additionalimpressions,meaningthatpeopleinthe\badads"conditionwouldneedtobepaidanadditional$1.15CPMtogenerateasmanyimpressionsasinthe\goodads"condition.4.CONCLUSIONThemainresultofthispaperisthatannoyingadsleadtositeabandonmentandthusfewerimpressionsthangoodadsornoads.Inwhatmightbeseenasgoodnewsforpublishers,goodadsandnoadsledtoroughlyequalnumbersofimpressions.Wecalculatedthecompensatingwagedi erentialinourexperimentofbadadstonoadstobe$1.53CPM,andbadadstogoodadstobe$1.15.Somecaremustbetakenininterpretingthesenumbers.Whilewepickedatask|classifyingemails|thatshouldbefamiliarandcommonformostinternetusers,thistaskmaynotberepresentativeofotherinternettaskslikereadingnewsstoriesorsearchingforproductstopurchase.Abandonmentratesmaydi erwithdi erenttasksandthee ectsofadvertisingmayvaryaswell.Whilevirtuallyeverywebservicefeaturescompetition,theswitchingcostsvaryfromverylowinconsumingnewstorelativelyhighinchangingemailservices.BecauseourusersonMechanicalTurkhaveanoutsideoptionofworkingonanalternativetask,weexpectourresultstobemostapplicabletosituationsinvolvinglowerswitchingcosts.Nevertheless,weexpectthatourndingthatannoyingadscosttheuseratleast$1CPMovermorepleasantadswillbeobtainedinsomeotherenvironments.Forthesereasons,wesuggestfurtherstudiesbedoneonMechanicalTurk,aseldexperiments,andinlaboratoriestomeasurethisdi erentialonsimilaranddi erenttasks.Ifstudiesacrossvariousdomainswithavarietyoftasksandoutsideoptionsarriveatsimilardi erentials,morecredencecanbeplacedonthesenumbers.Weviewthisworkasarststepinthisdirection.Iffutureworkarrivesatsimilarestimatesacrossavarietyofpublishers,suchestimatescouldserveasausefullowerboundforwhatapublishershouldchargetoruntheseads.Moreover,itwillbevaluabletousethecompensatingdi erentialsapproachtopricethevariousbadaspectsofads,includinganimationandpooraesthetics.Thisworkalsosuggestsavarietyofpolicyrecommendations.Mostdirectly,the$1CPMusercostofbadadshaspracticalconsequencesforpublishers,especiallyasbadadsoftencommandlowerCPMs.Itisareasonthatpublishersshouldinsistonasubstantialpremiumforannoyingadvertisements.Moreover,apublishercouldrandomizewhichusersseewhichadsandtrackbothtimespentonthepageandthefrequencywithwhichusersreturntothesite.Thistypeofexperimentationwouldcapturelongerterme ectsofannoyingadsthanthosestudiedhere.Also,publisherscouldgiveusersanoptiontocloseorreplaceanad.Areplacementeventwouldallowthepublishertoinferthatauserwouldpreferarandomadovertheadcurrentlyshown.Advertiserswithahighclosurerateshouldbechargedmoresincemoreannoyingadswouldbeclosedorreplacedfasterthanlessannoyingads.Adreplacementwouldhelptheuserbyremovingtheannoyingadandthepublisherbymakingitpossibletochargefortwoimpressions.ACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52 52D.G.Goldsteinetal.REFERENCESBurke,M.,Hornof,A.,Nilsen,E.,andGorman,N.2005.High-costbannerblindness:Adsincreaseperceivedworkload,hindervisualsearch,andareforgotten.ACMTransactionsonComputer-HumanInteraction12,4(December),423{445.Goldstein,D.G.,Suri,S.,McAfee,R.,Ekstrand-Abueg,M.,andDiaz,F.2014.Theeco-nomicandcognitivecostsofannoyingdisplayadvertisements.JournalofMarketingResearch.InPress.Riley,J.G.2001.Silversignals:Twenty-veyearsofscreeningandsignaling.JournalofEconomicLiteratureXXXIX,432{478.Toomim,M.,Kriplean,T.,Portner,C.,andLanday,J.A.2011.Utilityofhuman-computerinteractions:Towardascienceofpreferencemeasurement.InProceedingsofCHI2011:ACMConferenceonHumanFactorsinComputingSystems.Venables,W.N.andRipley,B.D.2002.ModernAppliedStatisticswithS.Springer,NewYork.Yoo,C.Y.andKim,K.2005.Processingofanimationinonlinebanneradvertising:Therolesofcognitiveandemotionalresponses.JournalofInteractiveMarketing19,4,18{34.ACMSIGecomExchanges,Vol.13,No.2,December2014,Pages47{52