stanfordedu sudhof jurafsky cgpottsstanfordedu Abstract We propose a computational framework for identifying linguistic aspects of polite ness Our starting point is a new corpus of requests annotated for politeness which we use to evaluate aspects of ID: 11392
Download Pdf The PPT/PDF document "A computational approach to politeness w..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
litenesstheory,includingnotonlythenegativepolitenessmarkersmentionedabovebutalsoel-ementsofpositivepoliteness(gratitude,positiveandoptimisticsentiment,solidarity,andinclusive-ness).Theclassierachievesnearhuman-levelac-curacyacrossdomains,whichhighlightsthecon-sistentnatureofpolitenessstrategiesandpavesthewaytousingtheclassiertostudynewdata.Politenesstheorypredictsanegativecorrelationbetweenpolitenessandthepoweroftherequester,wherepowerisbroadlyconstruedtoincludeso-cialstatus,authority,andautonomy(BrownandLevinson,1987).Thegreaterthespeaker'spowerrelativetoheraddressee,thelesspoliteherre-questsareexpectedtobe:thereisnoneedforhertoincurtheexpenseofpayingrespect,andfailingtomakesuchpaymentscaninvoke,andhencere-inforce,herpower.WesupportthispredictionbyapplyingourpolitenessframeworktoWikipediaandStackExchange,bothofwhichprovidein-dependentmeasuresofsocialstatus.WeshowthatpoliteWikipediaeditorsaremorelikelytoachievehighstatusthroughelections;however,onceelected,theybecomelesspolite.Similarly,onStackExchange,wendthatusersatthetopofthereputationscalearelesspolitethanthoseatthebottom.Finally,webrieyaddressthequestionofhowpolitenessnormsvaryacrosscommunitiesandso-cialgroups.Ourndingsconrmestablishedre-sultsabouttherelationshipbetweenpolitenessandgender,andtheyidentifysubstantialvariationinpolitenessacrossdifferentprogramminglanguagesubcommunitiesonStackExchange.2PolitenessdataRequestsinvolveanimpositionontheaddressee,makingthemanaturaldomainforstudyingtheinter-connectionsbetweenlinguisticaspectsofpo-litenessandsocialvariables.RequestsinonlinecommunitiesWebaseouranalysisontwoonlinecommunitieswherere-questshaveanimportantrole:theWikipediacommunityofeditorsandtheStackExchangequestion-answercommunity.1OnWikipedia,tocoordinateonthecreationandmaintenanceofthecollaborativeencyclopedia,editorscanin-teractwitheachotheronusertalk-pages;2re- 1http://stackexchange.com/about2http://en.wikipedia.org/wiki/Wikipedia:User_pagesquestspostedonausertalk-page,althoughpub-lic,aregenerallydirectedtotheownerofthetalk-page.OnStackExchange,usersoftencommentonexistingpostsrequestingfurtherinformationorproposingedits;theserequestsaregenerallydi-rectedtotheauthorsoftheoriginalposts.Bothcommunitiesarenotonlyrichinuser-to-userrequests,buttheserequestsarealsopartofconsequentialconversations,notemptysocialbanter;theysolicitspecicinformationorcon-creteactions,andtheyexpectaresponse.PolitenessannotationComputationalstudiesofpoliteness,orindeedanyaspectoflinguisticprag-matics,demandrichlylabeleddata.Wethere-forelabelalargeportionofourrequestdata(over10,000utterances)usingAmazonMechan-icalTurk(AMT),creatingthelargestcorpuswithpolitenessannotations(seeTable1fordetails).3Wechoosetoannotaterequestscontainingex-actlytwosentences,wherethesecondsentenceistheactualrequest(andendswithaquestionmark).Thisprovidesenoughcontexttothean-notatorswhilealsocontrollingforlengtheffects.Eachannotatorwasinstructedtoreadabatchof13requestsandconsiderthemasoriginatingfromaco-workerbyemail.Foreachrequest,theanno-tatorhadtoindicatehowpolitesheperceivedtherequesttobebyusingasliderwithvaluesrang-ingfrom“veryimpolite”to“verypolite”.4Eachrequestwaslabeledbyvedifferentannotators.Wevettedannotatorsbyrestrictingtheirresi-dencetobeintheU.S.andbyconductingalin-guisticbackgroundquestionnaire.Wealsogavethemaparaphrasingtaskshowntobeeffectiveforverifyingandelicitinglinguisticattentiveness(Munroetal.,2010),andwemonitoredthean-notationjobandmanuallylteredoutannotatorswhosubmitteduniformorseeminglyrandoman-notations.Becausepolitenessishighlysubjectiveandan-notatorsmayhaveinconsistentscales,weap-pliedthestandardz-scorenormalizationtoeachworker'sscores.Finally,wedenethepolitenessscore(henceforthpoliteness)ofarequestastheaverageofthevescoresassignedbytheannota-tors.Thedistributionofresultingrequestscores(showninFigure1)hasanaverageof0andstan- 3Publiclyavailableathttp://www.mpi-sws.org/˜cristian/Politeness.html4Weusednon-categoricalratingsfornergranularityandtohelpaccountforannotators'differentperceptionscales. litenessstrategiesofBrownandLevinson(1987),whichareattemptstomitigatethesesocialthreats.ThesestrategiesareprominentinTable3,whichdescribesthecorepolitenessmarkersweanalyzedinourcorpusofWikipediarequests.WedonotincludetheStackExchangedatainthisanalysis,reservingitasa“testcommunity”forourpredic-tiontask(Section4).Requestsexhibitingpolitenessmarkersareau-tomaticallyextractedusingregularexpressionmatchingonthedependencyparseobtainedbytheStanfordDependencyParser(deMarneffeetal.,2006),togetherwithspecializedlexicons.Forex-ample,forthehedgesmarker(Table3,line19),wematchallrequestscontaininganominalsubjectdependencyedgepointingoutfromahedgeverbfromthehedgelistcreatedbyHyland(2005).Foreachpolitenessstrategy,Table3showstheaver-agepolitenessscoreoftherespectiverequests(asdescribedinSection2;positivenumbersindicatepoliterequests),andtheirtoppolitenessquartilemembership(i.e.,whatpercentagefallwithinthetopquartileofpolitenessscores).AsdiscussedattheendofSection2,thetoppolitenessquartilegivesamorerobustandmoreintuitivemeasureofpoliteness.Forreference,arandomsampleofre-questswillhavea0politenessscoreanda25%topquartilemembership;inbothcases,largernum-bersindicatehigherpoliteness.Gratitudeanddeference(lines1–2)arewaysforthespeakertoincurasocialcost,helpingtobalanceouttheburdentherequestplacesontheaddressee.AdoptingKaplan(1999)'smetaphor,thesearethecoinoftherealmwhenitcomestopayingtheaddresseerespect.Thus,theyareindi-catorsofpositivepoliteness.Termsfromthesentimentlexicon(Liuetal.,2005)arealsotoolsforpositivepoliteness,eitherbyemphasizingapositiverelationshipwiththead-dressee(line4),orbeingimpolitebyusingnega-tivesentimentthatdamagesthispositiverelation-ship(line5).Greetings(line3)areanotherwaytobuildapositiverelationshipwiththeaddressee.TheremainderofthecuesinTable3areneg-ativepolitenessstrategies,servingthepurposeofminimizing,atleastinappearance,theimpositionontheaddressee.Apologizing(line6)deectsthesocialthreatoftherequestbyattuningtotheimpo-sitionitself.Beingindirect(line9)isanotherwaytominimizesocialthreat.Thisstrategyallowsthespeakertoavoidwordsandphrasesconvention-allyassociatedwithrequests.First-personpluralformslikeweandour(line15)arealsowaysofbeingindirect,astheycreatethesensethattheburdenoftherequestissharedbetweenspeakerandaddressee(Wereallyshould...).Thoughin-directnessisnotinvariablyinterpretedaspolite-nessmarking(Blum-Kulka,2003),itisnonethe-lessareliablemarkerofit,asourscoresindicate.What'smore,directvariants(imperatives,state-mentsabouttheaddressee'sobligations)arelesspolite(lines10–11).Indirectstrategiesalsocombinewithhedges(line19)conveyingthattheaddresseeisunlikelytoaccepttheburden(Wouldyoubyanychance...?,Woulditbeatallpossible...?).Thesetooservetoprovidetheaddresseewithaface-savingwaytodenytherequest.Weevenseesubtleeffectsofmodalityatworkhere:theirrealis,counterfac-tualformswouldandcouldaremorepolitethantheirability(dispositional)orfuture-orientedvari-antscanandwill;comparelines12and13.Thisparallelsthecontrastbetweenfactualitymarkers(impolite;line20)andhedging(polite;line19).Manyofthesefeaturesarecorrelatedwitheachother,inkeepingwiththeinsightofBrownandLevinson(1987)thatpolitenessmarkersareof-tencombinedtocreateacumulativeeffectofin-creasedpoliteness.Ourcorporaalsohighlightin-teractionsthatareunexpected(oratleastunac-countedfor)onexistingtheoriesofpoliteness.Forexample,sentence-medialpleaseispolite(line7),presumablybecauseofitsfreedomtocombinewithothernegativepolitenessstrategies(Couldyouplease...).Incontrast,sentence-initialpleaseisimpolite(line8),becauseittypicallysignalsamoredirectstrategy(Pleasedothis),whichcanmakethepolitenessmarkeritselfseeminsincere.Weseesimilarinteractionsbetweenpronominalformsandsyntacticstructure:sentence-initialyouisimpolite(Youneedto...),whereassentence-medialyouisoftenpartoftheindirectstrategieswediscussedabove(Would/Couldyou...).4PredictingpolitenessWenowshowhowourlinguisticanalysiscanbeusedinamachinelearningmodelforautomati-callyclassifyingrequestsaccordingtopoliteness.Aclassiercanhelpverifythepredictivepower,robustness,anddomain-independentgeneralityofthelinguisticstrategiesofSection3.Also,bypro-vidingautomaticpolitenessjudgmentsforlarge classicationtask,especiallyconsideringthelargeamountoftrainingdataavailable.Thelinguisti-callyinformedclassier(Ling.)isanSVMusingthelinguisticfeatureslistedinTable3inadditiontotheunigramfeatures.Finally,toobtainaref-erencepointforthepredictiontaskwealsocollectthreenewpolitenessannotationsforeachofthere-questsinourdatasetusingthesamemethodologydescribedinSection2.Wethencalculatehumanperformanceonthetask(Human)asthepercent-ageofrequestsforwhichtheaveragescorefromtheadditionalannotationsmatchesthebinarypo-litenessclassoftheoriginalannotations(e.g.,apositivescorecorrespondstothepoliteclass).ClassicationresultsWeevaluatetheclassi-ersbothinanin-domainsetting,withastandardleave-one-outcrossvalidationprocedure,andinacross-domainsetting,wherewetrainononedo-mainandtestontheother(Table4).Forbothourdevelopmentandourtestdomains,andinboththein-domainandcross-domainsettings,thelinguis-ticallyinformedfeaturesgive3-4%absoluteim-provementoverthebagofwordsmodel.Whilethein-domainresultsarewithin3%ofhumanper-formance,thegreaterroomforimprovementinthecross-domainsettingmotivatesfurtherresearchonlinguisticcuesofpoliteness.Theexperimentsinthissectionconrmthatourtheory-inspiredfeaturesareindeedeffectiveinpractice,andgeneralizewelltonewdomains.Inthenextsectionweexploitthisinsighttoautomat-icallyannotateamuchlargersetofrequests(about400,000)withpolitenesslabels,enablingustore-latepolitenesstoseveralsocialvariablesandout-comes.Fornewrequests,weuseclassprobabil-ityestimatesobtainedbyttingalogisticregres-sionmodeltotheoutputoftheSVM(WittenandFrank,2005)aspredictedpolitenessscores(withvaluesbetween0and1;henceforthpoliteness,byabuseoflanguage).5RelationtosocialfactorsWenowapplyourframeworktostudyingtherela-tionshipbetweenpolitenessandsocialvariables,focussingonsocialpowerdynamics.Encour-agedbytheclose-to-humanperformanceofourin-domainclassiers,weusethemtoassignpo-litenesslabelstoourfulldatasetandthencomparetheselabelstoindependentmeasuresofpowerandstatusinourdata.Theresultscloselymatchthoseobtainedwithhuman-labeleddataalone,therebyIn-domainCross-domainTrainWikiSEWikiSETestWikiSESEWiki BOW79.84%74.47%64.23%72.17%Ling.83.79%78.19%67.53%75.43% Human86.72%80.89%80.89%86.72%Table4:AccuraciesofourtwoclassiersforWikipedia(Wiki)andStackExchange(SE),forin-domainandcross-domainsettings.Humanper-formanceisincludedasareferencepoint.Theran-dombaselineperformanceis50%.supportingtheuseofcomputationalmethodstopursuequestionsaboutsocialvariables.5.1RelationtosocialoutcomeEarlier,wecharacterizedpolitenessmarkingsascurrencyusedtopayrespect.Suchlanguageisthereforecostlyinasocialsense,and,relatedly,tendstoincurcostsintermsofcommunicativeef-ciency(VanRooy,2003).Arethesecostsworthpaying?WenowaddressthisquestionbystudyingpolitenessinthecontextoftheelectoralsystemoftheWikipediacommunityofeditors.AmongWikipediaeditors,statusisasalientso-cialvariable(Andersonetal.,2012).Administra-tors(admins)areeditorswhohavebeengrantedcertainrights,includingtheabilitytoblockothereditorsandtoprotectordeletearticles.7Ad-minshaveahigherstatusthancommoneditors(non-admins),andthisdistinctionseemstobewidelyacknowledgedbythecommunity(BurkeandKraut,2008b;Leskovecetal.,2010;Danescu-Niculescu-Miziletal.,2012).Aspiringeditorsbecomeadminsthroughpublicelections,8soweknowwhenthestatuschangefromnon-admintoadminsoccurredandcanstudyusers'languageuseinrelationtothattime.Toseewhetherpolitenesscorrelateswitheven-tualhighstatus,wecompare,inTable5,thepo-litenesslevelsofrequestsmadebyuserswhowilleventuallysucceedinbecomingadministrators(Eventualstatus:Admins)withrequestsmadebyuserswhoarenotadmins(Non-admins).9Weob-servethatadmins-to-bearesignicantlymorepo- 7http://en.wikipedia.org/wiki/Wikipedia:Administrators8http://en.wikipedia.org/wiki/Wikipedia:Requests_for_adminship9Weconsideronlyrequestsmadeuptoonemonthbeforetheelection,toavoidconfusionwithpre-electionbehavior. arerevealedthroughaspectsoflinguisticaccom-modation.Thepresentpapercomplementsthisworkbyrevealingtheroleofpolitenessinsocialoutcomesandpowerrelations.7ConclusionWeconstructandreleasealargecollectionofpoliteness-annotatedrequestsanduseittoevalu-atekeyaspectsofpolitenesstheory.Webuildapolitenessclassierthatachievesnear-humanper-formanceanduseittoexploretherelationbetweenpolitenessandsocialfactorssuchaspower,status,gender,andcommunitymembership.Wehopethepubliclyavailablecollectionofannotatedrequestsenablesfurtherstudyofpolitenessanditsrelationtosocialfactors,asthispaperhasonlybeguntoexplorethisarea.AcknowledgmentsWethankJeanWuforrunningtheAMTan-notationtask,andalltheparticipatingturkers.WethankDianaMinculescuandtheanonymousreviewersfortheirhelpfulcomments.ThisworkwassupportedinpartbyNSFIIS-1016909,CNS-1010921,IIS-1149837,IIS-1159679,AROMURI,DARPASMISC,OkawaFoundation,Do-como,Boeing,Allyes,Volkswagen,Intel,AlfredP.SloanFellowship,theMicrosoftFacultyFel-lowship,theGordonandDaileyPatteeFacultyFellowship,andtheCenterforAdvancedStudyintheBehavioralSciencesatStanford.ReferencesMuhammadAbdul-MageedandMonaDiab.2012.AWATIF:Amulti-genrecorpusforModernStan-dardArabicsubjectivityandsentimentanalysis.InProceedingsofLREC,pages3907–3914.AshtonAnderson,DanielHuttenlocher,JonKleinberg,andJureLeskovec.2012.Effectsofusersimilarityinsocialmedia.InProceedingsofWSDM,pages703–712.LynneM.AnderssonandChristineM.Pearson.1999.Titfortat?thespiralingeffectofincivilityintheworkplace.TheAcademyofManagementReview,24(3):452–471.ShoshanaBlum-KulkaandGabrieleKasper.1990.Specialissueonpoliteness.JournalofPragmatics,144(2).ShoshanaBlum-Kulka.2003.Indirectnessandpo-litenessinrequests:Sameordifferent?JournalofPragmatics,11(2):131–146.PhilipBramsen,MarthaEscobar-Molana,AmiPatel,andRafaelAlonso.Extractingsocialpowerrela-tionshipsfromnaturallanguage.InProceedingsofACL,pages773–782.SusanEBrennanandJustinaOOhaeri.1999.Whydoelectronicconversationsseemlesspolite?thecostsandbenetsofhedging.SIGSOFTSoftw.Eng.Notes,24(2):227–235.PenelopeBrownandStephenC.Levinson.1978.Universalsinlanguageuse:Politenessphenomena.InEstherN.Goody,editor,QuestionsandPolite-ness:StrategiesinSocialInteraction,pages56–311,Cambridge.CambridgeUniversityPress.PenelopeBrownandStephenCLevinson.1987.Po-liteness:someuniversalsinlanguageusage.Cam-bridgeUniversityPress.MoiraBurkeandRobertKraut.2008a.MindyourPsandQs:theimpactofpolitenessandrudenessinonlinecommunities.InProceedingsofCSCW,pages281–284.MoiraBurkeandRobertKraut.2008b.Takingupthemop:identifyingfuturewikipediaadministrators.InCHI'08extendedabstractsonHumanfactorsincomputingsystems,pages3441–3446.AndrewSangpilByon.2006.Theroleoflinguisticin-directnessandhonoricsinachievinglinguisticpo-litenessinKoreanrequests.JournalofPolitenessResearch,2(2):247–276.PaulChilton.1990.Politeness,politics,anddiplo-macy.DiscourseandSociety,1(2):201–224.HerbertH.ClarkandDaleH.Schunk.1980.Politeresponsestopoliterequests.Cognition,8(1):111–143.CristianDanescu-Niculescu-Mizil,LillianLee,BoPang,andJonKleinberg.2012.Echoesofpower:Languageeffectsandpowerdifferencesinsocialinteraction.InProceedingsofWWW,pages699–708.Marie-CatherinedeMarneffe,BillMacCartney,andChristopherD.Manning.2006.Generatingtypeddependencyparsesfromphrasestructureparses.InProceedingsofLREC,pages449–454.ChristopherP.Diehl,GalileoNamata,andLiseGetoor.2007.Relationshipidenticationforsocialnetworkdiscovery.InProceedingsoftheAAAIWorkshoponEnhancedMessaging,pages546–552.KirkWDuthler.2006.ThePolitenessofRequestsMadeViaEmailandVoicemail:SupportfortheHy-perpersonalModel.JournalofComputer-MediatedCommunication,11(2):500–521.ManaalFaruquiandSebastianPado.2012.Towardsamodelofformalandinformaladdressinenglish.InProceedingsofEACL,pages623–633.