TOTALWORDStotalnumberofwordsPASTTENSEusesofpasttenseauxiliarieswaswerehadMETADATEhorndatebellsurveyspeedformquestionnairerushedstudyresearchYOUyouyoudyoullyouryoureyoursyouvenotc ID: 298816
Download Pdf The PPT/PDF document "privateAmericanuniversityin2005andinspir..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
privateAmericanuniversityin2005andinspiredbypriorspeed-datingresearch(Madanetal.,2005;Pentland,2005).Thegraduatestudentparticipantsvolunteeredtobeinthestudyandwerepromisedemailsofpersonswithwhomtheyreportedmutualliking.Eachdatewasconductedinanopensettingwheretherewassubstantialbackgroundnoise.Allparticipantsworeaudiorecordersonashouldersash,thusresultingintwoaudiorecordingsoftheapprox-imately11004-minutedates.Inadditiontotheau-dio,wecollectedpre-testsurveys,eventscorecards,andpost-testsurveys.Thisisthelargestsampleweknowofwhereaudiodataanddetailedsurveyinfor-mationwerecombinedinanaturalexperiment.Therichsurveyinformationincludeddateper-ceptionsandfollow-upinterest,aswellasgen-eralattitudes,preferences,anddemographicinfor-mation.Participantswerealsoaskedabouttheconversationalstyleandintentionoftheinterlocu-tor.Eachspeakerwasaskedtoreporthowof-tentheirdate'sspeechreecteddifferentconversa-tionalstyles(awkward,friendly,irtatious,funny,assertive)onascaleof1-10(1=never,10=con-stantly):“Howoftendidtheotherpersonbehaveinthefollowingwaysonthis`date'?”.Wechosethreeofthesevetofocusoninthispaper.Weacquiredacousticinformationbytakingtheacousticwavelefromeachrecorderandmanuallysegmentingitintoasequenceofwaveles,eachcor-respondingtoone4-minutedate.Sincebothspeak-ersworemicrophones,mostdateshadtworecord-ings,onefromthemalerecorderandonefromthefemalerecorder.Becauseofmechanical,opera-tor,andexperimentererrors,somerecordingswerelost,andthussomedateshadonlyonerecording.Transcribersataprofessionaltranscriptionserviceusedthetworecordingstocreateatranscriptforeachdate,andtime-stampedthestartandendtimeofeachspeakerturn.Transcriberswereinstructedtomarkvariousdisuenciesaswellassomenon-verbalelementsoftheconversationsuchaslaughter.Becauseofnoise,participantswhoaccidentallyturnedofftheirmikes,andsomesegmentationandtranscriptionerrors,anumberofdateswerenotpos-sibletoanalyze.19dateswerelostcompletely,andforanadditional130welostoneofthetwoaudiotracksandhadtousetheremainingtracktoextractfeaturesforbothinterlocutors.Thecurrentstudyfo-cusesonthe991remainingcleandatesforwhichwehadusableaudio,transcripts,andsurveyinfor-mation.3TheExperimentsOurgoalistodetectthreeofthestylevariables,inparticularawkward,friendly,orirtatiousspeakers,viaamachinelearningclassier.Recallthateachspeakerinadate(eachconversationside)wasla-beledbyhisorherinterlocutorwitharatingfrom1-10forawkward,friendly,orirtatiousbehavior.Fortheexperiments,the1-10Likertscaleratingswererstmean-centeredwithineachrespondentsothattheaveragewas0.Thenthetoptenpercentoftherespondent-centeredmeanedLikertratingsweremarkedaspositiveforthetrait,andthebottomtenpercentweremarkedasnegativeforatrait.Thuseachrespondentlabelstheotherspeakeraseitherpositive,negative,orNAforeachofthethreetraits.Werunourbinaryclassicationexperimentstopredictthisoutputvariable.Foreachspeakersideofeach4-minuteconversa-tion,weextractedfeaturesfromthewavelesandthetranscript,asdescribedinthenextsection.Wethentrainedsixseparatebinaryclassiers(foreachgenderforthe3tasks),asdescribedinSection5.4FeatureExtractionInselectingfeatureswedrewonpreviousresearchontheuseofrelativelysimplesurfacefeaturesthatcuesocialmeaning,describedinthenextsections.Eachdatewasrepresentedbythetwo4-minutewaveles,onefromtherecorderwornbyeachspeaker,andasingletranscription.Becauseoftheveryhighlevelofnoise,thespeakerwearingtherecorderwasmuchcleareronhis/herownrecording,andsoweextractedtheacousticfeaturesforeachspeakerfromtheirownmicrophone(exceptforthe130datesforwhichweonlyhadoneaudiole).Alllexicalanddiscoursefeatureswereextractedfromthetranscripts.Allfeaturesdescribethespeakeroftheconversa-tionsidebeinglabeledforstyle.Thefeaturesforaconversationsidethusindicatewhetheraspeakerwhotalksalot,laughs,ismoredisuent,hashigherF0,etc.,ismoreorlesslikelytobeconsideredir-tatious,friendly,orawkwardbytheinterlocutor.We TOTALWORDStotalnumberofwordsPASTTENSEusesofpasttenseauxiliarieswas,were,hadMETADATEhorn,date,bell,survey,speed,form,questionnaire,rushed,study,researchYOUyou,you'd,you'll,your,you're,yours,you've(notcountingyouknow)WElets,let's,our,ours,ourselves,us,we,we'd,we'll,we're,we'veII'd,I'll,I'm,I've,me,mine,my,myself(notcountingImean)ASSENTyeah,okay,cool,yes,awesome,absolutely,agreeSWEARhell,sucks,damn,crap,shit,screw,heck,fuck*INSIGHTthink*/thought,feel*/felt,nd/found,understand*,gure*,idea*,imagine,wonderANGERhate/hated,hell,ridiculous*,stupid,kill*,screwed,blame,sucks,mad,bother,shitNEGEMOTIONbad,weird,hate,crazy,problem*,difcult,tough,awkward,boring,wrong,sad,worry,SEXUALlove*,passion*,loves,virgin,sex,screwINGESTfood,eat*,water,bar/bars,drink*,cook*,dinner,coffee,wine,beer,restaurant,lunch,dishTable2:Lexicalfeatures.Eachfeaturevalueisatotalcountofthewordsinthatclassforeachconversationside;asterisksindicatethatsufxedformswereincluded(e.g.,love,loves,loving).AllexcepttherstthreearefromLIWC(Pennebakeretal.,2007)(modiedslightly,forexamplebyremovingyouknowandImean).Thelastveclassesincludemorewordsinadditiontothoseshown.turesareexpensivetohand-labelandhardtoauto-maticallyextract.Wechoseasuggestivediscoursefeaturesthatwefeltmightstillbeautomaticallyex-tracted.FourparticulardialogactswerechosenasshowninTable3.Backchannels(orcontinuers)andap-preciations(acontinuerexpressingpositiveaffect)werecodedbyhand-builtregularexpressions.Theregularexpressionswerebasedonanalysisofthebackchannelsandappreciationsinthehand-labeledSwitchboardcorpusofdialogacts(Jurafskyetal.,1997).Questionswerecodedsimplybythepres-enceofquestionmarks.Finally,repairquestions(alsocalledNTRIs;nextturnrepairindicators)areturnsinwhichaspeakersignalslackofhearingorunderstanding(Schegloffetal.,1977).Todetectthese,weusedasimpleheuristic:thepresenceof`Excuseme'or`Wait',asinthefollowingexample:FEMALE:Okay.Areyouexcitedaboutthat?MALE:Excuseme?Acollaborativecompletionisaturnwhereaspeakercompletestheutterancebegunbythealter(Lerner,1991;Lerner,1996).Ourheuristicforiden-tifyingcollaborativecompletionswastoselectsen-tencesforwhichtherstwordofthespeakerwasextremelypredictablefromthelasttwowordsofthepreviousspeaker.Wetrainedawordtrigrammodel1 1interpolated,withGoodTuringsmoothing,trainedontheTreebank3Switchboardtranscriptsafterstrippingpunctuation.andusedittocomputetheprobabilitypoftherstwordofaspeaker'sturngiventhelasttwowordsoftheinterlocutor'sturn.Wearbitrarilychosethethreshold.01,labelingallturnsforwhichp�:01ascollaborativecompletionsandusedthetotalnumberofcollaborativecompletionsinaconversationsideasourvariable.Thissimpleheuristicwaserrorful,butdidtendtondcompletionsbeginningwithandoror(1below)andwh-questionsfollowedbyanNPorPPphrasethatisgrammaticallycoherentwiththeendofthequestion(2and3):(1)FEMALE:Thedrivingrange.(1)MALE:Andthetenniscourt,too.(2)MALE:Whatyeardidyougraduate?(2)FEMALE:Fromhighschool?(3)FEMALE:Whatdepartmentareyouin?(3)MALE:Thebusinessschool.Wealsomarkedaspectsofthepreferencestruc-tureoflanguage.Adispreferredactionisoneinwhichaspeakeravoidstheface-threattotheinter-locutorthatwouldbecausedby,e.g.,refusingarequestornotansweringaquestion,byusingspe-cicstrategiessuchastheuseofwell,hesitations,orrestarts(Schegloffetal.,1977;Pomerantz,1984).Finally,weincludedthenumberofinstancesoflaughterfortheside,aswellasthetotalnumberofturnsaspeakertook.4.4DisuencyFeaturesAsecondgroupofdiscoursefeaturesrelatingtore-pair,disuency,andspeakeroverlaparesummarized weightswhichmaximizethefollowingoptimiza-tionproblem:argmaxXilogp(yijxi;)� R()(2)R()isaregularizationtermusedtopenalizelargeweights.WechoseR(),theregularizationfunction,tobetheL1normof.Thatis,R()=jjjj1=Pni=1jij.Inourcase,giventhetrainingsetStrain,testsetStest,andvalidationsetSval,wetrainedtheweightsasfollows:argmax accuracy( ;Sval)(3)whereforagivensparsityparameter =argmaxXilogp(yijxi;)� R()(4)WechoseL1-regularizationbecausethenumberoftrainingexamplestolearnwellgrowslogarithmi-callywiththenumberofinputvariables(Ng,2004),andtoachieveasparseactivationofourfeaturestondonlythemostsalientexplanatoryvariables.Thischoiceofregularizationwasmadetoavoidtheproblemsthatoftenplaguesupervisedlearninginsituationswithlargenumberoffeaturesbutonlyasmallnumberofexamples.Thesearchspaceoverthesparsityparameter isboundedaroundanex-pectedsparsitytopreventovertting.Finally,toevaluateourmodelonthelearned and weusedthefeaturesXofthetestsetStesttocomputethepredictedoutputsYusingthelogisticregressionmodel.Accuracyissimplycomputedasthepercentofcorrectpredictions.Toavoidanydataorderingbias,wecalculateda foreachrandomizedrun.Theoutputoftherunswasavectorofweightsforeachfeature.Wekeptanyfeatureifthemedianofitsweightvectorwasnonzero.4AsampleboxplotforthehighestweightedegofeaturesforpredictingmaleirtcanbefoundinFigure1. 4Wealsoperformedat-testtondsalientfeaturevaluessignicantlydifferentthanzero;thenon-zeromedianmethodturnedouttobeamoreconservativemeasureinpractice(intu-itively,becauseL1normedregressionpushesweightsto0). Figure1:Anillustrativeboxplotforirtationinmenshowingthe10mostsignicantfeaturesandonenotsignicant(`I').Shownaremedianvalues(centralredline),rstquartile,thirdquartile,outliers(redX's)andinterquartilerange(lledbox).6ResultsResultsforthe6binaryclassiersarepresentedinTable5.AwkFlirtFriendly MFMFMF Speaker63%51%67%60%72%68%+other64%64%71%60%73%75%Table5:Accuracyofbinaryclassicationofeachcon-versationside,wherechanceis50%.Therstrowusesfeaturesonlyfromthesinglespeaker;thesecondaddsallthefeaturesfromtheinterlocutoraswell.Theseaccu-racyresultswereaggregatedfrom25randomizedrunsof5-foldcrossvalidation.Therstrowshowsresultsusingfeaturesex-tractedfromthespeakerbeinglabeled.Here,allconversationalstylesareeasiesttodetectinmen.Thesecondrowoftable5showstheaccuracywhenusingfeaturesfrombothspeakers.Notsur-prisingly,addinginformationabouttheinterlocutortendstoimproveclassication,andespeciallyforwomen,suggestingthatmalespeakinghasgreaterswayoverperceptionsofconversationalstyle.Wediscussbelowtheroleofthesefeatures.Werstconsideredthefeaturesthathelpedclas-sicationwhenconsideringonlytheego(i.e.,there-sultsintherstrowofTable5).Table6showsfea-tureweightsforthefeatures(featureswerenormedsoweightsarecomparable),andissummarizedinthefollowingparagraphs:Menwhoarelabeledasfriendlyuseyou,col- MALEFRIENDLYMALEFLIRT backchannel-0.737question0.376you0.631f0meansd0.288intensityminsd0.552you0.214f0sdsd-0.446rate0.190intensitymin-0.445intensitymin-0.163completion0.337backchannel-0.142time-0.270appreciation-0.136Insight-0.249repairquestion0.128f0min-0.226intensitymax-0.121intensitymax-0.221laugh0.107overlap0.213time-0.092laugh0.192overlap-0.090turndur-0.059f0min0.089Sexual0.059Sexual0.082appreciation-0.054Negemo0.075Anger-0.051metadate-0.041FEMALEFRIENDLYFEMALEFLIRT intensityminsd0.420f0max0.475intensitymaxsd-0.367rate0.346completion0.276intensityminsd0.269repairquestion0.255f0meansd0.21appreciation0.253Swear0.156f0max0.233question-0.153Swear-0.194Assent-0.127wordcount0.165f0min-0.111restart0.172intensitymax0.092uh0.241I0.073I0.111metadate-0.071past-0.060wordcount0.065laugh0.048laugh0.054Negemotion-0.021restart0.046intensitymin-0.02overlap-0.036Ingest-0.017f0sdsd-0.025Assent0.0087Ingest-0.024f0maxsd0.0089MALEAWK restart0.502completion-0.141f0sdsd0.371intensitymax-0.135appreciation-0.354f0meansd-0.091turns-0.292Ingest-0.079uh0.270Anger0.075you-0.210repairquestion-0.067overlap-0.190Insight-0.056past-0.175rate0.049intensityminsd-0.173Table6:Featureweights(medianweightsoftherandom-izedruns)forthenon-zeropredictorsforeachclassier.Sinceouraccuracyfordetectingawkwardnessinwomenbasedsolelyonegofeaturesissoclosetochance,wedidn'tanalyzetheawkwardnessfeaturesforwomenhere.laborativecompletions,laugh,overlap,butdon'tbackchanneloruseappreciations.Theirutterancesareshorter(insecondsandwords)andtheyarequi-eterandtheir(minimum)pitchislowerandsome-whatlessvariable.Womenlabeledasfriendlyhavemorecollab-orativecompletions,repairquestions,laughter,andappreciations.Theyusemorewordsoverall,anduseImoreoften.Theyaremoredisuent(bothrestartsanduh)butlesslikelytoswear.Prosodicallytheirf0ishigher,andthereseemstobesomepatterninvolv-ingquietspeech;morevariationinintensitymini-mumthanintensitymax.Menwhoarelabeledasirtingaskmoreques-tions,includingrepairquestions,anduseyou.Theydon'tusebackchannelsorappreciations,oroverlapasmuch.Theylaughmore,andusemoresexualandnegativeemotionalwords.Prosodicallytheyspeakfaster,withhigherandmorevariablepitch,butqui-eter(lowerintensitymax).Thestrongestfeaturesforwomenwhoarela-beledasirtingareprosodic;theyspeakfasterandlouderwithhigherandmorevariablepitch.Theyalsousemorewordsingeneral,swearmore,don'taskquestionsoruseAssent,usemoreI,laughmore,andaresomewhatmoredisuent(restarts).Menwhoarelabeledasawkwardaremoredis-uent,withincreasedrestartsandlledpauses(uhandum).Theyarealsonot`collaborative'conversa-tionalists;theydon'tuseappreciations,repairques-tions,collaborativecompletions,past-tense,oryou,takefewerturnsoverall,anddon'toverlap.Prosod-icallytheawkwardlabelsarehardtocharacterize;thereisbothanincreaseinpitchvariation(f0sdsd)andadecrease(f0meansd).Theydon'tseemtogetquiteasloud(intensitymax).Thepreviousanalysisshowedwhatfeaturesoftheegohelpinclassication.Wenextaskedaboutfea-turesofthealter,basedontheresultsusingbothegoandalterfeaturesinthesecondrowofTable5.Hereweareaskingaboutthelinguisticbehaviorsofaspeakerwhodescribestheinterlocutorasirting,friendly,orawkward.Whilewedon'tshowthesevaluesinatable,weofferhereanoverviewoftheirtendencies.Forexampleforwomenwholabeledtheirmalein-terlocutorsasfriendly,thewomengotmuchqui-eter,used`well'muchmore,laughed,askedmore ReferencesJ.Ang,R.Dhillon,A.Krupski,E.Shriberg,andA.Stol-cke.2002.Prosody-BasedAutomaticDetectionofAnnoyanceandFrustrationinHuman-ComputerDia-log.InINTERSPEECH-02.P.BoersmaandD.Weenink.2005.Praat:doingpho-neticsbycomputer(version4.3.14).[Computerpro-gram].RetrievedMay26,2005,fromhttp://www.praat.org/.S.Brave,C.Nass,andK.Hutchinson.2005.Comput-ersthatcare:Investigatingtheeffectsoforientationofemotionexhibitedbyanembodiedconversationalagent.InternationalJournalofHuman-ComputerStudies,62(2):161–178.S.E.BrennanandM.F.Schober.2001.Howlisten-erscompensatefordisuenciesinspontaneousspeech.JournalofMemoryandLanguage,44:274–296.S.E.BrennanandM.Williams.1995.Thefeelingofanother'sknowing:Prosodyandlledpausesascuestolistenersaboutthemetacognitivestatesofspeakers.JournalofMemoryandLanguage,34:383–398.J.Coates.1996.WomenTalk.Blackwell.M.A.Cohn,M.R.Mehl,andJ.W.Pennebaker.2004.Linguisticmarkersofpsychologicalchangesurround-ingSeptember11,2001.PsychologicalScience,15:687–693.C.Edelsky.1981.Who'sgottheoor?LanguageinSociety,10:383–421.F.Enos,E.Shriberg,M.Graciarena,J.Hirschberg,andA.Stolcke.2007.DetectingDeceptionUsingCriticalSegments.InINTERSPEECH-07.D.Jurafsky,E.Shriberg,andD.Biasca.1997.Switch-boardSWBD-DAMSLLabelingProjectCoder'sMan-ual,Draft13.TechnicalReport97-02,UniversityofColoradoInstituteofCognitiveScience.D.Jurafsky,E.Shriberg,B.Fox,andT.Curl.1998.Lex-ical,prosodic,andsyntacticcuesfordialogacts.InProceedings,COLING-ACLWorkshoponDiscourseRelationsandDiscourseMarkers,pages114–120.D.Jurafsky.2001.Pragmaticsandcomputationallin-guistics.InL.R.HornandG.Ward,editors,Hand-bookofPragmatics.Blackwell.C.M.LeeandS.S.Narayanan.2002.Combiningacous-ticandlanguageinformationforemotionrecognition.InICSLP-02,pages873–876,Denver,CO.G.H.Lerner.1991.Onthesyntaxofsentences-in-progress.LanguageinSociety,20(3):441–458.G.H.Lerner.1996.Onthe“semi-permeable”characterofgrammaticalunitsinconversation:Conditionalen-tryintotheturnspaceofanotherspeaker.InE.Ochs,E.A.Schegloff,andS.A.Thompson,editors,Interac-tionandGrammar,pages238–276.CambridgeUni-versityPress.J.Liscombe,J.Venditti,andJ.Hirschberg.2003.Clas-sifyingSubjectRatingsofEmotionalSpeechUsingAcousticFeatures.InINTERSPEECH-03.A.MadanandA.Pentland.2006.Vibefones:Sociallyawaremobilephones.InTenthIEEEInternationalSymposiumonWearableComputers.A.Madan,R.Caneel,andA.Pentland.2005.Voicesofattraction.PresentedatAugmentedCognition,HCI2005,LasVegas.F.MairesseandM.Walker.2008.Trainablegenerationofbig-vepersonalitystylesthroughdata-drivenpa-rameterestimation.InACL-08,Columbus.F.Mairesse,M.Walker,M.Mehl,andR.Moore.2007.Usinglinguisticcuesfortheautomaticrecognitionofpersonalityinconversationandtext.JournalofArti-cialIntelligenceResearch(JAIR),30:457–500.C.NassandS.Brave.2005.Wiredforspeech:Howvoiceactivatesandadvancesthehuman-computerre-lationship.MITPress,Cambridge,MA.M.L.Newman,J.W.Pennebaker,D.S.Berry,andJ.M.Richards.2003.Lyingwords:Predictingdeceptionfromlinguisticstyle.PersonalityandSocialPsychol-ogyBulletin,29:665–675.A.Y.Ng.2004.Featureselection,L1vs.L2regulariza-tion,androtationalinvariance.InICML2004.J.W.PennebakerandT.C.Lay.2002.Languageuseandpersonalityduringcrises:AnalysesofMayorRudolphGiuliani'spressconferences.JournalofResearchinPersonality,36:271–282.J.W.Pennebaker,R.Booth,andM.Francis.2007.Lin-guisticinquiryandwordcount:LIWC2007operator'smanual.Technicalreport,UniversityofTexas.A.Pentland.2005.Sociallyawarecomputationandcommunication.Computer,pages63–70.A.M.Pomerantz.1984.Agreeinganddisagreeingwithassessment:Somefeaturesofpreferred/dispreferredturnshapes.InJ.M.AtkinsonandJ.Heritage,edi-tors,StructureofSocialAction:StudiesinConversa-tionAnalysis.CambridgeUniversityPress.A.RosenbergandJ.Hirschberg.2005.Acous-tic/prosodicandlexicalcorrelatesofcharismaticspeech.InEUROSPEECH-05,pages513–516,Lis-bon,Portugal.S.S.Rude,E.M.Gortner,andJ.W.Pennebaker.2004.Languageuseofdepressedanddepression-vulnerablecollegestudents.CognitionandEmotion,18:1121–1133.H.Sacks,E.A.Schegloff,andG.Jefferson.1974.Asimplestsystematicsfortheorganizationofturn-takingforconversation.Language,50(4):696–735.E.A.Schegloff,G.Jefferson,andH.Sacks.1977.Thepreferenceforself-correctionintheorganizationofre-pairinconversation.Language,53:361–382.