(a)`Aroma'wasthedominantconventionby2003,butitwassupplantedby`S'(for`S - PDF document

(a)`Aroma'wasthedominantconventionby2003,butitwassupplantedby`S'(for`Smell')around2007. (b)Userswhojoinedin2003hungontothe`Aroma'conventionoftheiryouth. (c)Userswhojoinedin2005weremorere-ceptivetotheemerging`S'norm.Figure1:ExampleofcommunityanduserevolutioninBeerAdvocate:onenormforreferringtothesmellofabeergavewaytoanother,withdifferenteffectsondifferentusersdependingonwhentheyjoinedthecommunity.Thispatternissurprisinglyconsistent,withusersfollowingthesamelifecyclepatternregardlessofhowmuchtimeorefforttheyactuallyspentinthecommunity.Longtimecontributorsandshorttermusersfollowthesamepattern,withthelatterevolvingatanac-celeratedpace.Crucially,thismeansthatatthemomenttheydepartfromthecommunitymostusersarelinguisticallyconservative.Buildingontheseobservations,weshowthatauser'spatternsoflinguisticchangecanbeharnessedtomakepredictionsabouthermembershipinthecommunity(Section4).Inspiredbyourempir-icalanalysis,wedesignfeaturesthatcapturethespeedatwhichamemberconformstocommunitynormsalongwithotherlinguisticchangepatterns,andusetheminsimplemachinelearningmodels.Wendthat,usingsolelythelanguageoftherstfewinitialre-views,wecanpredictauser'stotallifetimeinthecommunity.Ourmodelsgivesignicantperformanceimprovementoverabaselinemodelthatisbasedontraditionalactivitybasedfeatures[9].Ourresultsthushavepracticalsignicancefordesignersandmaintain-ersofonlinecommunities,whocanusethemtodetectearlyinauser'scareerhowlongshewillstayactiveinthecommunity.Implicationsforsocialnetworksandsociolinguisticsresearch.Thisworkalsoyieldsnewtheoreticalinsightsintotheevolutionoflinguisticnormsandthecomplexinterplaybetweencommunity-levelandindividual-levellinguisticchange,addressingimportantopenquestionsinthesocialnetworkandsociolinguisticliterature.Thereisextensiveresearchcoveringtheevolutionofonlinecom-munities[21,28],theevolutionofcommunitymembers[2,10,16,29,32],andtheirparticipationpatterns[1,3,26,30,31,48](seeSection6foradiscussion).Butlittleisknownabouttheinterplaybetweenuser-levelevolutionandtheevolutionofthecommunityatlarge,anissuewhichisthecruxofthiswork.Thisinterplayisalsoacentralopenresearchquestioninlinguis-tics.Muchsociolinguisticresearchhasreliedontheadultlanguagestabilityassumption:underthecriticalassumptionthatindividuals'speechpatternsarelargelyxedbyearlyadulthood,olderspeak-ers'languagecanbeemployedasaproxyforthelinguisticstateofthecommunityatanearlierstage[11,12,22,23,37,42].How-ever,studieshavealsoshownthatthisassumptioncanfailtohold.Forinstance,individualsmightchangetheirlanguageastheyage,whilethecommunityasawholeremainsstable;orachangemightbesimultaneouslyadoptedbyallmembersofthecommunity,orjustbyoldermembers[25,39,40,46,47].Distinguishingamongscenarioslikethesehasprovedtobeafundamentalchallengeinso-ciolinguisticsresearch[43].Ourframeworkopensupnewavenuesforthestudyingoftheseissuesinhighlydynamiccommunities,andourresultsshowhowtheadultlanguagestabilityassumptionandothertheoreticalmodelsoflinguisticchangeapplytoonlinesettings(seeSection5foradetaileddiscussion).Linguisticchange:Anexample.Thebulkofthispaperisded-icatedtodevelopingaframeworkfortrackingtheaggregateef-fectsofnumerous,ever-evolvinglinguisticchanges.Notallofthesechangesareintuitivelyaccessible;likethephonologicalef-fectsprimarilystudiedbylinguistsworkinginofinecommunities,thechangesareoftendifculttoidentifyandcharacterizeindivid-ually.Thus,inwhatfollows,werelyonquantitativeevidenceandhigh-levelevaluationstoassessourframework.Nonetheless,manyofthelinguisticchangesatworkinourdataarehighlyintuitiveandeasytodiscern.Itisillustrativetoreviewoneofthem,beforewemovetostudyingthematamoreabstractlevel.Thethemesofthisexampleplayoutmanytimesoverinexperimentstocome.Becauseourcommunitiesarebuiltaroundbeer,thediscussionfrequentlyturnstoassessingvariousaspectsofthebeer-tastingex-perience,sothisisalocusoflinguisticchangeatthelexicallevel.Oneprominentexampleconcernssmell.OverthelifeoftheBeer-Advocatecommunity,thereweretwoprominentconventionsusedtointroducediscussionsofsmell:AromaandS(shortfor`Smell').Figure1(a)summarizesthebasictrendforthislinguisticvariable:theAromaconvention(blue,thinnerline)rosequicklyinpopularitybetween2001and2003,whenitreacheditspeak.Around2003,theSconvention(green)beganitsriseinpopularity.TheAromaconventionlostgroundquicklyandwassoonovertakenbyS.Figures1(b)and1(c)showthatthislinguisticchangeaffectedoldusersdifferentlythanitaffectednewones.Userswhojoinedthesitein2003,attheheightoftheAromaboom,wereveryun-likelytoswitchtoS(Figure1(b)).Indeed,theymakehardlyanyuseofS(green),andevenincreasetheiruseofAroma(blue),pos-siblyasareactiontotheencroachingnormandthesocialchangesitpotentiallysignals[47].Thepictureisverydifferentforuserswhojoinedin2005,whenSwastakingoff.Figure1(c)suggeststhatthesenewusersaredriversofthischange;theirSusagerisessharplywhiletheirAromausagestartsandremainslow.Weturnnowtodeningourframeworkfortrackingsuchchangessystematically,seekingtouseittounderstandthesocialdynamicsofacommunityandconnectthemwithindividualmem-bers'behaviors. TestsetperformanceTestsetclasssizesCommunitywDepartedrangeLivingrangeModelPrecisionRecallF1DepartedLiving BeerAdvocate2020–50200+Activity77.041.253.6327(46%)387(54%)BeerAdvocate2020–50200+Full69.646.956.0327(46%)387(54%)BeerAdvocate4040–100200+Activity74.627.339.8218(36%)378(64%)BeerAdvocate4040–100200+Full66.431.142.2218(36%)378(64%)RateBeer2020–50200+Activity73.719.330.5261(36%)465(64%)RateBeer2020–50200+Full64.832.342.9261(36%)465(64%)RateBeer4040–100200+Activity65.919.630.0179(27%)470(73%)RateBeer4040–100200+Full61.326.336.7179(27%)470(73%) Table3:Predictingwhetheranewuserisabouttoleavethecommunityorwillremainasanactiveuser.Thenumberofpostsweanalyzeisdenotedbyw.The`full'modelsusesallofourfeatures,whilethe`activity'modelsusesonlyactivity-basedfeatures.Theprecision,recall,andF1numbersgivenareforthetarget`departing'class.Forallsitesandw,thefullmodelsignicantlyimprovesovertheactivity-onlymodelaccordingtoapairedWilcoxonsignedranktestontheF1scores(p0:001). FeaturesF1F1w=20w=40 Activity30.530.0+Cross-entropy37.432.2+Jaccardself-similarity38.033.5+Adoptionoflexicalinnovations40.935.3+First-personsingularpronouns41.235.0+Numberofwords42.936.7 Table4:Performanceimprovementresultingfromincremen-tallyaddingourlinguisticchangefeaturestothe`activity'model(forRateBeer,our`testcommunity').Featureanalysis.Tobetterunderstandtheextendtowhichtheseimprovementsareexplainedbytheexploitationoflinguis-ticchangepatterns,weconductabrieffeatureanalysisofourfulllogisticmodels.Wendthatthelearnedcoefcientsareingoodcorrespondencewiththenewinsightsbroughtforwardinthiswork;forexample,the`departureclass'ischaracterizedbynegativeco-efcientsforLexicalInnovation1(i.e.,alowinitialrateofadop-tionoflexicalinnovation)aswellasbynegativecoefcientsforLexicalInnovationmax(i.e.,anearlyendofthelinguisticadoles-cencestage),thuscorrespondingtotheobservationssummarizedinFigure8.Similarobservationsalsoholdforthecross-entropyandself-similarityfeatures.Furthermore,wendthateachofthevefamiliesoflinguis-ticfeaturesbringimprovementsinperformancewhenaddedincre-mentallytotheactivitymodel(Table4).Atthesametime,noneofthefeaturescanbythemselvesexplaintheimprovementreportedinTable3.Thisindicatesthatourlinguisticchangefeaturescom-plementeachotherinpredictingauser'sdeparturefromthecom-munity. ityclass,whichisalsoourtarget`departing'class.Forcomplete-ness,wenotethatourmodelsalsocomparefavorablyintermsofaccuracywhencomparedtothemajority-classbaseline.Itshouldbehoweveremphasizedthattheaccuracyoneachieveswiththemajority-classbaselinewouldnottranslatewelltoarealworldcon-text,wheresuchmodelcouldonlyadviseacommunitymaintainerthatallornoneofthecommunity'smemberswereleaving.Incon-trast,ourmodelsprovideactionableintelligenceinthattheycouldhelpcommunitymaintainerstoidentifyspecicgroupsofat-riskmembersandtrytore-involvetheminthecommunity.5.IMPLICATIONSFORSOCIOLINGUISTICSThedevelopmentoftheframeworkdiscussedinSection3wasguidedbyalargebodyofsociolinguisticresearchconcerningthepatternsoflinguisticchangeinofinecommunities.However,wedonotsimplyusetheseresultspassively.Rather,weshowthatstudyingverylargeonlinecommunitiescanleadtonewlinguis-ticinsightsandweaddresschallengingmethodologicalissuescon-cerninghowtotrackandmeasurechange.Herewediscusshowourworkrelatestosociolinguisticresearchoflanguagechangeinofinecommunities.Inoneoftheearliestandmostinuentialstudiesoflinguis-ticchange[22],WilliamLabovproposedtheapparenttimecon-struct:underthecriticalassumptionthatindividuals'speechpat-ternsarelargelyxedbyearlyadulthood,olderspeakers'lan-guagecanbeemployedasaproxyforthelinguisticstateofthecommunityatanearlierstage,thusprovidingthetemporalfac-tornecessaryforstudyinglinguisticchange.Thisassumption,calledtheadultlanguagestabilityassumption,wassupportedbynumeroussubsequentstudiesinawidevarietyofsocialsettings[11,12,23,37,40,42,47],butitiswidelyacknowledgedthatitcouldfailtoholdincertaincases[25,39,46].Forinstance,indi-vidualsmightchangetheirlanguageastheyage,whilethecom-munityasawholeremainsstable.Alternatively,achangemightbesimultaneouslyadoptedbyallmembersofthecommunity,orjustbyoldermembers.Distinguishingamongscenarioslikethesehasprovedtobeafundamentalchallengeoverthelastvedecades[43].Apriori,itisnotatallobviouswhethertheadultlanguagesta-bilityassumptionsuitstheonlineworld,wherecommunity-timeiswarped,withdrasticlinguisticchangesarisinginveryshortpe-riodsoftime(forexample,theTwitterRTconventionachievedmainstreamstatusinabouttwomonths[20]).Similarly,conceptslikeadolescenceandadulthoodneedtoberedenedtoaccountforthevastlydifferentinteractionratescharacteristicofonlineset-ting:memberswhointeractwithinthecommunityonadailybasisarelikelytomaturefasterthanmemberswhosignononlyonceamonth.Moreover,ithasbeensuggestedthatadultlanguagestabil-ityhasbiologicalexplanations[27],andthereforeistiedtoactualbiologicalaging,whichislessrelevantinthecontextoffast-pacedonlinecommunities.Ourframeworkconrmsthat,inspiteofthesefundamentaldif-ferences,theadultlanguagestabilityassumptiondoesindeedhold [6]H.H.Clark,D.Wilkes-Gibbs.Referringasacollaborativeprocess.Cognition,1986.[7]C.Danescu-Niculescu-Mizil,M.Gamon,S.Dumais.Markmywords!Linguisticstyleaccommodationinsocialmedia.WWW,2011.[8]C.Danescu-Niculescu-Mizil,L.Lee,B.Pang,J.Kleinberg.Echoesofpower:Languageeffectsandpowerdifferencesinsocialinteraction.WWW,2012.[9]G.Dror,D.Pelleg,O.Rokhlenko,I.Szpektor.ChurnpredictioninnewusersofYahoo!answers.WorkshoponCommunityQuestionAnsweringontheWebatWWW,2012.[10]N.Ducheneaut,N.Yee,E.Nickell,R.Moore.Thelifeanddeathofonlinegamingcommunities:alookatguildsinworldofwarcraft.CHI,2007.[11]P.Eckert.Adolescentsocialstructureandthespreadoflinguisticchange.Languageinsociety,1988.[12]P.Eckert.AgeasaSociolinguisticVariable.Thehandbookofsociolinguistics.Wiley-Blackwell,1998.[13]D.C.Feldman.Thedevelopmentandenforcementofgroupnorms.TheAcademyofManagementReview,1984.[14]M.Garley,J.Hockenmaier.Beefmoves:Dissemination,Diversity,andDynamicsofEnglishBorrowingsinaGermanHipHopForum.ACL,2012.[15]H.Giles,J.Coupland,N.Coupland.Accommodationtheory:Communication,context,andconsequence.Contextsofaccommodation:Developmentsinappliedsociolinguistics.CambridgeUnivPr,1991.[16]S.Kairam,D.Wang,J.Leskovec.Thelifeanddeathofonlinegroups:Predictinggroupgrowthandlongevity.WSDM,2012.[17]S.M.Katz.Estimationofprobabilitiesfromsparsedataforthelanguagemodelcomponentofaspeechrecognizer.IEEETrans.onAcoustics,Speech,andSignalProcessing,1987.[18]P.Koehn.Statisticalsignicancetestsformachinetranslationevaluation.EMNLP,2004.[19]F.Kooti,W.A.Mason,K.P.Gummadi,M.Cha.PredictingEmergingSocialConventionsinOnlineSocialNetworks.CIKM,2012.[20]F.Kooti,H.Yang,M.Cha,K.P.Gummadi,W.A.Mason.TheEmergenceofConventionsinOnlineSocialNetworks.ICWSM,2012.[21]R.Kumar,J.Novak,A.Tomkins.Structureandevolutionofonlinesocialnetworks.KDD,2006.[22]W.Labov.TheSocialStraticationofEnglishinNewYorkCity.CenterforAppliedLinguistics,1966.[23]W.Labov.Sociolinguisticpatterns.Philadelphia:UniversityofPennsylvaniaPress,1972.[24]W.Labov.PrinciplesofLinguisticChange,SocialFactors.Wiley-Blackwell,2001.[25]W.Labov.PrinciplesofLinguisticChange,CognitiveandCulturalFactors.Wiley-Blackwell,2001.[26]C.Lampe,E.Johnston.Followthe(slash)dot:effectsoffeedbackonnewmembersinanonlinecommunity.SIGGROUP.ACM,2005.[27]E.H.Lenneberg.Biologicalfoundationsoflanguage.Wiley,1967.[28]J.Leskovec,J.Kleinberg,C.Faloutsos.Graphevolution:Densicationandshrinkingdiameters.ACMTransactionsonKnowledgeDiscoveryfromData(TKDD),2007.[29]J.Leskovec,L.Backstrom,R.Kumar,A.Tomkins.Microscopicevolutionofsocialnetworks.KDD,2008.[30]K.Ling,G.Beenen,P.Ludford,X.Wang,K.Chang,X.Li,D.Cosley,D.Frankowski,L.Terveen,A.M.Rashid,P.Resnick,R.Kraut.UsingSocialPsychologytoMotivateContributionstoOnlineCommunities.JournalofComputer-MediatedCommunication,2006.[31]P.Ludford,D.Cosley,D.Frankowski,L.Terveen.Thinkdifferent:increasingonlinecommunityparticipationusinguniquenessandgroupdissimilarity.CHI,2004.[32]J.McAuley,J.Leskovec.Fromamateurstoconnoisseurs:modelingtheevolutionofuserexpertisethroughonlinereviews.WWW,2013.[33]J.McAuley,J.Leskovec,D.Jurafsky.Learningattitudesandattributesfrommulti-aspectreviews.ICDM,2012.[34]D.Nguyen,C.P.Rosé.Languageuseasareectionofsocializationinonlinecommunities.WorkshoponLanguageinSocialMediaatACL,2011.[35]T.Postmes,R.Spears,M.Lea.Theformationofgroupnormsincomputer-mediatedcommunication.Humancommunicationresearch,2000.[36]J.Preece,B.Nonnecke,D.Andrews.Thetopvereasonsforlurking:improvingcommunityexperiencesforeveryone.ComputersinHumanBehavior,2004.[37]S.Romaine.Thelanguageofchildrenandadolescents:Theacquisitionofcommunicativecompetence.Blackwell,1984.[38]D.M.Romero,B.Meeder,J.Kleinberg.Differencesinthemechanicsofinformationdiffusionacrosstopics:Idioms,politicalhashtags,andcomplexcontagiononTwitter.WWW,2011.[39]G.Sankoff.Cross-sectionalandlongitudinalstudiesinsociolinguistics.Sociolinguistics:Aninternationalhandbookofthescienceoflanguageandsociety,2005.[40]G.Sankoff,H.Blondeau.Languagechangeacrossthelifespan:/r/inMontrealFrench.Language,2007.[41]J.C.Sherblom.Organizationinvolvementexpressedthroughpronounuseincomputermediatedcommunication.CommunicationResearchReports,2009.[42]S.Tagliamonte.Sowho?Likehow?Justwhat?DiscoursemarkersintheconversationsofYoungCanadians.JournalofPragmatics,2005.[43]S.Tagliamonte.Variationistsociolinguistics:change,observation,interpretation,volume39.Wiley-Blackwell,2012.[44]S.Tagliamonte,A.D'Arcy.Peaksbeyondphonology:Adolescence,incrementation,andlanguagechange.Language,2009.[45]O.Tsur,A.Rappoport.What'sinahashtag?Contentbasedpredictionofthespreadofideasinmicrobloggingcommunities.WSDM,2012.[46]S.E.Wagner.AgeGradinginSociolinguisticTheory.LanguageandLinguisticsCompass,2012.[47]S.E.Wagner,G.Sankoff.AgegradingintheMontrealFrenchinectedfuture.LanguageVariationandChange,2011.[48]S.M.Wilson,L.C.Peterson.Theanthropologyofonlinecommunities.AnnualReviewofAnthropology,2002.[49]J.Yang,X.Wei,M.Ackerman,L.Adamic.Activitylifespan:Ananalysisofusersurvivalpatternsinonlineknowledgesharingcommunities.ProceedingofICWSM,2010.[50]E.Zheleva,H.Sharara,L.Getoor.Co-evolutionofsocialandafliationnetworks.KDD,2009.