Event Analytics on Social Media: Challenges and

slide1. Event Analytics on Social Media: Challenges and Solutions Yuheng Hu Committee Members
Dr. Subbarao Kambhampati, Chair
Dr. Eric Horvitz,
Dr. John Krumm
Dr. Huan Liu
Dr. Hari Sundaram<br>
slide2. Since the dawn of civilization, people congregated
in town squares to discuss events The emergence of social media has now created a sprawling virtual town square, whose scope is vast, and whose chatter can be captured! opening exciting possibilities for analyzing what people are actually saying..<br>
slide3. debate i-5 bridge collapse Superbowl Obama’s selfie<br>
slide4. Which part of the event did a tweet refer to? What’s the relation between event and tweets?<br>
slide5. What were the topics of the event and tweets? What were the sentiments of the event elicited on tweets?<br>
slide6. How to characterize the crowds’ tweeting behavior<br>
slide7. How to detect an event from social media responses ?<br>
slide8. How to predict the crowds' engagement in future events<br>
slide9. How to distill insights about event based on social media responses How to predict future development of event How to predict crowds’ engagement in future event How to find social media responses about the events How to model relations between event and its responses How to link social media responses to events How to infer topics and sentiments of social media responses How to characterize the crowds’ behavior in response to events How to address these challenges?<br>
slide10. Computational Journalism Political Campaign Potential applications<br>
slide11. “Fox News Unveils New State-Of-The-Art Newsroom” – the Verge, Oct 17, 2013 The event master<br>
slide12. The fact: vast amounts of
social media responses Tweets volume on Egypt & Morsi
12k ~ per hour We need automated solutions!<br>
slide13. Event Analytics on Social Media Most existing event analytics solutions are primitive. Simply combining other solutions ignores connections between events and responses Given the vast amounts of social media responses and complex nature of events, we need automated tools to conduct in-depth analysis<br>
slide14. In this proposal, we present Eventics Task 1: Event sensemaking Task 2: Event recognition Task 3: Event engagement prediction Trending events
with associated Twitter responses Predict user’s engagement in future events Event topics, segments, Event-tweet alignment,
Event sentiments<br>
slide15. ET-LDA
[AAAI’12, ICWSM’12, MMW’12] Specific Specific Specific General Specific General Specific General General<br>
slide16. ET-LDA
[AAAI’12, ICWSM’12, MMW’12] Frequency of specific tweets Event-tweets alignment Evolution of specific tweets<br>
slide17. SocSent
[IJCAI’13]<br>
slide18. Fire happened at 5 St and Pike, heard sirens, lots smoke DeMA
[CHI’13]<br>
slide19. Alice
[under review] Hey Mike: we found this event may be of interest to you based on our prediction on your potential engagement ! Our predictions were made based on your Twitter engagement history.

Regards,
Alice<br>
slide20. Summary of Contributions ET-LDA & SocSent for Event sensemaking
DeMA for Event recognition
Alice for Event engagement prediction Eventics, automated toolbox to conduct in-depth analysis of 3 core tasks in event analytics How people respond to events on Twitter
What factors affect crowd’s engagement in events Our toolbox enables a richer perspective about<br>
slide21. Event Characterization Event in Twitter
Real-world occurrence e with 1) associated time period Te, 2) a stream of corresponding Twitter messages Me about e, and published during time Te
Planned
e with 1) pre-known event context, e.g., topic or hashtags, 2) time, at which e is planned to occur
Trending events
e with one or more features (e.g., terms) of Me exhibiting bursty patterns during Te<br>
slide22. Event examples Planned events and trending events are not mutually exclusive event types – only represent an event along different dimensions.<br>
slide23. Event Sensemaking Republican Primary Debate, 09/07/2011 Tweets tagged with #ReaganDebate ? ? Motivation What’s the relation between an event and tweets?
Which part of the event did a tweet refer to?
What were the topics of the event and tweets?
How to characterize the crowds’ tweeting behavior?<br>
slide24. Event Sensemaking: the Problem Given an event’s transcript S, and its associated tweets T
– Characterize the event in terms of its topics and segments, and its influences (w.r.t the nature and magnitude) on the crowds’ Twitter responding behavior Requirements:
Extract topics in the event and tweets
Segment the event into topically coherent chunks
Establish the alignment between the event and tweets
Measure the influence of the event on its associated tweets<br>
slide25. Event Sensemaking: the Challenges Both topics and segments are latent Tweets are topically influenced by the content of the event. A tweet’s topics can be general (high-level and constant across the entire event), or specific (concrete and relate to specific segments of the event) An event is formed by discrete sequentially-ordered segments, each of which discusses a particular set of topics<br>
slide26. Event Sensemaking: Possible Approaches Applying existing event segmentation tools
e.g., time-windows For each <tweet, segment> pair, measuring similarities e.g., TF-IDF Counting related tweets for each segment<br>
slide27. Our Contribution: ET-LDA ET-LDA (joint Event and Tweets LDA) is a hierarchical fully Bayesian model, which jointly models an event and its Twitter responses via their inter-dependency, i.e., topical influences Yuheng Hu, Ajita John, Fei Wang, Subbarao Kambhampati. “ET-LDA: Joint Topic Modeling for Aligning Events and their Twitter Feedback.” In AAAI Conference on Artificial Intelligence (AAAI) 2012
Yuheng Hu, Ajita John, Doree Duncan Seligmann, Fei Wang. “What were the Tweets about? Topical Associations between Public Events and Twitter Feeds.” ICWSM’12
Yuheng Hu, Ajita John, Doree Duncan Seligmann. “Event Analytics via Social Media.” In Proc. ACM Multimedia 2011 Workshop on Social and Behavioral Networked Media Access (SBNMA) , 2011<br>
slide28. ET-LDA: Generative Process Foreach paragraph s in S
draw a segment choice indicate Cs
if Cs = 1 then draw a new topic mixture for s
else then topic of s is as same as the topic of previous paragraph s-1
Foreach tweet t in T
draw a topic changing indicate Ct
if Ct = 1 then draw a new topic for t
else then
draw a paragraph s
assign topic mixture of s to t<br>
slide29. ET-LDA: Graphical Model Event Tweets Determine tweet type
C(t)~Bernoulli(λ) Determine which segment a tweet (word) refers to
S(t) ~ Categorical(γ) Determine word’s topic in event
Zs~multinomial(θ) Tweets word’s topic
Zt~multinomial(ψ) or
Zt~multinomial(θ) General topics
Ψ(t)~Dirichlet(α)<br>
slide30. Inference in ET-LDA is HARD Unfortunately, model inference is intractable during coupling of hyperparameters. We need approximate inference algorithms. Here we use collapsed Gibbs sampling 30 Gibbs sampling approximates the posterior distribution by iteratively updating each latent variable given the
remaining variables We need to infer P(Zs, Zt, Cs , Cs, St | Ws, Wt ) How joint distribution looks like:<br>
slide31. Inference in ET-LDA: example<br>
slide32. Inference in ET-LDA: example<br>
slide33. Evaluation of ET-LDA Experimental Setup
Tweets for President Obama’s speech on the Middle East on May 19, 2011 (#MESpeech) and Republican Primary debate in the US on Sept 7, 2011 (#ReaganDebate)
Event transcripts from New York Times
Model settings: Gibbs sampling and pick #topics by maximizing log-likelihood
Baselines
LDA – Latent Dirichlet Allocation (LDA)
LCSeg – HMM-based event segmentation tool Tasks
Event segmentation
Topic extraction
Alignment<br>
slide34. Results: Event Segmentation Pk = probability that a randomly chosen pair of words from the event will be incorrectly separated by a hypothesized segment boundary<br>
slide35. Results: Topic Extraction Performance based on Likert scale<br>
slide36. Results: Alignment Goal: whether the specific tweets (i.e., tweets that are strongly influenced by the events) are correctly identified for each segment.
Procedure:
ET-LDA: sampled tweets when P(C(t)) > .5
LDA: run LDA on tweets corpus, and event transcripts; calculate distance between topic mixtures through JS-divergence Performance based on Likert scale<br>
slide37. Evolution of Specific Tweets most responses were either tangential or about the high-level themes rapid increase from 33% to 54% Controversial topic mentioned, the responses were pronounced Observation 1: crowds’ responses tended to be general and steady before the event; after the event, while during the event, they were more specific and episodic.<br>
slide38. Distribution of Segments Referred to by Specific Tweets People can also talk about things which are expected to be discussed later People can talk about things that have been discussed before or being discussed currently Observation 2: topical context of the tweets did not always correlate with the timeline of the event – an event segment can be referred to by specific tweets at any time irrespective of whether it has already occurred or is occurring currently or will occur later on ET-LDA alignment<br>
slide39. Examples of Specific/General tweets 39 Yes, we need to talk about jobs and teachers needing jobs! #Reagandebate Something the #GOP candidates won't mention about Reagan - Reagan grew the size of the federal government tremendously. #reagandebate Boring #GOPDebate #tcot #ReaganDebate Ron Paul. Gogogog :) . #reagandebate Specific General<br>
slide40. Summary of ET-LDA Motivated joint event-tweet modeling for event sensemaking
ET-LDA can concurrently segment an event and classify two types of tweets: general and specific
Demonstrated that ET-LDA significantly outperformed the traditional models
ET-LDA enables many insights which were never studied before<br>
slide41. ET-LDA is powerful, but there Remain Open Questions How does data incompleteness in event’s transcript affect the performance of ET-LDA in classifying the types of tweets
How does the volume of tweets affect the performance of ET-LDA in segmenting the event.

How well does ET-LDA predict future tweeting behavior given the topics covered in the event.
How does ET-LDA predict the future development of the event given the tweets seen so far. Proposed Work Robustness
of ET-LDA Predictive power
of ET-LDA<br>
slide42. Extension to ET-LDA Possible solutions:
Investigate the performance of different inference algorithms (e.g., the EM algorithm) in estimating ET-LDA’s parameters while data is incomplete
Investigate a training-testing scheme for the ET-LDA model
Outcome
Analyze what is currently happening rather than the “after-the-fact” analysis
Users can interact with the system and evaluate its effectiveness in predicting future development of the event as well as the tweeting behavior Proposed Work<br>
slide43. What other tasks can we do based on this alignment?<br>
slide44. Applications: Event analysis, Stock market, Advertisement<br>
slide45. Events Sensemaking via Aggregated Twitter Sentiment: the Problem Given an event’s transcript S and its associated tweets T
– Find the aggregated sentiments (positive or negative) about segment (s ∈ S) and topics of the event (k ∈ K) elicited on Twitter 45 45<br>
slide46. Events Sensemaking via Aggregated Twitter Sentiment: possible solution Main steps
Manually label tweets with their sentiment orientation as training data
Apply off-the-shelf sentiment classifiers, e.g., MinCut [Pang et al. 2002]
Relate aggregated Twitter sentiment to segments and topics of the event that occur within fixed time-windows around the tweets’ timestamps 46 Is this sufficient?
Unfortunately, NO..<br>
slide47. Events Sensemaking via Aggregated Twitter Sentiment: Challenges C1. Difficult to relate Twitter sentiment to segments and topics of the event
Fixed time-window approach is often not valid as presented in ET-LDA
C2. Manually annotating sentiments of a vast amount of tweets is error-prone
Present a bottleneck in learning high quality models
C3. Twitter sentiment is conveyed with highly domain-specific contextual cues
Can cause models to potentially lose performance and become stale 47 How to overcome these challenges?<br>
slide48. Our Contribution: SocSent Leverage prior knowledge to overcome the challenges
ET-LDA to align tweets to the event  C1
Sentiment lexicon  C3
Labels for small sets of tweets  C2
SocSent incorporates prior knowledge into a matrix factorization framework, that learns factors in latent dimensions – segments, topics and sentiments (positive or negative) – of the event, as elicited on Twitter Yuheng Hu, Fei Wang, Subbarao Kambhampati. “Listen to the Crowd: Automated Analysis of Events via Aggregated Twitter Sentiment.” In International Joint Conference on Artificial Intelligence (IJCAI) 2013<br>
slide49. SocSent: Framework terms tweet tweet segment segment topic topic sentiment sentiment term tweet segment sentiment term tweet sentiment Tweet-event alignment from ET-LDA Sentiment lexicon Labels for small tweets factorization Regulation
From prior Regulation
From prior Regulation
From prior We require that the factors respect the prior knowledge to the extent possible. T<br>
slide50. SocSent: Formal Formulation R0 regulates G, T and S together T X S represents segment-sentiment matrix G X T X S represents tweets-sentiment matrix<br>
slide51. Prior Knowledge in SocSent Obtain F0 sentiment lexicon from MPQA corpus. F0(i, 1) = 1 if word i is possible, and F0(i, 2) = 1 for negative sentiment Ask people to label the sentiment for a few tweets (e.g., less than 1000) for the purposes of capturing some domain-specific connotations sentiment term F0 R0 tweets sentiment tweet segment G0 Obtain G0 sentiment lexicon from ET-LDA inference. Each row represent nt tweets and its columns represent ns segments of the event. the content is the posterior probability of a tweet referring to the segments.<br>
slide52. SocSent: Model Inference The coupling between G, T, S, F makes it difficult to find optimal solutions for all factors simultaneously.

We adopt an alternating optimization scheme [Ding et al., 2006]<br>
slide53. Inference in SocSent Ψ is the Lagrangian multipliers which enforce non-negativity constraints on F, C represents terms irrelevant to F<br>
slide54. Evaluation of SocSent Classification performance of sentiment of event segment
Classification performance of sentiment of event Topics
Effectiveness of Prior Knowledge Tweets for President Obama’s speech on the Middle East (#MESpeech) & 2012 Presidential Debate in the US (#DenverDebate")
Event transcripts from New York Times
Ground truth:
Graduate students manually label the sentiment. Later applied ET-LDA to establish the alignment between the labeled tweets and the event segments
Label sentiment according to the majority aggregated Twitter sentiment that correlated to it Evaluation Plan for SocSent Experimental Setup<br>
slide55. Sentiment Classification ofEvent Segment SocSent utilizes the partially available knowledge on tweet-event alignment from ET-LDA to improve the quality of sentiment classification in both events. Baselines:
LexRatio: Counts the ratio of sentiment words from subjectivity lexicon in a tweet to determine its sentiment orientation [Wilson et al., 2009]
MinCuts: Utilizes contextual information via the minimum-cut framework to improve polarity-classification accuracy [Pang and Lee, 2004]
MFLK: Supervised matrix factorization method [Li et al. 2009] SocSent improves other approaches by 7.3% to 18.8%<br>
slide56. Sentiment Classification of the Event Topics SocSent improves the three baselines with a range of 6.5% to 17.3% for both datasets<br>
slide57. Effectiveness of Prior Knowledge Single type of prior knowledge is less effective than combining them. (combining all three leads to the most significant improvement)
Domain-specific knowledge (tweet labels) is more effective than domain-independent knowledge (lexicon)
Domain-specific knowledge is particularly helpful in its corresponding task. (F0 + G0 or R0 + G0 have better performance than F0 + R0 on sentiment segments), as G0 conveys prior knowledge on tweet-segment alignment F0 for sentiment Lexicon, R0 for tweets labels, G0 for prior tweet/event alignment knowledge from ET-LDA.<br>
slide58. Summary of SocSent Motivated low-rank representation of event-tweet sentiment analysis with prior knowledge acting as a regularizer
Developed SocSent Framework
Provided evaluations on two tweet datasets
Demonstrated that SocSent significantly outperformed the traditional models 58<br>
slide59. Are ET-LDA and SocSent Enough? ET-LDA and SocSent work on events with textual projections (e.g., transcripts). What if event transcripts are not available?

Challenges
How to establish the “alignment” between the event and its associated tweets, while the event transcribe is absent. Proposed Work<br>
slide60. Event Segmentation via Tweets Possible solutions
Automated split the entire volume of tweets posted around a given event into a sequentially-ordered segments, where each segment of tweets in fact represents a specific stage of the event (in which a set of topics can be discussed by the event attendees).
Build prior event ontology to characterize the event development
Build statistical approaches to model both the topical transitions and event stage transitions in the Twitter messages, with the help of the background knowledge from the event ontology.
Outcomes
What an event and its associated tweets are about
Which part of the event is referred by which tweets, can be a prior knowledge for SocSent
The corresponding tweeting behavior, only based the Twitter messages themselves. Proposed Work<br>
slide61. Fire happened at 5 St and Pike, heard sirens, lots smoke How to detect events from social media responses<br>
slide62. Event Recognition: the Problem Given a set of tweets
Find an event where it consists of a set of topically-related trending features extracted from tweets at a given time, where trending is a time interval over which the rate of change of momentum is positive Challenges
Be versatile
Locate the time periods when bursts happen
Differentiate whether the detected new event is trivial or not.<br>
slide63. Our Contribution: DeMA DeMA is an unsupervised feature-pivot online event detector, which recognizes trending events their associated Twitter responses from a stream of noisy Twitter message, with 3 steps:
Trending feature identification
Trending feature ranking
Trending feature grouping Yuheng Hu, Shelly Farnham, Andrés Monroy-Hernández. “Whoo.ly: Facilitating Information Seeking For Hyperlocal Communities Using Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI) 2013<br>
slide64. Detected Events<br>
slide65. Hey Mike: we found this event may be of interest to you based on our prediction on your potential engagement! Our predictions were made based on your Twitter engagement history.

Regards,
Alice How to predict crowds' engagement in future events<br>
slide66. Our Contribution: Alice How to find factors that affect these behavior?
Inspired by work in marketing
Five categories of engagement model
Why to use the engagement model?
Understand which factors affect people’s engagement
Based on the insights, build predictive model based on those important factors Alice is a statistical framework which can be used to understand people’s engagement with events that are trending in a local community, and to predict the engagement on unseen events Yuheng Hu, Shelly Farnham. “Understanding and Predicting People’s Community Engagement in Social Media” Under submission<br>
slide67. Predicting user’s engagement Binary classification: predicts whether or not a user will post tweets about a future event
Multi-class classification problem: predicts the volume of tweets a user will generate in the future event
SVM classification
Data: 1148 participants, over the course of 140 events
Results Accuracy of results on binary classification Accuracy of results on multiclass classification<br>
slide68. Event Recognition and Prediction for Geo-specific Events Motivation
Event happen locally
Mobile devices are ubiquitous
Foster civic engagement
Challenges
How to detect events at a specific geolocation
How to predict the growth and interestingness of the detected events
How to recommend events to potential users Proposed Work<br>
slide69. Proposed solutions
Extend DeMA by incorporating additional geo-specific features of density
Able to infer “at time T , there are some important things E going on at location X”
Develop tools that leverages content feature, social networks feature, as well as volume features, to predict the growth and popularity of these detected Twitter events.
Able to infer “at time point T , will the detected event E be trending and popular”
Extend Alice by incorporating additional geo-specific features
Outcomes
Build global event explorer on Twitter to track happening events in different regions around the world
Rank detected events according to their predicted growth and popular score
Alert potential users intelligently about the detected events Proposed Work Event Recognition and Prediction for Geo-specific Events<br>
slide70. ET-LDA is powerful, but there Remain Open Questions How does data incompleteness in event’s transcript affect the performance of ET-LDA in classifying the types of tweets
How does the volume of tweets affect the performance of ET-LDA in segmenting the event.

How well does ET-LDA predict future tweeting behavior given the topics covered in the event.
How does ET-LDA predict the future development of the event given the tweets seen so far. Proposed Work Robustness
of ET-LDA Predictive power
of ET-LDA<br>
slide71. Extension to ET-LDA Possible solutions:
Investigate the performance of different inference algorithms (e.g., the EM algorithm) in estimating ET-LDA’s parameters while data is incomplete
Investigate a training-testing scheme for the ET-LDA model
Outcome
Analyze what is currently happening rather than the “after-the-fact” analysis
Users can interact with the system and evaluate its effectiveness in predicting future development of the event as well as the tweeting behavior Proposed Work<br>
slide72. Are ET-LDA and SocSent Enough? ET-LDA and SocSent work on events with textual projections (e.g., transcripts). What if event transcripts are not available?

Challenges
How to establish the “alignment” between the event and its associated tweets, while the event transcribe is absent. Proposed Work<br>
slide73. Event Segmentation via Tweets Possible solutions
Automated split the entire volume of tweets posted around a given event into a sequentially-ordered segments, where each segment of tweets in fact represents a specific stage of the event (in which a set of topics can be discussed by the event attendees).
Build prior event ontology to characterize the event development
Build statistical approaches to model both the topical transitions and event stage transitions in the Twitter messages, with the help of the background knowledge from the event ontology.
Outcomes
What an event and its associated tweets are about
Which part of the event is referred by which tweets, can be a prior knowledge for SocSent
The corresponding tweeting behavior, only based the Twitter messages themselves. Proposed Work<br>
slide74. Summary of Contributions Event analytics on social media is an important problem
Technical contribution: proposed Eventics, a powerful toolbox<br>
slide75. Publications Yuheng Hu, Shelly Farnham. “Understanding and Predicting People’s Community Engagement in Social Media” Under submission
Yuheng Hu, Fei Wang, Subbarao Kambhampati. “Listen to the Crowd: Automated Analysis of Events via Aggregated Twitter Sentiment.” In International Joint Conference on Artificial Intelligence (IJCAI) 2013
Yuheng Hu, Kartik Talamadupula, Subbarao Kambhampati. “Dude, srsly?: The Surprisingly Formal Nature of Twitter's Language.” In In Proc. of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM'13)
Yuheng Hu, Shelly Farnham, Andrés Monroy-Hernández. “Whoo.ly: Facilitating Information Seeking For Hyperlocal Communities Using Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI) 2013 (Best Paper Honorable Mention )
Yuheng Hu, Ajita John, Fei Wang, Subbarao Kambhampati. “ET-LDA: Joint Topic Modeling for Aligning Events and their Twitter Feedback.” In AAAI Conference on Artificial Intelligence (AAAI) 2012
Yuheng Hu, Ajita John, Doree Duncan Seligmann, Fei Wang. “What were the Tweets about? Topical Associations between Public Events and Twitter Feeds.” In Proc. of the 6th International AAAI Conference on Weblogs and Social Media (ICWSM'12)
Yuheng Hu, Ajita John, Doree Duncan Seligmann. “Event Analytics via Social Media.” In Proc. ACM Multimedia 2011 Workshop on Social and Behavioral Networked Media Access (SBNMA) , 2011<br>
slide76. Timeline December 2013 Thesis proposal
January - February 2014 Extending ET-LDA by investigating different properties of ET-LDA,
including how it handles missing data, etc. Write up results for TKDD journal.
March - May 2014 Examine the solutions for segmentation the event using Twitter messages.
May - June 2014 Examine the solutions for classifying event types, and models to leverage the
results of the classification.
July - August 2014 Write up results for WWW 2015 conference.
August - October 2014 Thesis writing.
November 2014 Thesis defense.<br>
slide77. C1. Highly domain-specific C2. Unable to locate the time periods when bursts happen C3. Unable to differentiate whether the detected new event is trivial or not. Event Recognition: the Challenges State-of-the-art
First story detection [Petrovic et al.]
Clustering-based event detector [Becker et al.]
Wavelet-based detector [Weng et al]
Disasters event detection [Sakaki et al] How to overcome these challenges?<br>
slide78. Event Prediction: the Challenges State-of-the-art
Event prediction in news corpus [Radinsky et al]
Twitter Engagement in Occupy Wall Street [Chen at al]
Tie formation [Gilbert et al] C1. Highly domain-specific C2. Unable to model the engagement in local community C3. Lack of validation on large scale data How to overcome these challenges?<br>
slide79. C1. Highly domain-specific C2. Unable to locate the time periods when bursts happen C3. Unable to differentiate whether the detected new event is trivial or not. Event Recognition: the Challenges State-of-the-art
First story detection [Petrovic et al.]
Clustering-based event detector [Becker et al.]
Wavelet-based detector [Weng et al]
Disasters event detection [Sakaki et al] How to overcome these challenges?<br>
slide80. Understanding User’s Engagement Whether or not to get engaged in an event depending on:
Event is local
News friends already posted in
Event with more replies
More personal (1st and 2nd pro-nouns)
Smaller overlapping networks with other participants in the events<br>
slide81. Understanding User’s Engagement cont’d Level of participation of a user depending on:
Friends during event
How many news friends posted
How many friends post already
Hub friends posted
Topics are similar
More personal<br>
slide82. Identify Trending Feature Trending event identification
Find an event where it consists of a set of topically-related trending features, at a given time, where features are terms in each tweets
Definition of Trending
Trending – a time interval over which the rate of change of momentum is positive
Momentum = mass * velocity
Mass = current importance of an feature
Velocity = feature’s average frequency in tweets, during a time period
Inspired by work in finance
EMA (Exponential Moving Average)
MACD (Moving Average Convergence Divergence)
MACD histogram Unfortunately, measuring mass, velocity, momentum is HARD<br>
slide83. Challenges for Event Recognition Goals: detecting unplanned trending events and their associated Twitter responses, from a stream of noisy Twitter messages.
Connections to Event Sensemaking task: A pre-step
State-of-the-art
First story detection [Petrovic et al.]
Clustering-based event detector [Becker et al.]
Wavelet-based detector [Weng et al]
Disasters event detection [Sakaki et al] C1. Highly domain-specific C2. Unable to locate the time periods when bursts happen C3. Unable to differentiate whether the detected new event is trivial or not.<br>
slide84. Our Contribution: DeMA Document-pivot VS. feature-pivot methods
Document-pivot: detect events by clustering docs based on distance between documents
Feature-pivot: detect events by learning and combining features of words
Pros and Cons DeMA is an unsupervised feature-pivot online event detector, which recognizes trending events their associated Twitter responses from a stream of noisy Twitter message, with 3 steps:
Trending feature identification
Trending feature ranking
Trending feature grouping Yuheng Hu, Shelly Farnham, Andrés Monroy-Hernández. “Whoo.ly: Facilitating Information Seeking For Hyperlocal Communities Using Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI) 2013<br>
slide85. Identify Trending Feature Trending event identification
Find an event where it consists of a set of topically-related trending features, at a given time, where features are terms in each tweets
Definition of Trending
Trending – a time interval over which the rate of change of momentum is positive
Momentum = mass * velocity
Mass = current importance of an feature
Velocity = feature’s average frequency in tweets, during a time period
Inspired by work in finance
EMA (Exponential Moving Average)
MACD (Moving Average Convergence Divergence)
MACD histogram Unfortunately, measuring mass, velocity, momentum is HARD<br>
slide86. Identify Trending Feature: An Example Given a feature F and its time series S = S(F) = {f1, f2, … fm} #debate<br>
slide87. Identify Trending Feature: An Example Given a feature F and its time series S = S(F) = {f1, f2, … fm} #debate n1 EMA n2 EMA<br>
slide88. Identify Trending Feature: An Example Given a feature F and its time series S = S(F) = {f1, f2, … fm} #debate n1 EMA n2 EMA<br>
slide89. Identify Trending Feature cont’d MACD Signal line MACD histogram MACD – Difference between the n1- and n2- hour EMA for S(F) MACD histogram (difference between F’s MACD and its signal line) #debate Up-trends start Down-trends start<br>
slide90. Rank Trending Feature EMA, MACD, MACD-histogram can be computed in linear time
But features may be trending up repeatedly
“morning” can be trending from 8am – 11am everyday
Rank trending features by their novelty
R(h, d, w, F) = MACD histogram results during hour h, day d, and week w
Mean(h, d, F) = average trend score
SD(h, d, F) = standard deviation<br>
slide91. Group Trending Feature Multiple events can be trending within the same time period
We need topically-related features to form an event
Need a way to separate all trending features and group them into event clusters
Shared k-nearest neighborhood clustering with distance defined by JS-divergence with LDA C<br>
slide92. Evaluation of DeMA Setup
Randomly sampled 2600 tweets posted during August 2012
Tweets from the users who identified their location as Seattle
Tasks
Accuracy in identifying events
Accuracy in identifying importance of events
Baselines
Fastest
Random
Ground truth<br>
slide93. Accuracy in Event Recognition Detected events<br>
slide94. Accuracy in Identification of Importance of Events How accuracy is DeMA in identifying the importance of the events?
Importance = novelty score Statistical analysis shows the results of DeMA is significantly correlated with the participant’s rating of the importance of an event
(r=0.31, p<0.001) Human labels DeMA results<br>
slide95. Conclusion Developed DeMA, an effective event detector for recognition unplanned trending events from a noisy Twitter stream in real-time
Evaluated DeMA and demonstrated that it outperformed other approaches<br>
slide96. Outline Event Characterization
Challenges and Contributions for Event Sensemaking
Challenges and Contributions for Event Recognition
Challenges and Contributions for Event Prediction
Summary<br>
slide97. Challenges for Event Prediction Goals: forecasting the crowds’ behavior in response to future local events, based on their present responses of the past events.
State-of-the-art
Event prediction in news corpus [Radinsky et al]
Twitter Engagement in Occupy Wall Street [Chen at al]
Tie formation [Gilbert et al] C1. Highly domain-specific C2. Unable to model the engagement in local community C3. Lack of validation on large scale data<br>
slide98. Our Contribution: Alice What is a person’s local event engagement on Twitter looks like?
posting, retweeting, replying to Twitter messages about a local event on Twitter.
How to find factors that affect these behavior?
Inspired by work in marketing
Five categories of engagement model
Why to use the engagement model?
Understand which factors affect people’s engagement
Based on the insights, build predictive model based on those important factors Alice is a statistical framework which can be used to understand people’s engagement with events that are trending in a local community, and to predict the engagement on unseen events Yuheng Hu, Shelly Farnham. “Understanding and Predicting People’s Community Engagement in Social Media” Under submission<br>
slide99. Alice: Predictive Variables Involvement – Level of participation with the event)
Tweets volume, Max. tweets per hour, Avg. tweets per hour
Interaction – interaction with other people who are also involved in the event through the tweets content
Hashtag ratio, RT ratio, Links ratio, Reply ratio, Mentions ratio.
Interest – topics of a user’s tweets content
Topical similarity to past 5 engaged events
Intimacy – affect and personal nature of the person’s tweets content
Percentage of tweets classified as Joviality, Fatigue, Hostility, Sadness, Serenity, Fear, Guit, 1st , 2nd, 3rd personal pronouns,
Influence – person’s likelihood to be influenced by other people in her social networks during the event
No. of followers, friends, Attention, Common neighbors, Friends about news, Friends about neighborhood hubs, Friends posted prior to user’s engagement<br>
slide100. Alice: Dependent Variables The user’s event engagement on Twitter is operationalized as posting tweets about the events: the posts could easily from retweets, reply, or a new posts.
Dependent variable
A binary dependent measure: whether or not users post in a particular event
continuous dependent measure: how often they post
Statistical Methods
Linear combination of predictive variables Dependent variable Predictive variables<br>
slide101. Experimental Results Setup
228 participants, over the course of 28 events
Participants at least post 4 tweets
Understanding user’s engagement
Via regressions to understand which factors are important to people’s event engagement
Predicting user’s engagement
Use learned insights to build predictive framework, and use it to make predictions of user’s engagement on large scale data<br>
slide102. Understanding User’s Engagement Whether or not to get engaged in an event depending on:
Event is local
News friends already posted in
Event with more replies
More personal (1st and 2nd pro-nouns)
Smaller overlapping networks with other participants in the events<br>
slide103. Understanding User’s Engagement cont’d Level of participation of a user depending on:
Friends during event
How many news friends posted
How many friends post already
Hub friends posted
Topics are similar
More personal<br>
slide104. Predicting user’s engagement Binary classification: predicts whether or not a user will post tweets about a future event
Multi-class classification problem: predicts the volume of tweets a user will generate in the future event
SVM classification
Data: 1148 participants, over the course of 140 events
Results Accuracy of results on binary classification Accuracy of results on multiclass classification<br>
slide105. Conclusion Developed Alice, a predictive framework for predicting users’ engagement in forthcoming unseen events based on the engagement in present events
Insights about why and how people engaged in local community events were revealed<br>