Bing Liu University Of Illinois at Chicago liubcsuicedu KDD2012 Summer School August 10 2012 Beijing China Bing Liu KDD2012 Summer School Aug 10 2012 Beijing China ID: 582490
Download Presentation The PPT/PDF document "Modeling Opinions and Beyond in Social ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Modeling Opinions and Beyond in Social Media
Bing LiuUniversity Of Illinois at Chicagoliub@cs.uic.edu
KDD-2012 Summer School, August 10, 2012,
Beijing
, China Slide2
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
2Introduction
Why are opinions so important?
Opinions are key influencers of our behaviors.
Our beliefs and perceptions of reality are conditioned on how others see the world.
Whenever we need to make a decision we often seek out others’ opinions.
True for both individuals and organizations
It is simply the “human nature”
We
want
to express our opinions
We
also
want to
hear
others’ opinionsSlide3
Topics of this lecture
Sentiment analysis and opinion miningIt has been studied extensively in the past 10 years. A large number of applications have been deployed. We will
define/model this
task and introduce some core
research and challenges.
Going beyond:
comments, discussions/debates
Beyond expressing our opinions in isolation,
we also like to comment, argue, discuss and debate. They involve user interactions.These are opinions too but of a slightly different typeWe will try to model some of these interactive forums
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
3Slide4
Roadmap
Sentiment Analysis and Opinion MiningProblem of Sentiment AnalysisDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
4Slide5
Sentiment analysis and opinion mining
Sentiment analysis or opinion miningcomputational study of opinions, sentiments, appraisal, and emotions expressed in text.
Reviews, blogs, discussions,
microblogs
, social networks
Its inception and rapid growth coincide with those of the social media on the Web
For the first time in human history, a huge volume of opinionated data is recorded in digital forms.
A
core technology for social media analysis Because a key function of social media is for people to express views & opinionsBing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
5Slide6
A fascinating and challenging problem!
Intellectually challenging & many applications.A popular research topic in NLP, text and Web mining (Edited book: Shanahan,
Qu
, & Wiebe, 2006; Book Chapters: Liu, 2007 & 2011;
Surveys
: Pang & Lee 2008; Liu, 2012)
It has spread from computer science to management science and social sciences
(Hu,
Pavlou & Zhang, 2006; Archak, Ghose & Ipeirotis, 2007; Liu et al 2007; Park, Lee & Han, 2007; Dellarocas et al., 2007; Chen & Xie 2007).> 350 companies working on it in
USA.Almost no research before early 2000.
Either from NLP or Linguistics (no data?)
Potentially a major technology from NLP.
But it is very hard!
People grossly underestimated the difficulty earlier.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
6Slide7
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
7Slide8
Abstraction (1): what is an opinion?
Find a structure from the unstructured text. Id: Abc123 on 5-1-2008
“
I bought an
iPhone
a few days ago. It is such a nice
phone.
The touch screen is really cool. The voice quality is clear too. It is much better than my old Blackberry. However, my mother was mad with me as I did not tell her before I bought the phone.
She also thought the phone was too expensive, …” One can look at this review/blog from
Document level
, i.e., is this review + or -?
Sentence level
, i.e., is each sentence + or -?
Entity and feature/aspect level
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
8Slide9
Entity and feature/aspect level
Id: Abc123 on 5-1-2008 “I bought an
iPhone
a few days ago. It is such a nice
phone.
The
touch screen
is really cool. The voice quality is clear too. It is much better than my old Blackberry. However, my mother was mad with me as I did not tell her before I bought the
phone. She also thought the phone was too expensive, …”
What do we see?
Opinion targets:
entities and their features/aspects
Sentiments:
positive and negative
Opinion holders:
persons who hold opinions
Time:
when opinions are given
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
9Slide10
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
10Two main types of opinions
(Jindal and Liu 2006; Liu, 2010)
Regular opinions
: Sentiment/opinion expressions on some target entities
Direct opinions
:
“The
touch screen is really cool.”Indirect opinions: “After taking the drug, my pain has gone.” Comparative opinions: Comparisons of more than one entity.
E.g., “iPhone is better than Blackberry.”We focus on regular opinions in this talk, and just call them opinions. Slide11
Basic Definition of an Opinion
Definition: An opinion is a quadruple, (target,
sentiment
,
holder
,
time
)
This definition is concise, but is not easy to use in many applications.The target description can be quite complex.E.g., “I bought a Canon G12 camera last week. The picture quality is amazing.”Target = picture quality? (not quite)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
11Slide12
A More Practical Definition(Hu and Liu 2004; Liu, in NLP handbook, 2010)
An opinion is a quintuple
(
e
j
,
a
jk
, soijkl, hi, tl),ej is a target entity.
ajk is a feature/aspect of the entity e
j
.
so
ijkl
is the sentiment value of the opinion of the opinion holder
h
i
on aspect
a
jk
of entity
e
j
at time
t
l
.
so
ijkl
is +
ve
, -
ve
, or
neu
, or a
more granular rating.
h
i
is an opinion holder.
t
l
is the time when the opinion was expressed.
Still a simplified definition (see Liu, 2012 book)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
12Slide13
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
13Structure the unstructured
Objective
: Given an opinion document,
Discover all quintuples (
e
j
,
ak, soijkl,
hi,
t
l
)
,
Or, solve some simpler forms of the problem
E.g., sentiment classification at the document or sentence level.
With
the quintuples
,
Unstructured Text
Structured Data
Traditional data and visualization tools can be used to slice, dice and visualize the results.
Enable qualitative and quantitative analysis
.
Th
e definition/model is widely used in industrySlide14
Abstraction (2): Opinion Summary
With a lot of opinions, a summary is necessary.A multi-document summary taskDifferent from traditional summary of facts1 fact = any number of the same factOpinion summary has a quantitative
side
1 opinion
any number of the same opinion
The
quintuple representation
provides a basis for opinion summarization.Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 14Slide15
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
15
(Aspect)Feature-based opinion summary
(Hu & Liu, 2004)
“
“
I bought an
iPhone a few days ago. It is such a nice phone. The touch screen is really cool. The voice quality is clear too. It is much better than my old
Blackberry,. However,
my mother
was mad with me as I did not tell her before I bought the
phone.
She also thought the phone was too
expensive, …”
….
Feature Based Summary of iPhone
:
Feature1
:
Touch screen
Positive
:
212
The
touch screen
was really cool
.
The
touch screen
was so easy to use and can do amazing things.
…
Negative
: 6
The
screen
is easily scratched.
I have a lot of difficulty in removing finger marks from the
touch screen
.
…
Feature2
:
voice quality
…
Note: We omit opinion holdersSlide16
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
16Opinion observer - visualization
(Liu et al. 05)
Summary of reviews of
Cell Phone
1
Voice
Screen
Size
Weight
Battery
+
_
Comparison of reviews of
Cell Phone 1
Cell Phone 2
_
+Slide17
Feature/aspect-based opinion summary
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
17Slide18
Google Product Search
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
18Slide19
Not just ONE problem
(ej, ajk
,
so
ijkl
,
h
i
, tl),ej - a target entity: Named Entity Extraction (more)ajk - a feature/aspect of ej:
Information Extraction (more)so
ijkl
is sentiment:
Sentiment Identification
h
i
is an opinion holder:
Information
/
Data Extraction
t
l
is the time:
Information/
Data Extraction
Coreference
resolution
Synonym match (voice = sound quality)
…
A multifaceted and integrated problem!
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
19Slide20
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
20Slide21
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
21Document sentiment classification
Classify a whole opinion document
(e.g., a review) based on the overall sentiment of the opinion holder
(Pang et al 2002;
Turney
2002, …)
Classes: Positive, negative (possibly neutral)Neutral or no opinion is hard. Most papers ignore it. An example review: “I bought an iPhone a few days ago. It is such a nice phone, although a little large. The touch screen is cool. The voice quality is clear too. I simply love it!”
Classification: positive or negative?Classification methods: SVM, Naïve Bayes,
etcSlide22
Assumption and goal
Assumption: The doc is written by a single person and express opinion/sentiment on a single entity.
Goal
: discover
(
_
,
_
, so, _, _), where e, a, h, and t are ignoredReviews usually satisfy the assumption. Almost all papers use reviews
Positive: 4 or 5 stars, negative: 1 or 2 starsForum postings and blogs do not
They can mention and compare multiple entities
Many such postings express no sentiments
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
22Slide23
Features for supervised learning
The problem has been studied by numerous researchers Probably the most extensive studied problemIncluding domain adaption and cross-lingual, etc.
Key:
feature engineering. A large set of features have been tried by researchers. E.g.,
Terms frequency and different IR weighting schemes
Part of speech (POS) tags
Opinion words and phrases
Negations
Syntactic dependency, etcBing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
23Slide24
Domain adaptation (transfer learning)
Sentiment classification is sensitive to the domain of the training data. A classifier trained using reviews from one domain often performs poorly in another domain.
words and even language constructs used in different domains for expressing opinions can be quite different.
same word in one domain may mean positive but
negative
in another, e.g., “
this vacuum cleaner really
sucks
.” Existing research has used labeled data from one domain and unlabeled data from the target domain and general opinion words for learning (Aue and Gamon 2005; Blitzer et al 2007; Yang et al 2006; Pan et al 2010; Wu, Tan and Cheng 2009; Bollegala, Weir and Carroll 2011; He, Lin and Alani
2011).Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
24Slide25
Cross-lingual sentiment classification
Useful in the following scenarios: E.g., there are many English sentiment corpora, but for other languages (e.g. Chinese), the annotated sentiment corpora may be limited. Utilizing English corpora for Chinese sentiment classification can relieve the labeling burden.
Main approach:
use available language corpora to train sentiment classifiers for the target language data. Machine translation is typically employed
(
Banea
et al 2008; Wan 2009; Wei and Pal 2010;
Kim et al. 2010; Guo et al 2010;
Mihalcea & Wiebe 2010; Boyd-Graber and Resnik 2010; Banea et al 2010; Duh, Fujino & Nagata 2011; Lu et al 2011)
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
25Slide26
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
26Slide27
Sentence subjectivity classification
Document-level sentiment classification is too coarse for most applications. We now move to the sentence level. Much of the early work on sentence level analysis focuses on identifying
subjective sentences
.
Subjectivity classification:
classify a sentence into one of the
two classes
(Wiebe et al 1999)
Objective and subjective. Most techniques use supervised learning as well. E.g., a naïve Bayesian classifier (Wiebe et al. 1999).Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
27Slide28
Sentence sentiment analysis
Usually consist of two stepsSubjectivity classificationTo identify subjective sentencesSentiment classification of subjective sentences
Into two classes, positive and negative
But bear in mind
Many objective sentences can imply sentiments
Many subjective sentences do not express positive or negative sentiments/opinions
E.g.,”I believe he went home yesterday.”
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
28Slide29
Assumption
Assumption: Each sentence is written by a single person and expresses a single positive or negative opinion/sentiment. True for simple sentences
, e.g.,
“I like this car”
But not true for compound and “complex” sentences
, e.g.,
“I like the picture quality but battery life sucks.”
“Apple is doing very well in this lousy economy.”
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 29Slide30
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
30Slide31
We need to go further
Sentiment classification at both the document and sentence (or clause) levels are useful, but They do not find what people liked and disliked.
They do not identify the
targets
of opinions, i.e.,
Entities and their aspects
Without knowing targets, opinions are of limited use.
We need to go to the entity and aspect level.
Aspect-based opinion mining and summarization (Hu and Liu 2004). We thus need the full opinion definition.Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
31Slide32
Recall an opinion is a quintuple
An opinion is a quintuple
(
e
j
,
a
jk
, soijkl, hi, tl), where ej is a target entity.
ajk is an aspect/feature of the entity ej
.
so
ijkl
is the sentiment value of the opinion of the opinion holder
h
i
on feature
a
jk
of entity
e
j
at time
t
l
.
so
ijkl
is +
ve
, -
ve
, or
neu
, or a
more granular rating.
h
i
is an opinion holder.
t
l
is the time when the opinion is expressed.
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
32Slide33
Aspect-based sentiment analysis
Much of the research is based on online reviewsFor reviews, aspect-based sentiment analysis is easier because the entity (i.e., product name) is usually knownReviewers simply express positive and negative opinions on different aspects of the entity.
For blogs
,
forum discussions
, etc., it is harder:
both entity and aspects of entity are unknown,
there may also be many comparisons, and
there is also a lot of irrelevant information. Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 33Slide34
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
34Aspect extraction
Goal
:
Given an opinion corpus, extract all aspects
A frequency-based approach
(Hu and Liu, 2004)
:
nouns (NN) that are frequently talked about are likely to be true aspects (called frequent aspects) . Pruning based on part-of relations and Web search, e.g., “camera has” (Popescu and Etzioni, 2005).Supervised learning
, e.g., HMM and CRF (conditional random fields) (Jin and Ho, 2009; Jakob and Gurevych, 2010).
Using dependency parsing + “opinion has target”
(Hu and Liu 2004, Zhuang,Jing and Zhu, 2006; Qiu et al. 2009)Slide35
Extract Aspects & Opinion Words (Qiu et al., 2011)
A double propagation (DP) approach proposedUse dependency of opinions & features to extract both features & opinion words.
Knowing one helps find the other.
E.g., “
The
rooms
are
spacious
”It bootstraps using a set of seed opinion words, but no feature seeds needed.Based on the dependency grammar. It is a domain independent method!Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 35Slide36
Rules from dependency grammar
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 36Slide37
Aspect-sentiment statistical models
This direction of research is mainly based on topic models: pLSA:
Probabilistic Latent Semantic Analysis
(Hofmann 1999)
LDA
:
Latent
Dirichlet allocation (Blei, Ng & Jordan, 2003; Griffiths & Steyvers, 2003; 2004) Topic models:documents are mixtures of topics
a topic is a probability distribution over words. A topic model is a document generative model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
37Slide38
Aspect-sentiment model (Mei et al 2007)
This model is based on pLSA (Hofmann, 1999). It builds a topic (aspect) model, a positive sentiment model, and a negative sentiment model. A training data is used to build the initial models. Training data: topic queries and associated positive and negative sentences about the topics.
The learned models are then used as priors to build the final models on the target data.
Solution: log likelihood and EM algorithm
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
38Slide39
Multi-Grain LDA to extract aspects (Titov and McDonald, 2008a, 2008b)
Unlike a diverse document set used for traditional topic modeling. All reviews for a product talk about the same topics/aspects. It makes applying PLSA or LDA in the traditional way problematic. Multi-Grain LDA (MG-LDA) models global topics and local topics (Titov and McDonald, 2008a).
Global topics are entities (based on reviews)
Local topics are aspects (based on local context, sliding windows of review sentences)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
39Slide40
Aspect-rating of short text (Lu et al 2009)
This work makes use of short phrases, head terms (wh) and their modifiers (wm), i.e.(wm, w
h
)
E.g., great shipping, excellent seller
Objective: (1) extract aspects and (2) compute their ratings in each short comment.
It uses pLSA to extract and group aspects
It uses existing rating for the full post to help determine aspect ratings.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 40Slide41
MaxEnt-LDA Hybrid (Zhao et al. 2010)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 41Slide42
Graphical model
yd,s,n indicatesBackground wordAspect word, or
Opinion word
MaxEnt
is used to train a model using training set
d,s,n
x
d,s,n feature vectorud,s,n indicatesGeneral orAspect-specificBing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
42Slide43
Topic model of snippets (Sauper, Haghighi and Barzilay, 2011)
This method works on short snippets already extracted from reviews. “battery life is the best I’ve found”The model is a variation of LDA but with seeds for sentiment words as priors,
but it also has HMM for modeling the sequence of words with types (aspect word, sentiment word, or background word).
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
43Slide44
Semi-supervised model
(Mukherjee and Liu, ACL-2012)Unsupervised modeling is governed by “higher-order co-occurrence” (Heinrich, 2009), i.e., based on how often terms co-occur in different contexts.
It results in not so “meaningful” clustering because
conceptually different terms can co-occur in related contexts
e.g., in hotel domain
stain
,
shower, walls in aspect Maintenance; bed
, linens, pillows
in aspect
Cleanliness
,
are equally probable of emission for any aspect.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
44Slide45
Semi-supervised model (contd.)
Semi-supervised modeling allows the user to give some seed aspect expressions for a subset of aspects (topic clusters) In order to produce aspects that meet the user’s need.
Employ seeds to “guide” model clustering, not by “higher order co-occurrence” alone.
Standard multinomial => 2-level tree structured priors
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
45Slide46
Graphical model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 46Slide47
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
47Aspect sentiment classification
For each aspect, identify the sentiment or opinion expressed
about
it.
Classification based on sentence is insufficient. E.g.
“The
battery life
and picture quality are great (+), but the view founder is
small (-)”. “Apple
(+)
is
doing well in this bad
economy (-)
.”
“
Standard & Poor
downgraded
Greece's credit
rating (-)
”
Classification needs
to consider target and thus to segment
each sentence
Lexicon-based approach (e.g., Ding, Liu and Yu, 2008)
Supervised learning (e.g
., Jiang et al. 2011
)Slide48
Aspect sentiment classification
Almost all approaches make use of opinion words and
phrases.
But notice:
Some opinion words have context independent orientations, e.g., “good” and “bad” (almost)
Some other words have context dependent orientations, e.g., “small” and “sucks” (+
ve for vacuum cleaner)Lexicon-based methodsParsing is needed to deal with: Simple sentences, compound sentences, comparative sentences, conditional sentences, questions, etcNegation (not), contrary (but), comparisons, etc. A large opinion lexicon, context dependency, etc.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
48Slide49
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
49A lexicon-based method
(Ding, Liu and Yu 2008)
Input
: A set of opinion words and phrases. A pair (
a
,
s
), where a is an aspect and s is a sentence that contains a. Output: whether the opinion on a in s is +ve, -ve, or neutral. Two steps: Step 1: split the sentence if needed based on BUT words (but, except that, etc).
Step 2: work on the segment sf containing
a
. Let the set of opinion words in
s
f
be
w
1
, ..,
w
n
. Sum up their orientations (1, -1, 0), and assign the orientation to (
a
, s) accordingly.
where
w
i
.o
is the opinion orientation of
w
i
.
d
(
w
i
,
a
) is the distance from
a
to
w
i
.Slide50
Sentiment shifters (e.g., Polanyi and Zaenen 2004)
Sentiment/opinion shifters (also called valence shifters are words and phrases that can shift or change opinion orientations. Negation words like not,
never
,
cannot
, etc., are the most common type.
Many other words and phrases can also alter opinion orientations.
E.g., modal auxiliary verbs (e.g., would, should, could, etc) “The brake could be improved.” Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
50Slide51
Sentiment shifters (contd)
Some presuppositional items also can change opinions, e.g., barely and hardly
“It hardly works.” (comparing to “it works”)
It presupposes that better was expected.
Words like
fail
,
omit
, neglect behave similarly, “This camera fails to impress me.” Sarcasm changes orientation too “What a great car, it did not start the first day.”Jia, Yu and Meng (2009) designed some rules based on parsing to find the scope of negation.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
51Slide52
Basic rules of opinions (Liu, 2010
)Opinions/sentiments are governed by many rules, e.g.,Opinion word or phrase, ex: “I love this car”
P ::= a positive opinion word or phrase
N ::= an negative opinion word or phrase
Desirable or undesirable facts, ex:
“After my wife and I slept on it for two weeks, I noticed a mountain in the middle of the mattress”
P ::= desirable fact
N ::= undesirable fact
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
52Slide53
Basic rules of opinions
High, low, increased and decreased quantity of a positive or negative potential item
, ex:
“The battery life is long.”
PO ::= no, low, less or decreased quantity of NPI | large, larger, or increased quantity of PPI
NE ::= no, low, less, or decreased quantity of PPI
| large, larger, or increased quantity of NPI
NPI ::= a negative potential item
PPI ::= a positive potential itemBing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
53Slide54
Basic rules of opinions
Decreased and increased quantity of an opinionated item, ex: “This drug reduced my pain significantly.” PO ::= less or decreased N
| more or increased P
NE ::= less or decreased P
| more or increased N
Deviation from the desired value range
: “This drug increased my blood pressure to 200.”
PO ::= within the desired value range
NE ::= above or below the desired value range Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
54Slide55
Basic rules of opinions
Producing and consuming resources and wastes, ex: “This washer uses a lot of water” PO ::= produce a large quantity of or more resource
| produce no, little or less waste
| consume no, little or less resource
| consume a large quantity of or more waste
NE ::= produce no, little or less resource
| produce some or more waste
| consume a large quantity of or more resource
| consume no, little or less waste
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
55Slide56
Opinions implied by objective terms (Zhang and Liu, 2011)
For opinion mining, many researchers first identify subjective sentences and then determine if they are positive/negative. This approach
can be problematic
Many objective sentences imply opinions/sentiments
E.g., “After sleeping on the mattress for one month, a
valley
is formed in the middle.”
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
56Slide57
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
57Slide58
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
58Comparative Opinions
(Jindal and Liu, 2006)
Gradable
Non-Equal Gradable
: Relations of the type
greater
or
less thanEx: “optics of camera A is better than that of camera B”Equative: Relations of the type equal to Ex: “camera A and camera B both come in 7MP”
Superlative: Relations of the type greater or less than all others
Ex: “
camera A is the cheapest in market
”Slide59
Analyzing Comparative Opinions
Objective: Given an opinionated document d, Extract comparative opinions:
(
E
1
,
E
2, F, po, h, t), where E1 and E2 are the entity sets being compared based on their shared features/aspects F
, po is the preferred object set of the opinion holder
h
, and
t
is the time when the comparative opinion is expressed.
Note:
not positive or negative opinions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
59Slide60
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
60Slide61
Opinion Spam Detection (Jindal et al, 2008, 2010 and 2011)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
61Slide62
Supervised learning (fake reviews)
Training dataSame userid, same product
Different
userid
, same product
Same
userid
, different products
Different userid, different productsThe last three types are very likely to be spam!Other reviews, non-spamBuild a supervised classification model (Jindal and Liu 2008)(Ott
et al., 2011) and (Li et al., 2011)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
62Slide63
Finding Unexpected Behavior Patterns(Jindal and Liu 2010)
Opinion spam is hard to detect because it is very difficult to recognize fake reviews by manually reading them. i.e., hard to detect based on contentLet us analyze the
behavior of reviewers
identifying
unusual review patterns
which may represent suspicious behaviors of reviewers.
We formulate the problem as finding
unexpected rules and rule groups
.Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 63Slide64
Finding unexpected review patterns
For example, if a reviewer wrote all positive reviews on products of a brand but all negative reviews on a competing brand …Finding unexpected rules,
Data:
reviewer-id
,
brand-id
,
product-id
, and a class.Mining: class association rule miningFinding unexpected rules and rule groups, i.e., showing atypical behaviors of reviewers. Rule1: Reviewer-1, brand-1 -> positive (confid=100%)Rule2: Reviewer-1, brand-2 -> negative (
confid=100%)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
64Slide65
The example (cont.)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
65Slide66
Confidence unexpectedness
Rule: reviewer-1, brand-1 positive [sup = 0.1, conf = 1]
If we find that on average reviewers give brand-1 only 20% positive reviews (expectation), then reviewer-1 is quite unexpected.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
66Slide67
Support unexpectedness
Rule: reviewer-1, product-1 -> positive [sup = 5]Each reviewer should write only one review on a product and give it a positive (negative) rating (expectation). This unexpectedness can detect those reviewers who review the same product multiple times, which is unexpected.
These reviewers are likely to be spammers.
Can be defined probabilistically as well.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
67Slide68
Detecting group opinion spam (Mukherjee, Liu and Glance, WWW
-2012)A group of people who work together to promote an product or to demote another product. The algorithm has two stepsFrequent pattern mining: find groups of people who reviewed a number of products. These are candidate spammer groups.
A relational model is then formulated to compute a ranking of candidate groups based on their likelihood being fake.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
68Slide69
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
69Slide70
Modeling Review Comments(Mukherjee and Liu, ACL-2012)
Online reviews by consumers evaluate
products and services that they have used.
While certainly useful, reviews only provide part of the story: evaluations and experiences of the reviewers.
Hidden glitches:
Reviewer may not be an expert.
Misuses a product.
Doesn’t mention some product aspects of consumer interest.
Reviewer can be an opinion spammer writing fake reviews.Clearly, there is a room for
improvement of the online review system.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
70Slide71
Review Comments
To improve the reviewing system, popular review hosting sites (e.g., Amazon, Epinions, Wired.com, etc.) support reader-comments on reviews.
Comments on review are a richer way of “review profiling”, rather than just clicking whether the review is helpful or not.
Many reviews receive a large number of comments. (e.g., hundreds of them)
Reading them all to get a gist of them is not easy.
Some kind of summary will be very useful.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
71Slide72
What to model?
Topics/aspects and different types of commentsThumbs-up (e.g., “review helped me”)Thumbs-down
(e.g., “poor review”)
Question
(e.g., “how to”)
Answer acknowledgement
(e.g., “thank you for clarifying”).
Disagreement
(contention) (e.g., “I disagree”) Agreement (e.g., “I agree”).They are collectively called, C-expressions.Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
72Slide73
Summary and usefulness
Extracted topics and C-expressions from comments are quite useful in practice: Enable more accurate classification of comments, e.g., evaluating review quality and credibility.
Help identify
key
product aspects that people are troubled with in disagreements and in questions.
Facilitate
comments summarization. Summary
may include but not limited to:
% of people who giving a thumbs-up or thumbs-down% of people who agree or disagree with the reviewer Disagreed (contentious) aspects (or topics) Aspects that people
often have questions with
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
73Slide74
A graphical model – generative process
For each C-expression type
, draw
For each topic
t
, draw
For
each comment post
:
Draw
Draw
For each
term
,
:
Draw
Draw
if
(
//
is a C-expression
term
Draw
)
else
//
,
is a topical
term
Draw
)
Emit
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
74Slide75
The graphical model in plate notation
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 75Slide76
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
76Slide77
Modeling Online Discussions/Debates
(Mukherjee and Liu, KDD-2012)A large part of social media is about discussion and debate.
A large part of such contents is about
social
, political and religious issues.
On such issues, there are often heated discussions/debates, i.e., people argue and agree or disagree with one another.
We
can model
such interactive social media.Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
77Slide78
The Goal
Given a set of discussion/debate posts, we aim to perform the following tasks. Discover expressions often used to express Contention/Disagreement (e.g., “I disagree”, “you make no sense”) and
A
greement (e.g., “I agree”, “I think you’re right”). We collectively call them
CA-expressions
.
Determine contentious topics.
First discover discussion topics in the whole collection,
then for each contentious post, discover the contention points (or topics). Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
78Slide79
Joint modeling of debate topics and expressions (JTE)
We jointly model topics and CA-expressions Observation: A typical discussion/debate post mentions a few topics (using semantically related topical terms) and expresses some viewpoints with one or more CA-expression types (using semantically related contention and/or agreement expressions).
The above observation motivates the model
Posts are represented as random mixtures of latent topics and CA-expression types.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
79Slide80
A graphical model – generative process(the same as that for comments)
For each C-expression type
, draw
For each topic
t
, draw
For
each comment post
:
Draw
Draw
For each
term
,
:
Draw
Draw
if
(
//
is a C-expression
term
Draw
)
else
//
,
is a topical
term
Draw
)
Emit
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
80Slide81
JTE in plate notation(the same as that for comments)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
81Slide82
JTE-R: Encoding reply relations
Observation
: Whenever a
post
d
replies to the viewpoints
of
some other posts by quoting them, and the posts quoted by d should have similar topic distributions.Let
qd
be
the set of posts quoted by post
d
.
q
d
is
observed.
Key challenge: -
constrain
to be similar to
, where
during
inference while the topic distributions of both
and
,
are latent and unknown
apriori
.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
82Slide83
Exploiting Dirichlet
distribution
A simple solution: exploit the following salient features of the
Dirichlet
distribution:
Since
, we have
= 1. Thus, it suffices that
can act as a base measure for
Dirichlet
distributions of the same order.
Also
, the expected probability mass associated with each dimension of the
Dirichlet
distribution is proportional to the corresponding component of its base
measure
=
. Thus,
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
83Slide84
Exploiting Dirichlet
distribution (contd)
We need functional base measures
Thus for posts that quote:
we draw
, where
(the expected topical distribution of posts in
).
For
posts
that
do not quote any other post,
we
simply draw
.
The Gibbs sampling is, however, an approximation (see the paper for detail)
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
84Slide85
JTE-R in plate notation
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 85Slide86
JTE-P : Encoding Pair Structures
Observation: When authors reply to others’ viewpoints,
they typically direct their topical viewpoints with contention or agreeing expressions to those authors.
Such exchanges can go back and forth between author pairs.
The discussion topics and CA-expressions emitted are thus caused by the author-pairs’ topical interests and their nature of interactions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
86Slide87
The approach
Let
be
the author of a post
,
be the list of
target authors
to whom
replies
to or quotes in
.
The
pairs of the form
= (
),
c
∈
essentially
shapes both the topics and CA-expressions emitted in
d
as contention or agreement on topical viewpoints are almost always directed towards certain
authors
.
Thus
, it is appropriate to condition
and
over author-pairs
.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
87Slide88
The approach
To generate each term
, a
target author,
, is chosen at uniform from
forming a pair
= (
,
).
Then
, depending on the switch variable
, a topic or an expression type index
is chosen from a multinomial over topic distribution
or CA-expression type distribution
, where the subscript
denotes the fact that the distributions are specific to the author-target pair
which shape topics and CA-expressions.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
88Slide89
JTE-P graphical model
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 89Slide90
Roadmap
Sentiment Analysis and Opinion MiningSentiment Analysis ProblemDocument sentiment classification
Sentence subjectivity & sentiment classification
Aspect-based sentiment analysis
Mining comparative opinions
Opinion spam detection
Beyond Sentiments
Modeling review comments
Modeling discussions/debatesSummary
Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City
90Slide91
Summary
We first introduced some basics of sentiment analysis and opinion mining Current solutions are still inaccurate. Every sub-problem is hard
General NL understanding is probably
hopeless in near future
But can we understand this restricted aspect of semantics?
Endless applications due to the human nature
We also discussed the problem of modeling interactive social forums, such as
review comments
and debates/discussions.There is a lot of future work, e.g., linguistic knowledge.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China
91Slide92
References
All references are in the
New Book
Bing Liu.
Sentiment Analysis and Opinion Mining
.
Morgan & Claypool Publishers. May 2012.
Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 92