Opinion Mining and Sentiment Analysis - PowerPoint Presentation

Slide1

Opinion Mining and Sentiment Analysis

Slides from Bing Liu and Ronan FeldmanSlide2

Introduction

Two main types of textual information.

Facts and Opinions

Note: factual statements can imply opinions too.

Most current text information processing methods (e.g., web search, text mining) work with factual information.

Sentiment analysis or

opinion mining

computational study of opinions, sentiments and emotions expressed in text.

Why opinion mining now?

Mainly because of the Web; huge volumes of opinionated text.Slide3

Introduction – user-generated media

Importance of opinions:

Opinions are useful when making a decision, we want to hear others’ opinions.

In the past

,

Individuals

: opinions from friends and family

businesses

: surveys, focus groups, consultants …

Word-of-mouth on the Web

User-generated media

: One can express opinions on anything in reviews, forums, discussion groups, blogs ...

Opinions of global scale

:

No longer limited to:

Individuals:

one’s circle of friends

Businesses

: Small scale surveys, tiny focus groups, etc. Slide4

Sentiment analysis applications

Businesses and organizations

Benchmark products and services; market intelligence.

Businesses spend a huge amount of money to find consumer opinions using consultants, surveys and focus groups,

etc

Individuals

Make decisions to purchase products or to use services

Find public opinions about political candidates and issues

Ad placement

: e.g. in social media

Place an ad if one praises a product.

Place an ad from a competitor if one criticizes a product.

Opinion retrieval

: provide general search for opinions.Slide5

A Fascinating Problem!

Intellectually challenging & major applications

.

A popular research topic in recent years in NLP and Web data mining.

20-60 companies in USA alone

It touches every aspect of NLP and yet is restricted and confined.

Little research in NLP/Linguistics in the past.

Potentially a major technology from NLP

But

“not yet” and not easy!

Data sourcing and data integration are hard too!Slide6

Abstract Problem Statement

It consists of two parts

Opinion definition

What is an opinion?

Opinion summarization

Opinions are subjective. An opinion from a single person (unless a VIP) is often not sufficient for action.

We need opinions from many people, and thus opinion summarization.Slide7

An Example Review

“

I bought an

iPhone

a few days ago. It was such a nice phone. The touch screen was really cool. The voice quality was clear too. Although the battery life was not long, that is ok for me.

However, my mother was mad with me as I did not tell her before I bought the phone.

She also thought the phone was too expensive, and wanted me to return it to the shop

. …”

What do we see?

Opinions

,

targets of opinions

, and

opinion holdersSlide8

Entity and aspect/feature level

Id: Abc123 on 5-1-2008

“

I bought an

iPhone

a few days ago. It is such a

nice phone.

The

touch screen

is

really cool.

The

voice quality

is

clear

too.

It is much better than my old

Blackberry

, which was a terrible phone and so

difficult to type

with its

tiny keys

. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too

expensive

…”

What do we see?

Opinion targets:

entities and their features

Sentiments:

positive and negative

Opinion holders:

persons who hold the opinions

Time:

when opinions were expressedSlide9

Two main types of opinions

Regular opinions

: Sentiment/opinion expressions on some target entities

Direct opinions:

“The touch screen is really cool”

Indirect opinions

:

“After taking the drug, my pain has gone”

Comparative opinions:

Comparisons of more than one entity

E.g., “iPhone is better than Blackberry”

We focus on regular opinions first, and just call them opinions.Slide10

Entity and Aspect

Definition

(

entity

): An

entity e

is a product, person, event, organization, or topic.

e

is represented as

a hierarchy of

components, sub-components

, and so on.

Each node represents a component and is associated with a set of

attributes

of the component.

An opinion can be expressed on any node or attribute of the node.

For simplicity, we use the term aspects (features) to represent both components and attributes.

B. Liu, Handbook of NLP, 2010Slide11

Direct Opinion: definition

A

direct opinion

is a quintuple

(

o

j

,

f

jk

,

s

ijkl

,

h

i

,

t

l

), where oj is a target objectfjk is a feature of the object ojsijkl is the sentiment value of the opinion of the opinion holder h

i on feature fjk of object oj at time tl.

s

ijkl

is positive, negatve, or neutral, or a more granular rating hi is an opinion holdertl is the time when the opinion is expressed

B. Liu, Handbook of NLP, 2010Slide12

Our example in quintuples

(iPhone, GENERAL, +, Abc123, 5-1-2008)

(iPhone,

touch_screen

, +, Abc123, 5-1-2008)Slide13

Alternative terminology

Entity is also called object.

Aspect is also called feature, attribute, facet,

etc

Opinion holder is also called opinion sourceSlide14

Structure the unstructured

Goal

: Given an opinionated document

Discover all quintuples

(

o

j

,

f

jk

,

s

ijkl

,

h

i

,

t

l

) i.e., mine the five corresponding pieces of information in each quintuple, andOr, solve some simpler problemsE.g. classify the sentiment of the entire documentWith the quintuples, Unstructured Text 

Structured DataTraditional data and visualization tools can be used to slice, dice and visualize the results in all kinds of waysEnable qualitative and quantitative analysis. Slide15

Sentiment Classification: doc-level

Classify a document

(e.g., a review) based on the overall sentiment expressed by opinion holder

Classes

: Positive, or negative (and neutral)

In the model, (

o

j

,

f

jk

,

s

ijkl

,

h

i

,

t

l),It assumesEach document focuses on a single object and contains opinions from a single opinion holder.It considers opinion on the object, oj

(or oj = fjk)

Pang and Lee, et al 2002 and Turney 2002Slide16

Subjectivity

Sentence subjectivity

:

An

objective sentence

presents some factual information, while a

subjective sentence

expresses some personal opinions, beliefs, views, feelings, or emotions.

Not the same as emotionSlide17

Subjectivity Analysis

Sentence-level sentiment analysis has two tasks:

Subjectivity classification

: Subjective or objective.

Objective

: e.g.,

I bought an iPhone a few days ago.

Subjective

: e.g.,

It is such a nice phone.

Sentiment classification

: For subjective sentences or clauses, classify positive or negative.

Positive

:

It is such a nice phone.

However

(Liu, NLP handbook)

subjective sentences

≠ +ve or –ve opinionsE.g., I think he came yesterday. Objective sentence ≠ no opinionImply –

ve opinion: My phone broke in the second day. Wiebe et al 2004Slide18

Rational and emotional evaluations

Rational evaluation: Many evaluation/opinion sentences express no emotion

E.g. “The voice on this phone is clear”

Emotional evaluation

E.g. “I love this phone”

“The voice on this phone is crystal clear” (?)

Some emotion sentences express no (positive or negative) opinion/sentiment

E.g. “I am so surprised to see you”Slide19

Feature-Based Sentiment Analysis

Sentiment classification at both document and sentence (or clause) levels are

not sufficient

,

they do not tell

what people like and/or dislike

A positive opinion on an object does not mean that the opinion holder likes everything.

An negative opinion on an object does not mean …..

Objective:

Discovering all quintuples

(

o

j

,

f

jk

,

so

ijkl, hi, tl)With all quintuples, all kinds of analyses become possible.Slide20

Opinion Summary

With a lot of opinions, a summary is necessary.

A multi-document summarization task

For factual texts, summarization is to select the most important facts and present them in a sensible order while avoiding repetition

1 fact = any number of the same fact

But for opinion documents, it is different because opinions have a quantitative side & have targets

1 opinion ≠ a number of opinions

Aspect-based summary is more suitable

Quintuples form the basis for opinion summarizationSlide21

Feature-Based Opinion Summary

“I bought an

iPhone

a few days ago. It was such a nice

phone

. The

touch screen

was really cool. The

voice quality

was clear too. Although the

battery life

was not long, that is ok for me. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too

expensive

, and wanted me to return it to the shop. …”

Feature Based Summary

:

Feature1:

Touch screen

Positive: 212

The touch screen was really cool. The touch screen was so easy to use and can do amazing things. …

Negative

: 6

The screen is easily scratched.I have a lot of difficulty in removing finger marks from the touch screen. … Feature2:

battery life

…

Note: We omit opinion holders

Hu & Liu, KDD-2004Slide22

Visual Comparison

Summary of reviews of

Cell Phone

1

Voice

Screen

Size

Weight

Battery

+

_

Comparison of reviews of

Cell Phone 1

Cell Phone 2

_

+

Liu et al. WWW-2005Slide23

Aspect-based opinion summarySlide24

Google Product SearchSlide25

Comparing 3 GPSs on different features

Each bar shows the proportion of +

ve

opinionSlide26

Demo 1: Detail opinion sentences

You can click on any bar to see the opinion sentences. Here are negative opinion sentences on the maps feature of Garmin.

The pie chart gives the proportions of opinions. Slide27

# of feature mentions

People talked more about prices than other features. They are quite positive about price, but not bout maps and software.

27Slide28

Aggregate opinion trend

More complains in July - Aug, and in Oct – Dec!

28Slide29

Sentiment Analysis is Challenging!

“

This past Saturday, I bought a

Nokia

phone and my girlfriend bought a

Motorola

phone with

Bluetooth

. We called each other when we got home.

The

voice

on my phone was not so clear, worse than my previous phone

.

The battery life was long

.

My girlfriend was quite happy with her phone

.

I wanted a phone with good

sound quality

. So my purchase was a real disappointment. I returned the phone yesterday.”Slide30

Requires solving several IE problems

(

o

j

,

f

jk

,

s

ijkl

,

h

i

,

t

l

),

o

j

- a target object: Named Entity Extraction (more)fjk - a feature of oj: Information Extractionsijkl is sentiment: Sentiment determination hi is an opinion holder: Information/Data Extractiontl is the time:

Data ExtractionCo-reference resolutionRelation extractionSynonym match (voice = sound quality) …None of them is a solved problem!Slide31

Easier and harder problems

Tweets from Twitter are probably the easiest

short and thus usually straight to the point

Stocktwits

are much harder!

Reviews are next

entities are given (almost) and there is little noise

Discussions, comments, and blogs are hard.

Multiple

entities

,

comparisons

,

noisy

,

sarcasm

,

etc

Extracting entities and aspects, and determining sentiments/opinions about them are hard.

Combining them is harder.Slide32

Extraction of competing objects

The user first gives a few objects/products as

seeds

, e.g., BMW and Ford.

The system then identifies other competing objects from the opinion corpus.

The problem can be tackled with

PU learning

(Learning from positive and unlabeled examples) (Liu et al., 2002, 2003).

See Li et al., ACL-2010Slide33

Feature/Aspect-based sentiment analysisSlide34

Aspect-based sentiment analysis

Much of the research is based on online reviews

For reviews, aspect-based sentiment analysis is easier because the entity (i.e., product name) is usually known

Reviewers simply express positive and negative opinions on different aspects of the entity.

For blogs, forum discussions, etc., it is harder:

both entity and aspects of entity are unknown,

there may also be many comparisons, and

there is also a lot of irrelevant information.Slide35

Find entities

Although similar, it is somewhat different from the traditional named entity recognition (NER).

E.g., one wants to study opinions on phones given Motorola and Nokia, find all phone brands and models in a corpus, e.g., Samsung, Moto.Slide36

Feature/Aspect extraction

Extraction may use:

frequent nouns and noun phrases

Sometimes limited to a set known to be related to the entity of interest or using

part discriminators

e.g., for a scanner entity “of scanner”, “scanner has”

opinion and target relations

Proximity or syntactic dependency

Standard IE methods

Rule-based or supervised learning

Often HMMs or CRFs (like standard IE)Slide37

Double Propagation

Proposed in

Qiu

et al., IJCAI-2009

Like co-training

It exploits the dependency relations of opinions and features to extract features.

Opinions words modify object features, e.g.,

“This camera takes

great

pictures

”

The algorithm bootstraps using a set of seed opinion words (no feature input).

To extract features (and also opinion words)Slide38

The DP method

DP is a bootstrapping method

Input: a set of seed opinion words,

no aspect seeds needed

Based on dependency tree (

Tesniere

1959)

This phone has good screenSlide39

Rules from dependency grammarSlide40

Group synonym features

Features that are domain synonyms should be grouped together.

Many techniques can be used to deal with the problem, e.g.,

Topic modeling, distributional similarity, etc

Semi-supervised learning method

Z. Zhai, B. Liu, H. Xu and P. Jia. Grouping Product Features Using Semi-Supervised Learning with Soft-Constraints.

COLING-2010.Slide41

Coreference Resolution

Different from traditional coreference resolution

Important to resolve objects and features

E.g.., “

I bought a

Canon S500

camera yesterday.

It

looked beautiful. I took a few

photos

last night.

They

were amazing

”.

Some specific characteristics of opinions can be exploited for better accuracy. See

X. Ding and B. Liu, Resolving Object and Attribute Coreference in Opinion Mining.

COLING-2010

. Slide42

Coreference Resolution

Attardi et al. 2010. Coreference Resolution by Parse Analysis and Similarity Clustering.

Best performing at

SemEval

2010

Identifies mentions from dependency trees

Uses eager classifier to cluster mentions

Positive

and

negative

instances are created by pairing each mention with each of the preceding ones

Features extracted from pairs of mentions:

Lexical:

Same, Prefix, Suffix, Acronym

…

Distance:

Edit, mention, token, sentence

Syntax

:

same HeadPoS, pair of HeadPosPair of counts of mention occurrences

Same NE typeFor pronouns: type, pair of genders, pair of numbersSlide43

Accuracy at SemEval 2010

Mention

CEAF

B

3

Catalan

82.7

57.1

64.6

German

59.2

49.5

50.7

English

73.9

57.3

61.3

Spanish

83.1

59.3

66.0Slide44

Coreference Resolution

Method of (Lee,

Peirsman

et al. 2011)

Best-performing in CoNLL-2011

Based on locating all noun phrases, identifying their properties, and then clustering them in several deterministic iterations (called

sieves

), starting with the highest-confidence rules and moving to lower-confidence higher-recall ones.

Eager approach: any matching noun phrases with matching properties are immediately clustered together.Slide45

Identify opinion orientation

For each feature, we identify the sentiment or opinion orientation expressed by a reviewer.

Almost all approaches make use of

opinion words

and

phrases.

But notice again (a simplistic way):

Some opinion words have context independent orientations, e.g., “great”.

Some other opinion words have context dependent orientations, e.g., “small”

Many ways to use opinion words.

Machine learning methods for sentiment classification at the sentence and clause levels are also applicable.Slide46

Aggregation of opinion words

Input

: a pair

(

f

,

s

)

, where

f

is a product feature and

s

is a sentence that contains

f

.

Output

: whether the opinion on

f

in

s is positive, negative, or neutral. Two steps: Step 1: split the sentence if needed based on BUT words (but, except that, etc). Step 2: work on the segment sf containing f. Let the set of opinion words in sf be w

1, .., wn. Sum up their orientations (1, -1, 0), and assign the orientation to (f, s) accordingly.

In (Ding et al, WSDM-08), step 2 is changed to

with better results. wi.o is the opinion orientation of wi.

d

(

w

i

,

f

)

is the distance from

f

to

w

i

.

Ding and Liu, 2008Slide47

Basic Opinion Rules

Opinions are governed by some rules, e.g.,

Neg



Negative

Pos



Positive

Negation

Neg



Positive

Negation Pos



Negative

Desired value range



PositiveBelow or above the desired value range  NegativeB. Liu, NLP handbook, 2010Slide48

Basic Opinion Rules

Decreased

Neg



Positive

Decreased Pos



Negative

Increased

Neg



Negative

Increased Pos



Positive

Consume resource



NegativeProduce resource  PositiveConsume waste  PositiveProduce waste  NegativeB. Liu, NLP handbook, 2010Slide49

Two Main Types of Opinions

Direct Opinions

:

direct sentiment expressions on some target objects, e.g., products, events, topics, persons.

E.g., “the picture quality of this camera is great.”

(many are much more complex).

Comparative Opinions:

Comparisons expressing similarities or differences of more than one object. Usually stating an ordering or preference.

E.g., “car x is cheaper than car y.”Slide50

Comparative Opinions

Gradable

Non-Equal Gradable

: Relations of the type

greater

or

less than

Ex: “optics of camera A is better than that of camera B”

Equative

: Relations of the type

equal to

Ex: “

camera A and camera B both come in 7MP

”

Superlative

: Relations of the type

greater

or

less than all othersEx: “camera A is the cheapest camera available in market”Jindal and Liu, 2006Slide51

Mining Comparative Opinions

Objective

: Given an opinionated document

d

Extract comparative opinions

:

(

O

1

,

O

2

,

F

,

po

, h, t

), whereO1 and O2 are the object sets being compared based on their shared features Fpo is the preferred object set of the opinion holder ht

is the time when the comparative opinion is expressed.Note: not positive or negative opinions. Slide52

Sentiment LexiconSlide53

Sentiment (opinion) lexicon

Sentiment lexicon: lists of words and expressions used to express people’s subjective feelings and sentiments/opinions

sentiments/opinions.

Not just individual words, but also phrases and idioms, e.g.

“costs an arm and a leg”

Many sentiment lexica can be found on the web

They often have thousands of terms, and are quite usefulSlide54

Sentiment lexicon

Sentiment words or phrases (also called polar words,

opinion bearing words,

etc

). E.g.,

Positive: beautiful, wonderful, good, amazing,

Negative: bad, poor, terrible, cost an arm and a leg.

Many of them are context dependent, not just application domain dependent.

Three main ways to compile such lists:

Manual approach: not a bad idea for a one-time effort

Corpus-based approach

Dictionary-based approachSlide55

Corpus vs Dictionary-based method

Corpus-based approaches

Often use a double propagation between opinion words and the items they modify

require a large corpus to get good coverage

Dictionary-based methods

Typically use

WordNet’s

synsets

and hierarchies to acquire opinion words

usually do not give domain or context dependent meaningsSlide56

Corpus-based approaches

Rely on syntactic patterns in large corpora.

(

Hazivassiloglou

and

McKeown

, 1997;

Turney

, 2002; Yu and

Hazivassiloglou

, 2003;

Kanayama

and

Nasukawa

, 2006; Ding, Liu and Yu, 2008)

Can find domain dependent orientations (positive, negative, or neutral).

(

Turney

, 2002) and (Yu and

Hazivassiloglou, 2003)are similar.Assign opinion orientations (polarities) to words/phrases.(Yu and Hazivassiloglou, 2003) is slightly different from (Turney, 2002)use more seed words (rather than two) and use log-likelihood ratio (rather than PMI).Slide57

The Double Propagation method

The DP method can also use dependency of opinions & aspects to extract new opinion words.

Based on dependency relations

Knowing an aspect can find the opinion word that modifies it

E.g. “The rooms are spacious”

Knowing some opinion words can find more opinion words

E.g. “The rooms are spacious and beautiful”

Jijkoun

,

Rijke

and

Weerkamp

(2010) did similarlySlide58

Opinions implied by objective terms

Most opinion words are adjectives and adverbs, e.g., good, bad,

etc

There are also many subjective and opinion verbs and

nouns, e.g., hate (VB), love (VB), crap (NN).

But objective nouns can imply opinions too

E.g. “After sleeping on the mattress for one month, a body impression has formed in the middle”

How to discover such nouns in a domain or context?Slide59

Pruning

For an aspect with an implied opinion, it has a fixed opinion, either +

ve

or -

ve

, but not both.

We find two direct modification relations using a dependency parser.

Type 1: O → O-

Dep

→ A

This TV has

good

picture

quality

Type 2: O → O-

Dep

→ H ← A-

Dep

← AE.g. The springs of the mattress are badIf an aspect has mixed opinions based on the two dependency relations, prune it.Slide60

Dictionary-based methods

Typically use

WordNet’s

synsets

and hierarchies to acquire opinion words

Start with a small seed set of opinion words.

Bootstrap the set to search for synonyms and antonyms in

WordNet

iteratively (Hu and Liu, 2004; Kim and

Hovy

, 2004;

Valitutti, Strapparava and Stock, 2004; Mohammad, Dunne

and Dorr, 2009).

Kamps

et al., (2004) proposed a

WordNet

distance method to determine the sentiment orientation of a given adjectiveSlide61

Semi-supervised learning(Esuli and Sebastiani, 2005)

Use supervised learning

Given two seed sets: positive set P, negative set N

The two seed sets are then expanded using synonym and

antonymy

relations in an online dictionary to generate the expanded sets P’ and N’

P’ and N’ form the training sets

Using all the glosses in a dictionary for each term in P’



N’ and converting them to a vector

Build a binary classifier

SentiWordnetSlide62

Which approach to use?

Both corpus and dictionary based approaches are needed.

Dictionary usually does not give domain or context dependent meanings

Corpus is needed for that

Corpus-based approach is hard to find a very large set of opinion words

Dictionary is good for that

In practice, corpus, dictionary and manual approaches are all needed.Slide63

Deep Analysis for Sentiment Analysis

L’iPhone

è

il

mio

preferito

Android è

preferito

all’iPhone

Android è

meno

preferito

dell’iPhone

Il

gioco preferito per AndroidAndroid è l’obiettivo preferito dai piratiLo schermo non è tanto belloSlide64

Syntax Tree

COMP

SUBJ

PREP

MOD

MOD

PRED

ROOT

Android è l’ obiettivo preferito dai pirati informaticiSlide65

Deep AnalysisStarts from syntax tree

Identifies mentions and relations

Applies filters

Assigns scoreSlide66

Example

Mention 1:

il

prezzo

è

elevato

Concept:

prezzo

Attribute:

elevato

Value: -1.00

Mention 2:

la

qualità

è

notevole

Concept:

qualità

Attribute: elevatoValue: +4.00

SUBJ

SUBJ

CONJ

PRED

PRED

Il prezzo è elevato ma la qualità è notevoleSlide67

Spam DetectionSlide68

Opinion Spam Detection

Fake/untruthful reviews

:

Write undeserving positive reviews

for some target objects in order to promote them.

Write unfair or malicious negative reviews

for some target objects to damage their reputations.

Increasing number of customers wary of fake reviews (biased reviews, paid reviews)

Jindal and Liu, 2007, 2008Slide69

An Example Practice of Review Spam

Belkin International, Inc

Top networking and peripherals manufacturer | Sales ~ $500 million in 2008

Posted an ad for writing fake reviews on amazon.com

(

65 cents per review)

Jan 2009Slide70

Experiments with Amazon Reviews

June 2006

5.8mil reviews, 1.2mil products and 2.1mil reviewers.

A review has 8 parts

<Product ID> <Reviewer ID> <Rating> <Date> <Review Title> <Review Body> <Number of Helpful feedbacks> <Number of Feedbacks> <Number of Helpful Feedbacks>

Industry manufactured products “

mProducts”

e.g. electronics, computers, accessories, etc

228K reviews, 36K products and 165K reviewers.Slide71

Some Tentative Results

Negative outlier reviews tend to be heavily spammed

Those reviews that are the only reviews of some products are likely to be spammed

Top-ranked reviewers are more likely to be spammers

Spam reviews can get good helpful feedbacks and non-spam reviews can get bad feedbacksSlide72

Meeting Social Sciences

Extract and analyze political opinions.

Candidates and issues

Compare opinions across cultures and lang.

Comparing opinions of people from different countries on the same issue or topic, e.g.,

Internet diplomacy

Opinion spam (fake opinions)

What are social, culture, economic aspects of it?

Opinion propagation

in social contexts

How opinions on the Web influence the real world

Are they correlated?

Emotion analysis

in social context & virtual worldSlide73

WebSays + Tiscali

17/1/2013Slide74

Monitoring Brexit Referendum

Predicted by Web Analysis

Exit Polls

http://www.sense-eu.info/Slide75

Summary

We briefly defined sentiment analysis problem.

Direct opinions:

focused on feature level analysis

Comparative opinions:

different types of comparisons

Opinion spam detection

: fake reviews.

Currently working with Google

(Google research award).

A lot of applications.

Technical challenges are still huge.

But I am quite optimistic.

Interested in collaboration with social scientists

opinions and related issues are inherently social. Slide76

References

B. Liu, “Sentiment Analysis and Subjectivity.” A Chapter in

Handbook of Natural Language Processing

, 2nd Edition, 2010.

http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html