/
A Joint Model of Feature Mining and Sentiment Analysis for A Joint Model of Feature Mining and Sentiment Analysis for

A Joint Model of Feature Mining and Sentiment Analysis for - PowerPoint Presentation

giovanna-bartolotta
giovanna-bartolotta . @giovanna-bartolotta
Follow
391 views
Uploaded On 2016-02-19

A Joint Model of Feature Mining and Sentiment Analysis for - PPT Presentation

Jorge Carrillo de Albornoz Laura Plaza Pablo Gervás Alberto Díaz Universidad Complutense de Madrid NIL Natural Interaction based on Language 1 Jorge Carrillo de Albornoz ECIR 2011 Motivation ID: 223884

albornoz feature jorge carrillo feature albornoz carrillo jorge ecir 2011 product features set polarity reviews review sentence experiment classes

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "A Joint Model of Feature Mining and Sent..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

A Joint Model of Feature Mining and Sentiment Analysis for Product Review Rating

Jorge Carrillo de AlbornozLaura PlazaPablo Gervás Alberto Díaz

Universidad Complutense de MadridNIL (Natural Interaction based on Language)

1

Jorge Carrillo de Albornoz - ECIR 2011Slide2

Motivation

Product review forums have become commonplaceReviews are of great interestCompanies use them to exploit their marketing-mixIndividuals are interested in others’ opinions when purchasing a productManual analysis is unfeasible

Typical NLP tasks:Subjective detectionPolarity recognitionRating inference, etc.Jorge Carrillo de Albornoz - ECIR 20112Slide3

Motivation

Traditional approaches:Term frequencies, POS, etc.Polar expressionsThey do not take into account:The product features on which the opinions are expressedThe relations between them

Jorge Carrillo de Albornoz - ECIR 20113Slide4

Hypothesis

Humans have a conceptual model of what is relevant regarding a certain productThis model influences the polarity and strength of their opinionsIt is necessary to combine feature mining and sentiment analysis strategies toAutomatically extract the important featuresQuantify the strength of the opinions about such features

Jorge Carrillo de Albornoz - ECIR 20114Slide5

The HotelReview Corpus

25 reviews from 60 different hotels (1500 reviews)Each review:The cityThe reviewer nationalityThe date

The reviewer categoryA score in 0-10 ranking the opinionA free-text describing, separately, what the reviewer liked and disliked5Jorge Carrillo de Albornoz - ECIR 2011Slide6

The HotelReview Corpus

No relation between the score and the text describing the user opinion:

Two annotatorsExcellent, Good, Fair, Poor and Very poorGood, Fair and PoorAfter removing conflicting judgments =1000 reviewsJorge Carrillo de Albornoz - ECIR 20116Good location. Nice roof restaurant - (I have stayed in the baglioni more than 5 times before). Maybe reshaping/redecorating the lobby.Noisy due to road traffic. The room was extremely small. Parking awkward. Shower screen was broken and there was no bulb in the bedside light.Slide7

The HotelReview Corpus

Jorge Carrillo de Albornoz - ECIR 20117

Download: http://nil.fdi.ucm.es/index.php?q=node/456Slide8

Automatic Product Review Rating

Step I: Detecting Salient Product FeaturesIdentifying the features that are relevant to consumersStep II:

Extracting the User OpinionExtracting from the review the opinions expressed on such featuresStep III: Quantifying the User OpinionsPredicting the polarity of the sentences associated to each featureStep IV: Predicting the Rating of a ReviewTranslating the product review into a Vector of Feature Intensities (VFI)Jorge Carrillo de Albornoz - ECIR 20118Slide9

Step I: Detecting Salient Product Features

Objective: Identifying the product features that are relevant to consumers Given a set of reviews R={r

1, r2, …, rn}:The set of reviews is represented as a graphVertices = conceptsEdges = is a + semantic similarity relationsThe concepts are ranked according to its salience and a degree-based clustering algorithm is executedThe result is a number of clusters where each cluster represent a product featureJorge Carrillo de Albornoz - ECIR 20119Slide10

Step II: Extracting the User Opinion on Each Product Feature

Objective: Locating in the review all textual mentions related to each product featureMapping the reviews to WordNet concepts

Associating the sentences to feature clusters:Most Common Feature (MCF): more WordNet concepts in commonAll Common Features (ACF): every feature with some concept in commonMost Salient Feature (MSF): the sentence is associated to the highest score featureJorge Carrillo de Albornoz - ECIR 201110Slide11

Step III: Quantifying the User Opinions

Objective: Quantifying the opinion expressed by the reviewer on the different product featuresClassifying the sentences of each review into positive or negativeAny polarity classification system may be usedOur system:

Concepts rather than termsEmotional categoriesNegations and quantifiersJorge Carrillo de Albornoz - ECIR 201111Slide12

Step IV: Predicting the Rating of a Review

Objective: Aggregate all previous information to provide an overall rating for the reviewMapping the product review to a VFIA VFI is a vector of N+1 values representing the detected features and the

other featureTwo strategies for assigning values to the VFI:Binary Polarity (BP): the position in the VFI of the feature assigned to each sentence is increased or decreased in one according to the polarity of the sentenceProbability of Polarity (PP): the feature position is increased or decreased with the probability calculated by the classifierJorge Carrillo de Albornoz - ECIR 201112[-1.0, 0.0, 0.0, 0.0, …,-1.0, 0.0, 0.0,1.0, 0.0, 1.0, 0.0, …., 1.0]Slide13

Experimental Setup

HotelReview corpus: 1000 reviewsDifferent feature sets:Feature set 1: 50 reviews  24 feature clusters and 114 concepts

Feature set 2: 1000 reviews  18 feature clusters and 330 conceptsFeature set 3: 1500 reviews  18 feature clusters and 353 conceptsBaselines:Carrillo de Albornoz et al. (2010)Pang et al. (2002)Jorge Carrillo de Albornoz - ECIR 201113Slide14

Experiment 1

Objectives: To examine the effect of the product feature setTo determine the best heuristic for sentence-to-feature assignment (

Most Common Feature, All Common Features and Most Salient Feature)Task: Three classes classification (Poor, Fair and Good)We use the Binary Polarity strategy for assigning values to the VFI vectorJorge Carrillo de Albornoz - ECIR 201114Slide15

Experiment 1 - Results

Method

Feature set 1Feature set 2Feature set 3MCFACFMSFMCFACFMSFMCFACFMSFLogistic 69.8 67.7

69.8 70.4

67.4

70.8 69.1 67.4

70

LibSVM

69

67.1

69.2

69

67.8

69.2

68.8

67.7

69

FT

66.8

64.2

66.8

66.3

65.2

68.6

68.4

65.8

68.4

Jorge Carrillo de Albornoz - ECIR 2011

15

Average accuracies for different classifiers, using different feature sets

and

sentence-to-feature assignment strategiesSlide16

Experiment 1 - Discussion

Feature set 2 reports the best results for all classifiersAccuracy differs little across different feature sets and increasing the number of reviews used for extracting the features does not always improve accuracyThis is due to the fact that users are concerned about a small set of features which are also quite consistent among users

Jorge Carrillo de Albornoz - ECIR 201116Slide17

Experiment 1 - Discussion

The Most Salient Feature (MSF) heuristic for sentence-to-feature assignment produces the best outcomeThe Most Common Feature (MCF) heuristic reports very close results

But the All Common Features (ACF) one behaves significantly worseIt seems that only the main feature in each sentence provides useful information for the taskJorge Carrillo de Albornoz - ECIR 201117Slide18

Experiment 1I

Objectives:To check if the Probability of Polarity strategy produces better results than the Binary Polarity strategy

Test the system in a 5-classes prediction taskTasks: Three classes classification (Poor, Fair and Good)Five classes classification (Very Poor, Poor, Fair, Good and Excellent)We use the Feature set 2 and the MSF strategy for these experimentsJorge Carrillo de Albornoz - ECIR 201118Slide19

Experiment 1I - Results

Jorge Carrillo de Albornoz - ECIR 201119

Method3-classes5-classesLogistic71.7 46.9LibSVM69.4 45.3FT 66.9 43.7

Carrillo de Albornoz et al. [9]66.7

43.2

Pang

et al. [4]

54.2

33.5

Average accuracies for different classifiers in the 3-classes and 5-classes prediction task.Slide20

Experiment 1I - Discussion

The Probability of Polarity behaves significantly better than the Binary Polarity strategyIt allows to captures the degree of negativity/positivity of a sentence, not only its polarity

It is clearly not the same to say The bedcover was a bit dirty than The bedcover was terribly dirtyJorge Carrillo de Albornoz - ECIR 201120Slide21

Experiment 1I - Discussion

The results in the 5-classes prediction task are considerably lower than in the 3-classes taskThis was expected:The task is more difficultThe borderline between

Poor-Very poor and Good-Excellent instances is fuzzyOur system significantly outperforms both baselines in all tasksJorge Carrillo de Albornoz - ECIR 201121Slide22

Conclusions and Future Work

The system performs significantly better than previous approachesThe product features have different impact on the user opinionUsers are concerned about a relatively small set of product featuresThe salient features can be easily obtained from a relatively small set of product reviews and without previous knowledge

Differences between the various Weka classifiers are not markedJorge Carrillo de Albornoz - ECIR 201122Slide23

Conclusions and Future Work

Error propagation of the sentence polarity classifierError assigning sentences to featuresNot enough information:Dirty. Stinky. Unfriendly. NoisyCo-reference problem:Anyway, everybody else was nice

To evaluate the system over other domainsTo translate the system to other languageJorge Carrillo de Albornoz - ECIR 201123Slide24

Thank you!

Any question?Jorge Carrillo de Albornoz - ECIR 2011

24