Hidden Topic Sentiment Model - PowerPoint Presentation

Slide1

Hidden Topic Sentiment Model

Md Mustafizur Rahman and Hongning WangDepartment of Computer ScienceUniversity of Virginia, Charlottesville,Virginia, VA 22903Slide2

2

I especially like its portability (3 pounds with a smallfootprint) and the speed of its solid state drive. When it comelooks you have to give it to the Inspiron. It definitely has the sleek

look of an

ultrabook

. The

combination of brushed aluminumwith black trim, keys and bezel make for a very classy, “corporate” presence. The fit and finish are first rate. However, the sound sucks . I have owned 10 notebook and laptop computers over the past two decades and this Inspiron has the worst sound of any before it. It is weak, tinny and what low end it has is muddy and indistinct. While we’ve all come to expect pretty lousy sound from notebooks, this is subpar even considering those low standards.

Overall Rating or classification

does not give enough information

Aspect Level Sentiment Analysis

What do we need?

Motivation

Observation

Method

Conclusion

Experimental

Analysis

Sentiment topic models

Motivation

Figure 1: A review from AmazonSlide3

3 Figure 2: Result of applying Joint Sentiment Topic model [1]

Topic 1sleekfitlowTopic 2worst solidsound Topic 3weak computsmall Topic 4suckmovie

half

Motivation

MethodConclusionExperimental AnalysisObservationA document is a mixture of latent sentiment [1][2]Each sentiment is a mixture of latent topics [1][2]Given the sentiment label, selection of the topics are independent over words [1][2]Sentiment-Topic Models : AssumptionsBy: Kindle Customer Date: June 25, 2014I especially like its portability (3 pounds with a smallfootprint) and the speed of its solid state drive. When it comelooks you have to give it to the Inspiron. It definitely has the sleek look of an ultrabook . The combination of brushed aluminumwith black trim, keys and bezel make for a very classy, “corporate” presence. The fit and finish are first rate. However, the sound sucks . I have owned 10 notebook and laptop computers

over the past two decades and this Inspiron has the worst

sound of any before it. It is weak, tinny

and what low end it has is muddy and indistinct. While we’ve all come to expect pretty lousy sound from notebooks, this

is subpar even considering those low standards.

Limitations

Sentence is structured & coherentTopic independency assumption fails to capture

topic coherenceSentence independency assumptions cannot handle sentiment consistencySlide4

Assumptions Sentence is the basic structure unit

All the words in a sentence share the same topicEnforces topic coherenceTopic and sentiment of current sentence influence the topic and sentiment of the next sentence Enhances topic coherenceCaptures sentiment consistency Hidden Topic Sentiment Model4MotivationObservationConclusionExperimental AnalysisMethodSlide5

5

Portability,+PortabilitySmall…Appearance,+lookclassy...Sound,-soundWorst…Document-topic proportion θPortability,+Appearance,+Sound,-0.250.250.50

P(

w|z

,

β)P,+A+S,-Sentence positionP,+A+S,-P,+A+S,-P,+A+S,-I especially like its portability (3 pounds with a smallfootprint) {portability,+} .

Aspect switch

ψ

Sentiment switch τ

Ψ

= 0,

τ

= 0Keep same topic and sentimentΨ = 0, τ = 1Keep previous sentiment, change topicΨ = 1, τ = 1Change both topic and sentimentGenerative Explanation of HTSMIt definitely has the sleek look of anultrabook {appearance,+} . However, the sound sucks {sound,-} .

I have owned 10 notebook and laptop computers over the past two decades and this Inspiron has the worst sound of any before it

{sound,-} .

Z

1

Z

2

Z

3

Z

4

Sentiment Consistency

Topic Coherence

Motivation

Observation

Conclusion

Experimental

Analysis

Method

Observable

Hidden

HiddenSlide6

6

εσψτIt definitely has the sleek look of an ultrabook

{appearance,+}

.

However, the sound sucks

{sound,-} . ObservableP,+A+S,-P,+S,-A+Sentence 1It definitely has the sleek look of an ultrabook Positive SentimentSentence 2However, the sound sucks Negative SentimentSentiWordNet

Sentiment Transition Feature fs(d,i)

Bias TermContent-based cosine similarity

sentiWordNet scoreJaccard coefficient between POS tagNegation count

Topic Transition Feature

fa(d,i)Bias

TermContent-based cosine similarity Length ratio

Relative position of sentenceCan we do better than coin tossing?MotivationObservationConclusion

Experimental Analysis

MethodSlide7

7

First order Markov modelEnsure Topic Coherence & Sentiment ConsistencyOne sentence contains one topic :Topic coherenceSentence the basic unit of HTSM

Topic and sentiment transition selector

7

Motivation

ObservationConclusionExperimental AnalysisMethodSlide8

Sentence-level topic assignment

(discrete random variable)Document-level topic proportion (continues random variable)Exact posterior inference is not feasible!Solutionapproximate posterior inference via coordinate ascentwill converge to a local maximum of data likelihood function of the document d 8Posterior Inference (, )  

Motivation

Observation

Conclusion

Experimental AnalysisMethodSlide9

Model Parameters (β,

ε, σ)Expectation Maximization (EM) algorithmRandomly initialize (βt, εt σt) at time tExpectation Step (E step):Approximate inference for (z, θ) is computed for each document with model parameters (βt, εt σt) Maximization Step (M step):maximum likelihood estimator is used to compute the model parameters at time T+1 (βT+1, εT+1, σT+1) 9

Parameter Estimation

Motivation

Observation

ConclusionExperimental AnalysisMethodSome review websites (i.e. newegg.com) provides the sentiment switch explicitly (σ), where our model can be trained in a semi-supervised mannerSlide10

Product review dataset from 2 different websites Amazon and NewEgg website

Four (4) different products (Camera, Phone, Tablet, and TV)10DatasetDatasetAmazonNewEggVocabulary SizeCamera691930201406Tv472916621410Tablet61474071515Phone68992681282

Table 1: Description of Dataset

Motivation

Observation

ConclusionExperimental AnalysisMethodSlide11

Topic EvaluationPerplexity In Information theory, perplexity is a measurement of how well a probability distribution or probability model predicts a sample

Sentiment ClassificationF-1 measure11Performance MetricsMotivationObservationConclusionExperimental AnalysisMethodSlide12

12

Number of Topics: Empirical EvidenceFigure 3: Effects of varying the number of topics on perplexity (left) and F-1 measure (right)CategoryCameraPhoneTabletTv# topics26263016Table 2 : Number of Topics

Motivation

Observation

Conclusion

Experimental AnalysisMethodSlide13

13

Topic EvaluationFigure 4 : Perplexity with increasing training size on four different review document setsMotivationObservationConclusionExperimental Analysis

MethodSlide14

14

Word Intrusion Experiment [4] Topic 1ProbabilityCharge0.150Recharge0.135Battery0.124Life0.105…Dock0.003Topic 2ProbabilityLens0.161

Screen

0.159

Display

0.157Touch0.142View0.125Topic 1BatteryDockLifeScreenRechargeChargeIntra-topic Intruding WordInter-topic Intruding WordModel generated topicsAnnotators have to find out which two words are intruding.Successful identification of intruding words means the model can generate topics that are human interpretable MotivationObservationConclusionExperimental Analysis

MethodMerged test topicSlide15

15

Topic Evaluation (Word Intrusion) Table 3: Word intrusion measurement across different topicmodels of four categories of product reviewsPerformance MetricInter-topic MRCategoryLDAHTMMASUMHTSMCamera0.1670.2180.218

0.282

Tablet

0.356

0.2560.2440.389Phone0.1920.1790.2310.333Tv0.1880.1880.2710.313Intra-topic MRCategoryLDAHTMMASUMHTSMCamera0.4740.3850.4360.346Tablet0.4780.5330.4560.522Phone0.5510.5000.4870.346Tv0.6250.6460.5630.500MotivationObservationConclusionExperimental AnalysisMethodSlide16

16

Sentiment ClassificationFigure 5 : Sentiment classification performance with increasing training size on four different review document setsMotivationObservationConclusionExperimental AnalysisMethodSlide17

17

Topic Evaluation (Topic Transition) Figure 6 : Estimated topic transition and top words under selected topic on tablet data set

When users have

positive

sentiment

When users have negative sentimentMotivationObservationConclusionExperimental AnalysisMethodSlide18

18

Aspect-Based Contrastive SummaryTable 4 : Aspect-based contrastive summarization on tablet datasetTopicSamsung Galaxy Note 10.1Amazon Kindle Fire HDX(+, battery) Battery life is very good, it is easily an all day device with wifi on and high brightness while taking notes Battery life is ok - probably need to recharge every other day with normal use(-, battery)My only issue is that it takes a long time to take a full charge and does not charge rapidly enough to use while charging, but the battery life is not badEverything works great, but the battery life is not nearly as long as advertised(+, sound)it has pretty good battery life, it also has an excellent quality sounding speakers, which I wasn't expecting on any tabletSound is really good (not home theater quality or anything) but better than any phone I've heard(-, sound)

The audio became occasionally inoperative and the headphone jack would crackle when using my ear buds

Users can get confused with volume buttons on the other side

Motivation

ObservationConclusionExperimental AnalysisMethodSlide19

19

User Study for SummarizationInterleave the summarized sentences from all modelsReduce position bias Ask annotatorsE.g. Pick two sentences POSITIVE about PRICE Aspect Performance MetricAnnotators selection of a sentence generated from a topic model, means it can pick more informative sentence than other modelsMotivationObservationConclusionExperimental Analysis

Method

Table 5: Interleaved document summarization quality test

Category

ASUMHTMMHTSMCamera0.0780.3620.560Tablet0.1530.3700.477Phone0.1180.4390.443Tv0.1730.3380.489Slide20

20

Experimental AnalysisConclusionSummary of ContributionsA unified topic model to explicitly capturetopic coherenceBy enforcing the constraint that the sentence can have one topicsentiment consistency Using the constraint that the sentiment and topic of previous sentence affects the sentiment and topic of the next sentencecapture the sentiment and topic switch from observable text reviewsFlexible modeling assumption enables both unsupervised and semi-supervised estimation of model parametersMotivationObservation

MethodSlide21

21

Future WorksCapture long-term dependencySkip-chainIncorporate document-level ratingOverall rating from 1 to 5 star ratingValue of the product user care aboutExperimental AnalysisConclusionMotivationObservationMethodSlide22

References

[1] C. Lin and Y. He. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM conference on Information and knowledge management , pages 375–384. ACM, 2009. [2] Y. Jo and A. H. Oh. Aspect and sentiment unification model for online review analysis. In Proceedings of the fourth ACM international conference on Web search and data mining , pages 815–824. ACM, 2011. [3] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. the Journal of machine Learning research, 3:993–1022, 2003. [4] J. Chang, S. Gerrish, C. Wang, J. L. Boyd-Graber, and D. M. Blei. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems, pages 288–296, 2009. [5] A. Gruber, Y. Weiss, and M. Rosen-Zvi. Hidden topic markov models. In International Conference on Artificial Intelligence and Statistics , pages 163–170, 2007 [6] T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence , pages 289–296. Morgan Kaufmann Publishers Inc., 1999 [7] Mei, Qiaozhu, et al. "Topic sentiment mixture: modeling facets and opinions in weblogs." Proceedings of the 16th international conference on World Wide Web. ACM, 2007. 22Slide23

Thank you! Contact:

mr4xb@virginia.edu 23LARA