Bayes Models Outline I. Conditional independence - PowerPoint Presentation

1. Bayes ModelsOutlineI. Conditional independence * Figures are from the textbook site.II. Naïve Bayes modelIII. Revisiting the wumpus world

2. I. Combining EvidenceWhat happens when we have two or more pieces of evidences? Suppose we know the full joint distribution. Does not scale up to a large number of evidence variables.   

3. Exponential Growth  (Boolean) evidence variables: Toothache, Catch, X-rays, Diet, …  possible combinations of observed values are needed!  Apply Bayes’ rule:namely,    // elementwise product

4. Bayes’ rule:  Conditional Independence Toothache and Catch are independent given the presence or absence of a cavity.Each is directly caused by the cavity.Neither has a direct effect on the other.Toothache depends on the state of the nerves in the tooth.The probe’s accuracy depends on the dentist’s skill irrelevant to the toothache.   Conditional independence of toothache and catch given Cavity: 

5. Definition of Conditional Independence Conditional independence of two variables and given a third variable :   // asserts independence for all possible combinations of values of// Toothache and Catch. Equivalent forms:   andDecomposition into smaller conditional assertions.  

6. Decomposition of Joint Distribution    3 tables of dimensions , and with a total of independent numbers(which appear in the first row of every table). With symptoms, the size grows as instead of   Decomposition of large probabilistic domains through conditional independence is one of the most important recent developments in AI.  Conditional independence assertions allow probabilistic systems to scale up.  They are more commonly available than absolute independence assertions.

7. II. Naive Bayes Model  often works well in cases where the “effect” variables are not strictly independent.       Full joint distribution:   Observed effects:   Unobserved effects:  

8. (cont’d)  Take each possible cause.  Multiply its prior probability by the product of the conditional probabilities of the observed effects given that cause. Normalize the result.Calculate the probability distribution of the causes from observed effects: Linear run time in the number of observed effects only. The number of unobserved effects is irrelevant no matter how large it is (as in medicine).

9. Text ClassificationProblem Given a text, decide which of a predefined set of classes or categories it belongs to.“cause” Category with range, e.g.,  “effects” : presence or absence of the th keyword.  {news, sports, business, weather, entertainment }Example sentences: 1. Stocks rallied on Monday, with major indexes gaining 1% as optimism persisted over the first quarter earnings season.2. Heavy rain continued to pound much of the east coast on Monday, with flood warnings issued in New York City and other locations..Classify each sentence into a Category.

10. Classification (cont’d) Prior probabilities: Category  Conditional probabilities:   Category  fraction of all previously seen documents that are of . Category // 9% of articles are about weather.    fraction of documents of each category that contain word .  // 37% of articles about business contain word 6, “stocks”. 

11. Classification Algorithm Check which key words (i.e., effects) appear. Compute their posterior probability distribution over categories (i.e. causes).  e.g., HasWords  appearances/disappearances of the key words.

12. Other Applications of Naïve Bayes Models Language determination (to detect the language a text is written in)  Spam filtering (to identify spam e-mails) Real-time prediction (because they are very fast) Sentiment analysis (to identify positive and negative customer sentiments in social media) Recommendation systems (to filter unseen information and predict whether a user would like a given resource or not) Medical diagnosis (which requires more sophisticated models)Naïve Bayes models are not used in

13. III. The Wumpus World RevisitedEach of the three squaresmight contain a pit. Logical inference cannot conclude about which square is most likely to be safe.  So a logical agent has no idea and has to make a random choice. We want to calculate the probability that each of squares and contains a pit.  A pit causes breeze in all neighboring squares. Each square other than contains a pit with probability 0.2.  Identify the set of random variables. true if square contains a pit.   true if square is breezy – included only for the observed squares,  

14. Full Joint Distribution    for a configuration with pits. values in the distribution, for a given pitconfiguration, are 1 if all the breezy squares among are adjacent to pits and 0 otherwise.   prior probability of a pit configuration// false 

15. Evidence  Query // how likely does [1,3] contain a pit,// given the observations so far?  Unknown: a set of 12 s for squares other than the three known ones and the query one    summation over terms (if the full joint probabilities are available).  Exponential in the number of squares! ?  

16. Irrelevant Squares? OutlineFrontier: pit variables for the squares adjacent to visited ones.and  Other: pit variables for the other unknown squares.10 other squaresUnknown Frontier Other Insight: The observed breezes are conditionally independent of Other, given Known, Frontier, and the query variable.

17. Applying Conditional Independence     is independent of Other, given known, , and Frontier.  product ruleindependent of Other not “unknown” as in the text

18. Elimination of Other Squares (other)  factoring    and  

19. Probability of Containing a Pit   In the distribution : a probability is 1 if is consistent with the values of and the variables in r, and 0 otherwise.   For each value of , we sum over the logical models for the values of variables in r that are consistent with known.  Five consistent models for Frontier :   contains a pit with 31% probability.