Reasoning Under Uncertainty: - PowerPoint Presentation

Slide1

Reasoning Under Uncertainty: Independence

Jim Little

Uncertainty

3

Nov 5, 2014

Textbook §6.2Slide2

Lecture OverviewRecapConditioning & Inference by EnumerationBayes Rule & The Chain Rule

Independence

Marginal Independence

Conditional Independence

2Slide3

Recap: ConditioningConditioning: revise beliefs based on new observationsWe need to integrate two sources of knowledge

Prior probability distribution P(X)

: all background knowledge New

evidence eCombine the two to form a posterior probability distribution The conditional probability P(h|e)

3Slide4

Recap: Example for conditioningYou have a prior for the joint distribution of weather and temperature, and the marginal distribution of temperature

Now, you look outside and see that it

’

s sunny

You are certain that you’re in world w1, w2, or w3

4

Possible world

Weather

Temperature

µ(w)

w

1

sunnyhot0.10w2sunnymild0.20w3sunnycold0.10w4cloudyhot0.05w5cloudymild0.35w6cloudycold0.20

T

P(T|W=sunny)

hot

?

mild

?

cold

?Slide5

Recap: Example for conditioningYou have a prior for the joint distribution of weather and temperature, and the marginal distribution of temperature

Now, you look outside and see that it

’

s sunny

You are certain that you’re in world w1, w2, or w3 To get the conditional probability, you simply renormalize to sum to 10.10+0.20+0.10=0.40

5

Possible world

Weather

Temperature

µ(w)

w

1

sunnyhot0.10w2sunnymild0.20w3sunnycold0.10w4cloudyhot0.05w5cloudymild0.35w6cloudycold0.20

T

P(T|W=sunny)

hot

0.10

/

0.40

=0.25

mild

0.20

/

0.40

=0.50

cold

0.10

/

0.40

=0.25Slide6

Recap: Conditional probability 

6

Possible world

Weather

Temperature

µ(w)

w

1

sunny

hot

0.10

w2sunnymild0.20w3sunnycold0.10w4cloudyhot0.05w5cloudymild0.35w6cloudycold0.20TP(T|W=sunny)

hot

0.10

/

0.40

=

0.25

mild

0.20/0.40=0.50

cold

0.10/0.40=0.25Slide7

Recap: Inference by EnumerationGreat, we can compute arbitrary probabilities now!Given Prior joint probability distribution (JPD) on set of variables X

specific values e for the evidence variables E (subset of X)

We w

ant to compute

posterior joint distribution of query variables Y (a subset of X) given evidence eStep 1: Condition to get distribution P(X|e)Step 2: Marginalize to get distribution P(Y|e)Generally applicable, but memory-heavy and slow

7Slide8

Recap: Bayes rule and Chain Rule

8

 Slide9

Lecture OverviewRecapConditioning & Inference by EnumerationBayes Rule & The Chain Rule

Independence

Marginal Independence

Conditional Independence

9Slide10

Marginal Independence: exampleSome variables are independent:Knowing the value of one does not tell you anything about the otherExample: variables W (weather) and R (result of a die throw)Let

’

s compare P(W) vs. P(W | R = 6 )

10

Weather W

Result R

P(W,R)

sunny

1

0.066

sunny

2

0.066sunny30.066sunny40.066sunny50.066sunny60.066cloudy10.1cloudy20.1cloudy30.1cloudy40.1cloudy50.1cloudy60.1B0.1A 0.066C0.4D0.6Slide11

Marginal Independence: example

11

Weather W

Result R

P(W,R)

sunny

1

0.066

sunny

2

0.066

sunny

30.066sunny40.066sunny50.066sunny60.066cloudy10.1cloudy20.1cloudy30.1cloudy40.1cloudy50.1cloudy60.1Some variables are independent:Knowing the value of one does not tell you anything about the otherExample: variables W (weather) and R (result of a die throw)Let’s compare P(W) vs. P(W | R = 6 )What is P(W=cloudy) ?P(W=cloudy) = rdom(R) P(W=cloudy, R = r) = 0.1+0.1+0.1+0.1+0.1+0.1 = 0.6Slide12

Marginal Independence: exampleSome variables are independent:Knowing the value of one does not tell you anything about the otherExample: variables W (weather) and

R (result of a die throw)

Let

’

s compare P(W) vs. P(W | R = 6 )The two distributions are identicalKnowing the result of the die does not change our belief in the weather

12

Weather W

Result R

P(W,R)

sunny

1

0.066

sunny20.066sunny30.066sunny40.066sunny50.066sunny60.066cloudy10.1cloudy20.1cloudy30.1cloudy40.1cloudy50.1cloudy60.1Weather WP(W)sunny0.4cloudy0.6Weather WP(W|R=6)sunny0.066/0.166=0.4cloudy0.1/0.166=0.6Slide13

Marginal IndependenceIntuitively: if X and Y are marginally independent, thenlearning that Y=y does not change your belief in Xand this is true for all values y that Y could take

For example, weather is marginally independent

of the result of a dice throw

13

 Slide14

Examples for marginal independenceResults C1 and C2 of

two tosses of a fair coin

Are C

1

and C2 marginally independent?

14

C

1

C

2

P(

C

1 , C2)headsheads0.25headstails0.25tailsheads0.25tailstails0.25 Slide15

Examples for marginal independenceResults C1 and C2 of

two tosses of a fair coin

Are C

1 and C

2 marginally independent?Yes. All probabilities in the definition above are 0.5.

15

C

1

C

2

P(

C

1 , C2)headsheads0.25headstails0.25tailsheads0.25tailstails0.25 Slide16

Examples for marginal independenceAre Weather and Temperaturemarginally independent?

Weather W

Temperature T

P(W,T)

sunny

hot

0.10

sunny

mild

0.20

sunny

cold0.10cloudyhot0.05cloudymild0.35cloudycold0.20Slide17

Examples for marginal independenceAre Weather and Temperaturemarginally independent?No. We saw before that knowingthe Weather changes ourbelief about the TemperatureE.g. P(hot) = 0.10+0.05=0.15

P(

hot|cloudy

) = 0.05/0.6  0.083

 Weather WTemperature T

P(W,T)

sunny

hot

0.10

sunny

mild

0.20

sunnycold0.10cloudyhot0.05cloudymild0.35cloudycold0.20Slide18

Examples for marginal independenceIntuitively (without numbers):Boolean random variable “Canucks win the Stanley Cup this season”

Numerical random variable

“

Canucks

’ revenue last season”

18

 Slide19

Examples for marginal independenceIntuitively (without numbers):Boolean random variable “Canucks win the Stanley Cup this season”

Numerical random variable

“

Canucks

’ revenue last season” ?

19

 Slide20

Exploiting marginal independence 

20Slide21

Exploiting marginal independence 

21Slide22

Exploiting marginal independence

22Slide23

Exploiting marginal independence 

23Slide24

Lecture OverviewRecapConditioning & Inference by EnumerationBayes Rule & The Chain Rule

Independence

Marginal Independence

Conditional Independence

24Slide25

Follow-up ExampleIntuitively (without numbers):Boolean random variable “Canucks win the Stanley Cup this season

”

Numerical random variable

“Canucks

’ revenue last season” ?Are the two marginally independent? No! Without revenue they cannot afford to keep their best playersBut they are conditionally independent given the Canucks line-upOnce we know who is playing then learning their revenue last yearwon’t change our belief in their chances

25Slide26

Conditional Independence

26

Intuitively: if X and Y are conditionally independent given Z, then

learning that Y=y does not change your belief in X when we already know Z=z

and this is true for all values y that Y could take

and all values z that Z could takeSlide27

Example for Conditional IndependenceWhether light l1 is lit is conditionally independent from the position of switch s2

given whether there is power in wire w

0

Once we know Power(w

0) learning values for any other variable will not change our beliefs about Lit(l1)I.e., Lit(l1) is independent of any other variable given Power(w0)

27Slide28

Example: conditionally but not marginally independentExamGrade and AssignmentGrade are not marginally independentStudents who do well on one typically do well on the other

But conditional on UnderstoodMaterial, they are independent

Variable UnderstoodMaterial is a

common cause of

variables ExamGrade and AssignmentGradeUnderstoodMaterial shields any information we could get from AssignmentGrade

28

UnderstoodMaterial

Assignment Grade

Exam

GradeSlide29

Example: marginally but not conditionally independentTwo variables can be marginallybut not conditionally independent“Smoking At Sensor

”

S: resident smokes cigarette next to fire sensor

“Fire

” F: there is a fire somewhere in the building“Alarm” A: the fire alarm ringsS and F are marginally independentLearning S=true or S=false does not change your belief in FBut they are not conditionally independent given alarmIf the alarm rings and you learn S=true your belief in F decreases

29

Alarm

Smoking At Sensor

FireSlide30

Conditional vs. Marginal IndependenceTwo variables can be Both marginally and conditionally independentCanucksWinStanleyCup and Lit(l1)CanucksWinStanleyCup and Lit(l1) given Power(

w

0

)Neither marginally nor conditionally independentTemperature and CloudinessTemperature and Cloudiness

given WindConditionally but not marginally independentExamGrade and AssignmentGradeExamGrade and AssignmentGrade given UnderstoodMaterialMarginally but not conditionally independentSmokingAtSensor and FireSmokingAtSensor and Fire given Alarm

30Slide31

Exploiting Conditional IndependenceExample 1: Boolean variables A,B,CC is conditionally independent of A given BWe can then rewrite P(C | A,B) as P(C|B)

 Slide32

Exploiting Conditional IndependenceExample 2: Boolean variables A,B,C,DD is conditionally independent of A given CD is conditionally independent of B given CWe can then rewrite P(D | A,B,C) as P(D|B,C)

And can further rewrite P(D|B,C) as P(D|C)

 Slide33

Exploiting Conditional IndependenceRecall the chain rule

33

 Slide34

Define and use marginal independenceDefine and use conditional independenceAssignment 4 available on ConnectDue in 2.5 weeks

Do the questions

early

Right after the material for the question has been covered in class

This will help you stay on track

34

Learning Goals For Today

’

s Class