Probability Review Friday 115 Gates B03 Late days To be fair Start the pset early double late days Where we are Machine Learning Variable Based Search CS221 Machine Learning Variable Based ID: 933196
Download Presentation The PPT/PDF document "Generative Models Announcements" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Generative Models
Slide2Slide3Announcements
Probability Review (Friday, 1:15 Gates B03)
Late days…
To be fair…
Start the p-set early
double late days.
Slide4Where we are
Slide5Machine Learning
Variable Based
Search
CS221
Slide6Machine Learning
Variable Based
Search
CS221
Slide7Machine Learning
Search
Variable Based
CS221
Slide8Slide9Slide10Where We Left
O
ff
Slide11Where We Left
O
ff
Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged
0.108
0.012
0.072
0.008
Not Drugged
0.016
0.064
0.144
0.576
Slide12Key Idea
If we have a joint distribution over all variables, then given evidence (which could be multiple variables)
E =
e
, we can find the probability of any query variable X
=
x
.
Slide13These are values in our table!
Y is all variables that aren’t in X or E
Y is all variables that aren’t in E
Key Idea
If we have a joint distribution over all variables, then given evidence (which could be multiple variables)
E =
e
, we can find the probability of any query variable X
=
x
.
Slide14Key Idea
If we have a joint distribution over all variables, then given evidence (which could be multiple variables)
E =
e
, we can find the probability of any query variable X
=
x
.
Since we know that p(x | e)’s must sum to 1
Slide15Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged0.108
0.0120.0720.008
Not Drugged0.016
0.0640.1440.576
Key Idea
Slide16Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged0.108
0.0120.0720.008
Not Drugged0.016
0.0640.1440.576
Key Idea
Slide17Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged
0.108
0.0120.072
0.008Not Drugged0.016
0.0640.1440.576
Key Idea
Slide18Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged
0.108
0.0120.072
0.008Not Drugged0.016
0.0640.1440.576
Key Idea
Slide19Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged0.108
0.0120.0720.008
Not Drugged0.016
0.0640.1440.576
Key Idea
Slide20Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged0.108
0.0120.0720.008
Not Drugged0.016
0.0640.1440.576
Key Idea
Slide21Key Idea
Slide22Our joint
gets
too big
Slide23Where We Left
O
ff
Loopy
Not
loopy
Purple
Not Purple
Purple
Not Purple
Drugged
0.108
0.012
0.072
0.008
Not Drugged
0.016
0.064
0.144
0.576
Add variable
Snowden location: {
Hong Kong,
Sao Paulo,
Moscow,
Nairobi,
Caracas,
Guantanamo
}
Size of the table is now 2*2*2*6 = 48
But what does Snowden have to do with drugged out
rockstars
?
Really are independent…
Joint is exponential in size.
Slide24Independence
l
= loopy
p
= purple
d
= drugged
s
=
snowden
If we have two tables, one over
l, p, d and one for s, we could recreate the joint.
Slide25What else is independent?
Snowden
Drugged
Purple
Loopy
Slide26What else is independent?
Snowden
Drugged
Purple
Loopy
Purple and loopy?
Slide27What else is independent?
Snowden
Drugged
Purple
Loopy
Both caused by drugged
Slide28What else is independent?
Snowden
Drugged
Purple
Loopy
If you know drugged, purple and loopy are independent!
Slide29Conditional Independence
If you know drugged, purple and loopy are independent!
Slide30If you know drugged, purple and loopy are independent!
Conditional Independence
Joint
Slide31This is important!
Slide32If you know drugged, purple and loopy are independent!
Conditional Independence
Joint
Slide33If you know drugged, purple and loopy are independent!
Conditional Independence
Joint
Slide34Drugged
Purple
Loopy
No longer need the full joint.
Conditional Independence
Slide35We only need p(
var
| causes) for each var.
Slide36Model the world with variables
Slide37And what causes what
Slide38Bayesian Network
Slide39Bayesian Network
Slide40Bayesian Network
Cough
Fever
Vomit
Flu
Stomach Bug
Slide41Bayesian Network
Cough
Fever
Vomit
Flu
Stomach Bug
Slide42Bayesian Network
Cough (c)
Fever (t)
Vomit (v)
Flu (f)
Stomach bug (s)
Slide43Bayesian Network
Cough (c)
Vomit (v)
Flu (f)
Stomach bug (s)
Joint
Fever (t)
Slide44Bayesian Network
Joint
Slide45Bayesian Network
Cough (c)
Fever (t)
Vomit (v)
Flu (f)
Stomach bug (s)
Joint
Slide46Definition
:
Bayes Net
=
DAG
DAG
: directed acyclic graph (BN’s
structure
)
Nodes
: random variables (typically
discrete
, but methods also exist to handle continuous variables)
Arcs
: indicate probabilistic dependencies between
nodes. Go from cause to effect.
CPDs
:
conditional probability distribution (BN’s
parameters
) Conditional
probabilities at each node, usually stored as a table (conditional probability table, or
CPT)Root nodes are a special case – no parents, so just use priors in CPD:
Formally
Slide47Slide48What does NSA do with our data?
Slide49Real World Problem
Formal Problem
Solution
Model the problem
Apply an Algorithm
Evaluate
The AI Pipeline
Slide50Slide51Live Research
Slide52Research Project
g
3
t
1
t
2
t
3
e
1
e
2
e
3
g
1
g
2
b
i
?
Slide53Research Project
g
3
t
1
t
2
t
3
e
1
e
2
e
3
g
1
g
2
b
i
?
Slide54Research Project
g
1
g
1
*
?
Slide55Modeling Surprise
g
1
g
1
*
?
Slide56Competition
Chose top 5
Test how well they predict grades
Select a finalist (gets +)
TA Review
Actually re-grade
Publish?
Slide57On worst
pset
question
Prize
+
Due Tuesday before class (email staff. Subject: Modeling
Regrades
)
Slide58Novel Science
Slide59http://vimeo.com/60381274
Slide60What does NSA do with our data?
Slide61Slide62Research Project
g
3
t
1
t
2
t
3
e
1
e
2
e
3
g
1
g
2
b
i
?
Slide63Can someone fix this?
Slide64Peer Graders