November 10 2009 Erick Gong Thanks to Null amp Miguel Agenda Class Scheduling DiffinDiff Math amp Graphs Case Study STATA Help Class Scheduling Nov 10 DiffinDiff Nov 17 Power Calculations amp Guest Speaker ID: 675785
Download Presentation The PPT/PDF document "Differences-in-Differences" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Differences-in-Differences
November
10, 2009
Erick Gong
Thanks
to Null
&
Miguel Slide2
Agenda
Class Scheduling
Diff-in-Diff (Math & Graphs)
Case Study
STATA HelpSlide3
Class Scheduling
Nov 10: Diff-in-Diff
Nov 17: Power Calculations & Guest Speaker
Nov 24: Class poll: Who will be here?
Dec 1: Review & Presentations
Class Poll: Who will be presenting their research proposals? Slide4
The Big Picture
What is this class really about, anyway?Slide5
The Big Picture
What is this class really about, anyway?
CausalitySlide6
The Big Picture
What is this class really about, anyway?
Causality
What is our biggest problem?Slide7
The Big Picture
What is this class really about, anyway?
Causality
What is our biggest problem?
Omitted variable biasSlide8
Omitted Variable Bias
The actual cause is unobserved
e.g. higher wages for educated actually caused by motivation, not schooling
Happens when people get to choose their own level of the “treatment” (broadly construed)
Selection bias
Non-random program placement
Because of someone else’s choice, “control” isn’t a good
counterfactual
for treatedSlide9
Math Review
(blackboard)Slide10
Math Review
for those of you looking at these slides later, here’s what we just wrote down:
(1) Yi = a +
bTi
+ cXi + ei
(2) E(Yi | Ti=1) – E(Yi | Ti=0)
=
[
a + b + cE(Xi | Ti=1) + E(ei | Ti=1)
]
–
[
a + 0 + cE(Xi | Ti=0) + E(ei | Ti=0)
]
= b + c
[
E(Xi | Ti=1) – E(Xi | Ti=0)
]
True effect
“Omitted variable/selection bias” termSlide11
What if we had data from before the program?
What if we estimated this equation using data from before the program?
(1) Yi = a + bTi + cXi + ei
Specifically, what would our estimate of b be?
Slide12
What if we had data from before the program?
What if we estimated this equation using data from before the program?
(1) Yi = a +
bTi
+
cXi
+
ei
(2)
E(Y
i0
| T
i1
=1
) –
E(Y
i0
| T
i1
=0
)
=
[
a +
0
+
cE
(X
i0
|
T
i1
=1
) +
E(e
i0
| T
i1
=1
)
]
–
[
a + 0 +
cE
(X
i0
| T
i1
=0
) +
E(e
i0
| T
i1
=0
)
]
= c
[
E(Xi | Ti=1) – E(Xi | Ti=0)
]
“Omitted variable/selection bias” term
ALL THAT’S LEFT IS THE PROBLEMATIC TERM – HOW COULD THIS BE HELPFUL TO US?Slide13
Differences-in-Differences
(just what it sounds like)
Use two periods of data
add second subscript to denote time
= {E(Y
i1
| T
i1
=1) – E(Y
i1
| T
i1
=0)}
(difference
btwn
T&C, post)
– {E(Y
i0
| T
i1
=1) – E(Y
i0
| T
i1
=0)}
– (difference
btwn
T&C, pre)
=
b + c
[
E(X
i1
| T
i1
=1) – E(X
i1
| T
i1
=0)
]
–
c
[
E(X
i0
| T
i1
=1) – E(X
i0
| T
i1
=0)
]Slide14
Differences-in-Differences
(just what it sounds like)
Use two periods of data
add second subscript to denote time
= {E(Y
i1
| T
i1
=1) – E(Y
i1
| T
i1
=0)}
(difference
btwn
T&C, post)
– {E(Y
i0
| T
i1
=1) – E(Y
i0
| T
i1
=0)}
– (difference
btwn T&C, pre) = b + c [E(Xi1 | Ti1=1) – E(Xi1 | Ti1=0)] – c [E(Xi0 | Ti1=1) – E(Xi0 | Ti1=0)] = b YAY!Assume differences between X don’t change over time.Slide15
Differences-in-Differences, Graphically
Pre
Post
Treatment
ControlSlide16
Differences-in-Differences, Graphically
Pre
Post
Effect of program using only pre- & post- data from T group (ignoring general time trend).Slide17
Differences-in-Differences, Graphically
Pre
Post
Effect of program using only T & C comparison from post-intervention (ignoring pre-existing differences between T & C groups).Slide18
Differences-in-Differences, Graphically
Pre
PostSlide19
Differences-in-Differences,
Graphically
Pre
Post
Effect of program difference-in-difference (taking into account pre-existing differences between T & C and general time trend).Slide20
Identifying Assumption
Whatever happened to the control group over time is what would have happened to the treatment group in the absence of the program.
Pre
Post
Effect of program difference-in-difference (taking into account pre-existing differences between T & C and general time trend).Slide21
Graphing Exercise
Form Groups of 3-4
4 Programs
Pre-Post Treatment Effect
Take the difference of post-treatment outcome vs. pre-treatment outcome
Post-intervention (Treatment vs. Control) Comparison
Circle what you think is pre-post effect and post-intervention treat vs. control effect
Ask group volunteersSlide22
Uses of Diff-in-Diff
Simple two-period, two-group comparison
very useful in combination with other methodsSlide23
Uses of Diff-in-Diff
Simple two-period, two-group comparison
very useful in combination with other methods
Randomization
Regression Discontinuity
Matching (propensity score)Slide24
Uses of Diff-in-Diff
Simple two-period, two-group comparison
very useful in combination with other methods
Randomization
Regression Discontinuity
Matching (propensity score)
Can also do much more complicated “cohort” analysis, comparing many groups over many time periodsSlide25
The (Simple) Regression
Y
i,t
= a + bTreat
i,t
+ cPost
i,t
+ d(Treat
i,t
Post
i,t
)+ e
i,t
Treat
i,t
is a binary indicator (“turns on” from 0 to 1) for being in the treatment group
Post
i,t
is a binary indicator for the period after treatment
and
Treat
i,t
Post
i,t
is the interaction (product)
Interpretation of
a, b, c, d is “holding all else constant”Slide26
Putting Graph & Regression Together
Pre
Post
Y
i,t
= a + bTreat
i,t
+ cPost
i,t
+ d(Treat
i,t
Post
i,t
)+ e
i,t
d
is the causal effect of treatment
a
a + c
a + b
a + b + c + dSlide27
Putting Graph & Regression Together
Pre
Post
Y
i,t
= a +
bTreat
i,t
+
cPost
i,t
+ d(
Treat
i,t
Post
i,t
)+
e
i,t
a
a + c
a + b
a + b + c + d
Single Diff 2= (
a+b+c+d
)-(
a+c
) = (
b+d
)
Single Diff 1= (
a+b
)-(a)=bSlide28
Putting Graph & Regression Together
Pre
Post
Y
i,t
= a +
bTreat
i,t
+
cPost
i,t
+ d(
Treat
i,t
Post
i,t
)+
e
i,t
Diff-in-Diff=(Single Diff 2-Single Diff 1)=(
b+d
)-b=d
a
a + c
a + b
a + b + c + d
Single Diff 2 = (
a+b+c+d
)-(
a+c
) = (
b+d
)
Single Diff 1= (
a+b
)-(a)=bSlide29
Cohort Analysis
When you’ve got richer data, it’s not as easy to draw the picture or write the equations
cross-section (lots of individuals at one point in time)
time-series (one individual over lots of time)
repeated cross-section (lots of individuals over several times)
panel (lots of individuals, multiple times for each)
Basically, control for each time period and each “group” (fixed effects) – the coefficient on the treatment dummy is the effect you’re trying to estimateSlide30
DiD Data Requirements
Either repeated cross-section or panel
Treatment can’t happen for everyone at the same time
If you believe the identifying assumption, then you can analyze policies ex post
Let’s us tackle really big questions that we’re unlikely to be able to randomizeSlide31
Malaria Eradication in the Americas (Bleakley 2007)
Question: What is the effect of malaria on economic development
?
Data: Malaria Eradication in United States South (1920’s) Brazil, Colombia, Mexico (1950’s)
Diff-in-Diff: Use birth cohorts (old people vs. young people) & (regions with lots of malaria vs. little malaria)
Idea: Young Cohort X Region w/malaria
Result: This group higher income & literacySlide32
What’s the intuition
Areas with high pre-treatment malaria will most benefit from malaria eradication
Young people living in these areas will benefit most (older people might have partial immunity)
Comparison Group: young people living in low pre-treatment malaria areas (malaria eradication will have little effect here)Slide33
Robustness Checks
If possible, use data on multiple pre-program periods to show that difference between treated & control is stable
Not necessary for trends to be parallel, just to know function for each
If possible, use data on multiple post-program periods to show that unusual difference between treated & control occurs only concurrent with program
Alternatively, use data on multiple indicators to show that response to program is only manifest for those we expect it to be (e.g. the diff-in-diff estimate of the impact of ITN distribution on diarrhea should be zero)Slide34
Intermission
Come back if intro to PS4
STATA tipsSlide35
Effect of 2ndary School Construction in Tanzania
Villages
“Treatment Villages” got 2ndary schools
“Control Villages” didn’t
Who benefits from 2ndary schools?
Young People benefit
Older people out of school shouldn’t benefit
Effect: (Young People X Treatment Villages)