Statistical Methods for Observational Studies - PowerPoint Presentation

Slide1

Statistical Methods for Observational StudiesLecture 3 (of 4)

Steve Fienberg Memorial Lectures Series in Advanced AnalyticsNovember 2018

Dylan SmallUniversity of Pennsylvania

Slides posted at my web site: www-stat.wharton.upenn.edu/~

dsmallSlide2

Treatment (C-section vs. vaginal delivery)

Outcome (Mortality)

Unmeasured Confounders

Measured Confounders

IV (Hospital rate of

C-section)

1

2

3

IV Assumptions:

IV affects the treatment;

IV is independent of unmeasured confounders;

IV has no direct effect on the outcome

(exclusion restriction) Slide3

Treatment (C-section vs. vaginal delivery)

Outcome (Mortality)

Unmeasured Confounders

Measured Confounders

IV (Hospital rate of

C-section)

Subgroup Unaffected by Treatment

:

Suppose there is a subgroup known to be unaffected by the treatment.

If IV affects outcome in this subgroup, then it provides

evidence that IV assumptions are violated.

?

?Slide4

Testing the Exclusion RestrictionPresuming the literature’s claim that C-section has no effect for 30-34 week babies, we tested the exclusion restriction by redoing our study for 30-34 week babies. If literature’s claim is true and exclusion restriction is true, IV should have no effect on mortality for 30-34 week babies.We matched 30-34 babies delivered in high C-section rate and low C-section rate hospitals on the measured covariates as before, producing 23,631 pairs of 30-34 week babies.Slide5

Matching Balanced the Measured Covariates for 30-34 Week BabiesSlide6

Effect of IV on Mortality for 30-34 Week BabiesMortality rates are higher in the low C-section hospitals.Odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital for 30-34 week babies: 0.62 (95% CI: 0.58, 0.69)

[Compare with for 23-24 week babies, odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital: 0.51 (95% CI: 0.34, 0.69)].Slide7

AporiaOur results: Using hospital C-section rate as an IV, C-sections increase mortality for both 23-24 week babies and older 30-34 week babies.Previous literature, based on adjusting for measured confounders, claims that C-sections do not increase mortality for 30-34 week babies.Each of these claims sound plausible and reasonable if taken one at a time, but they cannot all be correct.We have produced an aporia

. (Oxford American Dictionary): “an aporia is an irresolvable [without more data], internal contradiction…in a text, argument or theory” with aporetic as an adjective.

A collection of propositions is an aporia if each is plausible on its own but they are jointly inconsistent, that is, is false or implausible.Slide8

A special case of an aporia is used when doing a proof by contradiction: One proves by showing that are certainly true and are an aporia.Slide9

In contrast, in a typical aporia, the identity of the culpable proposition(s) is unknown.In Plato’s early dialogues, Socrates would invalidate the views of his opponents by showing

that those views were an aporia.Slide10

Value of Finding an Aporia ContinuedSocrates, in Plato’s Meno, says of a befuddled young interlocutor who he put in an aporia:

At first he did not know what [he thought he knew], and he does not know even now: but at any rate he thought he knew then, and confidently answered as though he knew, and was aware of no difficulty; whereas now he feels the difficulty he is in, and besides not knowing does not think he knows…. [W]e have certainly given him some assistance, it would seem, towards finding out the truth of the matter: for now he will push on in the search gladly, as lacking knowledge; whereas then he would have been only too ready to suppose he was right….

 [Having] been reduced to the perplexity of realizing that he did not know… he will go on and discover something.Slide11

Value of Finding an AporiaTo recognize that one’s beliefs contain an aporia, is an advance in understanding, albeit an uncomfortable one.When one recognizes that one’s beliefs contains an aporia, one recognizes that one harbors at least one false belief, is motivated to identify what the false belief is, and is hesitant in deducing consequences from .

This is certainly better than believing without recognizing that they are an aporia. One can avoid an aporia

by arbitrarily discarding propositions until the remaining propositions are inconsistent, but there is nothing to ensure that one has discarded false proposition and kept true ones. One has just narrowed one’s beliefs sufficiently to be safe from accusations of inconsistency. For example, one can avoid an aporia in testing the assumptions of an IV by defining those assumptions so narrowly that they become untestable. Slide12

Value of Finding Aporia for C-section study The aporia tells us that our beliefs that hospital C-section rate is a valid IV and that C-sections do not affect 30-34 week babies cannot both be true.We are motivated to find out which belief(s) isn’t true.

Some Next Steps: Investigate how high C-section rate hospitals and low C-section rate hospitals differ in other aspects of care besides C-sections.Try to better understand important confounders of C-section-mortality relationship and improve the analyses that adjust for measured confounders. Slide13

Strong inference and quasi-experimentsSlide14

Bracketing in the Comparative InterruptedTime-Series Design to Address Concerns aboutHistory Interacting with Group: Evaluating Missouri’sHandgun Purchaser LawRaiden Hasegawa

University of PennsylvaniaDaniel WebsterJohns Hopkins UniversityDylan Small

University of PennsylvaniaSlide15

Repeal of Missouri’s Background Check LawU.S. federal gun law requires background checks and record keeping for gun sales by federally licensed firearm dealers but exempts these regulations for private sales (e.g., gun show sales). However, some states have laws requiring all purchasers of handguns from licensed dealers and

private sellers to acquire a permit-to-purchase (PTP) license that verifies the purchaser has passed a background check.Missouri passed a PTP law in 1921, requiring handgun purchasers to

obtain a license from the local sheriff’s office which facilitated the background check, but repealed the law on August 28, 2007.Webster et al. (2014) conducted a difference-in-difference study comparing Missouri to border states (Arkansas, Illinois, Iowa, Kansas, Kentucky, Oklahoma, Nebraska and Tennessee).Slide16
Slide17

Difference-in-Difference: Causal Inference Based on the Parallel Trends AssumptionSlide18
Slide19

History Interacting with GroupHistory interacting with group: Even if trends are parallel in before period, there could be historical events in the after period that affect the treated and control group differently.History interacting with group may be related to the level of the outcome in the before period.Missouri study:

Missouri had higher gun homicide rates than average border state in before period (1999-2007). Recession starting in 2008. Recession followed by downturn in gun homicide rates.Effect of recession on gun homicide rate might depend on starting level of gun homicide rate.Slide20

Bracketing for Difference-in-DifferenceConcern: History interacting with group related to starting level of group.Ideal control would be equal in before period to treated group. In the absence of ideal control, construct two control groupsLower Control Group: lower expected outcome than

treated group in before periodUpper Control Group: higher expected outcome than treated group in before period.Under assumptions about history interacting with group being related to starting level, expectations of

(i) difference-in-difference comparing treated to upper control group and (ii) difference-in-difference comparing treated to lower control group bracket true causal effect. Slide21

ModelSlide22

Assumptions for Bracketing MethodSlide23

Assumptions for Bracketing Method ContinuedSlide24

Bracketing PropositionSlide25

Constructing Upper and Lower Control GroupsSlide26

Missouri StudySlide27
Slide28
Slide29

Placebo StudySlide30
Slide31

Summary of Findings from Missouri StudyDifference-in-difference finds Missouri PTP repeal associated with increase in gun homicides.Provides evidence for causal effect of Missouri PTP repeal on gun homicides under parallel trends assumption. Bracketing method eliminates history interacting with group due to time-invariant unmeasured confounder having an increased or decreased effect in the after period as a plausible rival explanation to a causal effect. Slide32

Strong inference and quasi-experimentsSlide33

Limitations and Partially Addressing the LimitationsSlide34

Final LectureElaborate Theories and Evidence Factors.