Fixed Versus Random Effects Models for Multilevel and Longitudinal - PowerPoint Presentation

Slide1

Fixed Versus Random Effects Models for Multilevel and Longitudinal Data Analysis

Ashley H. Schempf, PhD

MCH Epidemiology Training Course

June 1, 2012Slide2

OutlineClustered DataFixed Effects ModelsRandom Effects ModelsGEE

Models

Hybrid Models

Applied Examples

Homework AssignmentSlide3

Clustered DataInvolves nesting/clustering of observations or data pointsMultilevel—clustering over space

Panel/Longitudinal—clustering over time

A B C D Neighborhood

j

1 2 3 4 5 6 7 8 9 10 11 12 Individual

i

Time

t

repeated measurements

A

1 2 3

Individual/

B 1 2 3

Unit

i

C 1 2 3 D 1 2 3Slide4

Unique featuresCorrelation of data within clustersViolation of independence; as a modeling assumption errors must be independent

Complexity/redundancy must be accounted for

Variation at multiple levels allows richer examination and distinction of effects

Neighborhood/family versus individual effects

Cross-sectional (selection) versus longitudinal (causation)

A lot of bias can be introduced with single-level data (omitted variables, selection)

Racial disparities when contextual differences aren’t examined (neighborhood level data omitted)

Association between dieting and weight (longitudinal data omitted)Slide5

Between versus Within Cluster EffectsFactors that only vary between clusters are cluster level effects

Multilevel:

walkability

, crime level

Longitudinal: race/ethnicity, sex

However, any factor that varies within cluster can also vary between cluster

Multilevel: income/poverty, raceIndividual-level and neighborhood aggregated (e.g. % poverty, % black)Longitudinal: smoking, activity, diet

At each time point but also averaged for an individual (e.g. average activity level over time) Slide6

Between versus Within Cluster EffectsIn multilevel cases, we may care about both between and within-cluster effects

Contextual effect of living in more versus less segregated neighborhoods (% Black)

Individual effect of race/ethnicity

In longitudinal cases, the between-cluster effects of within-cluster variables tend to represent confounded cross-sectional inference

Comparing a person who smokes to one who doesn’t

Comparing outcomes within a person when they smoke and after they quitSlide7

Handling Clustered DataMany ways of accounting for complex errors and violation of non-independenceRobust SEs, random effects, GEE, survey analysis

Many ways of disentangling between and within-cluster effects

Fixed effects, hybrid models

The correct choice lies in your purposeSlide8

Applied Data ExampleTo demonstrate these options, I’ll use a dataset of birth certificate information from two counties in North CarolinaMultilevel data structure: births nested within neighborhoods (Census block groups)

Covariate of interest: race (Black-White)

Continuous and dichotomous outcome: gestational age and preterm birth (<37 weeks)

Schempf AH, Kaufman JS. Accounting for context in studies of health inequalities:

a review and comparison of approaches. Ann

Epidemiol

.

forthcomingSchempf AH, Kaufman JS, Messer LC, Mendola P. The neighborhood contribution to black-white

perinatal disparities: an example from two north Carolina counties, 1999-2001. Am J Epidemiol. 2011;174(6):744-52.Slide9

Fixed EffectsAccount for all cluster-level variation by holding cluster constantAll inference is therefore within-cluster

Can be implemented either by

entering dummy variables for n-1 clusters

conditional approach

Continuous outcome: “de-meaning” or subtracting the cluster means from all variables before running model

Binary outcome: conditional logistic regressionSlide10

reg

ga_clean

race

i.b_group

i.b_group

_Ib_group_1-392 (_Ib_group_1 for

b_g~p

==370630001011 omitted)note: _Ib_group_51 omitted because of

collinearity

note: _Ib_group_155 omitted because of collinearity

Source | SS df

MS Number of obs = 31489

-------------+------------------------------ F(390, 31098) = 2.86 Model | 5064.7562 390 12.9865543

Prob > F = 0.0000

Residual | 141130.634 31098 4.53825437 R-squared = 0.0346-------------+------------------------------ Adj

R-squared = 0.0225 Total | 146195.391 31488 4.64289223 Root MSE = 2.1303

------------------------------------------------------------------------------

ga_clean |

Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- race | -.4692281 .032978 -14.23 0.000 -.5338663 -.40459 _Ib_group_2 | .0241545 .3980305 0.06 0.952 -.7560012 .8043102 _Ib_group_3 | .5450826 .3702998 1.47 0.141 -.1807199 1.270885 _Ib_group_4 | -.1346887 .4903915 -0.27 0.784 -1.095876 .8264983

_Ib_group_5 | .435551 .4606856 0.95 0.344 -.4674114 1.338513 _Ib_group_6 | -.2657666 .5106346 -0.52 0.603 -1.266631 .7350977

_Ib_group_7 | .3649276 .4704894 0.78 0.438 -.5572506 1.287106 _Ib_group_8 | -.6098332 .5706181 -1.07 0.285 -1.728268 .5086011

_Ib_group_9 | .30786 .6502678 0.47 0.636 -.9666911 1.582411_Ib_group_10 | .1086299 .5707573 0.19 0.849 -1.010077 1.227337

….↓Accounting for neighborhood differences (within-neighborhood inference), Black infants are delivered -.47 weeks earlier than White infantsSlide11

proc glm

data

=

nc.data_final

;

class

b_group ;model

ga_clean= race

b_group /clparm

solution;

quit;

R-Square Coeff

Var Root MSE ga_clean

Mean

0.034644 5.476830 2.130318 38.89692 Source DF Type I SS Mean Square F Value

Pr > F

race 1 2585.192623 2585.192623 569.64 <.0001

b_group 389 2479.563572 6.374199 1.40 <.0001

Source DF Type III SS Mean Square F Value Pr > F race 1 918.775440 918.775440 202.45 <.0001

b_group 389 2479.563572 6.374199 1.40 <.0001

Standard Parameter Estimate Error t Value Pr

> |t|

Intercept 39.45939128 B 0.32492837 121.44 <.0001 race

-0.46922813 0.03297796 -14.23 <.0001

b_group 370630001011 -0.67109834 B 0.45201887 -1.48 0.1376

b_group 370630001012 -0.64694387 B 0.40694049 -1.59 0.1119

b_group

370630001021 -0.12601570 B 0.37957976 -0.33 0.7399 b_group

370630002001 -0.80578708 B 0.49753595 -1.62 0.1053

b_group

370630002002 -0.23554737 B 0.46848328 -0.50 0.6151

Parameter 95% Confidence Limits

Intercept 38.82251860 40.09626396

race -0.53386626 -0.40459000

b_group

370630001011 -1.55707353 0.21487684

b_group

370630001012 -1.44456361 0.15067587

b_group

370630001021 -0.87000731 0.61797592

b_group

370630002001 -1.78097758 0.16940342

Accounting for neighborhood differences (within-neighborhood inference), Black infants are delivered -.47 weeks earlier than White infantsSlide12

xtreg

ga_clean

race,

i

(

b_group

)

fe

Fixed-effects (within) regression Number of obs

= 31489Group variable: b_group

Number of groups = 390

R-sq: within = 0.0065

Obs per group: min = 1 between = 0.3141

avg = 80.7

overall = 0.0177 max = 652 F(1,31098) = 202.45

corr(

u_i, Xb

) = 0.2308 Prob > F = 0.0000

------------------------------------------------------------------------------

ga_clean | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- race

| -.4692281 .032978 -14.23 0.000 -.5338663 -.40459 _cons | 39.04992 .0161171 2422.89 0.000 39.01833 39.08151

-------------+---------------------------------------------------------------- sigma_u

| .39512117

sigma_e | 2.1303179 rho | .03325698 (fraction of variance due to

u_i)------------------------------------------------------------------------------

F test that all u_i

=0: F(389, 31098) = 1.40 Prob > F = 0.0000

Same result without the fixed coefficient output for all the clustersSlide13

proc

glm

data

=

nc.data_final

;absorb

b_group;

model ga_clean

= race;quit;

The GLM ProcedureDependent Variable: ga_clean

Sum of Source DF Squares Mean Square F Value Pr > F Model 390 5064.7562 12.9866 2.86 <.0001

Error 31098 141130.6344 4.5383 Corrected Total 31488 146195.3906

R-Square Coeff Var

Root MSE ga_clean Mean 0.034644 5.476830 2.130318 38.89692

Source DF Type I SS Mean Square F Value Pr > F

b_group 389 4145.980755 10.658048 2.35 <.0001

race 1 918.775440 918.775440 202.45 <.0001 Source DF Type III SS Mean Square F Value Pr > F

race 1 918.7754401 918.7754401 202.45 <.0001

Standard Parameter Estimate Error t Value Pr > |t| race -.4692281302 0.03297796 -14.23 <.0001Slide14

Comparison to Conventional Regression

Use cluster-robust SEs to account for complex error (individual and cluster)

reg

ga_clean

race,

vce

(cl b_group

)Linear regression Number of

obs = 31489

F( 1, 389) = 351.42

Prob > F = 0.0000

R-squared = 0.0177 Root MSE = 2.1356

(Std. Err. adjusted for 390 clusters in

b_group)------------------------------------------------------------------------------

| Robust

ga_clean |

Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | -.6112235 .0326053 -18.75 0.000 -.6753281 -.5471189 _cons | 39.09623 .014152 2762.59 0.000 39.0684 39.12405------------------------------------------------------------------------------Crude effect: -0.61 weeks

Adjusting for neighborhood: -0.47 weeksNeighborhood explained 23% of the racial disparity (assuming there are no confounders of neighborhood)Slide15

Can use surveyreg for cluster-robust SEs in SAS

proc

surveyreg

data

=

nc.data_final;cluster

b_group;class

b_group ;

model ga_clean= race /

clparm solution;

run;

The SURVEYREG Procedure Regression Analysis for Dependent Variable

ga_clean

Estimated Regression Coefficients

Standard 95% Confidence Parameter Estimate Error t Value

Pr > |t| Interval

Intercept 39.0962254 0.01415202 2762.59 <.0001 39.0684014 39.1240495 race -0.6112235 0.03260526 -18.75 <.0001 -0.6753281 -0.5471189

NOTE: The denominator degrees of freedom for the t tests is 389.Slide16

Logistic Model for Binary Outcomelogit ptb_total i.race

i.b_group

, or

Logistic regression Number of

obs

= 31157

LR chi2(367) = 687.08

Prob > chi2 = 0.0000Log likelihood = -9091.5835 Pseudo R2 = 0.0364

------------------------------------------------------------------------------

ptb_total | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | 1.614217 .0849715 9.10 0.000 1.455979 1.789652

_Ib_group_2 | .7499313 .4644189 -0.46 0.642 .2227859 2.524383 _Ib_group_3 | .7817543 .4527046 -0.43 0.671 .2512752 2.432154

_Ib_group_4 | 1.837389 1.208127 0.93 0.355 .506426 6.66632 _Ib_group_5 | 1.012978 .6832426 0.02 0.985 .2700685 3.799497

_Ib_group_6 | 1.871619 1.284711 0.91 0.361 .4874589 7.186159 _Ib_group_7 | .6577316 .504969 -0.55 0.585 .1460644 2.961781

_Ib_group_8 | 4.113144 2.752025 2.11 0.035 1.108285 15.26498 _Ib_group_9 | .6818469 .7794273 -0.34 0.738 .0725551 6.407749

_Ib_group_10 | 1.130029 1.000033 0.14 0.890 .1994381 6.40281Accounting for neighborhood differences (within-neighborhood inference), the

odds of PTB are 1.61 times greater for Black than White infants23 clusters with 322 observations were dropped because of non-varying outcomes—all 0 or 1—division by 0 for an OR Slide17

margins race,

vce

(unconditional) post

Predictive margins Number of

obs

= 31157

Expression : Pr(

ptb_total

), predict()

(Std. Err. adjusted for 367 clusters in b_group2)------------------------------------------------------------------------------

| Unconditional | Margin Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race |

0 | .0753146 .0021845 34.48 0.000 .071033 .0795962 1 | .1154796 .0039339 29.35 0.000 .1077693 .12319

------------------------------------------------------------------------------

. lincom _b[1.race] - _b[0.race]

Risk Difference ( 1) - 0bn.race + 1.race = 0

------------------------------------------------------------------------------

| Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- (1) | .040165 .0046569 8.62 0.000 .0310377 .0492924

------------------------------------------------------------------------------. nlcom _b[1.race] / _b[0.race] Risk Ratio

_nl_1: _b[1.race] / _b[0.race]

------------------------------------------------------------------------------ | Coef

. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

_nl_1 | 1.533297 .0713801 21.48 0.000 1.393394 1.673199------------------------------------------------------------------------------Slide18

proc

logistic

data

=

nc.data_final

;class

b_group ;

model ptb_total

(desc)= race

b_group;run

; Odds Ratio Estimates

Point 95% Wald Effect Estimate Confidence Limits

race 1.614 1.456 1.790 b_group

370630001011 vs

371830544023 2.118 0.387 11.580 b_group

370630001012 vs 371830544023 1.588 0.314 8.034

b_group

370630001021 vs 371830544023 1.656 0.347 7.897 b_group 370630002001 vs 371830544023 3.891 0.727 20.828

b_group 370630002002

vs 371830544023 2.145 0.390 11.787

b_group 370630002003

vs 371830544023 3.964 0.709 22.160

b_group 370630003011

vs 371830544023 1.393 0.219 8.850

b_group 370630003012

vs 371830544023 8.710 1.599 47.453

b_group 370630003013 vs

371830544023 1.444 0.120 17.316

b_group 370630003021 vs

371830544023 2.393 0.311 18.388

b_group

370630003022

vs

371830544023 1.250 0.198 7.881

b_group

370630003023

vs

371830544023 2.515 0.431 14.684

b_group

370630004011

vs

371830544023 1.409 0.187 10.615

b_group

370630004012

vs

371830544023 2.206 0.347 14.033

…↓

Would need to use SUDAAN or binomial/

poisson

models for RD or RR in SASSlide19

xtlogit

ptb_total

race,

i

(

b_group

)

fe or

clogit ptb_total

race, group(b_group

) vce(

cl b_group

) or

note: multiple positive outcomes within groups encountered.note: 23 groups (332 obs

) dropped because of all positive or all negative outcomes.

Iteration 0: log pseudolikelihood

= -8466.3107 Iteration 1: log

pseudolikelihood = -8456.9979 Iteration 2: log

pseudolikelihood = -8456.9975

Conditional (fixed-effects) logistic regression Number of obs = 31157 Wald chi2(1) = 84.25 Prob

> chi2 = 0.0000Log pseudolikelihood

= -8456.9975 Pseudo R2 = 0.0047

(Std. Err. adjusted for 367 clusters in b_group

)------------------------------------------------------------------------------

| Robust ptb_total

| Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

1.race | 1.604089 .0825837 9.18 0.000 1.450126 1.774398------------------------------------------------------------------------------

The conditional approach is recommended for non-linear models because of the incidental parameters problem with dummy variables, leading to upward bias.

Mainly a problem for small clusters so not too different in this sample (avg cluster size ~80) 1.61 versus 1.60Slide20

proc

logistic

data

=

nc.data_final

;strata

b_group;

model ptb_total

(desc) = race;

run;

The LOGISTIC Procedure Conditional Analysis

Model Fit Statistics Without With Criterion Covariates

Covariates AIC 16994.399 16915.995

SC 16994.399 16924.352 -2 Log L 16994.399 16913.995 Testing Global Null Hypothesis: BETA=0

Test Chi-Square DF Pr > ChiSq

Likelihood Ratio 80.4035 1 <.0001 Score 82.2342 1 <.0001 Wald 81.7008 1 <.0001

Analysis of Maximum Likelihood Estimates Standard Wald

Parameter DF Estimate Error Chi-Square Pr > ChiSq race 1 0.4726 0.0523 81.7008 <.0001

Odds Ratio Estimates Point 95% Wald Effect Estimate Confidence Limits race 1.604 1.448 1.777Slide21

Comparison to Conventional Regression

logit

ptb_total

race,

vce

(

cl

b_group) or

Logistic regression Number of obs

= 31489 Wald chi2(1) = 278.47

Prob

> chi2 = 0.0000Log pseudolikelihood

= -9312.2722 Pseudo R2 = 0.0163

(Std. Err. adjusted for 390 clusters in b_group

)------------------------------------------------------------------------------

| Robust ptb_total

| Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race | 2.02922 .0860517 16.69 0.000 1.86738 2.205085------------------------------------------------------------------------------

Crude OR: 2.03Adjusting for neighborhood: 1.60Neighborhood explained ~40% of the racial disparity in PTB (assuming there are no confounders of neighborhood)N.B. For percent change in OR, you always need to subtract the null (1.0) first(0.6-1.03)/1.03 = -.41 or a drop of 41% after controlling for contextual differencesSlide22

proc surveylogistic

data

=

nc.data_final

;

cluster b_group

;model ptb_total

(desc)=

race;run;

Testing Global Null Hypothesis: BETA=0

Test Chi-Square DF Pr

> ChiSq

Likelihood Ratio 308.0644 1 <.0001 Score 324.9556 1 <.0001

Wald 278.4631 1 <.0001

Analysis of Maximum Likelihood Estimates

Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq

Intercept 1 -2.6016 0.0285 8352.0534 <.0001

race 1 0.7077 0.0424 278.4631 <.0001

Odds Ratio Estimates Point 95% Wald Effect Estimate Confidence Limits race 2.029 1.867 2.205 Association of Predicted Probabilities and Observed Responses

Percent Concordant 32.9 Somers' D 0.167

Percent Discordant 16.2 Gamma 0.340 Percent Tied 50.8 Tau-a 0.027

Pairs 80536248 c 0.584Slide23

Fixed Effects: Benefits & DisadvantagesBenefits: Provides within-cluster effects that are not confounded by cluster-level factors because all cluster variation is removed (accounts for unobservable confounding)

No minimum number of clusters

Disadvantages:

Does not allow estimation of observable cluster-level effects so often seen more in longitudinal analyses where between-cluster effects may not be of interest

Can be inefficient/less precise due to less degrees of freedom (each cluster counts as parameter) and it only exploits one level of variationSlide24

Random EffectsAlternative to fixed effects that models only one additional parameter (instead of k-1) by making greater assumptionsMore efficient but vulnerable to bias

Average cluster-specific intercept with the cluster-level variance estimated (

τ₀

2

)

Accounts for variability in the outcome across neighborhoods but not for covariates (

corr μoj

, xi = 0) Allows estimates of variance at both levels and of cluster-level covariates because the cluster-level variance isn’t completely removed from model

Slide25

xtreg

ga_clean

race,

i

(

b_group

) re mle

Random-effects ML regression Number of

obs

= 31489Group variable:

b_group Number of groups = 390

Random effects u_i

~ Gaussian Obs per group: min = 1

avg

= 80.7 max = 652

LR chi2(1) = 389.27Log likelihood = -68563.638

Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ga_clean | Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- race | -.5883143 .027936 -21.06 0.000 -.6430679 -.5335606

_cons | 39.07689 .0180206 2168.46 0.000 39.04157 39.11221-------------+----------------------------------------------------------------

/sigma_u | .1364134 .021792 .0997419 .1865676

/sigma_e

| 2.131475 .0085433 2.114796 2.148285 rho | .0040792 .0013011 .0021387 .0074679

------------------------------------------------------------------------------Likelihood-ratio test of

sigma_u=0: chibar2(01)= 18.07 Prob

>=chibar2 = 0.000

Cluster-specific, within-neighborhood interpretation but significantly higher than FE estimate of -0.47Intracluster

Correlation = 0.004 (proportion of variance that occurs at neighborhood level)0.1362

/(0.1362+2.132

) = 0.004- Significant neighborhood variation but a small fraction of overall variability (0.4%)Slide26

proc

glimmix

data

=

nc.data_final

method

=quad;class

b_group;model

ga_clean = race /

solution ;random

intercept/ subject=

b_group;run;

The GLIMMIX Procedure Optimization Information

Optimization Technique Dual Quasi-Newton Parameters in Optimization 4 Lower Boundaries 2

Upper Boundaries 0 Fixed Effects Not Profiled Starting From GLM estimates

Quadrature Points 1

Covariance Parameter Estimates Standard

Cov Parm Subject Estimate Error

Intercept b_group 0.01861 0.005946 Residual 4.5432 0.03642 Solutions for Fixed Effects Standard Effect

Estimate Error DF t Value Pr > |t|

Intercept 39.0769 0.01802 389 2168.69 <.0001 race -0.5883 0.02793 31098 -21.06 <.0001

Type III Tests of Fixed Effects

Num Den Effect DF DF

F Value Pr > F race 1 31098 443.54 <.0001Slide27

xtlogit

ptb_total

race,

i

(

b_group

) re or

Random-effects logistic regression Number of

obs = 31489

Group variable: b_group

Number of groups = 390Random effects

u_i ~ Gaussian

Obs per group: min = 1

avg = 80.7

max = 652

Wald chi2(1) = 265.00Log likelihood = -9310.5391

Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ptb_total | OR Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race | 1.992814 .0844133 16.28 0.000 1.834049 2.165323-------------+---------------------------------------------------------------- /lnsig2u | -4.059262 .6231052 -5.280526 -2.837999-------------+---------------------------------------------------------------- sigma_u

| .131384 .040933 .0713425 .241956 rho | .0052196 .0032354 .0015447 .0174837

------------------------------------------------------------------------------Likelihood-ratio test of rho=0: chibar2(01) = 3.47

Prob >= chibar2 = 0.031

Cluster-specific, within-neighborhood interpretation but significantly higher than FE estimate of 1.60

Intracluster

Correlation = 0.005 (most of variance occurs within neighborhood at the individual level, 99.5%, rather than between neighborhoods at neighborhood level, 0.5%)0.1314/(0.1314 + π2/3) = 0.005Slide28

proc

glimmix

data

=

nc.data_final

method

=quad;class

b_group;model

ptb_total (

descending) = race /solution

dist=bin link

=logit

oddsratio;random

intercept / subject=

b_group;run

; Covariance Parameter Estimates Standard

Cov Parm Subject Estimate Error

Intercept b_group 0.01746 0.01081

Solutions for Fixed Effects Standard Effect Estimate Error DF t Value Pr > |t|

Intercept -2.5945 0.02907 389 -89.26 <.0001 race 0.6894 0.04238 31098 16.27 <.0001

Odds Ratio Estimates 95% Confidence race _race Estimate DF Limits

1.3261 0.3261 1.992 31098 1.834 2.165

Type III Tests of Fixed Effects

Num Den Effect DF DF

F Value Pr > F race 1 31098 264.60 <.0001Slide29

Random Intercept + Slope Also possible to allow random normal variation in covariate effect across neighborhood, e.g. allowing racial disparity to vary by neighborhood

xtmixed

ga_clean

race ||

b_group

: race,

mle

Mixed-effects ML regression Number of

obs = 31489

Group variable: b_group

Number of groups = 390

Obs per group: min = 1

avg = 80.7

max = 652 Wald chi2(1) = 304.72

Log likelihood = -68537.949 Prob

> chi2 = 0.0000------------------------------------------------------------------------------

ga_clean

| Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | -.6060449 .0347181 -17.46 0.000 -.6740911 -.5379988 _cons | 39.09577 .0147177 2656.37 0.000 39.06693 39.12462------------------------------------------------------------------------------------------------------------------------------------------------------------

Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]-----------------------------+------------------------------------------------

b_group: Independent |

sd(race) |

.3447168 .0374273 .2786404 .4264626

sd

(_cons) | .0221739 .0600709 .0001096 4.485535-----------------------------+------------------------------------------------

sd

(Residual) | 2.127355 .0085317 2.110699 2.144143------------------------------------------------------------------------------

LR test vs. linear regression: chi2(2) = 69.45 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.

Appears to be significant variation across neighborhoods but the point estimate or average within-neighborhood disparity is not correct based on comparisons to FE models (-0.47)Slide30

proc

glimmix

data

=

nc.data_final

method

=quad;class

b_group;model

ga_clean = race /

solution ;random

intercept race/ subject=

b_group;Run;

The GLIMMIX Procedure Covariance Parameter Estimates

Standard

Cov Parm Subject Estimate Error

Intercept b_group 0.000494 0.003077

race b_group 0.1188 0.02584 Residual 4.5256 0.03631

Solutions for Fixed Effects

Standard Effect Estimate Error DF t Value Pr > |t| Intercept 39.0958 0.01498 389 2609.64 <.0001 race -0.6060 0.03488 352 -17.38 <.0001

Type III Tests of Fixed Effects

Num Den Effect DF DF

F Value Pr > F race 1 352 301.93 <.0001Slide31

xtmelogit

ptb_total

race ||

b_group

: race, or

Mixed-effects logistic regression Number of

obs

= 31489

Group variable: b_group

Number of groups = 390

Obs per group: min = 1

avg

= 80.7 max = 652

Integration points = 7 Wald chi2(1) = 255.50Log likelihood = -9310.2914

Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ptb_total | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | 1.991382 .0858163 15.98 0.000 1.830093 2.166887

------------------------------------------------------------------------------

------------------------------------------------------------------------------ Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]-----------------------------+------------------------------------------------b_group: Independent |

sd(race) | .1218997 .0899226

.0287138 .5175045

sd(_cons) | .1101542 .0562791 .040468 .2998403

------------------------------------------------------------------------------LR test vs. logistic regression: chi2(2) = 3.96

Prob > chi2 = 0.1380

Note: LR test is conservative and provided only for reference.

No indication of significant neighborhood variation in the PTB racial disparity; average neighborhood-specific disparity is biased relative to FE (1.60) Slide32

proc

glimmix

data

=

nc.data_final

method

=quad;class

b_group;model

ptb_total (

descending) = race /solution

dist=bin link

=logit

oddsratio;random

intercept race/ subject=

b_group;run

; Covariance Parameter Estimates Standard

Cov Parm

Subject Estimate Error Intercept b_group

0.01230 0.01248 race b_group 0.01495 0.02201 Solutions for Fixed Effects Standard Effect

Estimate Error DF t Value Pr > |t|

Intercept -2.5967 0.02875 389 -90.31 <.0001 race 0.6886 0.04312 352 15.97 <.0001

Odds Ratio Estimates 95% Confidence race _race Estimate DF Limits

1.3261 0.3261 1.991 352 1.829 2.167

Type III Tests of Fixed Effects Num Den

Effect DF DF F Value Pr > F

race 1 352 255.04 <.0001Slide33

When will RE approximate FE?When there is no between-cluster confoundingNo clustering of X

No variation in outcome by cluster

Even when confounding is present, RE can still approximate FE under certain conditions

Normally, a composite of within and between effects but weighted toward the within-effect when it is more precise

Large cluster size

High ICCSlide34

ExampleMost of variation is between rather than within cluster (ICC=0.99) so within-effect is going to be very precise (little variability)

----------------------------------------------------------------

Variable |

ols

olsc

gee

fe

re

-------------+--------------------------------------------------

x | -2.2152 -2.2152 2.0552 2.0563 2.0503

| 0.799 1.189 0.044 0.045 0.054

_cons | 166.0880 166.0880 67.3646 67.3404 67.4793 | 18.868 28.958 13.615 1.049 7.058

---------------------------------------------------------------- legend: b/seSlide35

For most multilevel neighborhood studies, ICC is quite low (<10%) so only a huge cluster size could compensate to get valid within-cluster estimates in the presence of neighborhood confounding

We can also control for observable cluster-level factors but there are many factors that may not be measured or imperfectly measured

e.g. built environment, air quality/toxins, health care access/quality, fresh foods, social cohesion

So it can be rare to get the FE estimate, which controls for all factors--observed and unobserved, by controlling only for a few observed factors in a RE modelSlide36

xtreg ga_clean

race poverty,

i

(

b_group

) re

mle

Random-effects ML regression Number of

obs

= 31489Group variable: b_group

Number of groups = 390

Random effects u_i ~ Gaussian

Obs per group: min = 1

avg = 80.7

max = 652

LR chi2(2) = 407.38Log likelihood = -68554.58 Prob

> chi2 = 0.0000

------------------------------------------------------------------------------ ga_clean

| Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | -.5368362 .030145 -17.81 0.000 -.5959193 -.477753 poverty | -.0064216 .0015111 -4.25 0.000 -.0093832 -.0034599 _cons | 39.1246 .0205574 1903.19 0.000 39.08431 39.16489-------------+---------------------------------------------------------------- /

sigma_u | .1304851 .0218083 .0940368 .1810607

/sigma_e | 2.131108 .0085401 2.114436 2.147913

rho | .003735 .0012466 .0018992 .007028------------------------------------------------------------------------------

Likelihood-ratio test of sigma_u

=0: chibar2(01)= 15.89 Prob>=chibar2 = 0.000

So controlling for poverty moves us closer to the within-cluster effect but doesn’t control for all important neighborhood factorsCrude: -0.61 FE: -0.47 Controlling for neighborhood poverty: -0.54Explains about half of the neighborhood contribution (0.07/0.14)

And 11.5% of overall disparity (0.07/-0.61)Slide37

Hausman test for consistency in estimates from FE and RE modelshausman

ga_clean_fe

ga_clean_re_poverty

---- Coefficients ----

| (b) (B) (b-B)

sqrt

(diag

(V_b-V_B))

| ga_clean_fe

ga_clean_r~y Difference S.E.

-------------+---------------------------------------------------------------- race | -.4692281 -.5500752 .080847 .0156404

------------------------------------------------------------------------------ b = consistent under Ho and Ha; obtained from

xtreg B = inconsistent under Ha, efficient under Ho; obtained from

xtreg

Test: Ho: difference in coefficients not systematic chi2(1) = (b-B)'[(

V_b-V_B)^(-1)](b-B)

= 26.72 Prob

>chi2 = 0.0000Rejects the null of equivalence between the FE and RE estimator**From a RE model that is based on generalized least squares, not maximum likelihoodSlide38

xtlogit ptb_total

race poverty,

i

(

b_group

) re or

Random-effects

logistic regression Number of

obs = 31489Group variable:

b_group Number of groups = 390

Random effects u_i

~ Gaussian Obs per group: min = 1

avg

= 80.7 max = 652

Wald chi2(2) = 316.05Log likelihood = -9297.2331

Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ptb_total | OR Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | 1.804966 .0834998 12.77 0.000 1.64851 1.976272

poverty | 1.010437 .0020192 5.20 0.000 1.006487 1.014403-------------+---------------------------------------------------------------- /lnsig2u | -4.32276 .7583429 -5.809085 -2.836435-------------+---------------------------------------------------------------- sigma_u | .1151661 .0436677 .0547739 .2421452

rho | .0040153 .0030328 .0009111 .0175106------------------------------------------------------------------------------

Likelihood-ratio test of rho=0: chibar2(01) = 2.21 Prob

>= chibar2 = 0.068So controlling for poverty moves us closer to the within-cluster effect but doesn’t control for all important neighborhood factorsCrude: 2.03 FE:

1.60 Controlling for neighborhood poverty: 1.80Explains about half of the neighborhood contribution (0.2/0.43)And

20% of overall disparity (0.2/1.03)Slide39

Hausman test for consistency in estimates from FE and RE models

hausman

ptb_fe

ptb_re_poverty

---- Coefficients ----

| (b) (B) (b-B)

sqrt(

diag(V_b

-V_B)) | ptb_fe

ptb_re_pov~y Difference S.E.

-------------+---------------------------------------------------------------- race | .4725559 .590542 -.1179861 .0243549

------------------------------------------------------------------------------ b = consistent under Ho and Ha; obtained from

xtlogit

B = inconsistent under Ha, efficient under Ho; obtained from xtlogit

Test: Ho: difference in coefficients not systematic

chi2(1) = (b-B)'[(V_b

-V_B)^(-1)](b-B) = 23.47

Prob>chi2 = 0.0000Rejects the null of equivalence between the FE and RE estimatorSlide40

Random Effects: Benefits & DisadvantagesBenefits:

Ability to estimate covariates both within and between cluster (level 1 and 2 effects)

Ability to partition variance at multiple levels

Examine variation in effects across cluster

Efficient/parsimonious

Disadvantages:

Within-cluster effects can be significantly biasedRequires ~30 clusters for estimation of cluster variance with random normal assumptionSlide41

GEEHandles clustered data with complex error treated as a nuisance rather than explicitly controlled (FE) or modeled as an interest (RE)Within-cluster correlation specified as

Independent (robust SEs, point estimates unchanged)

Exchangeable (similar to RE point estimates)

Unstructured (allows variation in correlation)

Inference is population-averaged rather than cluster-specific

(only difference is for odds ratio since the average of each cluster-specific OR ≠ overall OR; not collapsible

)Slide42

xtreg

ga_clean

race,

i

(

b_group

) pa

corr(

ind) vce

(robust)xtgee

ga_clean race,

corr(ind

) vce(robust)

Iteration 1: tolerance = 6.133e-15

GEE population-averaged model Number of

obs = 31489Group variable:

b_group Number of groups = 390

Link: identity Obs per group: min = 1

Family: Gaussian avg

= 80.7Correlation: independent max = 652 Wald chi2(1) = 351.43Scale parameter: 4.560647 Prob > chi2 = 0.0000

Pearson chi2(31489): 143610.20 Deviance = 143610.20Dispersion (Pearson): 4.560647 Dispersion = 4.560647

(Std. Err. adjusted for clustering on

b_group)------------------------------------------------------------------------------

| Semirobust

ga_clean | Coef

. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race | -.6112235 .0326047 -18.75 0.000 -.6751276 -.5473194 _cons | 39.09623 .0141518 2762.63 0.000 39.06849 39.12396

------------------------------------------------------------------------------

Results are very similar to OLS regression with cluster-robust SEsSlide43

xtreg

ga_clean

race,

i

(

b_group

) pa

corr(

exc)

xtgee ga_clean

race, corr(

exc)

vce(robust)

Iteration 1: tolerance = .00997269Iteration 2: tolerance = .00071555

Iteration 3: tolerance = .00004882Iteration 4: tolerance = 3.319e-06Iteration 5: tolerance = 2.256e-07

GEE population-averaged model Number of

obs = 31489Group variable:

b_group Number of groups = 390

Link: identity Obs

per group: min = 1Family: Gaussian avg = 80.7Correlation: exchangeable max = 652 Wald chi2(1) = 350.49Scale parameter: 4.560803

Prob > chi2 = 0.0000

(Std. Err. adjusted for clustering on b_group

)------------------------------------------------------------------------------

| Semirobust

ga_clean

| Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | -.5939304 .0317245 -18.72 0.000 -.6561093 -.5317514

_cons | 39.08105 .013988 2793.91 0.000 39.05364 39.10847

------------------------------------------------------------------------------Results are very similar to the random intercept model with cluster-robust SEsSlide44

proc

genmod

data

=

nc.data_final

;class

b_group;

model ga_clean

= race;repeated

subject=b_group

/corr=

ind;run;

Analysis Of GEE Parameter Estimates Empirical Standard Error Estimates

Standard 95% Confidence Parameter Estimate Error Limits Z Pr > |Z|

Intercept 39.0962 0.0141 39.0685 39.1239 2766.18 <.0001 race -0.6112 0.0326 -0.6750 -0.5474 -18.77 <.0001

proc genmod

data=

nc.data_final;class

b_group;model ga_clean = race;repeated

subject=b_group

/corr=exc

;run;

Exchangeable Working Correlation Correlation 0.0028899011

Analysis Of GEE Parameter Estimates Empirical Standard Error Estimates

Standard 95% Confidence Parameter Estimate Error Limits Z Pr > |Z|

Intercept 39.0811 0.0140 39.0537 39.1084 2797.50 <.0001 race -0.5939 0.0317 -0.6560 -0.5318 -18.75 <.0001Slide45

xtlogit

ptb_total

race,

i

(

b_group

) pa

corr(

ind) vce

(robust) orxtgee

ptb_total race,

fam(bin) link(logit

) corr(

ind)

vce(robust) eform

Iteration 1: tolerance = 1.545e-07

GEE population-averaged model Number of

obs = 31489Group variable:

b_group Number of groups = 390

Link: logit Obs per group: min = 1Family: binomial avg

= 80.7Correlation: independent max = 652

Wald chi2(1) = 278.47Scale parameter: 1

Prob > chi2 = 0.0000

Pearson chi2(31489): 31489.00 Deviance = 18624.54Dispersion (Pearson): 1 Dispersion = .5914619

(Std. Err. adjusted for clustering on

b_group)------------------------------------------------------------------------------

| Semirobust

ptb_total | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race | 2.02922 .0860517 16.69 0.000 1.867381 2.205086------------------------------------------------------------------------------

Results are very similar to logistic regression with cluster-robust SEsSlide46

xtlogit

ptb_total

race,

i

(

b_group

) pa

corr(

exc) vce

(robust) orxtgee

ptb_total race,

fam(bin) link(logit

) corr(

exc)

vce(robust) eform

Iteration 1: tolerance = .00895091

Iteration 2: tolerance = .00071972Iteration 3: tolerance = .00006077

Iteration 4: tolerance = 5.223e-06Iteration 5: tolerance = 4.495e-07

GEE population-averaged model Number of obs

= 31489Group variable: b_group Number of groups = 390Link: logit

Obs per group: min = 1

Family: binomial avg = 80.7

Correlation: exchangeable max = 652 Wald chi2(1) = 269.83

Scale parameter: 1 Prob

> chi2 = 0.0000

(Std. Err. adjusted for clustering on b_group)

------------------------------------------------------------------------------ |

Semirobust

ptb_total | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race | 1.995782 .0839587 16.43 0.000 1.837828 2.167313

------------------------------------------------------------------------------

Results are very similar to RE logistic regression with cluster-robust SEs

ICC is so low that marginal OR ≈ cluster-specific ORSlide47

proc

genmod

data

=

nc.data_final;

class b_group

;model

ptb_total = race /dist

=bin link=

logit;repeated

subject=b_group

/corr=

ind;estimate

'race or' race -

1 1 /

exp;

run; Contrast Estimate Results

Mean Mean L'Beta Standard L'Beta Label Estimate Confidence Limits Estimate Error Alpha Confidence Limits race or 0.6699 0.6513 0.6880 0.7077 0.0424 0.05 0.6246 0.7907 Exp(race or)

2.0292 0.0859 0.05 1.8676 2.2049

proc

genmod

data=nc.data_final

;class

b_group;model

ptb_total = race /

dist=bin link=

logit;repeated

subject=

b_group/corr

=

exc

;

estimate

'race or'

race -

1

1

/

exp

;

run

;

Contrast Estimate Results

Mean

Mean

L'Beta

Standard

L'Beta

Label Estimate Confidence Limits Estimate Error Alpha Confidence Limits

race or 0.6662 0.6476 0.6843 0.6910 0.0420 0.05 0.6087 0.7734

Exp(race or)

1.9958 0.0839

0.05 1.8380 2.1671Slide48

GEE: Benefits & DisadvantagesBenefits:Ability to estimate both within and between-cluster effects and adjust SEs for clustering

Examine cross-level interaction

Disadvantages:

Within-cluster effects can be significantly biased

Marginal inference leads to effect estimates closer to the null in logistic models (depending on ICC)

No variance componentsSlide49

Hybrid ModelsObtain the appropriate within-neighborhood effect in random effects, GEE, or general cluster-robust modelsContain the advantages of RE or GEE models without the bias in the within-cluster effects

Incorporate the cluster-mean of the covariate to account for all between-cluster variation related to the covariate (aggregated variable, % Black)

Centering (subtracting cluster-mean)

Centering + cluster-mean covariate adjustment

Cluster-mean covariate adjustmentSlide50

egen

race_bgc

=mean(race), by(

b_group

)

gen

race_c

=race-race_bgc

xtreg

ga_clean race_c

, i

(b_group) re

mle

Random-effects ML regression Number of obs = 31489

Group variable: b_group

Number of groups = 390Random effects u_i

~ Gaussian Obs

per group: min = 1

avg = 80.7 max = 652

LR chi2(1) = 201.73Log likelihood = -68657.406 Prob > chi2 = 0.0000------------------------------------------------------------------------------

ga_clean |

Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+---------------------------------------------------------------- race_c

| -.4692281 .0329833 -14.23 0.000 -.5338742 -.404582 _cons | 38.85962 .0206466 1882.13 0.000 38.81915 38.90008

-------------+---------------------------------------------------------------- /

sigma_u | .291513 .0203577 .2542229 .3342729

/sigma_e | 2.130663 .0085431 2.113985 2.147473

rho | .0183752 .0025312 .0139516 .0239413------------------------------------------------------------------------------

Likelihood-ratio test of sigma_u

=0: chibar2(01)= 193.82 Prob

>=chibar2 = 0.000Estimate now corresponds to within-neighborhood effect of race obtained in FE model (-0.47)

-Note that the ICC is now much larger than in the RE model (0.004

 0.018)

-Neighborhood variance increased because centering removed the association between race and neighborhood before estimation so this refers to the ICC from a null model

-No other variables in the model (including random intercept) will be adjusted for Slide51

PROC

SUMMARY

NWAY

DATA

=nc.data

;VAR race;

CLASS b_group;

OUTPUT OUT

=cluster MEAN=

race_bgc;RUN;

DATA

nc.data_final;MERGE

nc.data cluster;

BY b_group;

race_c=race-race_bgc

;RUN;

proc

glimmix data=nc.data_final method=quad;class

b_group;

model ga_clean

= race_c /

solution ;random intercept /

subject=b_group

;run;

Covariance Parameter Estimates Standard

Cov Parm

Subject Estimate Error Intercept b_group

0.08497 0.01187 Residual 4.5397 0.03641

Solutions for Fixed Effects Standard Effect

Estimate

Error

DF t Value Pr > |t|

Intercept 38.8596 0.02065 389 1882.23 <.0001

race_c

-0.4692 0.03298 31098 -14.23 <.0001

Slide52

xtlogit

ptb_total

race_c

,

i

(

b_group) re or

Random-effects logistic regression Number of

obs = 31489Group variable:

b_group Number of groups = 390

Random effects u_i

~ Gaussian Obs

per group: min = 1

avg = 80.7 max = 652

Wald chi2(1) = 83.13

Log likelihood = -9384.2229 Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ptb_total

| OR Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- race_c | 1.610407 .0841611 9.12 0.000 1.453621 1.784104-------------+---------------------------------------------------------------- /lnsig2u | -2.192009 .1819926 -2.548708 -1.83531

-------------+----------------------------------------------------------------

sigma_u | .3342037 .0304113 .2796115 .3994546

rho | .0328355 .0057796 .023213 .046258------------------------------------------------------------------------------

Likelihood-ratio test of rho=0: chibar2(01) = 83.11 Prob

>= chibar2 = 0.000

OR now corresponds to within-neighborhood effect of race obtained in FE model (1.61)ICC is now much larger than in the RE model (0.005

 0.032) because it is not adjusted for racial clustering/segregation across neighborhoodsSlide53

PROC SUMMARY

NWAY

DATA

=

nc.data

;VAR race;

CLASS b_group

;OUTPUT

OUT=cluster MEAN

=race_bgc;

RUN;

DATA

nc.data_final;

MERGE nc.data

cluster;BY

b_group;r

ace_c=race-race_bgc

;RUN;proc glimmix

data=nc.data_final

method=quad;

class b_group;

model ptb_total

(descending) =

race_c /solution

dist=bin

link=logit

oddsratio;random

intercept / subject=

b_group;run

;

Covariance Parameter Estimates

Standard

Cov

Parm

Subject Estimate Error

Intercept

b_group

0.1113 0.02016

Solutions for Fixed Effects

Standard

Effect

Estimate

Error

DF t Value Pr > |t|

Intercept -2.3408 0.02861 389 -81.82 <.0001

race_c

0.4764 0.05226 31098 9.12 <.0001

Odds Ratio Estimates

95% Confidence

race_c

_

race_c

Estimate DF Limits

1 -1E-9

1.610 31098 1.454 1.784Slide54

egen

race_bgc

=mean(race), by(

b_group

)

gen

race_c

=race-race_bgc

xtreg

ga_clean

race_c race_bgc

, i

(b_group) re

mle

Random-effects ML regression Number of obs

= 31489Group variable: b_group

Number of groups = 390Random effects

u_i ~ Gaussian Obs

per group: min = 1

avg = 80.7 max = 652 LR chi2(2) = 431.28Log likelihood = -68542.63 Prob > chi2 = 0.0000

------------------------------------------------------------------------------

ga_clean |

Coef. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race_c

| -.4692281 .0329838 -14.23 0.000 -.5338752 -.4045811

race_bgc | -.8450466 .0474493 -17.81 0.000 -.9380455 -.7520477 _cons | 38.89084 .0143968 2701.36 0.000 38.86263 38.91906

-------------+----------------------------------------------------------------

/sigma_u | .1207709 .021501 .0851965 .1711996

/sigma_e

| 2.130694 .008534 2.114033 2.147486 rho | .0032025 .0011389 .0015566 .0062758

------------------------------------------------------------------------------Likelihood-ratio test of

sigma_u

=0: chibar2(01)= 13.58

Prob

>=chibar2 = 0.000

Within-neighborhood effect of race corresponds to that obtained in FE model (-0.47)

Between-neighborhood effect is not adjusted for the individual level effect so it reflects the ecological effect (contextual and individual effect) comparing outcomes between neighborhoods with a higher versus lower % Black

β

b =

β

w +

β

c if

β

c=0 then

β

b =

β

w and no RE bias

-0.85 = -0.47 +

β

c so

β

c = -0.38

-Note that the ICC is now small again because it’s adjusted for cluster mean of race (composition/segregation)Slide55

egen

race_bgc

=mean(race), by(

b_group

)

xtreg

ga_clean race

race_bgc,

i(b_group

) re mle

Random-effects ML regression Number of

obs = 31489Group variable:

b_group Number of groups = 390

Random effects u_i ~ Gaussian

Obs per group: min = 1

avg = 80.7

max = 652 LR chi2(2) = 431.28

Log likelihood = -68542.63 Prob

> chi2 = 0.0000------------------------------------------------------------------------------ ga_clean | Coef

. Std. Err. z P>|z| [95% Conf. Interval]-------------+----------------------------------------------------------------

race | -.4692281 .0329838 -14.23 0.000 -.5338752 -.4045811

race_bgc | -.3758185 .0577872 -6.50 0.000 -.4890794 -.2625576 _cons | 39.04385 .0179707 2172.64 0.000 39.00863 39.07907

-------------+---------------------------------------------------------------- /

sigma_u | .1207709 .021501 .0851965 .1711996

/sigma_e

| 2.130694 .008534 2.114033 2.147486 rho | .0032025 .0011389 .0015566 .0062758

------------------------------------------------------------------------------Likelihood-ratio test of

sigma_u=0: chibar2(01)= 13.58 Prob

>=chibar2 = 0.000Without cluster –mean centering , the cluster mean variable now refers to the contextual effect of racial segregation because it is now correlated with and can be adjusted for the individual effect

Refers to the effect of living in a more segregated neighborhood regardless of whether you are black or white

β

c =

β

b -

β

w = -0.38

Because it is significantly different from zero, there is evidence of a contextual effect ,

β

w ≠

β

b

corr

μ

oj

, x

i

≠

0Slide56

CaveatsCluster-mean adjustment is a convenient way to account for cluster-level confounding but there are some important considerations

Cluster mean must always come from sample (not external population, e.g. census % Black)

Other covariates may also require cluster-mean adjustment if they’re confounders of the principal covariate and also vary by cluster

Should always compare to FE estimates

For complex survey data, weights may need to be scaled for multilevel models and the hybrid approach may only work if there is minimal sampling biasSlide57

ExamplesMultilevel data structureNeighborhood modelsSibling studies

Longitudinal data structure

Policy evaluation (state panels)Slide58

Neighborhood Contribution to Black White Perinatal Disparities

Used hybrid models (cluster-mean adjustment) to evaluate the impact of neighborhood inequalities to racial disparities

Adjusted for individual-level factors that could influence neighborhood selection (age, education, marital status, gravidity)

Neighborhoods made a significant contribution to PTB (~15% disparity reduction)

Effect of neighborhood segregation wasn’t fully explained by neighborhood poverty

May need to consider context across the life course

Schempf AH, Kaufman JS, Messer LC, Mendola P. The neighborhood contribution to black-white

perinatal disparities: an example from two north Carolina counties, 1999-2001. Am J

Epidemiol. 2011;174(6):744-52.Slide59

Neighborhood Contribution to Hypertension DisparitiesUsed cluster-mean centering for all covariates compared to RE model with neighborhood-level covariates

Neighborhood differences helped to entirely explain the residual black-white hypertension disparity after adjustment for individual-level factors

OR 1.5



1.0

Measured neighborhood variables (poverty/affluence, Hispanic, age composition) accounted for entire neighborhood contribution

OR still 1.0 without cluster-mean centering

Morenoff JD, House JS, Hansen BB, et al. Understanding social disparities in hypertension prevalence, awareness, treatment, and control: the role of neighborhood context. Soc

Sci Med. 2007;65(9):1853-1866.Slide60

Gestational Weight Gain and Child BMISiblings nested within mothers (CPP)Used FE models to account for unobserved family level factors (environmental/genetic) that may be associated with both GWG and child BMI

GEE (independent) models showed associations between GWG and child BMI

FE models, comparing only changes in GWG between pregnancies within the same mother, showed no effect on child BMI

May be nothing deterministic about in-

utero

exposures

Branum AM, Parker JD, Keim SA, Schempf AH.

Prepregnancy body mass index and gestational weight gain in relation to child body mass index among siblings. Am J Epidemiol. 2011 Nov 15;174(10):1159-65.Slide61

Breastfeeding and Child BMIBreastfeeding is often associated with many positive health outcomes but it’s unclear whether this is due to confounding by selection (unobserved characteristics)

Siblings nested within mother (add-health)

Used FE models to contrast outcomes for discordant siblings (one breastfed, one formula fed)

No effect of different breastfeeding exposure on adolescent overweight within siblings to the same mother

Changing a woman’s breastfeeding behavior is not likely to change child BMI; many benefits of breastfeeding but reducing obesity may not be one

Nelson MC, Gordon-Larsen P, Adair LS. Are adolescents who were breast-fed less likely to be overweight? Analyses of sibling pairs to reduce confounding. Epidemiology. 2005 Mar;16(2):247-53.Slide62

Policy EvaluationCross sectional analyses comparing jurisdictions with a policy to those without can be severely biased (between-cluster)States/counties/hospitals may institute policies in response to a problem (reverse causality)

States/counties/hospitals may institute policies if they are more progressive; already healthier (selection, omitted variable bias)

Need longitudinal inference, comparing outcomes before and after policy change within a given unit to contemporaneous controls for time trend irrespective of policy changeSlide63

Difference in Difference

T=Treatment C=Control B=Before A=After

DiD

1) (Y

T

– Y

C, A) – (YT – YC, B) = 1

2) (YA – Y

B, T) – (YA – Y

B, C) = 1 Cross-sectional: (YT –

YC, A) = 3 Pre-Post: (Y

A – YB, T) = 3

Could be an example of selection for a positive health indicatore.g. breastfeeding laws Slide64

Difference in Difference

T=Treatment C=Control B=Before A=After

DiD

1) (Y

T

– Y

C, A) – (YT – YC, B) = -1

2) (YA – YB, T) – (Y

A – YB, C) = -1

Cross-sectional: (YT – YC, A) =

1 Pre-Post: (YA – Y

B, T) = 1 Could be an example of reverse causality for a negative indicator

e.g. school nutrition/activity policies and obesitySlide65

Fixed Effects Panel AnalysisDesignSame as Difference in Difference but extends beyond two periods and requires multiple units

Y

jt

=

β

o + βPjt +

βj + βt

+ βXjt +

ejtWithin-unit contrasts with between-unit contemporaneous controls are achieved by entering unit level dummy variables (

βj) along with time indicator (βt)3+ time points can control for unit-specific trends that may be simultaneous to policy changes Slide66

Sample Software CodeSASProc

reg

; *Proc logistic;

Class state year;

Model Y= x state year;

Run;

Proc logistic;Class state year;Model Y= x year;Strata state;

Run;STATA

xi: reg y x i.year

i.statexi: logit y x

i.year i.state

xi: xtreg y x i.year,

i(state) fexi: xtlogit y x

i.year, i(state) fe

xi: clogit y x i.year, group(state) Slide67

Applied Example ObjectiveTo evaluate and compare the impact of state-specific changes in smoking-related policies on childhood asthma prevalence & severitySlide68

DataIndividual-level outcome data come from available waves of NSCH (2003 & 2007)

Parent-reported current asthma

Severity of current asthma (mild v. moderate/severe)

Chronic ear infection (3+ in past year)

Control factors: child age, sex, race/ethnicity, primary language, family structure, insurance status/type, household poverty, and parental education

Longitudinal state policy data from CDC

Cigarette TaxesClean air legislationMedicaid coverage of cessation servicesSlide69

Data StepsAppend both years of survey dataSet statement in SASAppend in STATA

Need to assign unique ID numbers

Merge policy level detail by state and year

If exploring lags, create in policy database prior to merging (manually or with lag function)

State

Year

Tax

Tax_Lag1

Tax_Lag2

CT

20030.59

0.340.34CT

20072.001.51

1.51Slide70

Model SpecificationUsed OLS instead of logistic because output is interpretable as probabilitiesY = β

0

+

β

sex +

β

age + βrace + βlanguage + βfam_structure + βeducation + β

poverty + βinsurance + βyear + βstate +

βtax Slide71

ResultsCigarette taxesAsthma prevalence: ↓16% per $1 increase, p=0.09

Moderate/severe asthma: ↓29% per $1 increase, p=0.04

Clean air legislation

No significant effects

Medicaid coverage for cessation therapy

Chronic ear infection: ↓60% with expansion, p<0.01Slide72

Effects for a $1 increase in cigarette taxes, relative decrease from 2003 outcome level 3.1%Interpretation

Fixed effects

: On average, states that increase their tax by $1 see a significant ~30% drop in moderate/severe asthma

Cross-sectional

: On average, tax differences between states are not associated with moderate/severe asthma

Comparison of Model Inference

Outcome

Fixed

Effects Panel Analysis

Cross-Sectional

Moderate/Severe Asthma Prevalence-29.8%(-57.7%

, -1.9%) 3.8%(-4.4%, 12%)Slide73

State-Specific Inference

Kentucky

2003

2007

∆

Moderate/Severe

Asthma

4.3%3.2%

-1.1%Cigarette Taxes

3¢30¢27¢

Predicted effect = 0.27 * -0.9% pts

= -0.24% pts so the tax hike was responsible for 22% of the decline in moderate/severe asthma (-.24%/-1.1%)If state had increased taxes by $1 (73¢ more than actual), decline would have been greater by 0.73 * -0.9% = -0.66% -- translates to a relative drop of 41% instead of 26% (absolute change of -1.76% instead of -1.1% relative to baseline of 4.3%) Slide74

Smoking Legislation and Children’s Secondhand Smoke ExposureCross-sectional models showed that clean air legislation was associated with reduced exposure

Fixed effect (longitudinal models) showed no association so it appears that more progressive states enacted legislation

However, there

w

as an impact of cigarette taxes that served to reduce racial/ethnic and income disparities

Greater effects for White and low-income children

Hawkins SS, Chandra A,

Berkman

L. The Impact of Tobacco Control Policies on Disparities in Children's Secondhand Smoke Exposure: A Comparison of Methods. Matern Child Health J. 2012 Mar 29. [Epub ahead of print]Slide75

Medical Marijuana Laws and Adolescent UseRE models showed marijuana laws to be associated with greater useFE models showed no association between medical marijuana laws and use

Likely a reverse causal association that states with greater use among the constituency are more likely to pass legislation

Harper S, Strumpf EC, Kaufman JS. Do medical marijuana laws increase marijuana use? Replication study and extension. Ann

Epidemiol

. 2012 Mar;22(3):207-12.Slide76

SummaryFor multilevel analyses, you typically want to examine effects at both levels so you need models that allow variation at multiple levels

Generally want to use RE or GEE models

Use hybrid models incorporating cluster means to account for cluster-level confounding and achieve “true” within-cluster effects

For panel/longitudinal analyses, you typically only care about the within-unit inference obtained from FE models

Generally want to use conventional FE modelsSlide77

Homework AssignmentDataset of births matched to same mother (momid)

Use this to compare inference and interpret the effects of smoking on

birthweight

(

birwt

is continuous and LBW, <2500 g) from these models

Conventional regression with cluster robust SEsRandom interceptGEE (exchangeable)Fixed effectsHybrid fixed effects (cluster mean adjusting)

Include smoke and the covariates of mage, male, married, hsgrad, somecoll, collgrad, black, pretri2, pretri3,

novisitDescribe the benefits and disadvantages of the above modelsSlide78

Conventional regression with cluster robust SEs

proc

surveyreg

data

=new;cluster

momid;

model birwt

= smoke mage male black pretri2 pretri3 novisit married hsgrad

somecoll

collgrad;run

;

Number of Observations 8604 Mean of

birwt 3469.9

Sum of birwt

29855287

Design Summary

Number of Clusters 3978

Fit Statistics R-square 0.09307 Root MSE 502.33

Denominator DF 3977

Estimated Regression Coefficients

Standard Parameter Estimate Error t Value

Pr > |t|

black -222.62920 28.9780315 -7.68 <.0001 pretri2 15.16062 17.9602368 0.84 0.3987

pretri3 48.01495 38.4015782 1.25 0.2112

novisit -192.22265 85.1568122 -2.26 0.0240

married 50.13834 26.7366384 1.88 0.0608

hsgrad 58.16388 25.6906615 2.26 0.0236

somecoll 87.25244 28.3130890 3.08 0.0021

collgrad

102.54592 29.1243183 3.52 0.0004

NOTE: The denominator degrees of freedom for the t tests is 3977.Slide79

Random Interceptproc

glimmix

data

=new

method=quad(

initpl=5

) noinitglm;

class momid;

model birwt

= smoke mage male black pretri2 pretri3 novisit married

hsgrad somecoll

collgrad /solution

;random intercept /

subject=momid;

run;

Covariance Parameter Estimates

Standard Cov Parm Subject Estimate Error

Intercept momid

114042 4252.84 Residual 138531 2904.36

Solutions for Fixed Effects

Standard

Effect

Estimate Error

DF t Value Pr > |t|

Intercept 3106.48 40.9177 3972 75.92 <.0001 smoke -219.73 18.2327 4620 -12.05 <.0001

mage 7.8386 1.3469 4620 5.82 <.0001

male 120.67 9.5870 4620 12.59 <.0001 black -216.99 28.2373 4620 -7.68 <.0001

pretri2 8.1155 15.5535 4620 0.52 0.6018 pretri3 43.7332 34.5642 4620 1.27 0.2058

novisit

-158.13 54.7582 4620 -2.89 0.0039

married 53.1767 25.4802 4620 2.09 0.0369

hsgrad

62.9070 24.9831 4620 2.52 0.0118

somecoll

88.8345 27.2365 4620 3.26 0.0011

collgrad

100.40 27.9142 4620 3.60 0.0003Slide80

GEE (Exchangeable)proc

genmod

data

=new;

class

momid;model

birwt=smoke mage male black pretri2 pretri3

novisit married hsgrad

somecoll collgrad

;repeated

subject=momid

/corr=exc

;run;

Exchangeable Working Correlation

Correlation 0.4428629745

GEE Fit Criteria

QIC 8624.9632 QICu 8616.0000

Analysis Of GEE Parameter Estimates Empirical Standard Error Estimates

Standard 95% Confidence

Parameter Estimate Error Limits Z Pr

> |Z|

Intercept 3108.079 42.6106 3024.564 3191.594 72.94 <.0001 smoke -220.554 19.1370 -258.062 -183.046 -11.53 <.0001

mage 7.7884 1.4092 5.0264 10.5504 5.53 <.0001 male 120.5962 9.6715 101.6403 139.5521 12.47 <.0001

black -217.102 29.2310 -274.394 -159.810 -7.43 <.0001

pretri2 8.2117 15.7123 -22.5838 39.0073 0.52 0.6012 pretri3 43.7602 36.4548 -27.6899 115.2103 1.20 0.2300

novisit

-158.761 73.4419 -302.705 -14.8177 -2.16 0.0306 married 53.1095 26.6567 0.8632 105.3557 1.99 0.0463

hsgrad 62.8929 25.7679 12.3887 113.3971 2.44 0.0147

somecoll

88.8847 28.3292 33.3604 144.4089 3.14 0.0017

collgrad

100.5458 29.0306 43.6469 157.4447 3.46 0.0005Slide81

Fixed Effectsproc

glm

data

=new;

absorb

momid;model

birwt=smoke mage male black pretri2 pretri3

novisit married hsgrad

somecoll collgrad

;run;

quit; Dependent

Variable: birwt

Standard

Parameter Estimate Error t Value Pr

> |t|

smoke -104.2849992 29.14818572 -3.58 0.0004 mage 22.7041539 3.00947000 7.54 <.0001

male 125.3693243 10.93838681 11.46 <.0001 black 0.0000000 B . . .

pretri2 2.0485058 18.62282276 0.11 0.9124 pretri3 45.4074853 41.48998267 1.09 0.2738 novisit -99.1457340 67.59271486 -1.47 0.1425 married 0.0000000 B . . .

hsgrad

0.0000000 B . . .

somecoll 0.0000000 B . . .

collgrad 0.0000000 B . . .

NOTE: The X'X matrix has been found to be singular, and a generalized inverse was used to solve

the normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely estimable.Slide82

Hybrid Fixed Effects in RE ModelPROC

SUMMARY

NWAY

DATA=mla.smoking2;

VAR smoke mage male pretri2 pretri3 novisit;

CLASS momid;

OUTPUT OUT

=cluster MEAN=m_smoke

m_age

m_male m_pretri2 m_pretri3 m_novisit

;RUN;

DATA new;MERGE

mla.smoking2 cluster;BY

momid;RUN

;proc

glimmix

data=new method=quad(initpl=5) noinitglm;

class momid;

model birwt

= smoke m_smoke mage male black pretri2 pretri3

novisit married hsgrad

somecoll collgrad

m_age

m_male m_pretri2 m_pretri3 m_novisit

/solution ;random

intercept / subject=

momid;run

;

Covariance

Parameter

Estimates

Standard

Cov

Parm

Subject Estimate Error

Intercept

momid

114019 4219.03

Residual 137212

2867.98

Solutions for Fixed Effects

Standard

Effect

Estimate

Error

DF t Value Pr > |t|

Intercept 3227.63 45.8709 3966 70.36 <.0001

smoke -104.28 29.2192 4620 -3.57 0.0004

m_smoke

-184.57 37.2826 4620 -4.95 <.0001

mage 22.7041 3.0168 4620 7.53 <.0001

male 125.37 10.9651 4620 11.43 <.0001

black -224.76 28.3522 4620 -7.93 <.0001

pretri2 2.0485 18.6682 4620 0.11 0.9126

pretri3 45.4075 41.5911 4620 1.09 0.2750

novisit

-99.1457 67.7575 4620 -1.46 0.1435

married 44.0668 26.0170 4620 1.69 0.0904

hsgrad

63.3957 25.2540 4620 2.51 0.0121

somecoll

91.8638 27.8175 4620 3.30 0.0010

collgrad

108.02 28.9992 4620 3.73 0.0002

m_age

-18.3473 3.3681 4620 -5.45 <.0001

m_male -20.9414 22.3241 4620 -0.94 0.3483

m_pretri2 22.7864 33.7079 4620 0.68 0.4991

m_pretri3 2.4144 74.8960 4620 0.03 0.9743

m_novisit

-154.42 114.85 4620 -1.34 0.1788