Nonresponse Bias in a Nationwide Dual-Mode Survey

Transcript:Nonresponse Bias in a Nationwide Dual-Mode Survey:
Nonresponse Bias in a Nationwide Dual-Mode Survey Matthew DeBell, Stanford University Natalya Maisel, Stanford University Ted Brader, University of Michigan Vanessa Meldener, Westat ITSEW 2018, Durham NC American National Election Studies 2016 Time Series Study Survey of adult US citizens Seeks to explain voter turnout & candidate choice in presidential elections Dual mode ABS Face-to-face Mail-to-internet Clustered not clustered n=1180 n=3090 RR1=50% RR1=44% This Talk What’s non-random about non-response? 1. Accuracy of estimates 2. “Easy to get” vs “hard to get” 3. Respondents vs non-respondents in a NRFU study What if we got more respondents from the Web sample? Conclusions or implications for… accuracy of estimates field effort field strategy & adaptive design 1. Accuracy of estimates Comparisons to benchmarks (Current Population Survey) Not necessarily non-response error, but indicative Age, gender, education, race/ethnicity, marital status, income, household size, home tenure, region, employment status, and nativity Accuracy of estimates Errors of 4 points or more (unweighted) 2. Hard to get? “Easy to get” Rs: make contact with few attempts & cooperate readily “Hard to get” Rs: require multiple contact attempts or refusal conversion Definitions of hard-to-get Web: if a refusal conversion letter was mailed or a refusal was recorded (39%) FtF: any refusal, >5 contacts, or late contact (41%) We assume non-respondents are more like hard-to-get Rs Differences between easy- and hard-to-get Rs would indicate field effort matters How “hard to get” Rs differ from easier Rs Mail-to-web Younger Less: education, white, married More: Hispanic, renters Lower income More Southern, less NE Less likely to vote More Trump voters Face-to-face Younger No difference No difference Higher income Less Southern, more NE No difference (ns) No difference (ns, opposite) 3. Non-Response Follow-Up (NRFU) Study Mail survey Both responding and non-responding households n=4,725 Study name & sponsorship differed from ANES One page paper questionnaire with 18 questions RR = 39% NRFU Results: no differences Ethnicity (Hispanic) Talk to neighbors Like college professors Like news reporters Children under 18 Party ID Presidential vote choice NRFU Results: differences Web Voter turnout (10 points) Like surveys (.11) Internet access (15 points) Worry personal privacy (.08) Interpersonal trust (.04) Free time (.02) Education Age (2.6 years) Face-to-face Voter turnout (10 points) Like surveys (.15) Like talking politics (.05) 4. What if we got NRFU Rs in the first place? Estimated effect on Web sample of adding 400 Rs by mail like NRFU Rs 4. What if we