Dru Rose Methods of Data Collection Surveys and Polls Experimental Studies Observational Studies Dru Rose Polls and Surveys Poll Few questions Multichoice type only Survey Many questions ID: 276298
Download Presentation The PPT/PDF document "Non-sampling Errors in Polls and Surveys" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Non-sampling Errors in Polls and Surveys
Dru
Rose Slide2
Methods of Data Collection
Surveys
and Polls
Experimental
Studies
Observational
Studies
Dru
Rose Slide3
Polls and Surveys
Poll
Few questions
Multi-choice type onlySurvey
Many questions
May have branches and skips
May have a “tick multiple boxes” option
May have open-ended, write a comment questionsDru Rose Slide4
Non-sampling
Errors
Sampling Errors
(random process)
Selection bias Non-response bias
Self selection bias Question
effects
Behavioural
considerations
Interviewer
effects
Survey-format effects
Transfer
of findings
SamplingSlide5
Target population
(
e.g. adults in NZ)
Sampling
frame
(e.g. households with a landline phone)
Not included in sampling
frame
Not eligible for survey
Cannot be contacted
Refuse to respond
Incapable of responding
SAMPLED
POPULATION
Selection Bias:
Population sampled is not exactly the population of interest.
Slide6
Sources of Non-sampling Errors
Non-response bias When people who have been targeted to be surveyed do not respond:
Potential b
ias if
n
on-respondents are
likley to behave differently to respondents with respect to the question being asked.e.g. Non-respondents in an employment survey are likely to be those who work long hours.Slide7
Sources of Non-sampling Errors Self-selection bias
People decide themselves whether to be surveyed or not.
Dru
Rose Slide8
Self-selection bias:
phone-in or internet pollsSlide9
Self-selection bias
“The Centre for the Study of Higher Education (University of Melbourne) has sought Macquarie's support to survey a representative sample of their academic staff.
Interested
members of staff should contact Professor John Loxton
about obtaining copies of the survey questionnaire.”
Staff Newsletter (Macquarie University)
13 November 1998 Slide10
Sources of Non-sampling Errors Question effects
Subtle variations in wording can have an effect on responses.
e.g. “Should euthanasia be legal?”
vs. “Should voluntary euthanasia be legal?”
People are more likely to favour “voluntary” euthanasia.Slide11
18 August 1980 New York Times/CBS News Poll
“Do you think there should be an amendment to the constitution prohibiting abortions?”
Yes 29%
No 62% Later the same people were asked:
“Do you think there should be an amendment to the constitution protecting the life of the unborn child?”
Yes
50% No 39%Slide12
Ethnicity
1986
1991
1996
Single
Ethnicity
94.6
94.3
81.0
European
81.2
78.1
65.8
Maori
9.1
9.6
7.6
Two Ethnicities
4.0
4.5
11.2
European & Maori
2.9
2.7
4.7
Two European gps
0.0
0.6
4.5
Question Effects in the NZ Census
1986: “What is your ethnic origin?
(Tick the box or boxes which apply to you.)
1991: “Which ethnic group do you belong to?”
(Tick the box or boxes which apply to you.)
1996: “Tick as many circles as you need to show which ethnic group(s) you belong to.”Slide13
Sources of Non-sampling Errors Behavioural considerations
People tend to answer questions in a way they consider to be socially desirable.
e.g. pregnant women being asked about their drinking habits
may be reluctant to admit that they drink alcohol
Dru
Rose Slide14
Sources of Non-sampling Errors Interviewer effects
Different interviewers asking the same question can obtain different results.
e.g. the sex, race, religion , manner of the interviewer may influence how people respond to a particular question.
Dru
Rose Slide15
Interviewer Effects in Racial Questions
In 1968, one year after a major racial disturbance in Detroit, a sample of black residents were asked: “Do you personally feel that you trust most white people, some white people or none at all?”
White interviewer
: 35% answered “most”Black interviewer
:
7% answered “most”Slide16
Sources of Non-sampling Errors Survey-format effects
-question order
e.g.
“To what extent do you think teenagers are affected by peer pressure when drinking alcohol ?”followed by:
“ Name the top 5 peer pressures you think teenagers face today.”
-survey layout
-
interviewed by phone or in-person or mail.Slide17
Sources of Non-sampling Errors Transferring findings
Taking the data from one population and transferring the results to another.
e.g. Auckland opinions may not be a good indication of New Zealand opinions.
Auckland
sample
New ZealandSlide18
Non-sampling Errorscan be much larger than sampling errors
are always presentcan be virtually impossible to correct for after the completion of survey
virtually impossible to determine how badly they will affect the resultg
ood surveys try to minimize them in the design of the survey (e.g. do a pilot survey first)Slide19
Surveys / Polls
A report on a sample survey/poll should include:Who carried it out and who funded it
target population (population of interest)sample selection method
the sample size and the margin of errorthe date of the surveythe exact question(s) being askedthe results the claims (inferences) made