An Experiment of Social Desirability Biases in Telephone Interviewer and Online Surveys Indeok Song Joongbu University Korea Sung Kyum Cho Chungnam National University Contents ID: 262489
Download Presentation The PPT/PDF document "Effects of Survey Mode on Responses:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Effects of Survey Mode on Responses: An Experiment of Social Desirability Biases in Telephone, Interviewer, and Online Surveys
Indeok
Song
Joongbu
University, Korea
Sung
Kyum
Cho
Chungnam
National UniversitySlide2
ContentsBackground: 3 Surveys on Gambling Addiction Research Question & PurposeReview of Previous ResearchMethods
Results
Discussions & SuggestionsSlide3
Results of 2010’s SurveysSlide4
Results of 2010’s SurveysSlide5
Research Question What is the reason for the inconsistencies among 3 surveys’ results? Which mode does produce the most accurate estimates?Sampling?
Survey Mode?Slide6
Previous ResearchSurvey modes “can be equivalent but are not always identical” (Riva et al., 2003, p. 79)Sensitivity & Social desirability (Booth-Kewley, Larson, & Miyoshi, 2007; Couper
, 2000; Huang, 2006;
Joinson
, 1999;
Kays
,
Gathercoal
, &
Buhrow
, 2011;
Tourangeau
& Yan, 2007, 2011 and so on)
Mode of admin.: Self- vs. Interviewer-administered
Self-administration increases reporting of socially undesirable behaviors
(
Corkrey
& Parkinson, 2002;
Currivan
et al., 2004;
Hochstim
, 1967;
Moskowitz
, 2004; Mott, 1985; Richman et al., 1999, and so on)
∴
Paper-Pencil, Online
> Telephone, InterviewerSlide7
Previous ResearchInterviewer or 3rd party presenceSocial desirability bias is worse in telephone survey than in face-to-face interviews (Aquilino, 1994; Groves & Kahn, 1979; Holbrook et al., 2003; Johnson, Hougland
, & Clayton, 1989;
Leeuw
& van
der
Zouwen
, 1988, and so on)
The presence of others affects responses to sensitive questions
(
Aquilino
, Wright, & Supple, 2000; Harrison, 2001; Lombard &
Ditton
, 1997;
Sudman
, 2001;
Tourangeau
& Yan, 201, and so on
)
Online survey yields the lowest level of other people’s presence
∴ Online
>
Interviewer
> TelephoneSlide8
Research Purpose Controlling for variances from sampling and different groups of samples, to examine the effects of survey modes on the responses to sensitive questions.Slide9
MethodsPretest-posttest 3 between-group experimental design93 undergraduate students 66 analyzedOut of 66, male 38
(57.6%),
female 28
(42.4%),
mean age 20.61
(
SD
=1.97)
Paper-pencil
(
N
=93)
2
weeks
Random
Assignment
Online
(
n
=21)
Telephone
(
n
=24)
Interview
(
n
=21)Slide10
MethodsQuestionnaire DesignAddiction ScalesGambling addiction (4pt., 9 items) M=9.24, SD=2.54, α=.84Internet addiction (4pt., 15 items)
M
=29.13,
SD
=7.24,
α
=.90
Game addiction (4pt., 20 items)
M
=26.80,
SD
=8.84,
α
=.96
Dissatisfaction with self-appearance (5pt., 6 items)
M
=14.43,
SD
=3.71,
α
=.66
Sexual values (5pt., 10 items)
M
=27.88,
SD
=7.14,
α
=.89Slide11
MethodsSexual experienceAge of the first experiencen=27, M=18.59, SD=1.67Number of sexual partnersn
=26,
M
=10.81,
SD
=3.68
Sexual behaviors (physical contact, cuddle, kiss, caress, intercourse) experienced in current relationship
n
=19,
M
=3.68,
SD
=1.42
Demographics
Gender, age, household-incomeSlide12
ResultsHomogeneity among groups (pretest: paper-pencil)Sourcedf
F
p
Gambling Addiction
(2, 63)
1.29
.28
Internet Addiction
(2, 63)
.09
.91
Game Addiction
(2, 63)
2.13
.13
Dissatisfaction with Self
(2, 63)
.31
.73
Sexual Values
(2, 63)
.28
.76
Age of 1
st
Experience
(2, 24)
1.89
.19
No. of Partners
(2, 23)
.59
.56
Sexual Behaviors
(2, 16)
.24
.79Slide13
Responses for Game AddictionSlide14
Responses for Dissatisfaction with Self-AppearanceSlide15
2 x 3 Factorial ANOVAsIndependent Variables2 groups (‘Low’ & ‘High’) by median split3 modes (Online, Telephone, Interview)Dependent VariablesChanges in responses = Pretest – PosttestDVsNM
SD
Gambling Addiction
66
.27
2.30
Internet Addiction
66
1.08
6.78
Game Addiction
66
1.14
8.05
Dissatisfaction with Self
66
1.18
3.13
Sexual Values
66
1.44
4.65
Age of 1
st
Experience
22
.
68
1.49
No. of Partners
22
-3.95
16.76
Sexual Behaviors
16-.131.46Slide16
Changes in Responses for Gambling AddictionSlide17
Changes in Responses for Internet AddictionSlide18
Changes in Responses forGame AddictionSlide19
Changes in Responses for Dissatisfaction with Self-AppearanceSlide20
Changes in Responses for Sexual ValuesSlide21
Changes in Responses for Age of the 1st IntercourseSlide22
Summary of the ResultsResponses for ‘Game Addiction’ & ‘Dissatisfaction with Self-Appearance’Increased in the online survey modeDecreased in the telephone & interview survey modesThose who scored higher on the sensitive questions are more likely to change their responses in a socially desirable directionLess addicted (gambling, Internet, game)Less dissatisfied with self-appearanceLess liberal (more conservative) sexual valuesYounger in the age of the 1st
sexual intercourseSlide23
Discussions & SuggestionsSurvey responses to sensitive (privacy-related) questions are affected by different survey modes.Online survey in which respondents are less concerned about their privacies is more likely to elicit franker (more accurate) responses.Paper-pencil (group administration), telephone, interview modes lead respondents to privacy concern & misreport.Suggestion: Dual-mode survey
Telephone Sampling
Online QuestionnaireSlide24
Any idea or suggestion would be welcomed.