Effects of Survey Mode on Responses An Experiment of Social Desirability Biases in Telephone Interviewer and Online Surveys Indeok Song Joongbu University Korea Sung Kyum Cho Chungnam National University ID: 772848
Download Presentation The PPT/PDF document "Effects of Survey Mode on Responses:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Effects of Survey Mode on Responses: An Experiment of Social Desirability Biases in Telephone, Interviewer, and Online Surveys Indeok Song Joongbu University, Korea Sung Kyum Cho Chungnam National University
ContentsBackground: 3 Surveys on Gambling Addiction Research Question & PurposeReview of Previous ResearchMethods Results Discussions & Suggestions
Results of 2010’s Surveys
Results of 2010’s Surveys
Research Question What is the reason for the inconsistencies among 3 surveys’ results? Which mode does produce the most accurate estimates?Sampling? Survey Mode?
Previous ResearchSurvey modes “can be equivalent but are not always identical” (Riva et al., 2003, p. 79)Sensitivity & Social desirability (Booth-Kewley, Larson, & Miyoshi, 2007; Couper , 2000; Huang, 2006; Joinson , 1999; Kays , Gathercoal , & Buhrow , 2011; Tourangeau & Yan, 2007, 2011 and so on) Mode of admin.: Self- vs. Interviewer-administered Self-administration increases reporting of socially undesirable behaviors ( Corkrey & Parkinson, 2002; Currivan et al., 2004; Hochstim , 1967; Moskowitz , 2004; Mott, 1985; Richman et al., 1999, and so on) ∴ Paper-Pencil, Online > Telephone, Interviewer
Previous ResearchInterviewer or 3rd party presenceSocial desirability bias is worse in telephone survey than in face-to-face interviews (Aquilino, 1994; Groves & Kahn, 1979; Holbrook et al., 2003; Johnson, Hougland , & Clayton, 1989; Leeuw & van der Zouwen , 1988, and so on) The presence of others affects responses to sensitive questions ( Aquilino , Wright, & Supple, 2000; Harrison, 2001; Lombard & Ditton , 1997; Sudman , 2001; Tourangeau & Yan, 201, and so on ) Online survey yields the lowest level of other people’s presence ∴ Online > Interviewer > Telephone
Research Purpose Controlling for variances from sampling and different groups of samples, to examine the effects of survey modes on the responses to sensitive questions.
MethodsPretest-posttest 3 between-group experimental design93 undergraduate students 66 analyzedOut of 66, male 38 (57.6%), female 28 (42.4%), mean age 20.61 ( SD =1.97) Paper-pencil ( N =93) 2 weeks Random Assignment Online ( n =21) Telephone ( n =24) Interview ( n =21)
MethodsQuestionnaire DesignAddiction ScalesGambling addiction (4pt., 9 items) M=9.24, SD=2.54, α=.84Internet addiction (4pt., 15 items) M =29.13, SD =7.24, α =.90 Game addiction (4pt., 20 items) M =26.80, SD =8.84, α =.96 Dissatisfaction with self-appearance (5pt., 6 items) M =14.43, SD =3.71, α =.66 Sexual values (5pt., 10 items) M =27.88, SD =7.14, α =.89
MethodsSexual experienceAge of the first experiencen=27, M=18.59, SD=1.67Number of sexual partnersn =26, M =10.81, SD =3.68 Sexual behaviors (physical contact, cuddle, kiss, caress, intercourse) experienced in current relationship n =19, M =3.68, SD =1.42 Demographics Gender, age, household-income
ResultsHomogeneity among groups (pretest: paper-pencil)Sourcedf F p Gambling Addiction (2, 63) 1.29 .28 Internet Addiction (2, 63) .09 .91 Game Addiction (2, 63) 2.13 .13 Dissatisfaction with Self (2, 63) .31 .73 Sexual Values (2, 63) .28 .76 Age of 1 st Experience (2, 24) 1.89 .19 No. of Partners (2, 23) .59 .56 Sexual Behaviors (2, 16) .24 .79
Responses for Game Addiction
Responses for Dissatisfaction with Self-Appearance
2 x 3 Factorial ANOVAsIndependent Variables2 groups (‘Low’ & ‘High’) by median split3 modes (Online, Telephone, Interview)Dependent VariablesChanges in responses = Pretest – PosttestDVsNM SD Gambling Addiction 66 .27 2.30 Internet Addiction 66 1.08 6.78 Game Addiction 66 1.14 8.05 Dissatisfaction with Self 66 1.18 3.13 Sexual Values 66 1.44 4.65 Age of 1 st Experience 22 . 68 1.49 No. of Partners 22 -3.95 16.76 Sexual Behaviors 16-.131.46
Changes in Responses for Gambling Addiction
Changes in Responses for Internet Addiction
Changes in Responses forGame Addiction
Changes in Responses for Dissatisfaction with Self-Appearance
Changes in Responses for Sexual Values
Changes in Responses for Age of the 1st Intercourse
Summary of the ResultsResponses for ‘Game Addiction’ & ‘Dissatisfaction with Self-Appearance’Increased in the online survey modeDecreased in the telephone & interview survey modesThose who scored higher on the sensitive questions are more likely to change their responses in a socially desirable directionLess addicted (gambling, Internet, game)Less dissatisfied with self-appearanceLess liberal (more conservative) sexual valuesYounger in the age of the 1st sexual intercourse
Discussions & SuggestionsSurvey responses to sensitive (privacy-related) questions are affected by different survey modes.Online survey in which respondents are less concerned about their privacies is more likely to elicit franker (more accurate) responses.Paper-pencil (group administration), telephone, interview modes lead respondents to privacy concern & misreport.Suggestion: Dual-mode survey Telephone Sampling Online Questionnaire
Any idea or suggestion would be welcomed.