Collection and Research Initiatives for Non-Response - PowerPoint Presentation

1. Collection and Research Initiatives for Non-ResponseCindy UbartasChief, Research and Innovation Section, Collection Planning and Research DivisionStatistics Canada

2. OutlineData Collection Operations at Statistics CanadaContextFactors that contributed to declining response rates in social surveysCurrent state of collectionInitiatives to better understand non-response and improve response rates

3. Data Collection Operations at Statistics CanadaStatistics Canada conducts all national social and economic surveys across the country More centralized than US statistical organizations5 Computer-Assisted Telephone Interview (CATI) call centresNational sample for CATI as opposed to sample assigned to a specific call centreMost interviewers are still working from home5 Computer-Assisted Personal Interview (CAPI) offices Vast majority of surveys are multi-mode (e.g. Web/CATI/CAPI) Web collection: Electronic Questionnaire (EQ)One single questionnaire for all modes

4. ContextDownward trend in response rates in social surveysExample: Labour Force Survey (LFS) is mandatory in CanadaStable response rate observed in 2015-2016 (≈ 90%)Impact of the PandemicSlight increase since the beginning of 2022

5. Factors/challenges that contributed to declining response rates in social surveysChanges in respondent behaviour and preference in terms of communication channels Requirements for timely data increased with the pandemic New data requirements identified by the Disaggregated Data InitiativeLaunched in fall 2021Objective: obtain statistical information that highlights the experiences of specific population groups (e.g. Indigenous peoples, racialized populations, people living with disabilities) Need to maximize responses in short collection periods with large sample sizesLarge number of interviewers hired during the pandemicLarge employee turnover throughout the pandemic

6. Current state of collectionEQ works really well in generalBetween 15% to 35% response rates in EQGeneric electronic questionnaire (EQ) platform developed many years agoIn addition to traditional paper letters, the ability to send mass emails and SMS was developed Modes of contacts added to the collection strategiesBack to regular CAPI since the fall 2022Results mitigated Requirements now different for conducting in-person interviewsCATI collectionLarge variations between surveysImprovement observed for some of them in the last few months

7. What are the short/mid-term plans?Action items to improve CAPI resultsHiring still ongoingFocus on training and monitoring (including recordings of parts of interviews eventually)Improved reports to help the regional offices keep track of performance of their interviewersPotentially offer virtual interviewing (e.g., videoconferences) as an option Research project this year: explore people’s interest according to their characteristics & determine how it could work in practiceReintroduce the use of CATI Adaptive Collection DesignCurrently testing new functionalities in our collection system to facilitate this processLFS: units are in collection for 6 months, so could take advantage of collection characteristics from the previous cyclesPursue the use of a mobile application to collect dataConvenient way for respondents to complete some of the most burdensome surveysExample: Diary used for the Survey of Household Spending Pilot project done in the fall 2021 with subjective well-being in Canada (Vitali-T-Stat)Requires strong communication between a wide range of internal partners and App StoreNeed experienced developers with mobile-phone applications and extensive testing at each phase of the project

8. Research projects on non-response SegmentationUsed with success for the 2016 and the 2021 CensusesMain ideas:Divide population into homogeneous groups in terms of response/contact rates Target more difficult groups using public relations communication campaignExamples for surveys: Short video clips highlighting our survey data, on topics selected to attract the attention of these segmentsShort articles made available to the public through social media and community-level mediaSpecial landing page on our website where all the tools for this campaign are availableDevelop an insert to mail to selected households at the same time than the introduction lettersDevelop partnerships with organisations that could make these groups more aware of our surveys and their benefitsObjective: apply this campaign for the LFSCurrently looking at characteristics of non-respondents to identify targeted subgroups

9. Research projects on non-response Analysis of feedback from the interviewersNotes provided during the attempts and debriefing questionnairesCould potentially add observations on the field (presence of children, neighbourhood, etc.)Need parsing/use of artificial intelligence methods to better analyze results Can help understand:Characteristics of non-respondentsReasons for non-response, including understanding/interpretation of the questions AdvantagesShort-term: can improve the overall quality of the next iteration of a questionnaire for a surveyLong-term: could modify our communication material (letters, introduction messages in CATI/CAPI) based on the respondent’s profile

10. Research projects on non-response Non-response Bias StudyNon-response follow-up (NRFU) survey to get more insights on the characteristics of non-respondentsShort (15-minutes) EQ developed in our collection system with accelerated timelinesCould potentially help detecting the presence of bias among respondentsFirst candidate: International Study of Adults (ISA) in May 2023Will also serve as an example to establish a framework (standard collection vehicle) for future NRFU evaluations

11. Thank you!Questions?cindy.ubartas@statcan.gc.ca