Session 11 Planning for adoption of electronic data collection technologies

Session 11

Planning for adoption of electronic data collection technologies

UNSD presentation

Overview of the presentation

Growing dependency on technology

Paper-based

vs

electronic data collection approaches

Advantages and disadvantages of

using electronic data collection

technologies

Decision making

Planning considerations

Implementing handheld electronic devices for data collection

Implementing Internet for data collection

Conclusions

Growing dependency on technology

Recent round of censuses have introduced new technologies in conducting censuses

Optical data capture and Internet in 1990s

GIS in 2000s

Electronic data collection technologies in 2010sTechnology adoption through out census operationCoverage, Data quality, Timeliness, Accessibility

Main considerations in adoption of new technology

Cost-effectiveness

– adoption of technology in effective way to minimize the cost

Cost-benefit- efficient use of technology for maximizing the benefits taking into account all possible statistical activitiesRe- designing census procedures – new technology has impacts on all phases Capacity building – human resource, technological infrastructureSustainability of introducing new technology – long term plan for ensuring efficient useRequires developing a global plan

for modernization of national statistical system,

very

early

planning for census and more time for planning and preparation

Paper-based

vs

Electronic collection approach

Manual

Computer-assisted, automaticKey entry/computer-assistedOMR/OCR/ICRManualcomputer-assisted, automaticAutomated/built-inPaper-based approachData CollectionData Entry/Capture

Data Collection/Entry/Coding/Editing

Electronic

collection

a

pproach

Data Coding/Editing

Data Coding/Editing

(

partial)

Advantages of using electronic data collection technologies

Improve data quality through built-in

consistency checks

and logical validation rules during the interview

Allow a more accurate progression through the questionnaire due to automated skip patterns and mandatory itemsOptimize time spent by enumerator and respondentsImprove field management and real time monitoring of enumeration activities (eg. built-in date and time stamp could be useful for supervision)Allow collecting additional information that cannot be collected using paper-based approaches, such as: GPS co-ordinates, pictures, videos, date/time stamps, etc.Avoid manual data entry errors;

Advantages of using electronic data collection technologies

Significant improvement in timeliness of releasing census results

Allow the provision of on-screen

help

- for enumerators from the screen of electronic device - easier for the enumerators to access definitions or other items linking to each question Allow computerized case management, capacity to monitor and control over various census operationsUtilize GPS features and digital mapping capabilities such as EA maps and/or address information can be loaded onto the device to help the enumerator find the correct housing units to visit

Disadvantages of electronic data collection technologies

There are a number of

risks/disadvantages associated

with the introduction of

new technologyTechnology increases dependence on technology providers and introduces new challenges and risksLarge investment (eg high equipment costs with limited long-term use)More time needed during the planning and preparation stages (eg. programming application, setting up system, testing)Needs more skilled programmers who are able to do sophisticated programming specific to the mobile device (e.g., Android) and all the necessary components of an electronic questionnaire applicationRequires technologically skilled enumerators with more training and field support

Disadvantages of electronic data collection technologies

There are more infrastructure constraints to

consider

Electricity

needs to be available to charge the devices; which is challenging in areas with limited electricity.Needs a system to transfer the data from the handheld devices in a timely and secure manner. This could be the cellular network, Internet, local area network, etc.Limited Internet availability may cause difficulties in data transmission Data security must be assured during data collection, transfer, and storageNeeds a system for backing up the data to prevent data lossMultiple data collection might be necessary

Plans for 2020 round – Country example

Name

of

country

EgyptEthiopiaLibyaSouth Africa South SudanUganda9 Which methods of enumeration do you plan to use for field based data collection in the 2020 round? 3 Personal face-to-face interview, using electronic questionnaire (CAPI), 9 Internet self-response option (CAWI), on-line2 Personal face-to-face interview, using paper questionnaire, 3 Personal face-to-face interview, using electronic questionnaire (CAPI)3 Personal face-to-face interview, using electronic questionnaire (CAPI), 9 Internet self-response option (CAWI), on-line2 Personal face-to-face interview, using paper questionnaire,

3 Personal face-to-face interview, using electronic questionnaire (CAPI),

5 Personal telephone interview, using electronic questionnaire (CATI),

8 Self-completion of paper questionnaire by respondents (delivery/collection by enumerator),

9 Internet self-response option (CAWI), on-line,

10 Internet self-response option (CAWI), off-line (using downloadable form)

2 Personal face-to-face interview, using paper questionnaire,

8 Self-completion of paper questionnaire by respondents (delivery/collection by enumerator)

Personal face-to-face interview, using paper questionnaire,

3 Personal face-to-face interview, using electronic questionnaire (CAPI)

Decision making

Decision should

be done in consultation with all stakeholders and with a clear understanding of the

technology

Decision requires taking into consideration a number of critical factors, including: Information on national context Such as size of country in terms of area, population; diversity; households with access to electricity and Internet; persons with access to Internet; literacy/education level; etc. Institutional factorsPolicy/strategies for modernisation of NSO, transferring to electronic data collection; support of government/senior managers; culture for changes; legal base -investment in tech required by lawTechnological factors ICT infrastructure in country; compatibility with existing investment; extent of investment to GIS technology; multi-language support, etc.Economic factors - pressure for cost reduction; resource mobilization Socio-cultural factors - public acceptance and trust, cultural diversity, etc.

Information required for

decision-making

Operational aspects

# of modes being used; # of days to complete survey; # of enumerators and EAs; complexity of survey (# of questions);

Resources Cost estimates; total cost of ownership (TCO)-including training, maintenance Benefits - improved coverage, timeliness, response burden, efficiency, etc.Cost-benefit analysis- Comparison of total cost, cost savings compared to previous method of enumeration, cost saving as same technology can be used for other statistical activities (agriculture census/industry census/surveys/registersRisks - changes to business process; field staff considerations; procurement; IT infrastructure, impact on other statistical processes; integration into existing systems; etc.) Time required –for procurement, system development, human resource development, testing, training, etc.Plan for reuse/disposition of devices

Planning considerations for electronic data collection

It is critical to identify all requirements for carrying out the census using electronic data collection and

develop plans early in the census life cycle

.

Census timetable - generally, more time is needed for Setting up IT infrastructure and testingAllocating enough time for testing whole system is necessary including data collection, MIS, data transmission, data integration, data processing, etc. (pre-field tests and field tests)Enough time should be allotted for training the enumerators since the training must include the use of the devices, including making enough practices in the field

Planning considerations for electronic data collection

Capacity development –

organizational restructuring, IT skills and human capacity development,

Budget Considerations –

Careful analysis of all costs of use of electronic data collection versus saving costs due to absence of printing, data capture, etc.Infrastructure Considerations - Infrastructure issues such as availability of electricity and Internet access can affect the success of mobile data captureEarly feasibility analysis for applicability of hand-held devices/Internet in a country conditions taking into account possibility of use of multi-modes Data collection application: iterative process - develop, test (design, performance), revise, test, …. until make sure that it fits with the requirements. Good collaboration between subject specialist and programmers is necessary

Planning considerations for electronic data collection

Need

to

build partnerships

with:Application development partners Device manufacturer (to provide the devices as per specification)Connectivity providers (to provide connectivity for the device so that the data can be transferred seamlessly to the data center)Capacity building partners (training on using not only the forms and the entire process of data collection but also on the basics of the device and what to do for trouble shooting).

Implementing handheld electronic devices

Considerations for selection of handheld electronic devices

Important features that could impact the choice of device

include:

Portability – weight, size, ease of use and

transportBattery life – battery life should be sufficient to perform several hours of fieldwork without rechargingScreen size, resolution and visibility – preferable to use a monochrome screen not easily affected by bright sunlightProcessors and random-access memory (RAM) – for better performance and rapid navigation, a faster processor and a good amount of storage are neededData storage and backup possibilities – external data cards and flash drives (SD/MMC cards) for adequate storage and for backing up dataConnectivity options for data transfer (such as cellular communication, WiFi, Bluetooth, USB and other types of connectivity between devices)In-built functions (e.g. GPS receiver, camera, voice recorder, etc)Cost

CAPI

software

-

Essential functional

featuresCAPI software packages should be evaluated to assess whether they are robust enough in performance and broad enough in functionality to support a census op. The evaluation criteria for performance assessment of each CAPI software package should include the following desirable characteristics and functionalities:User-friendly development environment for modifying, and updating the survey instrument Simple but powerful interface Data capture and quality control modules Questionnaire navigation Skipping/branching Progress reporting and problem reporting Case management,Transfer and export facilities Support and documentation

CAPI – build, buy or freeware

T

he build versus buy decision is a critical one - choosing incorrectly could result in an inadequate solution or a poor return on investment

Some considerations in any build-versus-buy decision:

Are there sufficient internal programming resources? Building your own solution requires a set of developers with the technical ability to integrate your solution to back-end systems, otherwise it might make sense to buy or get freewareWhat is the time available to enumeration? Purchasing a solution eliminates the software development process, leaving only implementation, testing , and deployment phases to complete.Total cost - implementing a packaged data collection application will be more cost-effective than building it from scratch. When assessing cost it is important to look at Total Cost of Ownership (TCO)

Implementing handheld electronic devices

DATA SECURITY, TRANSMISSION, AND STORAGE CONSIDERATIONS

An important consideration is how to transmit data securely from electronic devices to a central server not to take any risk for data loss

Assessing alternatives for secure data transfer-considering variation in IT infrastructure within a country and establishing more than one system

Feasible solution in all areas of the country when setting up a system -reliable Internet may be available in cities, but not in rural areasAllocating time and resources to test the data transfer and storage system

Implementing handheld electronic devices

Field Case Management

Use of electronic questionnaire allows to linked to a computerized field case management system

Once data from each device are sent to a central database, the data can be used to monitor the progress of the enumeration and identify which households the interviewers need to visit, revisit, follow-up

Data transferred to central database can be used for preparing progress reports by small geographic unitsField operation staff and programmers should work together to ensure that the case management system is capable to identify problematic units and geographical areas taking full advantage of a computerized case management system

Implementing handheld electronic devices

STAFF SKILL REQUIREMENTS

Extensive training

for enumerators and supervisors

On census questionnaire and technical aspects of the data collection process, such as how to operate a tablet PC, transmit data, and navigate through the applicationWell- trained field supervisors for using field case management and reporting the progress and problems Help desks at headquarters and field technical support staff are needed to handle technical queries and troubleshoot problems during data collection

Implementing handheld electronic devices

Other factors to consider include:

Integration with geospatial tools/digital maps to assist enumeration and monitoring

Procurement and IT acquisition management

(understanding TCO, developing specifications, financial rules/standards )Logistics for storage, distribution and return of handheld electronic devices and accessories (eg. asset tagging for traceability; transportation; etc.)Device use policy (eg. authorized uses; misuses; proper handling (eg. during inclement weather); linking final payment to enumerators to return of device; etc.)Reuse/disposition of devices

Implementing Internet based data collection

Implementing Internet based data collection

Main advantages

Cost-effective method for enumeration of population

Better solution for enumerating people who are difficult to reach especially living in cities

High data quality with electronic data collectionRequires support systems such as call centers / toll-free telephone helpline for assisting respondents Requires high technical capacity to build sophisticated control system for avoiding duplications and undercounting and to ensure the security of the data

Implementing Internet based data collection

Requirements for data collection with

Internet include:

Reliable information and communication technology (ICT) infrastructure

High rate of internet penetration/access coverageHigh literacy ratePublic trust and acceptability of internet self-enumeration by the populationMaking reasonable assumptions and projections for response/take-up rates (considerations for duration of completing questionnaire, complicated design, etc.) Availability of address/building/dwelling registers/lists (eg. to organize enumeration, generate unique access codes, avoid duplication, contact respondents, etc.)

Internet collection -

data collection application and portal

Most

of the questionnaire design considerations for CAPI are also applicable to Internet-based data

capture – in terms of features and functions of data collection application (navigation, consistency controls, validation, etc.) as well as in terms of the considerations for the design of data collection application Additional requirements include the need to: optimize the application for use on variety of devices, screen resolutions, operating systems, browsersprovide instructions for respondents on the portalprovide respondent authentication procedurestesting IT infrastructure stability, security and capacity to handle anticipated response loads

Internet collection

Contact and communication strategies to motivate Internet response

Push/pull methods

(Pull: publicity campaigns, incentives; Push: internet first, face-to-face later)

Contact approaches (letters, post cards, FAQs brochures, e-mails, texts, phone calls, etc.)Contact approaches for targeting difficult to reach demographic groups and geographic areasCommunication/publicity to promote response (eg. targeting, multi-channel outreach, national and local partnerships, awareness campaigns via traditional and new media, etc.)Support to respondents Instructional and contextual help materials for completing online questionnaire, particularly for “difficult” census questionsProvide census questionnaire assistance via call centres/toll-free telephone helpline, social mediaPartnership with community facilities (schools, libraries, other public facilities) to set up PCs and Internet to facilitate response by those without personal Internet access

Conclusions

Electronic data collection technologies are being recognized as an option worth considering in the 2020 round of population and housing censuses

The adoption of electronic

data collection technologies

require early planning and through preparation including prototyping, use in small survey projects, and pilot exercises in order to validate all the stages Testing, testing, testing !!!!As for any new technology-based approach, there is a strong need to seek partnerships and strengthening capacity Successful adoption of new technologies requires global plan for modernisation of statistical systemStrong project management skills and ability is necessary to anticipate potential challenges and thinking ahead about alternative ways to solve problems