Session 11 Planning for adoption of electronic data collection technologies

Session 11

Planning for adoption of electronic data collection technologies

UNSD presentation

Overview of the presentation

Paper-based

vs

electronic data collection approaches

Advantages and disadvantages of

using electronic data collection

technologies

Transitioning from paper to electronic data

collection

Decision

making

Planning considerations

Implementing handheld electronic devices for data collection

Implementing Internet for data collection

Paper-based

vs

Electronic collection approach

Manual

Manual

computer-assisted

Key entry/computer-assistedOMR/OCR/ICR

Manualcomputer-assisted

Automated/built-in

Paper-based approach

Data Collection

Data Entry/Capture

Data Collection/Entry/Coding/Editing

Electronic collection

a

pproach

Data Coding/Editing

Data Coding/Editing

(

partial)

Advantages of using electronic data collection technologies

Allow for the collection of much of the same information collected via paper-based

approaches, AND that

of information that cannot be collected using paper-based

approaches,

such as: GPS co-ordinates, pictures, videos, date/time stamps, etcImprove data quality through built-in consistency checks and logical validation rules, allowing errors and inconsistencies to be resolved at interview time

Allow a more accurate progression through the questionnaire due to automated skip patterns and mandatory items. Avoid manual data entry errors; avoid problems due to illegible writingOptimize time spent by enumerator and respondents

Improve field management and real time monitoring of enumeration activities (eg. built-in date and time stamp could be useful for supervision)Reduce time and costs by eliminating a separate process for data capture, automated coding, and reducing the volume of post-enumeration data checking.

Decrease time between data collection and release of the results.

Advantages of using electronic data collection technologies

Reduce

costs of data capture, questionnaire printing, storage, and transportation. However, the increased equipment costs may outweigh the savings

Allow the provision of on-screen

help

- for enumerators from the screen of electronic device. A help feature can makes it easier for the enumerators to access definitions or other items needing clarification during the interview. Unlike in paper manual, the help feature in an electronic questionnaire can be linked to each question or a particular term that often needs clarification.

Allow computerized case management, capacity to monitor and control over various census operations

Can utilize GPS features and digital mapping capabilities. For example:Captured GPS locations can be used as a reference point for future census activities, eg.

help track location of data entry EA maps and/or address information can be loaded onto the device and even aerial or satellite photos to help the enumerator find the correct housing units to visit.

Disadvantages of electronic data collection technologies

There are a number of risks or drawbacks associated with the introduction of technology that should be considered and managed, otherwise these could lead to increases in expenditure, delays to the census timetable or impacts on the quality of the census.

Technology

increases dependence on technology providers and introduces new challenges and risks

Large investment (

eg high equipment costs with limited long-term use)More time needed during the preparation stage (eg. programming application, setting up system, testing)Needs more skilled programmers who are able to do sophisticated programming specific to the mobile device (e.g., Android) and all the necessary components of an electronic questionnaire application.

Requires technologically skilled enumerators with more training and field support.

Disadvantages of electronic data collection technologies

There are more infrastructure constraints to consider.

Electricity

needs to be available to charge the devices;

which is challenging in areas with limited electricity.

Needs a system to transfer the data from the handheld devices in a timely and secure manner. This could be the cellular network, Internet, local area network, or through manual data transfer.Limited Internet availability may cause difficulties in data transmission and other functions of the mobile data capture software application that rely on the Internet. A realistic assessment of Internet connectivity in all areas should be conducted during the planning stage.

Data security must be assured during data collection, transfer, and storage.Needs a system for backing up the data to prevent data loss.

Transitioning from paper to electronic data collection

Handheld devices

Paper-based

d

ata

c

ollection

Paper-and-Pencil

interviewing

Self-administrated

questionnaire

Internet

Plans for 2020 round - Southern African countries

Name

of

country

Botswana

Malawi

Namibia

South AfricaUgandaZambia

Zimbabwe

9 Which methods of enumeration do you plan to use for field based data collection in the 2020 round? 3 Personal face-to-face interview, using electronic questionnaire (CAPI)

3 Personal face-to-face interview, using electronic questionnaire (CAPI)

2 Personal face-to-face interview, using paper questionnaire, 3 Personal face-to-face interview, using electronic questionnaire (CAPI)

2 Personal face-to-face interview, using paper questionnaire, 3 Personal face-to-face interview, using electronic questionnaire (CAPI),

5 Personal telephone interview, using electronic questionnaire (CATI),

8 Self-completion of paper questionnaire by respondents (delivery/collection by enumerator),

9 Internet self-response option (CAWI), on-line,

10 Internet self-response option (CAWI), off-line (using downloadable form)2 Personal face-to-face interview, using paper questionnaire,

3 Personal face-to-face interview, using electronic questionnaire (CAPI)3 Personal face-to-face interview, using electronic questionnaire (CAPI)

2 Personal face-to-face interview, using paper questionnaire,

3 Personal face-to-face interview, using electronic questionnaire (CAPI)

Decision making

The decision on whether to use electronic data collection technologies should begin in the initial census planning stages -- t

he

decision

should be done in consultation with all stakeholders and with a clear understanding of the

technologyTo the extent possible, the determination as to whether digital collection meets the needs of a census operation, should be based on information generated by hands-on experience and field tests; best practices from other statistical collections/organizations/countries could also be useful in making a decision

The decision requires taking into consideration a number of critical factors, including: National context size of country in terms of area, population; diversity, socio-econ characteristics; investment in tech required by law? Institutional factors

Policy environment, support of government/senior managers, org cultureTechnological factors (ICT infrastructure, extent of existing investment in tech)Socio-economic factors (literacy, public trust, cultural diversity, etc)

Information required for

decision-making

Operational aspects

# of modes being used; # of days to complete survey; # of enumerators and EAs; complexity of survey (# of questions);

Stakeholders

’ requirementsResources Cost estimates; total cost of ownership (TCO)Benefits

“why” use technology: data quality, improved coverage, timeliness, response burden, efficiency, cost savings, etc.Risks

impact on/risks to: schedule (eg. testing plan); budget (positive & negative); changes to business process; field staff considerations; procurement; technical; managerial; impact on other statistical processes; integration into existing systems; etc.)

Timetable for procurement, system development, testing, training, enumeration, etc

.Plan for reuse/disposition of devices

Planning considerations for electronic data collection

In order to ensure the success of the census, it is critical to identify all requirements for carrying out the census using electronic data collection and develop plans early in the census life cycle.

Census timetable -

generally, more time is needed to develop and test the collection application, set up the data transfer and processing systems, and procure, program, and test the devices.

If edit checks are to be added to the application, the edit programming must be completed before the enumeration begins, rather than programming those separately in the post-enumeration stage.

Furthermore, time should be allotted for training the enumerators since the training must include the use of the devices. Therefore, it is critical to determine all the steps needed to take place in order to set up an electronic data collection system and allow sufficient time prior to the enumeration in the census timetable.

Planning considerations for electronic data collection

Budget Considerations -

Using an electronic questionnaire may save costs in the printing of the questionnaire and data capture, but the costs of the electronic equipment may be higher than the savings. Carefully consider all costs of conducting mobile data capture and account for them in the budget.

Infrastructure Considerations -

Infrastructure issues such as availability of electricity and Internet access can affect the success of mobile data capture. Early in the planning stage, identify places that lack electricity and/or Internet access. If relying on a cellular or Wi-Fi data connection to transmit data, it is important to research the speed of the data transmission and plan accordingly. Mapping the areas without electricity and/ or Internet access is helpful in planning. Plans should be developed for charging and backing up the devices. It is critical to have contingency plans for when electricity and/ or Internet access are not available.

Planning considerations for electronic data collection

Questionnaire development

p

rocess -

As with a paper questionnaire, developing an electronic questionnaire is an iterative process. It must be developed, tested, revised, and then tested again, repeating the cycle until the questionnaire works as intended. When designing an electronic questionnaire, it is important to make sure that the subject matter specialists

work closely with the programmers so that they do not lose control over the wording, layout, and design of the instrument.Need to build partnerships with:Application development partners

Device manufacturer (to provide the devices as per specification)Connectivity providers (to provide connectivity for the device so that the data can be transferred seamlessly to the data center)Capacity building partners (training on using not only the forms and the entire process of data collection but also on the basics of the device and what to do for trouble shooting).

Implementing handheld electronic devices

Considerations for selection of handheld electronic devices

Important features that could impact the choice of device

include:

Portability – weight, size, ease of use and

transport

Battery life – battery life should be sufficient to perform several hours of fieldwork without rechargingScreen size, resolution and visibility – preferable to use a monochrome screen not easily affected by bright

sunlightProcessors and random-access memory (RAM) – for better performance and rapid navigation, a faster processor and a good amount of storage are neededStability of the operating system

Data storage and backup possibilities – external data cards and flash drives (SD/MMC cards) for adequate storage and for backing up dataConnectivity options

for data transfer (such as cellular communication, WiFi, Bluetooth,

USB and other types of connectivity between devices)Ruggedness

and durability in field conditions (such as hot and cold temperatures, rain, sand storms, dust, heavy wind)In-built functions (e.g. GPS receiver, camera, voice recorder,

etc)Location accuracy (with respect to geographic location) for fieldwork application, especially if GPS functionality is requiredCost

CAPI

software

-

Essential functional features

CAPI software packages should be evaluated to assess whether they are robust enough in performance and broad enough in functionality to support a census

op. The evaluation criteria for performance assessment of each CAPI software package should include the following desirable characteristics and functionalities:User-friendly development environment for creating, modifying, and updating the survey instrument

Simple but powerful interface Data capture and quality control Questionnaire navigation Skipping/branching Case management Data management, transfer and export facilities Support and documentation

Testing

process for evaluating

CAPI software

The test process for evaluating software should include at least the following steps:

Obtain test copies;

Develop test prototypes for testing ability of CAPI to satisfy key functionality requirements;Detailing implications on the organization’s existing computing

environment;Gauging user satisfaction – assess whether the supplier is responsive and provides reliable support

mechanismConduct tests according to previously established criteria;Assess and document upgrade policy; Determine full costing;Produce a report on the evaluation process.

CAPI – build, buy or freeware

T

he build versus buy decision is a critical one - choosing incorrectly could result in an inadequate solution or a poor return on investment

Some considerations in any build-versus-buy decision:

Will building your own solution deliver a unique advantage that helps you achieve business objectives?

What is the time available to enumeration? -- Purchasing a solution eliminates the software development process, leaving only the implementation, testing and deployment phases to complete.Are there sufficient internal programming resources? -- Building your own solution requires a set of developers with the right coding skills and the technical ability to integrate your solution to back-end systems. If NSO does not currently have employees with this skill set, it might make sense to buy or get freewareTotal cost - In most cases, implementing a packaged data collection application will be more cost-effective than building it from scratch.

When assessing cost it is important to look at Total Cost of Ownership (TCO)

Implementing handheld electronic devices

DATA SECURITY, TRANSMISSION, AND STORAGE CONSIDERATIONS

An important consideration is how to transmit data securely from electronic devices to a central server. Secure data storage is also critical at every level where data are stored

Most electronic devices can store data either locally or on a network. Storing data locally risks data loss if the device is lost or stops working. Storing data on a network requires a way to transfer the data securely from each device to the main server.

Some ways to transfer data securely include:

Secure Internet connection; Local area network (LAN), a computer network that connects computers and mobile devices within a limited area (such as an office building); Bluetooth or Wi-Fi

that connects tablet PC to tablet PC; Cable connection (for example, connecting an enumerator’s tablet PC to her supervisor’s laptop); Wide area network (WAN), or a private network that extends beyond the geographic reach of LAN, but is not connected to the Internet (for example, a dedicated mobile network).

Implementing handheld electronic devices

It is important to consider what is feasible in all areas of the country when setting up a system. Reliable Internet may be available in cities, but not in rural areas.

It may be necessary to establish more than one system according to infrastructure constraints. For example, data may be uploaded to the server through secure Internet connection in cities, but in rural areas, data may be transferred using cable connection.

It is important to allocate time and resources to test the data transfer and storage system as failure to do so may have costly consequences.

Implementing handheld electronic devices

COMPUTERIZED FIELD CASE MANAGEMENT

One feature of using an electronic questionnaire is that it can be linked to a computerized field case management system.

As data from each device are sent to a central database, the data can be used to monitor the progress of the enumeration and identify which households the interviewers need to visit. Quality metrics and audits also can be incorporated in the field case management system.

Field operations and programmers should work together to ensure that the case management system meets the needs of the field operations unit and be able to take full advantage of a computerized case management system.

Implementing handheld electronic devices

STAFF SKILL REQUIREMENTS

Extensive training will be needed

f

or enumerators and supervisors. Training will be needed to familiarize them with the technical aspects of the data collection process, such as how to operate a tablet PC, transmit data, and navigate through the application.

Field supervisors may have fewer quality check responsibilities, since most of the completeness and consistency checks will be done by the data collection software. However, they will be required to conduct field case management and reporting using a computerized system. So they need training too.Help desks at headquarters and field technical support staff may be needed to handle technical queries and troubleshoot problems during data collection. In addition, existing programmers may not have the skills to program the questionnaire application and set up extensive data systems, requiring further training or hiring new staff.

Implementing handheld electronic devices

Other factors to consider include:

Integration with geospatial tools/digital maps to assist enumeration and monitoring

Procurement and IT acquisition management

(understanding TCO, developing specifications, financial rules/standards )

Logistics for storage, distribution and return of handheld electronic devices and accessories (eg. asset tagging for traceability; transportation; etc.)Device use policy

(eg. authorized uses; misuses; proper handling (eg. during inclement weather); linking final payment to enumerators to return of device; etc.)Reuse/disposition of devices

Implementing Internet based data collection

Implementing Internet based data collection

The main advantage of Internet-based data capture is that it can be cost effective since enumerators are not used. It can also be used for residents who are hard to reach in person.

However, in order for it to be cost effective, there needs to be a large number of people with reliable access to the Internet.

Another disadvantage is that a highly sophisticated control system is required to avoid duplications and undercounting and to ensure the security of the data.

Implementing Internet based data collection

Requirements for data collection with

Internet include:

Reliable information and communication technology (ICT) infrastructure

High rate of internet penetration/access coverage

High literacy ratePublic trust and acceptability of internet self-enumeration by the populationMaking reasonable assumptions and projections for response/take-up rates

Availability of address/building/dwelling registers/lists (eg. to organize enumeration, generate unique access codes, avoid duplication, contact respondents, etc.)

Internet collection -

data collection application and portal

Most

of the questionnaire design considerations for CAPI are also applicable to Internet-based data

capture – in terms of features

and functions of data collection application (navigation, consistency controls, validation, etc.) as well as in terms of the considerations for the design of data collection application Additional requirements include the need to:

optimize the application for use on variety of devices, screen resolutions, operating systems, browsersprovide instructions for respondents on the portalProvide respondent authentication

procedurestesting IT infrastructure stability, security and capacity to handle anticipated response loads

Internet collection

Contact and communication strategies to motivate Internet response

Push/pull methods

(Pull: publicity campaigns, incentives; Push: internet first, face-to-face later)

Contact

approaches (letters, post cards, FAQs brochures, e-mails, texts, phone calls, etc.)Contact approaches for targeting difficult to reach demographic groups and geographic areasCommunication/publicity to promote response

(eg. targeting, multi-channel outreach, national and local partnerships, awareness campaigns via traditional and new media, etc.)Support to respondents Instructional and contextual help materials for completing online questionnaire, particularly for “difficult” census questions

Provide census questionnaire assistance via call centres/toll-free telephone helpline, social mediaPartnership with community facilities (schools, libraries, other public facilities) to set up PCs and Internet to facilitate response by those without personal Internet access

Conclusions

Electronic data collection technologies are being recognized as an option worth considering in the 2020 round of population and housing censuses

The adoption of electronic

data collection technologies

require early planning and through preparation including prototyping, use in small survey projects, and pilot exercises in order to validate all the stages and anticipate alternative solutions

As for any new technology-based approach, there is a strong need to seek partnershipsMaintaining data security and confidentiality is also of paramount

concernSuccessful adoption of new technologies requires managers to have strong project management skills and ability to anticipate potential challenges and thinking ahead about alternative ways to solve problems