Coverage Evaluation for Preventive Chemotherapy - PowerPoint Presentation

Slide1

Coverage Evaluation for Preventive ChemotherapyDescription of the method

1

Department of Control of Neglected Tropical Diseases, WHO-HQ

Rationale: Background & ContextAchieving uniformly high treatment coverage in every treatment round is critical for the attainment of established NTD disease

control and elimination goals.monitoring treatment coverage using administrative / routinely reported data during mass drug administration (MDA) activities can be unreliable:

Incomplete tallying or reporting poorly documented shifts in populationreliance

on outdated census data

treatment

of individuals outside the targeted age group or geographic area

Coverage EvaluationDefinition: are population-based surveys that are designed to provide

precise statistical estimates of coverage that overcome many of the biases and errors that can undermine routinely reported coverage. Objective: To determine if the target coverage threshold has been met

To validate the reported coverage

Justification for preventive chemotherapy:

Sustained high coverage crucial to elimination and control

Reported coverage often inaccurate

Coverage evaluation surveys can save programs time and money

MDA round with low/below target coverage is not effective

Uses for Coverage Evaluation for preventive chemotherapyEstimation of PC coverage –

to obtain a precise estimate of PC coverage that can be compared with the target coverage threshold to determine if the MDA was effective.Validation of reported coverage – to check the accuracy of the data recording and reporting system and take corrective actions where necessary.

Identifying reasons for non-compliance – by identifying common reasons for not swallowing the drugs,

programme

managers can improve social mobilization prior to the next MDA round.

Detecting problems with the supply chain and distribution systems

– can identify clusters of individuals for whom the drugs were never offered and corrective action can be taken

Measuring coverage in specific population

:

subpopulations, e.g.. Rural vs. urban

post-MDA coverage evaluations acknowledged as very important but seldom conducted:limited time and financial resources, poor accessibility of households,

lack of available transportation Lack of M&E stafflack of expertiseEtc. The NTD Experience 2002 - 2012

"…… identify

a coverage survey sampling methodology that is feasible for national NTD programs to implement, produces valid point estimates of coverage, and can be standardized for use across the PC

NTDs".

Expert Consultation Meeting, 2012

Review of methodsWhen coverage evaluations are conducted, Expanded Programme on Immunization (EPI) cluster-survey method most common

6

+ Practicality

+ Simplicity

+ Widespread use

- Falls short of probability sampling

- Results in biased estimates

- EPI program replacing method in favor of rigorous probability sampling

Updating current methods to a standard approach for coverage evaluations that is statistically rigorous while feasible for programs to implement

EPI cluster-survey

7

Lot Quality Assurance Sampling (LQAS)

Probability Sampling with Segmentation (PSS)

Coverage Evaluation Common Methodologies:

Overall Pros and Cons

Pros

Proven feasibility

Precise

Widely

used

Feasible

Small sample size

Classification

of Survey Areas

Feasible

Precise

Unbiased

Cons

Biased

results

EPI

programme

moving away from method

Must visit

at least 95 different villages

Imprecise

Cannot directly calculate

coverage est.

Requires HH weighting or individual enumeration to avoid HH-size bias

Segmentation can be

time consuming when

maps

not available

Difficult to segment large villages; EAs are much easier to use

EPI

cluster-survey

Lot Quality Assurance Sampling (LQAS)

Probability Sampling with Segmentation (PSS)

Feasible

Yes

Yes

Yes

Unbiased

No

Depends/subjective

Yes

Clusters (EAs

3

)

30

95

30

Sample Size

1500

1

95

2

1500

1

Precise

Yes

No

Yes

PreparationList of EAs & pop(optional) mapsList of EAs & pop.5 Supervisory Areas(optional) mapsList of EAs & pop.(optional) maps

Key features of coverage evaluation methods

1

Sample sizes will vary based on the parameters used, but typically range from 1,000 - 1,600 individuals

2

Only one individual is selected per cluster, which is why the sample size and number of clusters are the same

3

EAs = census Enumeration Areas

WHO/STAG NTD-WG M&E recommended:

Probability sampling with segmentation (PSS)

●

Path

Stream

Road

House

School

Segment 1

Segment 2

4. Walk through segment and sample houses systematically according to the sampling interval

2. Divide EA into segments of ~50 House Holds

(HH)

3. Randomly select 1 segment

1. Select 30 EAs using PPES

EA = census enumeration area

PPES = probability proportional to estimated size HH = household

Main Difference Between EPI Method10

2. Visit nearest neighbor household, sampling all eligible within the household, until sample size is reached

1. Spin a bottle or pen in the center of village, then choose one random house between the center of the village and the edge of the village, in the direction of the spin, as the starting household

Limitations: EPIDo all individuals in a village have an equal probability of selection?

11

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

No, HH towards the center more likely to be selected.

Limitations: EPIDo individuals in different clusters have the same probability of selection? No. The probability that an individual’s village is selected is based on its

estimated size (PPES) but within the selected village (cluster) the probability that an individual is chosen is based on the actual village size.

P(Individualij) =

Estimated village size

Actual village size

Probability Sampling with Segmentation (PSS)Does everyone have an equal probability of selection?-

YES 13

P(individual) =

=

Example of Results

Coverage Evaluation 3 methods common compared in 4 countries in 2015 15

LQAS

EPI

PSS

EPI

LQAS

PSS

EPI

LQAS

PSS

1

2

3

EPI

LQAS

PSS

Different district & team for each method

Same district & team for each method

16

Some surveys exceed the target coverage threshold, while others fail.

Target threshold (LF, STH)

17

In 14 of the 16 surveys, reported coverage was

greater

than surveyed coverage.

Comparative costs for Coverage Evaluation Surveys(2014 – 2015)

Country

EPI

LQAS

PSS

Days to complete

Cost

Days to complete

Cost

Days to complete

Cost

Burkina

18

$ 4,385

19

$ 4,816

17

$ 4,525

Honduras

22

$

1,867

a

9

$

1,167

a

18

$

1,520

a

Malawi

14

$ 4,113

10

$ 3,247

16

$ 4,546

Uganda

23

$ 4,040

21

$ 3,835

26

$ 4,535

AVERAGE

19.25

$ 3,601

14.75

$ 3,266

19.25

$ 3,782

Acknowledgement of Contributors19

MOH Burkina FasoRoland BougmaDistrict health teams in Batie

, Dano and Diebougou MOH UgandaEdridah

Tukahebwa

Harriet

Lwanga

(RTI Envision, Uganda)

Survey teams from MOH

MOH Malawi

Square

Mkwanda

District health teams in

Balaka

,

Zomba

, and

Machinga

Secretary of Health Honduras

Reina Teresa

PAHO (Honduras & DC)

Rosa Elena Mejia & Romeo Montoya

Martha

Saboya, Laura Catala-Pascual & Ana MoricePamela Mbabazi (WHO)

Michael Deming (formerly CDC)Kristen Renneker (NTD-SC)Abdel Direny (RTI Envision)FundingBill & Melinda Gates FoundationUSAID

Supervisor’s Coverage Tool

Coverage Evaluation Survey

Data Qualit

y Assessment

Purpose:

To

improve

performance during current MDA

To

validate

reported coverage

(obtain a statistical

point estimate)

To

assess capacity

of data management and reporting systems

Administrative level:

Supervision Area

(

sub-district)

Implementation Unit (district)National and/or DistrictSample size:20 people>500 peopleN/ASites visited:

1 supervision area30 villages

12 service

delivery points

Survey team:

Internal, self-assessment

External to programmeInternal and external to programmeTiming:Within 2 weeks of MDAWithin 6 months of MDAAfter MDA data have been reported (3-6 months post-MDA)Cost:$0 - $1,000 per SA~$2,000 – $10,000 per district$ 12,000 – 15,000 nationally$1,000 – 2,500 per districtDuration:<1 week2-3 weeks~ 2-3 weeksM&E tools for improving quality of data reported by national NTD programmes implementing preventive chemotherapy

Integrated NTD Database

DQA

SCT

Coverage Surveys

WHO/NTD M&E Tool kit