Summarized by BRAC USA Why this exercise Q Why do donors contribute to an organization Awards word of mouth testimonials charity ratings preferred habitat programs geography culture ID: 805866
Download The PPT/PDF document "Impact BRAC Research Impact Assessment" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Impact
BRAC Research Impact Assessment
Summarized by BRAC USA
Slide2Why this exercise?
Q. Why do donors contribute to an organization?
Awards, word of mouth, testimonials, charity
ratings
, preferred habitat (programs,
geography, culture
, religion,
ethnicity), vanity,
emotive photos/stories
, scale,
quantifiable
impact.
Slide3Quantifiable impact is (becoming) the gold standard for NGO evaluation
More and more funders are asking for proof of effectiveness by demonstrating a quantifiable impact
BRAC’s
independent
research department is unrivaled in the magnitude of research produced
Development organizations ignore quantitative impact at their peril
Easier said than done
Slide4What is Impact?
Impact
“
The
positive and negative,
primary and
secondary long-term effects produced by
a development
intervention, directly or
indirectly, intended
or unintended
.”
-- OECD Development Assistance Council
Impact Evaluation
“A
study of the attribution of changes in the outcome to the intervention
.“
-- 3ie
Slide5Impact research should be independent
Real life example of research bias
Slide6Reports Used
Program
Title
Time Period
Agriculture
Impact of BRAC’s Agriculture & Livestock
Programme
in Uganda
2009-2010
Education
Economic and Social Analysis of Primary Education in Bangladesh: A Study of BRAC Interventions
2009-2010
Education
An Assessment of BRAC Pre-Primary Schools Environment
2012
Health
Manoshi
: Community Health Solutions in Bangladesh. Impact Evaluation Surveys in Dhaka Urban Slums, 2007, 2009, & 2011
2007-2011
Health
Maternal, Neonatal, and Child Health
Programmes
in Bangladesh: Review of Good Practices and Lessons Learned
2006
Health
The Pilot Maternal, Neonatal and Child Health Project (MNCH) at Nilphamari: Profiling the Changes During 2006-07
2006-2007
Health
Sustaining Health: The Role of BRAC’s Community Health Volunteers in Bangladesh, Afghanistan, and Uganda
2008-2009
Health
Impact of Community-based Tuberculosis Programme: Ex
N/A
Microfinance
The Wider Impacts of BRAC Poverty Alleviation Programme in Bangladesh
1992-2004
Microfinance
An Impact Evaluation of BRAC’s Microfinance Program in Uganda
2008-2009
Ultra-Poor
Addressing Extreme Poverty in a Sustainable Manner: Evidence from Ultra-Poor Programme In Bangladesh
2002-2008
Ultra-Poor
Can Basic Entrepreneurship Transform the Economic Lives of the Poor?
2007-2011
Ultra-Poor
Asset Transfer Programme for the Ultra Poor: A Randomized Control Trial Evaluation
2007-2011
Water & Sanitation
Achievements of BRAC Water, Sanitation, & Hygiene Programme
2006-2011
Youth
Social Network and Financial Literacy among Rural Adolescent Girls: Qualitative Assessment of BRAC’s SoFEA Programme
2011-2012
Youth
Empowering Adolescent Girls: Evidence from a Randomized Control Trial in Uganda
2008-2010
Slide7Positive
impact on usage of modern inputs, cultivation methods, and vaccination
services
(Uganda)
Program participants are 27 percent more likely to use improved
seeds
than the comparison
group
(Uganda)
Impact Synopsis
Agriculture & Livestock
BRAC Pre-primary schools:
Scores 7.7% higher than non-BRAC schools
Reduced Dropout and Repetition, Rate, Higher Pass
Rate: BRAC schools are twice as effective as government schools in terms of lower dropout, higher pass rates and repetition rate
Education
Slide8Impact Synopsis
Increases in MNCH knowledge, including danger signs for pregnant women, new mothers, and newborns
Increase in uptake of services, practicing essential newborn care, and increased access and follow-through of referral
Improved health metrics including: ANC and PNC visits, increase in use of
Post-partum
hemorrhage or PPH
(the leading cause of maternal mortality)
medication, increase in percentage of institutional delivery and an increase in percentage of women practicing exclusive breast feeding.
Reduction of MMR (Maternal Mortality Rate) from 236 per 100,000 live births in 2008 to 141 live births in 2010.
Health
Slide9Positive
long-term impact on the lives of the ultra poor, even four years after the end of the program’s two-year support phase, in terms of per capita income, employment, food security and asset
holding
Among participants that entered the two-year program in 2002,
92% satisfied
at least
7
out of
10
indicators of extreme poverty in
2008.
By 2008, six years after the baseline survey, the program’s impact on participants’ per capita income had almost doubled compared to the impact from 2002 to 2005.
Impact Synopsis
Microfinance
Better child survival, nutritional status, family planning, and education in households served by the program.
In
Uganda, a Study published in 2013 revealed positive impacts including increase in savings, improved food consumption and increased financial and social resources leading to improved business startup and
investment
Ultra Poor
Slide10Water & Sanitation
Sanitary latrine use increase: Households using sanitary latrines increased significantly from the baseline
(32%) to midline (41%) to end line (53%)
Reduction in water related diseases
: The prevalence of water related diseases significantly reduced from 9.4% in the baseline to 7.1% in midline and 2.3% at the end line.
Impact Synopsis
Slide11An RCT shows
Uganda's ELA program having positive impacts on life and vocational skills, health-related knowledge, risky behavior, early childbearing, and engagement in income generating
activities.
A study of the
SoFEA
program in Bangladesh found that the program
was
successful
in increasing self-confidence and motivation, and in strengthening the relationship between girls and their parents and within the community.
Initiative to encourage adolescent girl’s socialization and improve financial literacy has been found to be “quite effective”.
After
receiving Financial Literacy Training (FLT),
SoFEA members were more financially aware and many were involved in Income Generating Activities, such as livestock and poultry rearing.
Youth
Impact Synopsis
Slide12Agriculture and Livestock
Slide13Timeframe
of
study:
2009-2010
Positive impact
on
usage
of modern inputs, cultivation
methods
and vaccination
services
Participants
27 percent more likely to use improved seeds than the comparison
groupImproved cultivation methods: Participants are six percent more likely to follow line sowing cultivation and 10 percent less likely to follow inferior mixed-cropping
Participants are 24 percent more likely to utilize vaccination servicesParticipants are 16 percent more likely to be ‘market-oriented’ resulting in higher revenues from salesImpact of BRAC’s Agriculture & Livestock Programme in UgandaAgriculture & Livestock
Slide14Education
Slide15BRAC Education Program Model
Slide16Economic and Social Analysis of Primary Education in Bangladesh: A Study of BRAC
Interventions
Education
Timeframe of study:
December 2009 – February
2010
Value for Money:
Cost per
student completed is
2x in formal schools.
BRAC schools
boost test scores:
BRAC pre-primary
students enjoyed ↑ in final exam scores
in primary school
by
7.7
% co
mpared to comparison group higher as well as ↑ secondary school enrollment (89% v. 82%).BRAC schools have reduced dropout rates & higher completion rates, making them twice as effective as government schools.
Cost-Benefit:
Private returns to primary education are higher for BRAC graduates, due to the 4 year cycle & lower cost of BRAC schools, compared to the 5 year cycle of mainstream
schools
Internal Efficiency:
BRAC schools advantage includes intensive monitoring of teachers & teacher input.
Slide17Economic and Social Analysis of Primary Education in Bangladesh: A Study of BRAC Interventions
Slide18An Assessment of BRAC Pre-Primary Schools Environment
Timeframe
of
study:
July – September 2012
Poor Overall Environment:
In a
7
-point scale, BRAC schools received an average score of 1.47
On the other hand, POs considered that the majority of schools were rated as good or very good.
Education
Do BRAC Pre-Primary Schools Have a Positive Impact?
Shahjamal
and
Nath
(2008)
compared performance
of former pre-primary students enrolled in formal primary schools and a comparable group who did not have a pre-primary background, and found that former BRAC students did significantly better in Grade I, but no
difference
was found in other grades
Ahmad and
Haque
(2011) also find that scores achieved in primary school examinations are higher for students with BRAC pre-primary education
Are these results contradictory?
Personal care, lang. learning, interaction, parents
&
staff, parts of activities can be improved with little additional monetary investment.
Slide19Reaching Primary Education at the Doorstep of the Poor: The BRAC Experience
Education -
Summary from Andrew Jenkins
Timeframe
of study:
2005
Over
2.4 million children completed the primary education curriculum, due to BRAC’s educational interventions
6.5% of total primary school students were enrolled in BRAC
schools and over
90% of BRAC students
went
on to enroll in formal secondary schools
BRAC students had
a better knowledge of health &
development issues than
formal school students
Socioeconomic Impact of BRAC’s Non-Formal Primary Schools
Timeframe
of study:
2005
BRAC primary students enjoyed better performance compared to formal school students.
NFPE school attendees
were significantly more knowledgeable on child immunization/prevention of disease than those who attended government primary schools or never attended school at all
A smaller proportion
of adults who attended government primary schools had immunization cards (57.9%) compared to NFPE school enrolled households (64.3%)
NFPE school enrolled couples collected birth control at a higher rate (37.4%) than those who attended government primary schools (32.1%) and who never attended a school (26.9%)
Slide20Health
Slide21Manoshi
: Community Health Solutions in Bangladesh. Impact Evaluation Surveys in Dhaka Urban Slums, 2007, 2009, & 2011
Timeframe
of
study:
2007-2011
Increases
in MNCH
knowledge. Though improvement varied depending on the particular aspect of knowledge. For example, knowledge
about
recommended 4
+ ANC visits increased from 45% in 2007
to 59% in 2011 in the project area v. 50 to 52% in comparison area.Increase in uptake of services, practicing essential newborn care and increased access and follow-through of referral. For example, colostrum feeding increased 32 absolute percentage points (PP) in the project area v. 18 PP for the comparison area.
Improvement in multiple health practices: ↑ ANC & PNC visits, ↑ use of Misoprostol (drug used for post-partum hemorrhage -leading cause of maternal mortality), ↑ in percent of institutional delivery & % of women breast practicing exclusive breast feeding.For example: Women receiving PNC after delivery improved 28PP in project area v. 7 PP in comparison area. ↓Maternal Mortality Rate from 236 per 100,000 live births in 2008 to 141 in 2010, as a result of household services, timely referral, & coordinated support at referral facilities. (Source: Manoshi: A Community-based Solution to Avert Maternal Death, 13th ASCON 2011).
Health
Slide22MMR declined by 50% in
Manoshi
areas
Slide23Maternal, Neonatal, and Child Health
Programmes
in Bangladesh: Review of Good Practices and Lessons Learned
Timeframe
of
study:
2006
Saving
Newborn Lives
Program:
A
collaborative program Funded by BMG implemented by BRAC, et al demonstrated changes
in many areas of maternal care:Post-Natal Checkup increased by 37.5 to 61% from the baseline surveyNewborn checkup within 24 hours increased from 14.4% in 2002 to 27.3% in 2004ANC increased from 22% in 2002 to 92% .
Health*National averages sourced from UNICEFhttp://www.unicef.org/infobycountry/bangladesh_bangladesh_statistics.html) and the World Bank (http://data.worldbank.org/indicator/SP.DYN.CONU.ZS?page=1
)
SNL Program has Positive Impact on Post-Natal Checkup
Slide24The Pilot Maternal, Neonatal and Child
Health Project (MNCH) at
Nilphamari
:
Profiling the Changes During 2006-07
Timeframe of study:
2006-2007
↓ in induced abortion
. However
, self-treatment and treatment-seeking
from informal
providers for complication of abortion remains high.
Mean
number of ANC
visits ↑ with increased proportion attended by skilled providers relative to baseline. Intake of iron tabs and TT vaccination increased.Majority of deliveries continued to take place at home. Though deliveries by trained TBAs increased since baseline, the proportion delivered by the TBAs remained substantial.
For
treatment of neonatal complications, mothers increasingly
preferred UHCs/district
hospitals.Management of illnesses of the under-fives is fragmentary and the thrust is on prevention only.
Slide25Sustaining Health: The Role of BRAC’s Community Health Volunteers in Bangladesh, Afghanistan,
and Uganda
Timeframe
of
study:
2008-2009
Dropout
rate of 12%
low compared to
similar programs
internationally.
Self-reported motivations for
CHPs primarily related to financial incentives in all settings except Afghanistan - women see this role as income-generating opportunity:75% of SSs reported that their income makes a big difference. In Uganda and Afghanistan these numbers were 80% and 15%, respectively. 97% of SS in Bangladesh reported that being an SS gave them financial independenceBRAC is meeting the demand for medicines,
but CHWs in all three countries felt competition from pharmacies, private clinics, and other providers.Importance of increased Social recognition: 18% in Bangladesh, 11% in Uganda and 16% in AfghanistanHealth
Slide26Sustaining Health: The Role of BRAC’s Community Health Volunteers in Bangladesh, Afghanistan,
and Uganda
Health
Slide27Health
Timeframe of Study:
2008-2009
Cost effectiveness:
In BRAC areas, cost was $64 compared to $96 in other areas.
BRAC jointly expanded DOTS services to cover a population of 89.5 million, with services mainly provided by CHWs
In 2009, 96,427 patients were diagnosed, of them, 71,946 were new sputum-positive
Strong case-detection rate of ~80% and treatment success rate of 93%.
CHPs fill critical gaps in human resources at community level; reduce delays in diagnosis, bring doorstop DOTS service, and ↑ case-detection and cure rates
Impact of Community-based Tuberculosis
Programme
: Experience of BRAC
Slide28The Bangladesh Miracle –
Significant drop in mortality rates and increase in life expectancy
Slide29The Bangladesh Paradox –
Source: the Lancet
Success may be attributed to a multi-stakeholder pluralistic health system.
Women-focused, equity-oriented, nationally targeted
programs in
family planning, immunization, oral rehydration therapy, maternal and child health, tuberculosis
, and vitamin a supplementation were critical success factors
B
angladesh’s approach to equity and its widespread deployment of
female community health workers
to bring essential health services to local households is noteworthy and a “testament to how many services can be provided when there is a scarcity of
credentialled
health professionals”
BRAC and The Bangladesh Paradox
After the War of Liberation, the government created the space and donors provided the financial support for NGOs like BRAC, which, as a group “have innovated to address issues or poverty, unemployment health education, and the environment, and in many cases, the government and NGOs have worked together to achieve a common goal.”The national implementation of an oral rehydration therapy program by BRAC enables mothers to prepare homemade oral rehydration saline
Scaling up of innovations has been strengthened
by investment in research,
which has enhanced program design, monitoring & evaluation, and implementation
The Bangladesh Paradox: Exceptional health achievement despite economic poverty
Slide30Microfinance
Slide31The Wider Impacts of BRAC
Poverty Alleviation
Programme
in
Bangladesh
Timeframe of Study: 1992-1995
↑
child survival and nutritional status in
households.
Protein-Energy Malnutrition (PEM) prevalence declined from 23% to 12%.
Also improvements in expenditure patterns, per capita calorie consumption, family
planning practices and children’s education.
BRAC households % using family planning was 57% v. 49.6% for poor-non BRAC members.
Violence against women initially increased when credit was introduced but later tapered after other services like training were introduced.
Microfinance
Slide32Child survival rate better for BRAC members
Slide33An Impact Evaluation of BRAC’s Microfinance Program in Uganda
Timeframe of Study:
January 2008- May 2009
The study finds that BRAC’s program “seems to confer significant positive benefits to borrowers” including:
Increase in total savings (1.63 log change in savings) and assets (0.76 log change)
Greater consumption in the form of more expensive and nutritious food (increase of 0.3 meals per week consumed with fish)
The resources and incentives to start a household business (Household business ownership increased by 27%)
Results vary considerably depending
on the statistical technique used. This coupled with data issues makes it difficult to make causal claims.
One year
between baseline and follow up,
may not be adequate
.
Microfinance
Slide34Ultra Poor
Slide35Addressing Extreme Poverty in a Sustainable Manner: Evidence from
Ultra-Poor
Programme
In Bangladesh
Timeframe
of
study:
2002-2008
Key
Findings:
Positive
long-term
impact, four
years after the end of the two-year support phase, in terms of per capita income, employment, food security and asset holding92% of the ultra-poor households satisfied at least 7 out of 10 indicators of extreme poverty in
2008 v. 64%.Ultra Poor
Other Findings
Income/Employment:
By 2008,
participants’ per capita income had almost doubled
. The study calls this “remarkable,” because it
“indicates that impact on per capita income was not only sustainable but also increased over time”
Among working-aged female participants,
farm self-employment rose from 0.2% in 2002 to 37.0
% in 2008, versus 0.3% to 11% in the control sample.
Positive Impact on Farm Self-Employment
Slide36Addressing Extreme Poverty in a Sustainable Manner: Evidence from
Ultra Poor
Programme
In Bangladesh (Cont.)
Assets
:
Financial assets increased sharply compared to the control sample,
for years
after the end of the 2
-
year
program
, attributable to a “dramatic” change in savings
behavior & participation in the microcredit market (11, 24)The percentage of women with cash savings increased from 8% at the start of the program in 2002 to 98% in 2008, versus 21% to 34% in the control group
Ultra Poor
Food security:
By 2005
, self-perceived food insecurity “had
fallen tremendously” for participants, or 47% compared to the 2002 baseline versus only 11% drop in the control
Housing & sanitation:
By 2008, 94% of participants had roofs made of tin and 57% of participants had a sanitary latrine, compared to 44% and 2% in 2002
Education:
The program “
did not have significant impact on education in the
short term” but “in the long-term a modest positive impact on boys’ net primary enrollment was observed.”
Slide37Ultra Poor
Addressing Extreme Poverty in a Sustainable Manner: Evidence from Ultra-Poor
Programme
In Bangladesh (Cont.)
Slide38Can Basic Entrepreneurship Transform the Economic Lives of the Poor
?
Timeframe
of
study:
2007-2011
Cost of evaluation:
~$2 million
Annual income increased 38% over 4 years.
Food
security
improved 18pp after two years, and 8pp after four
years, corresponding to a 39% and 18% increase from the baseline.After 2 years
, self-employed women in the treated communities increased 20pp from the baseline, compared to only 6.2pp increase in control group. This difference further improved at 4 year point.After 2 years: ↑ ownership of livestock despite being free to dispose of these assets. Chicken ownership ↑ 2.3pp, v. a .2pp increase
in the control group. Most chickens were retained at the four-year point. Businesses are successfully retaining and expanding livestock assets even as BRAC has withdrawn direct assistance. Ultra Poor
Slide39Change in Food and Nutrient Consumption Among the Ultra Poor: Is the
CFPR/TUP
Programme
Making a Difference
Timeline of Study:
2001-2004
Key
Findings:
A
significantly higher
%
of
TUP households consumed meat and fish regularly (>=
4 times/week), compared to control households. A relatively large number of TUP households never consumed eggs, milk or fruit, however, which has an impact on the overall nutrient intake.
Ultra-Poor - Summary from Andrew Jenkins
Impact Assessment of CFPR/TUP: A Descriptive Analysis based on 2002-2005 Panel Data
Timeline
of Study:
2002-2005
Key Findings:
Participants are more likely to have savings
,
taken a loan, & be
informed
of laws. Selected
households
more likely to possess assets such as cows &
furniture.
Savings and asset accumulation are precursors for ownership of land and significant improvements in housing
“The
program
has contributed significantly
to
improving the lives of the ultra poor and helping them help themselves”
Slide40Water and Sanitation
Slide41Achievements of BRAC Water, Sanitation, &
Hygiene
Programme
Timeframe of
study: 2006-2011
Key
Findings:
Sanitary latrine use increase
: from
the
baseline (32%) to
midline (41%)
to end line (53%)Hand washing ↑: Hand washing practice with soap at ‘critical times’ significantly increased from baseline to end line. At end line 97% of respondents had knowledge about hand washing with soap before eating and after defecation, and 22% and 88% reportedly did so at that particular time.
Reduction in water related diseases: The prevalence of water related diseases ↓ from 9.4% in the baseline to 7.1% in midline and 2.3% at the end line (see detail next slide).Water & Sanitation
Slide42WASH program significant reduction in water related disease prevalence
Water & Sanitation
Slide43Achievements of BRAC Water, Sanitation, and Hygiene
Programme
(Cont.”)
Other
Findings:
↑ in hygienic
practice in terms of covering
water
vessels during transportation and
storage.
The improvement in knowledge and practice in drinking pure water could be a result of the health education provided by the WASH program.
Although
the prevalence of water-related diseases significantly reduced amongst children over and under five-years-old, the prevalence of water-related diseases continued to be highly pronounced among children under
five. Evidence suggests it is likely that a substantial number of mother/caregivers do not wash their hands with soap/ash after fecal contact and before feeding a child.
Water & Sanitation
Slide44Youth
Slide45Social Network and Financial Literacy among Rural Adolescent Girls: Qualitative Assessment of BRAC’s
SoFEA
Programme
Timeframe
of
study: April 2011- September 2012
Key Findings:
↑
in self-confidence
and
motivation
and in strengthening the relationship between girls and their parents and within the
community and improved socialization. After receiving Financial Literacy Training (FLT), SoFEA members were more financially aware and many were involved in Income Generating Activities, such as livestock and poultry
rearingThe fact that many of the girls are now supporting themselves financially has empowered them immensely, not only within their own families but also within the community as a whole.Concerns:Time
constraints for active participation—girls stopped attending regularly during exam periods and harvest season. Early Marriage The fact that rural Bangladeshi women tend to marry quite young poses a socio-cultural challenge to the SoFEA program; after they marry they tend to drop out of school to focus on post wedding responsibilities and no longer maintain a wider network of friends. Father opposition: biggest opponents to club participation were the fathers. Girls mentioned during sessions that even some educated fathers forbid daughters from going.
Youth
Slide46Empowering Adolescent Girls: Evidence from a Randomized Control Trial in
Uganda
Timeframe of Study: 2008-2010
Cost Effectiveness of program:
Cost of $17.9 v income gains of $26.7
Life
skills:
Condom usage
↑ 50%
points
among
sexually active.
Fertility
rates were 29% lower compared to a control sample.Participants’ reports of
having sex unwillingly decreased by 83% from the baseline – a clear indicator of success in empowering youth girls.Girls in treated communities who agree that “females should earn money for the family” increased by 18 pps from the baseline level of 37%.Vocational Skills:The likelihood of being engaged in income generating activity ↑ 35% over baseline levels.Personal consumption expenditures ↑ 33% from the baseline. No adverse effect on schooling: The ELA program does not appear to have an adverse effect on schooling investments made by participants.
Youth
Slide47