Making Sense of Outcomes - PowerPoint Presentation

Slide1

Making Sense of OutcomesUsing the signification process of SenseMaker to enrich Outcome Harvesting

Goele ScheersAEA conference 2016, Atlanta3:30 PM–4:15 PMOct 28, 2016Room A701Slide2

Outcome Harvesting A tool to identify, formulate, analyse and interpret outcomes to answer useable questions.

How did a programme or initiative contribute to outcomes?Slide3

Behavioural change outcomesAn observable and significant change in a social actor’s behaviour, relationships, activities, actions, policies or practice that has been achieved and that has been influenced by the change agent.Slide4

6 stepsDesign the harvestReview documentation and draft outcomes

Engage with change agentsSubstantiate Analyse, interpretSupport use of findingsSlide5

5

Steff Deprez (April 2016)

Outcome changes / fragments / statements

SenseMaker

SenseMaker

is a complexity-aware monitoring and evaluation approach

Collection of large numbers of micro-narratives (experiences, anecdotes, events, moments, etc.)

Self-signified by the storyteller (at the point of origin)

SenseMaker

® comes with software for collection and

analysis

(Cognitive Edge)Slide6

6

Steff Deprez (April 2016)

T

hink about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.

Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?

Please, tell us more about your story … Slide7

7

Steff Deprez (April 2016)

T

hink about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.

Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?

Please, tell us more about your story … Slide8

8

Steff Deprez (April 2016)

T

hink about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.

Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?

Please, tell us more about your story …

In your experience, attitudes revealed were…

wanting things to change

K

eep

things as they areSlide9

Cases

Case 1: The signification process applied during an Outcome Harvesting Evaluation of a Global Network dedicated to improving the life of vulnerable childrenCase 2 : Collective signification

during an outcome harvesting process in a Cereal Value Chain project in Mali (USAID). Slide10

The signification process applied during an Outcome Harvesting Evaluation of a Global Network dedicated to improving the life of vulnerable childrenSlide11

Family for Every Child is a global alliance of national civil society organisations working together to mobilise knowledge, skills and resources to build a world where every child grows up in a permanent, safe and caring family,

and to provide quality alternative care where neededSlide12

Outcome Harvesting for Family for Every ChildOH Evaluation: outcomes 2012-2016Integrate OH into the M&E system of the networkSlide13

6

OH steps applied to Family for Every Child

2. Review documentation and draft outcomes

1. Design the Harvest

4

.

Analyse

and interpret

3. Engage network members, board and staff

6

. Support use

of findings and make strategic decisions

5

. SubstantiationSlide14

Outcomes were gathered during focus groups with secretariat staff, board and members  95 outcomes

Harvesting the outcomesSlide15

Harvesting the outcomesSlide16

Example of an outcome from FamilyOutcome description: In June 2015, The Ministry of Women and Child Development of India invited Butterflies (Family’s Indian member) to lead the Foster Care Committee in developing guidelines, training modules and monitoring tools on foster

care and included their contribution in the Foster Care guidelines that were launched in October 2015.Significance: This was a key moment in the development of state policy and guidance in India, when not only international NGOs and experts, but also national NGOs, including Butterflies were asked to contribute. Previously, there had been piecemeal and patchy development of foster care in India, led by internationally supported NGOs, but not strongly or consistently coordinated by the government.

Contribution: The exposure

of Butterflies to foster care through its membership of Family enabled it to respond to a request it received from its government, and thereby make a significant contribution to the government’s development of policy and guidance for foster care providers. Butterflies was able to respond

within tight deadlines

because of the help of network members finding

and sharing with them the policy and practice examples from various countries

.Slide17

Harvesting the outcomesSlide18

Tagging of outcomes: Nature of the outcomeSlide19

Tagging of outcomes: SignificanceSlide20

Tagging of outcomes: ContributionSlide21

Tagging of outcomes: Effects of the outcomesSlide22

Analysis: triadSlide23

Analysis/Triads filteredSlide24

Analysis: contribution dyadSlide25

CONTRIBUTION OF THE NETWORK

Change

has happened

without

influence

of

the

network

Change

only

happened

because of the interventions of the network Internal outcomes

68%

External outcomes

14 outcomes

10 outcomes

Analysis: Dyad filteredSlide26

Analysis: dyads filteredSlide27

27

Governance

Membership

Better Outcomes Children

Income

55%

8

%

37%

STRATEGIC OBJECTIVES

Analysis: useless triadSlide28

Collective signification of outcomes

Outcome Harvesting Evaluation Cereal Value Chain project in Mali Slide29

The Cereal Value Chain Project

Focus area’s (1) Improve Farmer Cooperative Management (2) Improve Production Quality and Quantity (3) Increase Market Sales of Millet/Sorghum and Rice Five-year program (2014-2019)

101 communes: Sikasso & MoptiOne key implementer:

Bamako Two local partner organisations: Mopti,

SikassoSlide30

DOCUMENT REVIEW

Harvesting of Outcomes

OUTCOME HARVESTING WORKSHOP

Outcome Formulation

Outcome description

Significance description

Contribution description

Signification

> 60 outcomes formulated

2-daysSlide31
Slide32

Farmer CooperativesFrom 2014 until April 2016, the cooperative Rizicultrice de Niéna (512 members) has applied System Rice Intensification (SRI) on an area of 40ha, while the original forecast was 10 ha.

Farmer CooperativesIn 2014, the cooperative of Zébala 7 (12 members) has managed to do a group sale of 3.300T of sorghum and 7,24T millet for an amount of 1.551.460 FCFA. The extra revenues were used to start building a warehouse for cereals.

BanksOn 10/10/2014 the Mopti BMS financed the

Fédération pour le Développement Rural de Youwarou

(FDRY) for the sum of 246,500,000 FCFA.

OMA

The ‘Office de Observation du Marché

Agricole

’ (OMA) expanded the zones in 2014 in which market information (weekly price and quality of cereals sold in the

Mopti

,

Sikasso

and

Ségou

regions) is distributed

District Agriculture AuthorityIn 2015, the DRA used 7 local radio stations to broadcast at large-scale agricultural production technologies and the use of pesticides. The radio stations are ORTM Sikasso, Radio Kenedougou de Sikasso, Yeredon de koutiala, Wassoulou de Yanfolila, Folona de Kadiolo and Kafo Kan de Bougouni.Research Institutes As of June, 2014, the Regional Center for Agricultural Research (Centre Régional

de Recherche Agricole CRRA) in

Mopti introduced 4 new varieties of ameliorated rice adapted to the village parameters. These four varieties are: WAPMO, SUTURA, SAKU and SK 95-4. In 2014, seed production for 4 varieties were 4 tonnes (base seeds of G4). During the two last years (2015, 2016) the CRRA self-funded to pursue these activities of producing seeds of the 4 varieties. The seeds were sold in seed vendor stores.Slide33

OH template

Guides discussions

Collective Indexing Facilitates documentation Slide34
Slide35
Slide36
Slide37
Slide38

Collective tagging &

signifcation Slide39

AnalysisSlide40

Comparison of the nature of outcomes in 2014 and 2015

The CMDT invested in the capacity building of cooperatives in 2014 and 2015, to help prevent erosion. This was done through the

restauration of the soil and the increase in agricultural yield across the 117 cooperatives and more precisely in N’gounkan

and Bandiagara in the

Koutiala

subsidiary.

In 2015, three cooperatives (

Kendé

,

Ogodiré

,

Souley

) finalized the process for formalizing their cooperatives according to the law of the Organization for the Harmonization of Business Rights in Africa (OHADA)

In 2015, Faso

Kaba

of Bamako, a seed company, expanded its distribution network in Koutiala in the region of Sikasso by including the producer organization, Yeredon of Kadiaradougou (commune of Loulouni), and

Benkadi of Parasso

(commune of Loulouni). Slide41

16 outcomes

7 outcomes

Analysis

Contribution of the CVC Activity to the outcomesSlide42

Analysis

Agro-Dealers

Farmer Organisation

Processing Companies

Grain Traders

BanksSlide43

Reflections on Sensemaker for OH - Strenghts

COLLECTIVE SIGNIFICATION Stimulates discussion and joint reflection during harvestingCommon understanding about program, outcomes and the role of the implementerTriads are easy to understand, useful, possibility to make nuances

ANALYSISMore depth, new layers of meaning/connectionsEasy visualization of patterns

Change agents can relate to the analysisSlide44

SUPPORTING USE OF FINDINGS Feedback of the results of the OH evaluationFurther collective sensemaking

on the outcomesGenerate actionable insights

Reflections on Sensemaker for OH - StrenghtsSlide45

Three corners need to be well-definedTriads have to be connected to the use and the evaluation questionsNeed to have a substantial amount of outcomes

Reflections on Sensemaker for OH - ChallengesSlide46

Thank you!Goele ScheersIndependent consultantgoelescheers@gmail.com

www.goelescheers.be