Four Rules of the Art AND SOME POKING AROUND FOR
Description: Four Rules of the Art AND SOME POKING AROUND FOR PRACTICAL SOLUTIONS TO THE CHALLENGES OF JOINT PEACE INTERVENTION EVALUATIONS Emery Brusset NUPIIPI Workshop, New York 7 8 May 2009 Channel Research 2 Four Theoretical Challenges to the
Related Topics
Download Presentation
"Four Rules of the Art AND SOME POKING AROUND FOR" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Four Rules of the Art AND SOME POKING AROUND FOR PRACTICAL SOLUTIONS TO THE CHALLENGES OF JOINT PEACE INTERVENTION EVALUATIONS Emery Brusset
NUPI/IPI Workshop, New York
7 & 8 May 2009<br>
slide2. Channel Research 2 Four Theoretical Challenges to the evaluation of conflict interventions 1. Seizing Opportunities
Programming requires a response to constraints and opportunities as they arise.
These cannot be planned for in advance and plans are not a good evaluation reference 2. Attribution
It is particularly difficult to separate the impact of peace-building projects from other impacts on conflict
The time required to cover other interventions is limited 3. Data Quandaries
Selecting evidence requires prioritisation and demands a good understanding of the nature of the conflict
The verification methods must be applicable in a conflict situation 4. End State?
Problems in defining terms such as peace and conflict.
Usually depends on the identification of an end-state.
This may not be explicit or shared by all stakeholders or those within a single programme Options<br>
slide3. Channel Research 3 The Practical Challenge of Joint Evaluation: Evaluation needs to provide a convincing set of conclusions to a growing diversity and size of audience “Ideal single actor evaluation” TOR enable the evaluator to identify unplanned and unintended effects
A way is found to identify comparative weight of intervention
Good conflict analysis is possible
The nature of “peace” is agreed “Reality of Joint Evaluation” Expectations evolve
The questions focus on the management of interventions
There is no strategic baseline or agreed assessment
Peace is defined generically, or not at all 4 Rules of Good Evaluation:
Independence of team to focus on situation as it emerged
Team has means to analyse societal interface
Team is capable of participatory assessments
Conflict objectives are reconstructed to From<br>
slide4. Channel Research 4 Challenge 1: Capturing unintended and unexpected aspects
Rule 1: The team is enabled to define what issues to focus on,
and allows for Steering Committee consultation milestones
to secure alignment of all stakeholders.<br>
slide5. Challenge 2: Evaluation questions tend to focus on institutional issues Evaluation, particularly real time evaluation, tends to be linked to decisions and the need to understand and support the management of an intervention. This risks reducing the time available to analyse the impact .<br>
slide6. Conflict Analysis Channel Research 6 Rule 2: Evaluations are given time to enable an iterative but systematic process (six months to a year?) – scope is contained. There is agreement that attribution of impact is replaced with contribution. Threat Threat Threat Threat Threat Threat ? Success depends on sufficient resources<br>
slide7. Challenge N° 3: Get the readers and audience to agree on a consistent analysis of the conflict, and on key drivers of peace Map of situation Intervention footprint Situation with
intervention factored in –
how does it affect
situation, how does
situation affect it? Channel Research<br>
slide8. Channel Research 8 Participatory Mapping of Immediate Conflict Environments (MICE) works by looking at linkages between key trends and events = drivers Event: occurs at
a point in time Trend: change
in a condition or
characteristic Arrow of change:
increased
probability or
intensity The elements of
conflict mapping Growing
Unemployment Coup Corruption
more apparent Officers inc.
disgruntled Battle in which
war was lost Growing leftist
radicalisation Dissent
increasingly
suppressed Many insurgent
Attacks in XX Hypothetical and partial example: Country with
military regime in conflict with leftist insurgents<br>
slide9. Channel Research 9 The aim is the identification of key issues that need to be addressed to mitigate conflict or instability 5 4 4 4 To identify priority issues, we count the linkages leading to
and from relevant events and trends – those with the most
linkages have the most reverberation in the situation Priority issues:
1. Leftist radicalisation
2. Militarism, state repression, military dictatorship
CAVEAT
Conflict mapping is not root cause or systems analysis. Although corruption and unemployment might be major causes of leftist radicalisation, we do not identify them as priority issues – ranking is not intuitive at all<br>
slide10. Channel Research 10 Representation is qualitative and culturally neutral. For example… Military
coup Leftist
radicalisation Leftist
uprising Military
Politicisation increases Military
repression Labour
activism More Poverty Peasant
revolt Leftist – peasant
cooperation US company
oil contract Leftist
perception of
collusion Company
unpopularity
on ground Rebel threats
to company Strikes among
project staff Increasing
employment Increasing
state revenues Military regime
corruption Human rights
NGO scrutiny Shareholder
scrutiny US gov’t
criticism of
military regime Pressure for
democratisation Military steps up
project security General
strikes Oil production
increases _ _ _<br>
slide11. Channel Research 11 An example from a real mapping exercise in a post-conflict African country<br>
slide12. Channel Research 12 Mapping of Cross-Border Issues in the Conflict in NI Table 1: Key ‘nodes’ emerging in different mapping workshops, ranked according to emphasis given in each workshop<br>
slide13. Channel Research 13 Challenge N° 4: objectives and
their connection to
peace are unclear
or outdated<br>
slide15. Complexity in PLANNING is not well captured by prevailing tools “Project” Channel Research LogFrame “Strategy” ???<br>
slide16. Rule 3: allow the team to innovate and interpret in mapping objectives and their meaning in terms of peace “Project” Channel Research LogFrame “Strategy” “Effect Diagram”<br>
slide17. Channel Research 17 The intervention tree in its simplest form – it is an adapted logic or decision tree We need to achieve this, and to achieve this we need to have obtained… … this, and to see this, we need to do… … this, and to see this, we need to do… … this, and to see this, we need to do… This This This This This This This This This Each new branch needs to be a separate and
complete initiative to the extent possible, to
ensure no redundancy, and to spot any gaps<br>
slide18. Poverty reduced Alternative rural livelihoods created Equitable access to productive assets, markets and services (e.g. microfinance) established EC activities Outputs / Outcomes Impacts Rural economy strengthened Poppy production reduced or stabilized State of law and order improved (incl. drug enforcement Sustainable management of natural resources achieved / improved Vulnerability of rural population decreased Rural production and gainful employment strengthened More effective provincial / community based institutions developed Governmental / admin. structure strengthened Civil Service System improved (including police) Governmental physical infrastructure re-created Gov / ATA budget stabilized National payment system created Revenue position of Gov strengthened Reform of civil service promoted Access to BHCP improved / increased Health sector management on central and provincial level improved Positive tendency in health status of population (especially U5 and maternal mortality) Rural Development Public Administration Reform Foundation for public administration reform created (NOTE: in 2nd NIP, increased focus on sub-national level) (Participation of) rural communities in dev. Strengthened Health Example: EC Effect Diagram Afghanistan<br>
slide19. Channel Research 19 Actions Outcomes Outputs Global Impact Conflicts in partner countries prevented Regional co-operation and integration strengthened Support measures for regional integration mechanisms and trade co-operation implemented Direct Impact CSP used as central instrument for mainstreaming conflict prevention Trade links between countries strengthened. Macro-economic environment in at risk countries stabilized. Equal participation of men & women in social, economic & political life promoted (14). 1) Elements of Structural Stability: sustainable economic development, democracy & respect for Human Rights, viable political structures, healthy environmental & social conditions, capacity to manage change without resorting to conflict. Potential for more favourable democratic environment increased(14). Rule of Law, Good Governance & Civil Society strengthened & (13) Security sector prepared for post-conflict situation Peace in post-conflict countries consolidated (16). Potential for renewed conflicts in post-conflict countries reduced (16). Structural Stability improved 1) Support to security sector increased (I.e. police training, HR training) (14). Rehabilitation and reconciliation programmes implemented (16). Targeted actions implemented to support electoral process, parliamentary activities, administration of justice (14). Macro-economic assistance operations implemented (12). Integrated Approach Ownership of process by partner country established Cross-cutting issues efficiently addressed in EC activities Mechanisms developed to appropriately address cross-cutting issues in relevant areas:
social conditions (drugs, small arms, Health / communicable diseases (HIV / Aids), Population flows / trafficking)
environment & natural resources (water)
economy (Private sector in unstable areas) Early-warning mechanism & procedures created EU decision-making & operational centres alerted to imminent crisis EC enabled to formulate appropriate response to developing crisis situation (incl. preventive sanctions) information on cause & potential consequences of conflict provided EC enabled to react quickly to developing crisis situation Rapid Reaction Mechanism (RRM) introduced EC enabled to carry out wide-ranging short term missions on short notice Co-ordination of Member States on common political line intensified (incl. sanctions)(23). More focused, time-flexible & robust political dialogue & diplomacy enabled (23) Degeneration of political difficulties / tension into armed conflict prevented Traditional community instruments optimised to respond to conflict (including ESDP) Reform of aid management allows more responsiveness to needs in line with the performance of partner countries Coherence of approaches for conflict prevention improved among partners (countries, international organisations, NGOs) Dialogue with partners intensified & focused on conflict prevention (incl. NGOs)(25) Common early warning systems, monitoring procedures & conflict indicators developed Appropriate mechanisms & procedures for coordination with international partners developed. Cooperation with international partners intensified Tensions between countries reduced Poverty reduced (14). Economic growth stimulated Societal post-conflict tension reduced Potential for new conflicts in at at-risk countries reduced. Potential for conflicts between partner countries reduced<br>
slide20. Analysis of Effect Diagrams What is their purpose?
Visualize hierarchy of objectives / expected effects
Reveal logical links, help to define certain Evaluation Questions & assess internal coherence of strategy
What are their weaknesses?
Do not capture changes over time or feedback loops
“Other actor” / “Implementation” / “Context” not captured
Conclusion?
For complex interventions: can be used for sub-diagrams
Shows missing elements
Does not provide the final leap from objective to situation change Channel Research<br>
slide21. The diagram can be turned into a narrative: theories of change Theories of change can take the simple format: “We believe that by doing X (action) successfully, we will produce Y (movement towards peace).”
Or the chain of logic might involve multiple steps: “if we do X, that will lead to Y, which will encourage Z, which will exert an influence on key actor A.”
In order to create, uncover or test a theory of change, it is essential to conduct a comprehensive analysis of the conflict, identify the key factors of conflict and the relationship between them, and spell out how the intervention (project, programme or policy) will create change.
Making theories of change explicit is a central part of planning and programme design and can facilitate evaluation. Channel Research 21<br>
slide22. List of Identifiable Theories of Change: The Individual Change Theory: Peace comes through transformative change of a critical mass of individuals, their consciousness, attitudes, behaviors and skills. [Methods: investment in individual change through training, personal transformation/consciousness-raising workshops or processes; dialogues and encounter groups; trauma healing.]
The Healthy Relationships and Connections Theory: Peace emerges out of a process of breaking down isolation, polarization, division, prejudice and stereotypes between/among groups. Strong relationships are a necessary ingredient for peacebuilding. [Methods: processes of inter-group dialogue; networking; relationship-building processes; joint efforts and practical programs on substantive problems.]
The Withdrawal of the Resources for War Theory: Wars require vast amounts of material (weapons, supplies, transport, etc.) and human capital. If we can interrupt the supply of people and goods to the war making system, it will collapse and peace will break out. [Methods: anti-war campaigns to cut off funds/national budgets; conscientious objection and/or resistance to military service; international arms control; arms (and other) embargoes and boycotts.] Channel Research 22<br>
slide23. The Reduction of Violence Theory: Peace will result as we reduce the levels of violence perpetrated by combatants or their representatives. [Methods: cease-fires, creation of zones of peace, withdrawal/retreat from direct engagement, introduction of peacekeeping forces/interposition, observation missions, accompaniment efforts, promotion of nonviolent methods for achieving political/social/economic ends.]
The Root Causes/Justice Theory: We can achieve peace by addressing the underlying issues of injustice, oppression/exploitation, threats to identity and security, and peoples’ sense of injury/victimization. [Methods: long-term campaigns for social and structural change, truth and reconciliation; changes in social institutions, laws, regulations, and economic systems.]
The Institutional Development Theory: Peace is secured by establishing stable/reliable social institutions that guarantee democracy, equity, justice, and fair allocation of resources. [Methods: new constitutional and governance arrangements/entities; development of human rights, rule of law, anti-corruption; establishment of democratic/equitable economic structures; economic development; democratization.]
The Political Elites Theory: Peace comes when it is in the interest of political (and other) leaders to take the necessary steps. Peacebuilding efforts must change the political calculus of key leaders and groups. [Methods: raise the costs and reduce the benefits for political elites of continuing war and increase the incentives for peace; engage active and influential constituencies in favor of peace; withdraw international support/funding for warring parties.] Channel Research 23<br>
slide24. Channel Research 24 The Grassroots Mobilization Theory: “When the people lead, the leaders will follow.” If we mobilize enough opposition to war, political leaders will have to pay attention. [Methods: mobilize grassroots groups to either oppose war or to advocate for positive action. Use of the media; nonviolent direct action campaigns; education/mobilization effort; organizing advocacy groups; dramatic events to raise consciousness.]
The Economics Theory: As a politician once said, “It’s the economy, stupid!” People make personal decisions, and decision-makers make policy decisions based on a system of rewards and incentives and punishment/sanctions that are essentially economic in nature. If we can change the economies associated with war-making, we can bring peace [Methods: use of government or financial institutions to change supply and demand dynamics; control incentive and reward systems; boycotts.]
The Public Attitudes Theory: War and violence are partly motivated by prejudice, misperceptions, and intolerance of difference. We can promote peace by using the media (television and radio) to change public attitudes and build greater tolerance in society. [Methods: TV and radio programs that promote tolerance; modeling tolerant behavior; symbolic acts of solidarity/unity; dialogues among groups in conflict—with subsequent publicity.]<br>
slide25. 25 OTI HQ Outcome Mapping: Focusing on actors and behaviour<br>
slide26. Channel Research 26 Challenge N°5: Bringing it all back together E T E E E T T T Linkage mapping Issue 1 Issue 2 Issue 3 Outcomes Outcomes Outcomes Outcomes Relevance
x
Extent
x
Duration Focus of evaluation The Significance of Impact methodology
has been designed to provide solid evidence
on the contribution which an actor makes
to a particular issue<br>
slide27. Issues identified with the mapping tools can provide the starting point Priority issues
Poor relations with local community
Rivalry between central and provincial administration
Criminal economy
Ethnic cleavages lead to growing fear of threats Extent and Duration
What proportion of the target population is covered?
Were the key people accessed?
How long did the influence last?
Can the influence be sustained after the ending of the intervention? Duration Extent a b c d SIGNIFICANCE Priority risks<br>
slide28. Leading to three dimensional assessments… Priority issues
Poor relations with local community
Rivalry between central and provincial administration
Criminal economy
Ethnic cleavages lead to growing fear of threats Extent and Duration
What proportion of the target population is covered?
Were the key people accessed?
How long did the influence last?
Can the influence be sustained after the ending of the intervention? Relevance:
What do others do?
Are there other options?
Could a clearer objective have been designed?
Does the intervention correspond to the need assessed? Degree
of
relevance Extent and Duration Extent Duration Significance D C B A D. . . C. . . B. . . A. . . The tools is designed to highlight what has been done and test its connection to
the issues which are recognised as issues to be addressed to achieve ‘progress’<br>
slide29. Channel Research 29 All levels of the result chain can yield important results or outcomes: demobilisation Activities and process deliver output
Outputs are defined and easily verified
Example: “a workshop is held” First level
Operational level Specific objectives are the results of activities and ideally are how outputs are to be used by the beneficiary groups – user interface
Example: “skills and tools are taught” Second level
Programme level General objectives are the statement of overall verifiable change intended
Example: “Armed groups are able to design a new force concept” Third Strategy
Level
Society & State Process can be the output
The issue can be that groups have lost contact and need to build mutual respect
Example: “Familiarity is achieved”. New capacities can lead to changes in the command structure and the reintegration of groups.
Example: “Specialisation and opportunities are required for conversion to civilian life” Changes in the overall environment can lead to a better notion of security based on smaller forces
Example: “Security imperatives require larger armed groups”. Strategy Outcome<br>
slide30. The Table of Contents of the Evaluation Report! Issues / Entry Points have been identified
Outcomes achieved have been identified
Assessment of the value of the contribution is possible:
Relevance and efficiency
Effectiveness
Sustainability
Coherence with other interventions
Overall conclusions on « posture »
Recommendations Channel Research 30<br>
NUPI/IPI Workshop, New York
7 & 8 May 2009<br>
slide2. Channel Research 2 Four Theoretical Challenges to the evaluation of conflict interventions 1. Seizing Opportunities
Programming requires a response to constraints and opportunities as they arise.
These cannot be planned for in advance and plans are not a good evaluation reference 2. Attribution
It is particularly difficult to separate the impact of peace-building projects from other impacts on conflict
The time required to cover other interventions is limited 3. Data Quandaries
Selecting evidence requires prioritisation and demands a good understanding of the nature of the conflict
The verification methods must be applicable in a conflict situation 4. End State?
Problems in defining terms such as peace and conflict.
Usually depends on the identification of an end-state.
This may not be explicit or shared by all stakeholders or those within a single programme Options<br>
slide3. Channel Research 3 The Practical Challenge of Joint Evaluation: Evaluation needs to provide a convincing set of conclusions to a growing diversity and size of audience “Ideal single actor evaluation” TOR enable the evaluator to identify unplanned and unintended effects
A way is found to identify comparative weight of intervention
Good conflict analysis is possible
The nature of “peace” is agreed “Reality of Joint Evaluation” Expectations evolve
The questions focus on the management of interventions
There is no strategic baseline or agreed assessment
Peace is defined generically, or not at all 4 Rules of Good Evaluation:
Independence of team to focus on situation as it emerged
Team has means to analyse societal interface
Team is capable of participatory assessments
Conflict objectives are reconstructed to From<br>
slide4. Channel Research 4 Challenge 1: Capturing unintended and unexpected aspects
Rule 1: The team is enabled to define what issues to focus on,
and allows for Steering Committee consultation milestones
to secure alignment of all stakeholders.<br>
slide5. Challenge 2: Evaluation questions tend to focus on institutional issues Evaluation, particularly real time evaluation, tends to be linked to decisions and the need to understand and support the management of an intervention. This risks reducing the time available to analyse the impact .<br>
slide6. Conflict Analysis Channel Research 6 Rule 2: Evaluations are given time to enable an iterative but systematic process (six months to a year?) – scope is contained. There is agreement that attribution of impact is replaced with contribution. Threat Threat Threat Threat Threat Threat ? Success depends on sufficient resources<br>
slide7. Challenge N° 3: Get the readers and audience to agree on a consistent analysis of the conflict, and on key drivers of peace Map of situation Intervention footprint Situation with
intervention factored in –
how does it affect
situation, how does
situation affect it? Channel Research<br>
slide8. Channel Research 8 Participatory Mapping of Immediate Conflict Environments (MICE) works by looking at linkages between key trends and events = drivers Event: occurs at
a point in time Trend: change
in a condition or
characteristic Arrow of change:
increased
probability or
intensity The elements of
conflict mapping Growing
Unemployment Coup Corruption
more apparent Officers inc.
disgruntled Battle in which
war was lost Growing leftist
radicalisation Dissent
increasingly
suppressed Many insurgent
Attacks in XX Hypothetical and partial example: Country with
military regime in conflict with leftist insurgents<br>
slide9. Channel Research 9 The aim is the identification of key issues that need to be addressed to mitigate conflict or instability 5 4 4 4 To identify priority issues, we count the linkages leading to
and from relevant events and trends – those with the most
linkages have the most reverberation in the situation Priority issues:
1. Leftist radicalisation
2. Militarism, state repression, military dictatorship
CAVEAT
Conflict mapping is not root cause or systems analysis. Although corruption and unemployment might be major causes of leftist radicalisation, we do not identify them as priority issues – ranking is not intuitive at all<br>
slide10. Channel Research 10 Representation is qualitative and culturally neutral. For example… Military
coup Leftist
radicalisation Leftist
uprising Military
Politicisation increases Military
repression Labour
activism More Poverty Peasant
revolt Leftist – peasant
cooperation US company
oil contract Leftist
perception of
collusion Company
unpopularity
on ground Rebel threats
to company Strikes among
project staff Increasing
employment Increasing
state revenues Military regime
corruption Human rights
NGO scrutiny Shareholder
scrutiny US gov’t
criticism of
military regime Pressure for
democratisation Military steps up
project security General
strikes Oil production
increases _ _ _<br>
slide11. Channel Research 11 An example from a real mapping exercise in a post-conflict African country<br>
slide12. Channel Research 12 Mapping of Cross-Border Issues in the Conflict in NI Table 1: Key ‘nodes’ emerging in different mapping workshops, ranked according to emphasis given in each workshop<br>
slide13. Channel Research 13 Challenge N° 4: objectives and
their connection to
peace are unclear
or outdated<br>
slide15. Complexity in PLANNING is not well captured by prevailing tools “Project” Channel Research LogFrame “Strategy” ???<br>
slide16. Rule 3: allow the team to innovate and interpret in mapping objectives and their meaning in terms of peace “Project” Channel Research LogFrame “Strategy” “Effect Diagram”<br>
slide17. Channel Research 17 The intervention tree in its simplest form – it is an adapted logic or decision tree We need to achieve this, and to achieve this we need to have obtained… … this, and to see this, we need to do… … this, and to see this, we need to do… … this, and to see this, we need to do… This This This This This This This This This Each new branch needs to be a separate and
complete initiative to the extent possible, to
ensure no redundancy, and to spot any gaps<br>
slide18. Poverty reduced Alternative rural livelihoods created Equitable access to productive assets, markets and services (e.g. microfinance) established EC activities Outputs / Outcomes Impacts Rural economy strengthened Poppy production reduced or stabilized State of law and order improved (incl. drug enforcement Sustainable management of natural resources achieved / improved Vulnerability of rural population decreased Rural production and gainful employment strengthened More effective provincial / community based institutions developed Governmental / admin. structure strengthened Civil Service System improved (including police) Governmental physical infrastructure re-created Gov / ATA budget stabilized National payment system created Revenue position of Gov strengthened Reform of civil service promoted Access to BHCP improved / increased Health sector management on central and provincial level improved Positive tendency in health status of population (especially U5 and maternal mortality) Rural Development Public Administration Reform Foundation for public administration reform created (NOTE: in 2nd NIP, increased focus on sub-national level) (Participation of) rural communities in dev. Strengthened Health Example: EC Effect Diagram Afghanistan<br>
slide19. Channel Research 19 Actions Outcomes Outputs Global Impact Conflicts in partner countries prevented Regional co-operation and integration strengthened Support measures for regional integration mechanisms and trade co-operation implemented Direct Impact CSP used as central instrument for mainstreaming conflict prevention Trade links between countries strengthened. Macro-economic environment in at risk countries stabilized. Equal participation of men & women in social, economic & political life promoted (14). 1) Elements of Structural Stability: sustainable economic development, democracy & respect for Human Rights, viable political structures, healthy environmental & social conditions, capacity to manage change without resorting to conflict. Potential for more favourable democratic environment increased(14). Rule of Law, Good Governance & Civil Society strengthened & (13) Security sector prepared for post-conflict situation Peace in post-conflict countries consolidated (16). Potential for renewed conflicts in post-conflict countries reduced (16). Structural Stability improved 1) Support to security sector increased (I.e. police training, HR training) (14). Rehabilitation and reconciliation programmes implemented (16). Targeted actions implemented to support electoral process, parliamentary activities, administration of justice (14). Macro-economic assistance operations implemented (12). Integrated Approach Ownership of process by partner country established Cross-cutting issues efficiently addressed in EC activities Mechanisms developed to appropriately address cross-cutting issues in relevant areas:
social conditions (drugs, small arms, Health / communicable diseases (HIV / Aids), Population flows / trafficking)
environment & natural resources (water)
economy (Private sector in unstable areas) Early-warning mechanism & procedures created EU decision-making & operational centres alerted to imminent crisis EC enabled to formulate appropriate response to developing crisis situation (incl. preventive sanctions) information on cause & potential consequences of conflict provided EC enabled to react quickly to developing crisis situation Rapid Reaction Mechanism (RRM) introduced EC enabled to carry out wide-ranging short term missions on short notice Co-ordination of Member States on common political line intensified (incl. sanctions)(23). More focused, time-flexible & robust political dialogue & diplomacy enabled (23) Degeneration of political difficulties / tension into armed conflict prevented Traditional community instruments optimised to respond to conflict (including ESDP) Reform of aid management allows more responsiveness to needs in line with the performance of partner countries Coherence of approaches for conflict prevention improved among partners (countries, international organisations, NGOs) Dialogue with partners intensified & focused on conflict prevention (incl. NGOs)(25) Common early warning systems, monitoring procedures & conflict indicators developed Appropriate mechanisms & procedures for coordination with international partners developed. Cooperation with international partners intensified Tensions between countries reduced Poverty reduced (14). Economic growth stimulated Societal post-conflict tension reduced Potential for new conflicts in at at-risk countries reduced. Potential for conflicts between partner countries reduced<br>
slide20. Analysis of Effect Diagrams What is their purpose?
Visualize hierarchy of objectives / expected effects
Reveal logical links, help to define certain Evaluation Questions & assess internal coherence of strategy
What are their weaknesses?
Do not capture changes over time or feedback loops
“Other actor” / “Implementation” / “Context” not captured
Conclusion?
For complex interventions: can be used for sub-diagrams
Shows missing elements
Does not provide the final leap from objective to situation change Channel Research<br>
slide21. The diagram can be turned into a narrative: theories of change Theories of change can take the simple format: “We believe that by doing X (action) successfully, we will produce Y (movement towards peace).”
Or the chain of logic might involve multiple steps: “if we do X, that will lead to Y, which will encourage Z, which will exert an influence on key actor A.”
In order to create, uncover or test a theory of change, it is essential to conduct a comprehensive analysis of the conflict, identify the key factors of conflict and the relationship between them, and spell out how the intervention (project, programme or policy) will create change.
Making theories of change explicit is a central part of planning and programme design and can facilitate evaluation. Channel Research 21<br>
slide22. List of Identifiable Theories of Change: The Individual Change Theory: Peace comes through transformative change of a critical mass of individuals, their consciousness, attitudes, behaviors and skills. [Methods: investment in individual change through training, personal transformation/consciousness-raising workshops or processes; dialogues and encounter groups; trauma healing.]
The Healthy Relationships and Connections Theory: Peace emerges out of a process of breaking down isolation, polarization, division, prejudice and stereotypes between/among groups. Strong relationships are a necessary ingredient for peacebuilding. [Methods: processes of inter-group dialogue; networking; relationship-building processes; joint efforts and practical programs on substantive problems.]
The Withdrawal of the Resources for War Theory: Wars require vast amounts of material (weapons, supplies, transport, etc.) and human capital. If we can interrupt the supply of people and goods to the war making system, it will collapse and peace will break out. [Methods: anti-war campaigns to cut off funds/national budgets; conscientious objection and/or resistance to military service; international arms control; arms (and other) embargoes and boycotts.] Channel Research 22<br>
slide23. The Reduction of Violence Theory: Peace will result as we reduce the levels of violence perpetrated by combatants or their representatives. [Methods: cease-fires, creation of zones of peace, withdrawal/retreat from direct engagement, introduction of peacekeeping forces/interposition, observation missions, accompaniment efforts, promotion of nonviolent methods for achieving political/social/economic ends.]
The Root Causes/Justice Theory: We can achieve peace by addressing the underlying issues of injustice, oppression/exploitation, threats to identity and security, and peoples’ sense of injury/victimization. [Methods: long-term campaigns for social and structural change, truth and reconciliation; changes in social institutions, laws, regulations, and economic systems.]
The Institutional Development Theory: Peace is secured by establishing stable/reliable social institutions that guarantee democracy, equity, justice, and fair allocation of resources. [Methods: new constitutional and governance arrangements/entities; development of human rights, rule of law, anti-corruption; establishment of democratic/equitable economic structures; economic development; democratization.]
The Political Elites Theory: Peace comes when it is in the interest of political (and other) leaders to take the necessary steps. Peacebuilding efforts must change the political calculus of key leaders and groups. [Methods: raise the costs and reduce the benefits for political elites of continuing war and increase the incentives for peace; engage active and influential constituencies in favor of peace; withdraw international support/funding for warring parties.] Channel Research 23<br>
slide24. Channel Research 24 The Grassroots Mobilization Theory: “When the people lead, the leaders will follow.” If we mobilize enough opposition to war, political leaders will have to pay attention. [Methods: mobilize grassroots groups to either oppose war or to advocate for positive action. Use of the media; nonviolent direct action campaigns; education/mobilization effort; organizing advocacy groups; dramatic events to raise consciousness.]
The Economics Theory: As a politician once said, “It’s the economy, stupid!” People make personal decisions, and decision-makers make policy decisions based on a system of rewards and incentives and punishment/sanctions that are essentially economic in nature. If we can change the economies associated with war-making, we can bring peace [Methods: use of government or financial institutions to change supply and demand dynamics; control incentive and reward systems; boycotts.]
The Public Attitudes Theory: War and violence are partly motivated by prejudice, misperceptions, and intolerance of difference. We can promote peace by using the media (television and radio) to change public attitudes and build greater tolerance in society. [Methods: TV and radio programs that promote tolerance; modeling tolerant behavior; symbolic acts of solidarity/unity; dialogues among groups in conflict—with subsequent publicity.]<br>
slide25. 25 OTI HQ Outcome Mapping: Focusing on actors and behaviour<br>
slide26. Channel Research 26 Challenge N°5: Bringing it all back together E T E E E T T T Linkage mapping Issue 1 Issue 2 Issue 3 Outcomes Outcomes Outcomes Outcomes Relevance
x
Extent
x
Duration Focus of evaluation The Significance of Impact methodology
has been designed to provide solid evidence
on the contribution which an actor makes
to a particular issue<br>
slide27. Issues identified with the mapping tools can provide the starting point Priority issues
Poor relations with local community
Rivalry between central and provincial administration
Criminal economy
Ethnic cleavages lead to growing fear of threats Extent and Duration
What proportion of the target population is covered?
Were the key people accessed?
How long did the influence last?
Can the influence be sustained after the ending of the intervention? Duration Extent a b c d SIGNIFICANCE Priority risks<br>
slide28. Leading to three dimensional assessments… Priority issues
Poor relations with local community
Rivalry between central and provincial administration
Criminal economy
Ethnic cleavages lead to growing fear of threats Extent and Duration
What proportion of the target population is covered?
Were the key people accessed?
How long did the influence last?
Can the influence be sustained after the ending of the intervention? Relevance:
What do others do?
Are there other options?
Could a clearer objective have been designed?
Does the intervention correspond to the need assessed? Degree
of
relevance Extent and Duration Extent Duration Significance D C B A D. . . C. . . B. . . A. . . The tools is designed to highlight what has been done and test its connection to
the issues which are recognised as issues to be addressed to achieve ‘progress’<br>
slide29. Channel Research 29 All levels of the result chain can yield important results or outcomes: demobilisation Activities and process deliver output
Outputs are defined and easily verified
Example: “a workshop is held” First level
Operational level Specific objectives are the results of activities and ideally are how outputs are to be used by the beneficiary groups – user interface
Example: “skills and tools are taught” Second level
Programme level General objectives are the statement of overall verifiable change intended
Example: “Armed groups are able to design a new force concept” Third Strategy
Level
Society & State Process can be the output
The issue can be that groups have lost contact and need to build mutual respect
Example: “Familiarity is achieved”. New capacities can lead to changes in the command structure and the reintegration of groups.
Example: “Specialisation and opportunities are required for conversion to civilian life” Changes in the overall environment can lead to a better notion of security based on smaller forces
Example: “Security imperatives require larger armed groups”. Strategy Outcome<br>
slide30. The Table of Contents of the Evaluation Report! Issues / Entry Points have been identified
Outcomes achieved have been identified
Assessment of the value of the contribution is possible:
Relevance and efficiency
Effectiveness
Sustainability
Coherence with other interventions
Overall conclusions on « posture »
Recommendations Channel Research 30<br>