From the Presients Desk - PDF document

1 1 From the Presient
1 From the Presient’s Desk It gives me great pleasure to bring you a short message through this fourth Newsletter of IOCE. First, I would like to thank Natalia Kosheleva for efficiently taking over the editorial responsibilities. This year commenced with a very significant activity that would promote collaboration and cooperation among our members. This is the launching of EvalPartners, a collaborative initiative with UNICEF and with assistance from the Government of Finland. Many leading organisations and institutions have already joined the initiative as partners. One main purpose of this initiative is to facilitate and catalyse strategies of IOCE members to bring about stronger evaluation capacities in their respective areas of influence. A primary strategy would be promoting int eractions and collaboration among members such as through peer to peer learning. Good practices and lessons learnt will be analysed a s part of knowledge management strategy. An appropriate forum for exchange of such experiences will be available at a global conference which will be a milestone in the EvalPartners process. We invite all V oluntary Organisations of Professional evaluators to join this initiative and lead a process of professionalization of evaluation Soma De Silva President - IOCE                                               In this issue: TOP NEWS EvalPartners Initiative Page 2 Eval Partners at VOPE conferences Page 3 NEWS from the IOCE BOARD 2011 Annual General Meeting Page 3 IOCE Board Meeting in Accra, Ghana Page 4 Meet a New Member of the IOCE Board: Doha Abdelhamid Page 5 IOCE at the NONIE Meeting in Rome Page 5 IOCE Former President at the Busan Forum Page 7 Connecting with evaluators around the globe Page 8 IOCE MEMBERS A brief look at the Brazilian Monitoring and Evaluation Network Page 9 A message from the Australasian Evaluation Association 2012 International Conference Page 9 Issue No. 4 . M arch 201 2 2 TOP NEWS: EvalPartners Initiative IOCE and UNICEF have launched a significant global initiative to promote multi - party partnerships that will collaborate in strategic ways to enhance the capacities of Voluntary Organizations of Professional Evaluators (VOPEs) to enhance their capacities to engage in national evaluation processes, and to influence country - led evaluation systems. We’re calling it the EvalPartners Initiative . See http://www.mymande.org/evalpartners . Note: There have been a variety of ways of describing organizations formed by evaluators. Some call those that are formally organized associations, some call them societies. More informal ones are referred to as networks. We are proposing the introducti on of the term VOPE (Voluntary Organizations of Professional Evaluators) to be inclusive of the many organizational forms manifest by these groups, and recognize that what we are referring to are not commercial, for - profit consultancy firms, nor communitie s of practice open only to employees of particular agencies, but voluntary groupings open to any individual evaluator

2 s who see value in mutual learning and
s who see value in mutual learning and working together to enhance the evaluation profession. And we have evidence that this is, indeed, a growing profession. There are currently 1 3 6 national, regional or international VOPEs on the IOCE database (see www.IOCE.net ). But we also realize that there is great need to strengthen the capacities and effectiven ess of individual VOPEs; great strength within our collective networks to do so, and many international development agencies interested in supporting evaluation capacity building through VOPEs. Through this EvalPartners Initiative we intend to promote pa rtnerships among these many organizations in order to achieve these three major outcomes: A. Voluntary Organizations of Professional Evaluators (VOPEs) have strengthened institutional capacities; B. Individual members of VOPEs have stronger evaluation capacities in terms of skills to conduct quality evaluations; C. VOPEs are able to play strategic roles within their countries, contributing to country - led evaluation policies and systems, including enhanced demand for and use of evaluations. Means for achieving these objectives will include , but will not be limited to: a mapping survey that will update and deepen the information known about VOPEs around the world; promoting better access to support by regional and international networks/associations (including IOCE and the more developed VOPEs) and supportive institutions; sharing lessons learned of similar experiences in other countries; offering peer to peer mutual support; attending live webinars with international keynote speakers; taking part in e - learning program mes; mentoring programmes; and training organized by local institutions and more developed VOPEs. Within this new partnership, major stakeholders engaged in evaluation capacity development are being invited to contribute to the conceptualization and implem entation of the Initiative. These include regional VOPEs; UNEG and UN agencies; the OECD/DAC Evaluation group and bilateral donors; the World Bank and Regional development banks; foundations; international NGOs and others. Agencies and institutions that h ave already signed up as Core Partners to support the EvalPartners Initiative include the UNEG Task Force on National Evaluation Capacity, UN Women, the Ministry of Foreign Affairs of Finland, AfrEA, ReLAC , EES , CES, EvalMENANet , CoE/South Asia and AE S . Readers of this newsletter who want to learn more, or who have names of potential partner institutions to suggest, are encouraged to write to IOCE@earthlink.net . Or join u s at Facebo ok: www.facebook.com/EvalPartners . 3 Eval Partners at VOPE conferences European Evaluation Society EES 10 th International Conference: Evaluation in the net worked society: New concepts, New challenges, New solution ( 3 - 5 October 2012) Location: Helsinki, Finland The IOCE Board has submitted a proposal for a panel on EvalPartners to the EES Conference in Helsinki. It will introduce and discuss the internation al evaluation partnership initiative (EvalPartners) The panel will discuss the initiative and invite contributions from the floor in four ways: 1. Examine the concept of EvalPartners an the context of evaluation’s role in civil

3 society 2. Ways in which participati
society 2. Ways in which participation in the initiative might take place 3. What activities might be part of EvalPartners 4. Future steps These issues will be debated by members of the IOCE Board (Murray Saunders), representatives from UNICEF (Marco Segone) and from the Finnish Government and by participants from a regional VOPE (Maria Bustelo from the EES) . American Evaluation Association 26th Annual Conference : Evaluation in Complex Ecologies : Relationships, Responsibilities, Relevance ( 24 - 27 October 2012 ) Location: Minne apolis, Minnesota, USA The IOCE Board has also submitted a proposal for a panel on EvalPartners to the AEA Conference in Minne apolis . The p anel will include four presentations:  Introducing the EvalPartners Initiative  South to South Exchanges for Building Evaluation Capacity  Evaluation at Scale: the AEA's Experience in Raising the Profile of Evaluation  The AEA's Potential Role in EvalPartner Paneli sts are Marc o Segone , IOCE Board members Tessie Tzavaras Catsambas and Issaka Herman TRAORE , Martha McGuire ( Cathexis Consulting Inc. , Toronto, Canada) and George Julnes ( Professor, School of Public & International Affairs , University of Baltimore , USA) .                                               NEWS from the IOCE BOARD : 2011 Annual General Meeting IOCE Annual General Meeting (AGM) started with a webinar and continued through an online discus sion. AGM agenda included the following items: 1. Membership definition and matters ; 2. IOCE representation and partnerships ; 3. UNICEF proposal on national associations ; 4. E - j ournal and knowledge production ; 5. Development Evaluation ; 6. Suggestions for upcoming AGM meetings . The AGM has agreed to adopt a new definition of the membership of IOCE and change the by laws . According to this new definition o rganizations can become recognized as Members of the IOCE Network if they: - have accepted an invitation from IOCE to be officially recognized, - are formal societies or associations, or informal networks representing volunta ry memberships of evaluators, i.e. not commercial for - profit agencies or consultancy firms, - have identified one or more contact persons, - have submitted a profile (e.g. response to the IOCE questionnaire), and 4 - have joined the IOCE - Network listserv, or, i n the case of Contributing Members, have joined the IOCE - EvaLeaders listserv. AGM also agreed to add two new representatives to IOCE Board – one from East Asia/Oceania and one from Middle East and North Africa. AGM participants again confirmed the importance of IOCE independence, impartiality and transparency. Participants also reminded the role of IOCE in young networks and associations that do not yet have any formal structure.                                               IOCE Board Meeting in Accra, Ghana On January 14, 2012, IOCE Board

4 conducted a face - to - face meeting in
conducted a face - to - face meeting in Accra, Ghana, - right after the African Evaluation Association conference. Meeting’s agena included the following items: 1. The Evaluation Partners Initiative – IOCE - UNICEF cooperation; 2. IOCE representation at international events; 3. New appointments and roles and responsibilities of IOCE Board Members ; 4. Budget Report ; 5. Workplan ; 6. Listservs ; 7. IOCE Regions ; 8. 2015 as the Year of International Evaluation . The Board continued the discussion of the EvalPartners ini tiative that started at the AGM and developed a w orkplan for its implementation. The Board has decided to develop and branded IOCE presentation for the use at various international events so that even non - board members would be able to represent IOCE in conferences upon request from the Board . The Board has approved IOCE partic ipation at the NONIE meeting in Rome on 19 - 20 April 2 012 and UNEG General Assembly in Rome on 25 - 27 April 2012 . The Board has approved the new distribution of responsibilities among the Board members : Role Name President Soma de Silva Vice President Murray Saunders Treasurer Francois Dumaine Secretary Tessie Catsambas - minutes Issaka Traore – convening of meetings Membership Murray Saunders, Marcia Paterno, Pablo Rodriguez - Bilella International Representation Issaka Traore Website/Branding Tessie Catsambas Newsletter Natalia Kosheleva Membership database Jim Rugh The Board agreed to add the Mid - East & North Africa (MENA) to the IOCE Board. The MENA Evaluators Group nominated Doha Abdelhamid to be their representative to IOCE. The Board has agreed that designating the year 201 5 the Year of International Evaluation could contribute to the development of evaluation capacity worldwide. IOCE can play an overview and coordinat ing role for stimulating important conversations on evaluation across boundaries. 5 Meet a New Member of the IOCE Board: Doha Abdelhamid Doha Abdelhamid has recently joined the IOCE Board as the representative of the Mid - East & North Africa (MENA) . Doha Abdelhamid is an economist and policy evaluation expert with 27 years of experience in academia and consulting. She has numerous local, regional and internationally published books, academic and non - academic articles and policy papers. Her areas of re search interest are in development economics, finance, gender and monitoring and evaluation. She held positions as senior policy advisor to the former minister of finance, minister of planning and economic development, and finally in the Cabinet of ministe rs (with the Minister - President of the Central Agency for Organization and Administration) of Egypt. Doha is also active in civil society work, namely in the International Development Evaluation Association (IDEAS), in which she was re - elected Executive B oard Member for two terms and represented Egypt and the MENA region (2004 - 11); and in the Media Society for Consumer Protection and Development as Executive Board Member. Doha is co - founder of the MENA Regional Network for Development Evaluation, supported by the International Development Research Center (IDRC) of Canada a

5 nd the American University of Beirut (A
nd the American University of Beirut (AUB) of Lebanon, and co - founder of the Egyptian Development Evaluation Society (EgyDeval). She is a member of the African Evaluation Association (A fr EA). In addition to serving at the IOCE Board, Doha is an International Advisor to the African E valuation Association (A fr EA); a member of the International Editorial Board of the African Evaluation Journal; independent best - practices for quality assurance international expert to the International Atomic Energy Agency (IAEA); and panel peer reviewer of the evaluation function of the Food and Agriculture Organization (FAO) by the UN Evaluation Group (UNEG).                                               IOCE at the NONIE Meeting in R ome The IOCE Board has prepared a paper that outlines a broad position on the evaluation of impact adopted by the IOCE to be presented at the NONIE meeting in Rome on 19 - 20 April 2012 . IOCE argues for an open and inclusive approach to impact evaluation. This newsletter presents a shortened version of this paper. The full text of the paper is available at http://ioce. net/download/ioce_docs/IOCEsta tementonimpact.pdf . This new emphasis on assessing the effects of interventions and policies is prompted by three drivers (Saunders, 2011). The first is geared to a better understanding of complex changes brought about by diff erent kinds of intervention, the second aims at better economic and social stewardship and the third has to do with improved evaluation methods. Improving understanding The urge to ‘sense make’ in complex environments is an imperative when evaluations seek to build knowledge of the conditions under which interventions are designed to bring about i mprovements in citizen’s lives. Eleanor Chelimsky ’s formulation based on discussions with practising evaluators speaks of ‘evaluation for knowlege’ (1997). The e merging requirement that development interventions or cohesion policies in Europe, for example, are shown to have positive effects of a broad - based kind, prompts the search for ways of ‘knowing’. The source of this impulse is the need to know more about how 6 complex changes are brought about by a variety of policy levers, instruments and mechanisms. Better economic and social stewardship The second driver is derived from an increasingly acute recognition that judicious and well managed public expenditures geared to social and economic change are needed to tackle obstacles to peace and prosperity. We are now entering an era during which pub lic money will be under close and rigorous scrutiny. There will be less funding for the delivery of global, regional, national and local public goods. In these circumstances, the need for transparency and equity in the distribution, use and effects of p ublic spending has become increasingly important. It is here that a robust and legitimate set of evaluative practices can play a useful role. Some policy domains are slippery, ambiguous and unformed, relying on enabling networks, collaborations and partn erships (see Stern 2009). Increasingly, evaluation is being understood as p

6 art of the process by which ‘policy l
art of the process by which ‘policy learning’ or institutional growth an development is encouraged. Improving method The third driver is methodological. While a recurring theme , the need for more certainty about what counts as a positive change and how we might know its attribution while elusive, is growing in intensity. This explains the renewed interest in experimental designs as a way of producing ‘certainty’ runs counter to the ethical constraints inherent in experimental methods (Stryczynski 2008). Appropriately the Barca report suggests that the complex methodological issues which characterise the use of a counterfactual approach in the context of policy suggest that the effort to promote impact evaluation as one of the methodological backbones of cohesion policy must at the same time be visionary and humble. (Barca 2009 p180) The Future So, in the context of the Paris Declaration and the Millennium Development Goals, it i s appropriate for the international evaluation community to work together in supporting the development of local capacity to undertake the evaluation of effects and impact by acknowledging these contextual factors. The IOCE endorses the following foci for evaluations adapted from statements by OECD/DAC. Evaluations should take account of: Relevance: the extent to which the aid activity and policy is suited to the priorities and policies of the target group, recipient and donor, Effectiveness: the extent to which an aid or policy activity attains its objectives ; Efficiency: the outputs and outcomes in relation to the inputs ; Impact and effects: the positive and negative changes, (practices, systems, cultures and wellbeing) produced by a development intervention, directly or indirectly, intended or unintended ; Sustainability: whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable ; Inclu sivity: the extent to which intended beneficiaries and other stakeholders have participated in aspects of evaluation design, implementation and use. The IOCE encourages diverse yet rigorous methodological solutions to impact evaluation ( IE ) in order to strengthen the quality of evaluations of impact to expand buy - in and support for evaluation and for subsequent action. The IOCE welcomes the increased attention and funding for improving IE. It champions a multi - method approach drawing from the rich diversity of existing frameworks and one that engages both the developed and developing world. We would be pleased to join with others in this endeavour. Your comments and c ontributions are welcome 7 References Bamberger, M., Rugh, J. and Mabry, L. (20 11 ) RealWorld Evaluation. Thousand Oaks, CA: Sage Publications. Barca, F., (2009) An agenda for a reformed cohesion policy (EU: http://ec.europa.eu/regional_policy/policy/future/barca_ en.html ) Chelimsky, E. (1997) Thoughts for a new evaluation society, Evaluation, 3:p 97 - 109. Ramalingham B. (2011) Learning how to learn: eight lessons for impact evaluations that make a difference (O DI, London) Saunders, M. (2011) Capturing effects of interventions, policies and programmes in the European context: a social

7 practice perspective in Evaluation, Vol
practice perspective in Evaluation, Vol. 17, No 1, pp 89 - 103. Stern, E. (2009), Evaluation policy in the European Union and its in stitutions. New Directions for Evaluation , 2009: 67 – 85. doi: 10.1002/ev.306 Stryczynski K. (2008) Rigorous impact evaluation: the magic bullet for evaluation of Cohesion Policy? (Paper delivered at the Lisbon EES International Conference, 2008) White, H. ( 2011) ‘Five challenges facing impact evaluation on NONIE’ (http://nonie2011.org/?q=content/post - 2).                                               IOCE Former President at the Busan Forum Former IOCE President Oumoul Khayri Ba Tall represented IOCE at the Fourth High Level Forum on Aid Effectiveness in Busan, Korea, in No vember 2012 where over 3000 delegates met to review progress on implementing the principles of the Paris Declaration and to discuss how to maintain t he relevance of the aid effectiveness agenda in the context of the evolving development landscape . The ministers of developed and developing nations, emerging economies, providers of South - South and triangular co - operation and civil society signed the Busa n Partnership for Effec tive Development Co - operation . Upon returning from Busan , Oumoul Khayri Ba Tall shared her thoughts about the possible impact of the Forum on the field of evaluation through Al BordedelCaos blog maintained by Pablo Rodriguez - Bilella , a c urrent member of the IOCE Board: “ Post Busan, I foresee Evaluation to be predominant at three levels: 1. The Busan Outcome Document (BOD) or whatever official name will be chosen to replace the Paris Declaration (PD) and the Accra Age nda for Action (AAA) lays out a certain number of basic principles and strategic orientations, but there is no binding commitment yet as I understand. Binding commitment s and operational action plans will have to be decided at the country level, something like a “compact” to be designed based on agreements between development partners involved at the country level. Therefore, Monitoring and Evaluation plans will have to be set at the national level, and a global framework would have to be set in order to as sess the BOD as well. This duality brings in some methodological challenge when it comes to evaluate the BOD as compared to the PD, that Evaluators could reflect on from now. 2. At the country level, if the decision to use country systems is enforced as the BOD calls for, this should include evaluation systems which are almost in existen ce in most recipient countries. Such systems will have to be built (including not only the institutions, but also the capacity of national evaluators) and I see a major role for the Evaluation community. 3. The reference to “mutual accountability” also involves evaluation in many ways. The one that I would like to highlight is the new idea of accountability to the citizens of recipient countries, not only “the Government - to - Government” level, which also calls for a wide agenda of Evaluation Awareness and Evaluation Culture. C itizens need to be able to use Evaluation to h o ld their governments

8 accountable, which means appropriate to
accountable, which means appropriate tools and meth ods, and the right environmen t.” For the full text of the post see http://albordedelcaos.com/2011/12/12/desde - busan - y - mas - alla - escenarios - para - la - evaluacion - from - busan - and - beyond - evaluation - scenarios/ 8 Connecting with evaluators around the globe Jim Rugh, who served as the AEA Representative to the IOCE Board for four years, does a lot of international travel as a consultant for a variety of development agencies. Wherever he goes he likes to meet with evaluation colleagues, en couraging the formation and strengthening of national voluntary organizations of professional evaluators (VOPEs) . Here are two recent examples: During the RealWorld Evaluation workshop let by Jim and Oumou Khayri Ba Tal l in Accra as part of the recent AfrE A conference, he met Paulaw Kitheka. Noting that Paulaw came from Malawi, Jim told him that he would be coming to Lilongwe in two weeks to do a training workshop for USAID staff. While there he would like to take the opportunity to meet with local evalu ators. Paulaw offered to call together some of his colleagues to meet with Jim during his visit. Later on LinkedIn Jim discovered 5 names of persons with M&E in their titles who are based in Malawi, and passed them on to Paulaw. Immediately after arriv ing in Lilongwe Jim went straight to the hotel where Paulaw had r eserved a conference room, and found 12 colleagues gathered there. After eloquent introductions by Paulaw and his World Vision colleague Charles, Jim shared the PowerPoint introduction to IO CE, including brief summaries of the lessons learned through the 14 case studies, ending with the slide promoting their own discussion on i f and how they might want to get organized, and how they might want to relate to others such as AfrEA, IOCE, etc. Th ey decided that they see definite advantages in organizing a network here, will reach out to 13 others who ha respone to Paulaw’s invitation but were unable to come to this meeting due to short notice, and invite these and others to a 2nd meeting in two weeks. Thus Jim played a catalytic role in helping to give birth (or re - birth) to yet another national evaluation network. A few months ago Jim was in Almaty, Kazakhstan. In that case there is already a well - organized Kazakhstan Community of Evaluators, which is an active part of IPEN. KCoE leader Jamila Asanova invited Jim to lead a workshop for some of her colleagues, where he shared a brief summary of the RealWorld Evaluation approach and also shared information about IOCE and ot her evaluation networks around the world. Later Jamila and some of her friends took Jim up to the mountains above beautiful Almaty (see photo). Jim describes this as his passion for “spreaing the gospel of VOPEs” aroun the world! He has enjoyed similar meetings (ofte n including workshops and p resentations on IOCE) with colleagues in Bangladesh, Thailand, Nepal, Sierra Leone, Brazil (BMEN), Sri Lanka (SLEvA), India (CoE), Ukraine (IPEN), Costa Rica (ReLAC), South Africa (SAMEA), AfrEA (Nairobi, Cape Town, Niamey, Cair o and Accra) … an others, incluing of course AEA, CES, EES – some na

9 scent, others already well - de veloped
scent, others already well - de veloped VOPEs. 9 IOCE MEMBERS: t ext by Marcia Paterno Joppert A brief look at the Brazilian Monitoring and Evaluation Net work The Brazilian Monitoring and Evaluation Network (BMEN) was launched in November 2008. BMEN ’s mission is to promote interaction, learning and professionalization of public and private agents through the dissemination of knowledge and information on M& E, aiming to improve the performance of organizations, policies, programs and projects with transparency and social responsibility. The basic principles that govern BMEN are openness, partnership, theoretical and methodologi cal plurality and sectorial diversity. By now BMEN already has more than 2400 affiliates (individual and institutions) from Brazil and abroad . The openness of the Network has attracted the main stakeholders involved in the field M& E field in Brazil . BMEN members represent 27 states of Brazil and more than 600 Brazilian municipalities . One of BMEN achievements is the development of a governance model. The Steering Committee has become bigger, more representative and engaged in the development of the networ k. Since August 2011 BMEN ha s been conducting, in a very professional way, the Strategic Planning, which is now in its detailing phase. BMEN III National Seminar Since 2008 th e BMEN has held three national seminars in different cities, with an average of 300 participants. At these seminars there were also one - day workshops and mini - courses. In addition, a lot of news, information on training and work opportunities, events an d knowledge have been published on the BMEN web site. This year the IV National Seminar will be held on August 13, 14 and 15 in Rio de Janeiro . I ts key theme is “National Capacities in M&E”. BMEN representatives participate in many events striving to ac quire state - of - the - art knowledge in the sector. For more information see BMEN website at www. redebrasileirademea.ning.com .                                               A message from the Australasian Evaluation Association 2012 International Conference The Australasian Evaluation Society (AES) will hold its 2012 International Conference in Adelaide, South Australia. Starting with workshops on Monday 27 & Tuesday 28 August, the Conference itself will be held 29 – 31 August 2012. This year’s theme is ‘Evaluation in a Changing Worl’. It invites all those with an interest in evaluation to think about the ways our world is changing and the implications for our work roles, issues and competencies. For example, what opportunities and challenges are posed for evaluation by technological change; the changing balance of world power; climate change; and wicked social problems? It also provides an opportunity to highlight and examine new developments in evaluation methodology and technologies. Finally, it invites the AES and the conference itself to trial some innovative sessions, approaches, and technologies. For more information see c onferenc e website: www.aes2012.com.au