MONITORING and EVALUATION M&E establishes clear
Description: MONITORING and EVALUATION ME establishes clear links for past, present and future initiatives and development results Helps to extract relevant information from past and on-going activities Provides evidence for fine-tuning or
Related Topics
Download Presentation
"MONITORING and EVALUATION M&E establishes clear" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. MONITORING and EVALUATION<br>
slide2. M&E establishes clear links for past, present and future initiatives and development results
Helps to extract relevant information from past and on-going activities
Provides evidence for fine-tuning or re-orientation of a project/programme
Determines if work is going in the right direction 04/11/2013 PURPOSE OF M&E<br>
slide3. Monitoring is the on-going process by which stakeholders obtain regular feedback on the progress being made towards achieving their goals and objectives / targets and outcomes 04/11/2013 What is Monitoring?<br>
slide4. Clarifies program objectives
Links activities and their resources to objectives
Translates objectives into performance indicators and set targets
Routinely collects data for these indicators, compares actual results with targets
Reports progress to managers and alerts them to problems 04/11/2013 Monitoring<br>
slide5. Evaluation is the use of social science methods to collect, analyze, interpret and communicate data about the effectiveness of a policy, program or project
Evaluation is a rigorous and independent
assessment of either completed or ongoing activities to determine the extent to which they are achieving stated objectives and contributing to Decision-making. 04/11/2013 What is EVALUATION?<br>
slide6. Monitoring is focused on tracking evidence of movement towards the achievement of specific, predetermined targets
Evaluation takes a broader view of an intervention, considering not only progress toward stated goals, but the logic of the initiative, as well as its consequences
Both are needed to be able to better manage policies, programs, and projects 04/11/2013 Differences between Monitoring and Evaluation<br>
slide7. Define the project work.
Develop success indicators and their measures
Collect the evaluation data.
Analyze and interpret the data.
Use the evaluation results 04/11/2013 5 key processes<br>
slide8. The purpose of the review is to enable the health sector to undertake a stock taking exercise so that it can assess its performance in key technical programmes in relation to set objectives and targets over the years within the 5YPOW.
Lessons learnt should enable it to take corrective actions to improve performance in the following year. 04/11/2013 NATIONAL REVIEW PROCESS<br>
slide9. 1.To assess the progress made on the implementation of recommendations of the previous reviews
2.To review the technical performance of priority programmes in the past year
3. To identify challenges and enabling factors for implementation
4. To identify best practices (innovations that influence attainment of results).
5. To agree on modalities for scaling up. 04/11/2013 Specific objectives of monitoring<br>
slide10. FINANCING THE HEALTH SECTOR
SERVICE DELIVERY, PRIORITY HEALTH INTERVENTIONS AND INSTITUTIONAL CARE
ORGANIZATION AND MANAGEMENT OF HEALTH SERVICES
planning and budgeting; financial management; procurement
INVESTMENTS
Capital Investment Human resource dev., health information 04/11/2013 Key Areas for the Main Review<br>
slide11. An indicator is a specific measure, that when tracked systematically over time indicates progress (or not) toward a specific target
• Indicators asks the question:
– How will we know success when we see it?
Indicator Development should be
“CREAM”
C - Clear
R - Relevant
E - Economic
A - Adequate
M - Monitorable 04/11/2013 INDICATORS<br>
slide12. Makes explicit the intended objectives of government action
– (“Know where you are going before you get moving”)
• Outcomes are what produce benefits
• Clearly setting outcomes is key to designing and building results-based M&E system
• Important! Budget to outputs, management to outcomes! 04/11/2013 OUTCOME indicators<br>
slide13. Routine medical and other records - part of the broader health information management system;
Mortality records and reports from health facilities.
Vital systems registration and community registration
Data from organizations providing community‐based services
Procurement systems (forecasting quantification, storage and distribution systems)
Financial disbursements 04/11/2013 DATA SOURCES<br>
slide14. Mapping available services in health facilities and other health settings;
Health facility surveys;
Population‐based surveys;
Surveillance data (e.g. behavioural and biological surveys) 04/11/2013 Other sources of data include :<br>
slide15. Data collection,
storage and management including data recording tools,
data reporting forms,
databases,
Electronic systems for data‐sharing and analysis. 04/11/2013 RESOURCES<br>
slide16. Possible sources of Baseline data
Written records (paper & electronic)
Individuals involved with the intervention
The general public
Trained observers
Mechanical measurements and tests
Geographic information systems
Sources are who or what provide data NOT the method of collecting data 04/11/2013 Baseline data:<br>
slide17. Performance reviews –
This is the first step of the review process
Specifically, each department conducts a self-review by collecting, collating, analyzing and reporting on its performance using data generated routinely within the health delivery system. 04/11/2013 National Review Process<br>
slide18. Assessment of capacity within the health sector for planning, monitoring and evaluation, and support to NGOs (including private sector) in light of existing and future needs
Review of current arrangements for leading and guiding the financing approach and for external aid coordination, and recommendations for enhancing communications and working arrangements
Assessment of status of health sector/agencies working relationship 04/11/2013 Partnership and management arrangements between Health sector and agencies<br>
slide19. Decide how to obtain the data you need from each source
2. Prepare data collection instruments
3. Develop procedures for use of the data collection instruments 04/11/2013 Data Collection<br>
slide20. Desk top study to review all relevant policy documents underpinning POW
Field visits to Regions, Districts, and Tertiary institutions
Reconciliation of releases and payments from the Accounts Division
Briefing meetings with relevant stakeholders 04/11/2013 Organization & Methodology<br>
slide21. Gives information on the status of projects, programs, and policies
Provides important information over time on trends and directions
Helps confirm or challenge theory of change
Provides clues to problems 04/10/2013 Data Analysis<br>
slide22. Data quality issues should include
• Timeliness
• Periodicity
• Consistency
• Confidentiality, data security and data accessibility – 04/11/2013 Data Quality<br>
slide23. Purpose of the data auditing process for the Monitoring of the National Response activities is:
To ensure increased credibility and validity of primary data provided by implementing agencies.
To ensure that accurate data reporting is done 04/11/2013 DATA QUALITY AUDIT<br>
slide24. Data quality assessments (Local quality controls and checks) should be carried out periodically and regularly to identify weaknesses in data collection and reporting systems, and to constantly improve data quality and accuracy. 04/11/2013 DATA VERIFICATION<br>
slide25. Outcome and impact indicators can be derived from :
Population based or household surveys DHS, MICS, and AIS
Behavioural survey KAP
Facility Based surveys
Community based surveys
Client satisfaction surveys
Cost effective and cost benefit studies
An analytical approach known as ‘triangulation’ integrates multiple data sources to improve the understanding of a public‐health problem. It is used to guide programmatic decision‐making to address such problems. 04/11/2013 Evaluation processes<br>
slide26. The Health responses can be strengthened through different types of research
Clinical/Epidemiologic;
Socio-Cultural/Behavioural;
Health systems. 04/11/2013 RESEARCH<br>
slide27. EXAMPLES:
Factors that influence adherence to medical regimens,
Factors that influence uptake of diagnosis, testing and decisions. 04/11/2013 Operations Research<br>
slide28. INFORMATION DISSEMINATION
A dissemination process should be structured and encouraged for information to be shared at all levels with project managers, data collectors and beneficiaries (PLWHA, community leaders) and the general public
Dissemination platforms may include the documentation centres established at national and provincial levels. 04/11/2013<br>
slide29. Communication spaces available to disseminate the data to the general public include the radio, television and print media and SMS messages.
Regular production of newsletters and the development of an effective web based information system with chat rooms , blogs. etc 04/11/2013<br>
slide2. M&E establishes clear links for past, present and future initiatives and development results
Helps to extract relevant information from past and on-going activities
Provides evidence for fine-tuning or re-orientation of a project/programme
Determines if work is going in the right direction 04/11/2013 PURPOSE OF M&E<br>
slide3. Monitoring is the on-going process by which stakeholders obtain regular feedback on the progress being made towards achieving their goals and objectives / targets and outcomes 04/11/2013 What is Monitoring?<br>
slide4. Clarifies program objectives
Links activities and their resources to objectives
Translates objectives into performance indicators and set targets
Routinely collects data for these indicators, compares actual results with targets
Reports progress to managers and alerts them to problems 04/11/2013 Monitoring<br>
slide5. Evaluation is the use of social science methods to collect, analyze, interpret and communicate data about the effectiveness of a policy, program or project
Evaluation is a rigorous and independent
assessment of either completed or ongoing activities to determine the extent to which they are achieving stated objectives and contributing to Decision-making. 04/11/2013 What is EVALUATION?<br>
slide6. Monitoring is focused on tracking evidence of movement towards the achievement of specific, predetermined targets
Evaluation takes a broader view of an intervention, considering not only progress toward stated goals, but the logic of the initiative, as well as its consequences
Both are needed to be able to better manage policies, programs, and projects 04/11/2013 Differences between Monitoring and Evaluation<br>
slide7. Define the project work.
Develop success indicators and their measures
Collect the evaluation data.
Analyze and interpret the data.
Use the evaluation results 04/11/2013 5 key processes<br>
slide8. The purpose of the review is to enable the health sector to undertake a stock taking exercise so that it can assess its performance in key technical programmes in relation to set objectives and targets over the years within the 5YPOW.
Lessons learnt should enable it to take corrective actions to improve performance in the following year. 04/11/2013 NATIONAL REVIEW PROCESS<br>
slide9. 1.To assess the progress made on the implementation of recommendations of the previous reviews
2.To review the technical performance of priority programmes in the past year
3. To identify challenges and enabling factors for implementation
4. To identify best practices (innovations that influence attainment of results).
5. To agree on modalities for scaling up. 04/11/2013 Specific objectives of monitoring<br>
slide10. FINANCING THE HEALTH SECTOR
SERVICE DELIVERY, PRIORITY HEALTH INTERVENTIONS AND INSTITUTIONAL CARE
ORGANIZATION AND MANAGEMENT OF HEALTH SERVICES
planning and budgeting; financial management; procurement
INVESTMENTS
Capital Investment Human resource dev., health information 04/11/2013 Key Areas for the Main Review<br>
slide11. An indicator is a specific measure, that when tracked systematically over time indicates progress (or not) toward a specific target
• Indicators asks the question:
– How will we know success when we see it?
Indicator Development should be
“CREAM”
C - Clear
R - Relevant
E - Economic
A - Adequate
M - Monitorable 04/11/2013 INDICATORS<br>
slide12. Makes explicit the intended objectives of government action
– (“Know where you are going before you get moving”)
• Outcomes are what produce benefits
• Clearly setting outcomes is key to designing and building results-based M&E system
• Important! Budget to outputs, management to outcomes! 04/11/2013 OUTCOME indicators<br>
slide13. Routine medical and other records - part of the broader health information management system;
Mortality records and reports from health facilities.
Vital systems registration and community registration
Data from organizations providing community‐based services
Procurement systems (forecasting quantification, storage and distribution systems)
Financial disbursements 04/11/2013 DATA SOURCES<br>
slide14. Mapping available services in health facilities and other health settings;
Health facility surveys;
Population‐based surveys;
Surveillance data (e.g. behavioural and biological surveys) 04/11/2013 Other sources of data include :<br>
slide15. Data collection,
storage and management including data recording tools,
data reporting forms,
databases,
Electronic systems for data‐sharing and analysis. 04/11/2013 RESOURCES<br>
slide16. Possible sources of Baseline data
Written records (paper & electronic)
Individuals involved with the intervention
The general public
Trained observers
Mechanical measurements and tests
Geographic information systems
Sources are who or what provide data NOT the method of collecting data 04/11/2013 Baseline data:<br>
slide17. Performance reviews –
This is the first step of the review process
Specifically, each department conducts a self-review by collecting, collating, analyzing and reporting on its performance using data generated routinely within the health delivery system. 04/11/2013 National Review Process<br>
slide18. Assessment of capacity within the health sector for planning, monitoring and evaluation, and support to NGOs (including private sector) in light of existing and future needs
Review of current arrangements for leading and guiding the financing approach and for external aid coordination, and recommendations for enhancing communications and working arrangements
Assessment of status of health sector/agencies working relationship 04/11/2013 Partnership and management arrangements between Health sector and agencies<br>
slide19. Decide how to obtain the data you need from each source
2. Prepare data collection instruments
3. Develop procedures for use of the data collection instruments 04/11/2013 Data Collection<br>
slide20. Desk top study to review all relevant policy documents underpinning POW
Field visits to Regions, Districts, and Tertiary institutions
Reconciliation of releases and payments from the Accounts Division
Briefing meetings with relevant stakeholders 04/11/2013 Organization & Methodology<br>
slide21. Gives information on the status of projects, programs, and policies
Provides important information over time on trends and directions
Helps confirm or challenge theory of change
Provides clues to problems 04/10/2013 Data Analysis<br>
slide22. Data quality issues should include
• Timeliness
• Periodicity
• Consistency
• Confidentiality, data security and data accessibility – 04/11/2013 Data Quality<br>
slide23. Purpose of the data auditing process for the Monitoring of the National Response activities is:
To ensure increased credibility and validity of primary data provided by implementing agencies.
To ensure that accurate data reporting is done 04/11/2013 DATA QUALITY AUDIT<br>
slide24. Data quality assessments (Local quality controls and checks) should be carried out periodically and regularly to identify weaknesses in data collection and reporting systems, and to constantly improve data quality and accuracy. 04/11/2013 DATA VERIFICATION<br>
slide25. Outcome and impact indicators can be derived from :
Population based or household surveys DHS, MICS, and AIS
Behavioural survey KAP
Facility Based surveys
Community based surveys
Client satisfaction surveys
Cost effective and cost benefit studies
An analytical approach known as ‘triangulation’ integrates multiple data sources to improve the understanding of a public‐health problem. It is used to guide programmatic decision‐making to address such problems. 04/11/2013 Evaluation processes<br>
slide26. The Health responses can be strengthened through different types of research
Clinical/Epidemiologic;
Socio-Cultural/Behavioural;
Health systems. 04/11/2013 RESEARCH<br>
slide27. EXAMPLES:
Factors that influence adherence to medical regimens,
Factors that influence uptake of diagnosis, testing and decisions. 04/11/2013 Operations Research<br>
slide28. INFORMATION DISSEMINATION
A dissemination process should be structured and encouraged for information to be shared at all levels with project managers, data collectors and beneficiaries (PLWHA, community leaders) and the general public
Dissemination platforms may include the documentation centres established at national and provincial levels. 04/11/2013<br>
slide29. Communication spaces available to disseminate the data to the general public include the radio, television and print media and SMS messages.
Regular production of newsletters and the development of an effective web based information system with chat rooms , blogs. etc 04/11/2013<br>