Monitoring and Evaluation

Definitions

Monitoring and Evaluation: Monitoring refers to an ongoing process of assessing whether the process of planning and implementation is proceeding on target; evaluation, usually done at the end of a specified phase or at the end of the project/process, is an assessment of whether the various goals and objectives have been met. (home.graffiti.net/rustom/kv/nbsap_/key_terms.doc)

Evaluation is systematic determination of merit, worth, and significance of something or someone using criteria against a set of standards. Evaluation often is used to characterize and appraise subjects of interest in a wide range of human enterprises. Wikipedia.

A Brief Review of 20 Tools to Assess Capacity by Capacity Development. 08/2005

A Brief Review of 20 Tools to Assess Capacity by Capacity Development Group Bureau for Development Policy at UNDP.

This Resource Catalogue on Capacity Assessment Tools is a brief review of twenty tools is meant to serve as a preliminary resource for development practitioners in the area of organizational capacity assessment. It is exhaustive in neither breadth nor depth of tools studied.

Twenty tools were surveyed, drawing from publicly available resources of non-profit organizations, management consultancies, United Nations agencies and other donor organizations. The purpose or application of these tools varies from thematic, institutional or enabling environment levels. Inclusion of a tool in this review does not imply endorsement by the UNDP. Capacity Development Group Bureau for Development Policy.(12p;110.18 KB) Download

A Framework for Monitoring and Evaluating HIV Prevention Programmes for Most-At-Risk Populations. 2008

(UNAIDS), is intended for program managers and others involved in planning and implementing M&E programs for MARP at both national and sub-national levels. It is also aimed at those who use information to plan and improve policies and programs. The MARP guide presents an organizing framework that can be used to identify the information necessary for planning, monitoring, and evaluating HIV prevention programs for most-at-risk populations. The framework is also a useful tool for organizing a collective, coordinated and unified response to information gathering by national or sub-national programs and all their partners and donors. Rather than focusing solely on indicators, as many previous guides have done, the MARP guide promotes a strategic planning approach and the use of M&E data for decision making at all levels.

Original 2007 Download (PDF, 338KB)

1st reprint, December 2008. Download (PDF, 96p; 1.49 MB)

A Tool for Sharing Internal Best Practices. 2005

This tool, developed by the INFO Project, includes a step-by-stop process, tips, case studies and links to additional resources that explain how an organization can more effectively share its own best practices internally. This tool reviews what a best practice is, the benefits of sharing best practices, and some obstacles to sharing. It then outlines a process for identifying your organization's best practices, validating and documenting them, and preparing a plan to share them throughout your organization.

Download (PDF 212,21 KB, 23p.)

Building Monitoring, Evaluation and Reporting Systems for HIV/AIDS Programs. 04/2005

This publication is an easy-to-use manual designed to assist NGOs/FBOs/CBOs in developing a monitoring, evaluation and reporting (MER) framework for HIV programming with “how-to” sections on adopting this framework to the President’s Emergency Plan for AIDS Relief M&E requirements. The document includes chapters on the basic principles of MER, results-based MER, and indicators as well as providing worksheets that serve as a useful guide. The manual additionally provides practical data protocol sheets that guide NGOs/FBOs/CBOs on adapting their MER systems to the President’s Emergency Plan indicators. This manual was developed and field-tested through partial funding from Pact’s Global Community REACH and Pact South Africa programs which provides support to over 80 local organization partners implementing HIV/AIDS programs.

Download (PDF 587.75KB, 136p.)

Conducting In-Depth Interviews: A Guide for Designing and Conducting In-Depth Interviews for Evaluation Input. Monitoring & Evaluation 2.05/2006.

Pathfinders International. In-depth interviewing is a qualitative research technique that involves conducting intensive individual interviews with a small number of respondents to explore their perspectives on a particular idea, program, or situation. They are useful when you want detailed information about a person’s thoughts and behaviors or want to explore new issues in depth. Interviews are often used to provide context to other data (such as outcome data), offering a more complete picture of what happened in the program and why.

Download PDF (151.98KB, 16p.)

DFID Guidance On Using The Revised Logical Framework. 02/2009

How To Note. A DFID Practice Paper. February 2009

The UK Department for International Development (DFID) works with a wide range of partners from long-term arrangements with partner governments and multilateral organisations to short-term humanitarian aid projects funded through NGOs. DFID's interest is in ensuring that each is devised and delivered in the most efficient and effective way and links to identified objectives set out in a Divisional Performance Framework or Country / Regional Plan.

This guidance has been written for DFID project workers and DFID partners, and focuses on helping to make the best use of the logical framework (logframe) in designing and managing projects. It was partly informed by reports that highlighted weaknesses in existing logframes. The new designed format aims to address those weaknesses by encouraging the identification of objectives at the right level, more robust specification of indicators, increased coverage of baseline and target information and better quantification of results.

The guidance applies to any one involved in the design approval or active use of the logframe and all DFID projects of a value of one million pounds and above. Additional guidance in annexes has been provided to help the reader form a broader picture of what is involved in putting together a logframe.

Download PDF (37p; 740.45 KB)

Embracing Innovative Practice. Monitoring And Evaluating Capacity And Capacity Development. 09/2006

Many years of experience in the field had led David Watson to question the value of monitoring and evaluation. Recently, a range of innovative to M&E approaches has given him new hope. He explains why in this article.

Download PDF (38.33KB), 8p.)

FHI Inventory of Program Evaluation Tools and Guidelines

Family Health International has compiled an inventory of guidelines, frameworks, and manuals that provide users with instructions and step-by-step guidance to conduct program evaluations and inform program design and improvement.

The material is grouped by category of user:

* all users,

* program managers,

* evaluation specialists in community-based organizations,

* international and nongovernmental agencies,

* researchers, and

* governments.

This inventory is specifically relevant because it specifically addresses how to facilitate the use of data that is generated by evaluation research.

The Inventory of Program Evaluation Tools and Guidelines can be found here.

INTRODUCTION

Program evaluation is a complex necessity in all health programming. All program funders now require that programs have strong evaluations to guide their development, implementation, and ongoing fine tuning or improvement; and that these evaluations go beyond the simple reporting of activities and results to the funding agency. Implementing agencies are being asked to collect information to guide their program design and to use ongoing evaluation data to shape and mold the program as it is being implemented.

Implementing agencies are also being asked to gather information and data that can tell the story of their communities and programs and the context of observed changes. Implementers are being asked to answer a series of key questions: Who is this community? What does this community need? How is this program addressing these needs in a culturally competent manner? How is the community being affected or changed as a result of the implementation of this program or set of programs?

However, not enough people in the field have had the educational training or in-the-field experience to know what information to collect and how to do it in a way that yields accurate and useful results. In order for agencies to state that their programs are being successfully carried out, it is no longer sufficient to merely ask a handful of clients or program participants what they liked and did not like. More structure and rigor is being demanded of all implementing agencies.

Handbook on Monitoring and Evaluation for the CINDI Network. 2006

The design of the M&E framework is an initiative that follows on from an initial 2 year project funded by the Rockefeller Brothers Fund (RBF) which assisted Members with organisational development, strategic planning and monitoring and evaluation. The different sections draw on information that was distributed at learning sessions to promote understanding of M&E concepts and processes. Some of the issues raised by participants are also included as there was a genuine effort to make the system relevant to the CINDI Network and the environment in which it works.

Download PDF (370KB)

Handbook on Planning, Monitoring and Evaluating for Development Results. 2009

United Nations Development Programme (2009)
Authors: Kasturiaracchi,A. ; Eriksson,T. ; Rodriques,S. ; Kubota,A.

Planning, monitoring and evaluation in development requires a focus on nationally owned development priorities and results and should reflect the guiding principles of national ownership, capacity development human development. This handbook is aims to enhance the results-based culture within UNDP and improve the quality of planning, monitoring and evaluation. While written with UNDP staff, stakeholders and partners in mind, the handbook provides a useful overview of why and how to evaluate for results which can be used in other contexts.
This handbook concentrates on planning, monitoring and evaluating of results in development and is designed to be used as a reference throughout the programme cycle. The handbook covers the following areas:

* the integrated nature of planning, monitoring and evaluation, and describes the critical role they play in managing for development results

* the conceptual foundations of planning and specific guidance on planning techniques and the preparation of results frameworks that guide monitoring and evaluation

* how to plan for monitoring and evaluation before implementing a plan and issues related to monitoring, reporting and review

* an overview of the UNDP evaluation function and the policy framework, including key elements of evaluation design and tools and describe practical steps in managing the evaluation process

The handbook also presents practical steps and examples in using knowledge from monitoring and evaluation in managing for development results. Annexes include an evaluation terms of reference template, a list of international evaluation networks, guidance on how to select evaluators, a management response template, an evaluation report template and some useful further resources for evaluation.

Download PDF (232p; 2.19MB)

Monitoring and Evaluation Network of Training Online Resources

Through MENTOR (Monitoring and Evaluation Network of Training Online Resources), MEASURE Evaluation makes available free training materials and tools on M&E topics for use by researchers, program managers, trainers, policy makers, students, and other public health professionals.

These materials were developed by global experts in order to provide state-of-the-art information on monitoring and evaluation (M&E) topics.

Materials include a Interactive mini-course on M&E Fundamentals, Downloadable Training Materials and Population Research Materials.

Monitoring and Evaluation Systems Strengthening Tool. 01/2007

MEASURE Evaluation, January 2007.

By Ronald Tran Ba Huy, Karen Hardee, J. Win Brown et al.
This publication provides an monitoring and evaluation (M&E) systems strengthening tool which can help all reporting entities under government programmes and donor projects to assess the strengths and weaknesses of their M&E systems, including data collection and reporting, and highlighting areas for improvement that might require additional focus, funds and/or technical assistance.

Download PDF (50 p. 2.2 MB)

Participatory Monitoring and Evaluation of Community - and Faith-Based Programs : Second Edition CORE Initiative. 2006

Designed for use by local implementing agencies, this book demonstrates how using monitoring and evaluation can improve the impact of community interventions on HIV/AIDS. It is a step-by-step guide to making community-level HIV and AIDS services.

Download (3.67MB, 94p.)

Preparing a Case Study: A Guide for Designing and Conducting a Case Study for Evaluation Input. Monitoring & Evaluation 1. 05/2006

Pathfinders International Tool Series.

A case study is a story about something unique, special, or interesting—stories can be about individuals, organizations, processes, programs, neighborhoods, institutions, and even events. The case study gives the story behind the result by capturing what happened to bring it about, and can be a good opportunity to highlight a project’s success, or to bring attention to a particular challenge or difficulty in a project.

Download PDF (154.96KB, 16p.)

Program Evaluation.

Part of the National Minority AIDS Council “Organizational Effectiveness Manuals Series”. This manual demonstrates the importance of program evaluation with regard to the development of HIV/AIDS interventions.

A current challenge facing HIV/AIDS service providers is to document, validate, coordinate and integrate their programmatic activities. All of these components can be addressed with the implementation of a comprehensive evaluation program. Evaluation is the system used to assess the worth or merit of a program. This general definition, however, does not take
into account descriptive studies, implementation analyses, and formative evaluations. A better-suited definition would incorporate the information processing and feedback role of evaluation. An alternative definition of evaluation is the systematic acquisition and assessment of information to provide useful feedback about a program.

Download PDF (5.43MB, 69 p.)

Result Based Project Monitoring and Evaluation: Using the Logical Framework. 2007

Local Livelihoods Ltd.,

Edited by Sally Hunt.

This toolkit provides an outline for the monitoring and evaluation of development projects and programmes. It uses a Result Based Management, Monitoring and Evaluation system focussing on the higher level objectives/outputs and not the lower level activities. This approach uses the Logical Framework as the basis for the project design, the monitoring indicators and the assumption and risk analysis.

Download PDF (31p.; 397 kB)

Template for HIV Prevention Evaluation. 2009

Template for HIV Prevention Evaluation Terms of Reference. Quick Reference Guide

The UNAIDS technical working group on prevention evaluation developed a guidebook to facilitate the planning of evaluations for HIV prevention. NGO participants at a review and testing workshop (carried out in Nigeria, September 2008) expressed the need for there to be a shorter, more simplified version of the terms of reference (ToR) template that could be referred to and used by NGO staff commissioning evaluations, but not necessarily evaluation specialists.

Using the template as a basis, the International HIV/AIDS Alliance in collaboration with the World Bank has compiled this six page quick reference guide to be used as a stand alone document to write a ToR for a prevention evaluation.

The guide is primarily intended for use by HIV and AIDs service providers from civil society organisations such as programme managers/officers and monitoring and evaluation advisers.

Download PDF; (548.0KB; 6p)

Using Mystery Clients: A Guide to Using Mystery Clients for Evaluation Input . Monitoring & Evaluation 3. 05/2006

Pathfinders International.

Mystery clients are trained people (usually community members) who visit program facilities in the assumed role of clients, and then report (by completing a survey or through an interview) on their experience. They might be used in an effort to avoid the bias in the service delivery process that often results from having service transactions observed. Mystery clients can also serve to gather a sufficient number of observations of service transactions when the actual volume of service visits is low.

Download PDF (159.08KB, 20p.)

W. K. Kellogg Foundation Evaluation Toolkit.

This toolkit is designed to provide our grantees with guidance as they undertake evaluating and learning from their work. It is targeted primarily at those grantees who will be working with an external evaluator or conducting their own rigorous internal evaluations, but we believe anyone who is seeking to design a useful evaluation can benefit from it.

The tools and approaches in this toolkit are based on our mission and evaluation philosophy and built upon our Evaluation Handbook. The toolkit has seven sections (the links to these sections are located in the left hand column and are available on every page of the toolkit):

Where to start: Describes some of the first issues to address in beginning an evaluation.
Evaluation approaches: Gives some detail on different approaches to evaluation and how they are related to hiring an evaluator and designing the evaluation.
Evaluation questions: Gives suggestions on developing the questions that will guide the evaluation work.
Evaluation plan: Provides details on the major components of an evaluation plan, including data collection and analysis, reporting, and assuring use of findings.
Budgeting: Shows how to create an initial evaluation budget.
Hiring and managing evaluators: Suggests things to consider in selecting and managing the best evaluator for your project.
Additional resources: Links to useful on-line resources on evaluation.