Definitions of monitoring and evaluation

Introduction

Definitions of monitoring and evaluation

Monitoring is the continuous collection of data on specified indicators to assess for a development intervention (project, programme or policy) its implementation in relation to activity schedules and expenditure of allocated funds, and its progress and achievements in relation to its objectives. (Kusek JZ, Rist RC 2004)

Evaluation is the periodic assessment of the design, implementation, outcomes and impact of a development intervention. It should assess the relevance and achievement of objectives, implementation performance in terms of effectiveness and efficiency, and the nature, distribution and sustainability of impacts. (UNDP 2002)

Monitoring and Evaluation is a process of continual gathering of information and assessment of it in order to determine whether progress is being made towards pre-specified goals and objectives, and to highlight whether there are any unintended (positive or negative) effects from a project and its activities. It is an integral part of the project cycle and of good management practice. Monitoring is carried out in order to track progress and performance as a basis for decision-making at various steps in the process of an initiative or project. Evaluation, on the other hand is a more generalized assessment of data or experience to establish to what extent the initiative has achieved its goals or objectives. (Kusek JZ, Rist RC 2004)

The following are components involved in planning for result based monitoring and evaluation of any project

Assess the existing readiness and capacity for monitoring and evaluation. Review current capacity within the organization and its partners which will be responsible for project implementation, covering: technical skills, managerial skills, existence and quality of data systems, available technology and existing budgetary provision. Identify any barriers to Monitoring and evaluation of the project such as a lack of political will, expertise or experience. What other organizations such as universities, private consultants or government agencies have the capacity to provide technical assistance and/or training? (Rogers P 2009)

Establish the purpose and scope. Why is Monitoring and evaluation needed and how comprehensive should the system be?  What are national requirements with regard to Monitoring and evaluation In particular, what should be the scope and degree of rigor of the evaluation of final project impact? Should the Monitoring and evaluation process be participatory? In planning and implementing project Monitoring and evaluation it is important to recognize the potential benefits of stakeholder participation. There can be benefits from this at all stages of the project cycle including monitoring and evaluation. . (Kusek JZ, Rist RC 2004)

Identify and agree with main stakeholders. The project’s outcomes and development objectives Setting a development goal and the project purpose or expected outcomes is essential in building a Monitoring and evaluation system. In project design the specification of outputs, activities and inputs follows from this, and the expectation that achievement of outcomes will contribute to the higher level development goals provides the justification for the project. In Monitoring and evaluation design, indicators, baselines and targets (Rogers P 2009)

Select key indicators and an evaluation framework. Indicators are the qualitative or quantitative variables that measure project performance and achievements. Indicators should be developed for all levels of project logic. Indicators are needed to monitor progress with respect to inputs, activities, outputs, outcomes and impact, to feedback on areas of success and where improvement is required. Each indicator initially selected for inclusion in the Monitoring and evaluation programme needs to be carefully scrutinized and tested before acceptance. Criteria against which indicators can be tested to ensure that they are suitable for inclusion .The evaluation framework sets out the methods to be used to address the question of whether change observed through monitoring indicators can be attributed to the project interventions. The depth and rigor of impact evaluation required for a specific project given available resources needs to be carefully considered. A range of approaches are possible, but all require careful planning in conjunction with the selection of indicators if data omissions and weaknesses are to be avoided, and valid and reliable results produced. Assuming use of an experimental or quasi-experimental evaluation design, determination of which population units will receive the intervention and which will not, and establishing baseline information for all units are two reasons for detailed planning of impact evaluation in advance. (Kusek JZ, Rist RC 2004)

Set baselines and plan data collection and analysis. The baseline is the first measurement of an indicator, which sets the pre-project condition against which change can be tracked and evaluated. A single point in time or current value may not be representative and it may be better to use an average, for example, for the three previous years if such data are available. Baseline data must be gathered for the key indicators and this may require implementation of a baseline survey unless existing data sources are adequate. Subsequent data gathering and repeat surveys for the implementation period of the project and beyond should then be planned. Data collection may be continuous or periodic depending on the nature and purpose of an indicator. A wide range of data collection methods are applicable. The analytical approaches that will be required to match the needs of managers for information and of the evaluation framework must also be considered. In projects concerned with land use changes, use of modern technologies such as remote sensing should be considered. Ideally there should be sufficient capacity and resources to allow ad hoc special studies or investigations to be carried out to address specific problems or issues revealed by the on-going evaluate capacities storing data. These will be one-off, focused investigations of the issue at hand.  (Rogers P 2009)

Select results targets.  Indicators and baselines, target setting is a key step in building a results-based approach. A target is a specification of the quantity, quality, timing and location to be realized for a key indicator by a given date. Starting from the baseline level for an indicator the desired improvement is defined taking account of planned resource provision and activities, to arrive at a performance target for that indicator. Most targets are set annually, but some could be set quarterly or for longer periods. Targets do not have to be single numerical values and sometimes a range of achievement may be more appropriate. Targets should also be kept under review and revised flexibly as necessary to take account of changing resource availability or other factors beyond the control of project management, but not to disguise poor project performance. It is important to be realistic, taking account of what is feasible and being sensitive to the political issues associated with targets that are publicly announced. As outcomes are typically longer term it is usually necessary to establish targets as short-term objectives on the path to achievement of an outcome. For project management, targets for ‘leading indicators’ are particularly useful. Interim targets over shorter time periods for which inputs can be better known or estimated, and set with reference to desired outcomes and impact, are also important for process-orientated interventions for which work plans and resource provision are not fully planned in detail in advance. (Kusek JZ, Rist RC 2004)

Plan monitoring, data analysis, communication, and reporting Implementation monitoring’ tracking the inputs, activities and outputs in annual or multiyear work plans, and ‘results monitoring’ tracking achievement of outcomes and impact, are both needed. The demands for information at each level of management need to be established, responsibilities allocated, and plans made for, what data to be collected and when, how data are collected and analyzed, who collects and analyses data, who reports information, and in what form, to whom and when An assessment of the flow of information and degree of detail needed by each level of management will help to clarify the indicators to be measured. The agency managing the project will require different types of information for its own internal management, compared to the reporting requirements of higher levels of government and development agencies. (Turrall S, Pasteur K 2006)

Plan the form and timing of critical reflection and interim evaluations. For managers evaluation should be a continuously available mode of analysis utilized whenever evaluation results can be useful. Scheduling of events such as management team meetings can, however, be useful to ensure that analysis of progress and critical reflection takes place. Similarly, periodic project review workshops to facilitate analysis and discussion with project partners and other stakeholders may be necessary. Supervision requirements of governments and funding agencies may require periodic and formalized evaluations to take place. The data needs and analysis requirements for mid-term, terminal and ex post evaluations should be considered, and planning for these linked to the planning of monitoring and choice of evaluation framework. A timetable of formal evaluation reports should be set out. An indication also needs to be given at the design stage about feedback mechanisms for evaluation results beyond donor formalities such as mid-term and completion reviews. This is linked both to the development of accountability within the project, sector and higher levels of government, and the need to provide information to support decision-making. For example, flows of information may need to be timed to fit into national budget planning activities, and should inform and influence identification and appraisal of any similar future projects or programmes. (Woodhill J 2006)

Plan for the necessary conditions and capacities. It is necessary to plan the organizational structure for Monitoring and evaluation including whether a Monitoring and evaluation unit specific to the project is needed. Appropriate organizational structures for Monitoring and evaluation should be discussed with partners and other stakeholders. Each partner’s responsibilities and information requirements should be considered. Planning should cover: staffing levels and types, responsibilities and internal linkages, incentives and training needs, relationships with partners and stakeholders, horizontal and vertical lines of communication and authority, physical resource needs and budget. Monitoring and ongoing evaluation should normally be the responsibility of the project managers. Impact evaluation may often require the expertise and capacity of external specialists. (Woodhill J 2006)

Conclusion

Monitoring and evaluation are distinct though related activities. Monitoring is a regular, ongoing activity which provides information for managers so that they can ensure that project implementation is on-track. Evaluation asks whether the project is achieving its aims, how well the project is being, or has been, implemented and whether there are any unforeseen effects of the project. The results of evaluation may feed back into the design and implementation of the evaluated project and/or the design of future projects. Evaluation may be ongoing in which case it is likely to be carried out by project management, terminal or ex post. Understanding and designing project monitoring and evaluation systems can be informed by the concepts and method of logical framework analysis. This provides the means to structure indicators and the organization of all monitoring and evaluation activities. The steps involved in planning and implementation of a project M&E system are relatively straightforward, but the challenges arise in determination of the right balance of approaches and methods to be used. The approaches for data collection, evaluation and reporting that are selected must be well tailored to a project’s characteristics and to local conditions and capacities. It is important to understand monitoring and evaluation as an essential part of good management, and one that requires acceptance of the notion of learning and of the creation of a learning environment.

REFERENCES

Kusek JZ, Rist RC (2004) A Handbook for Development Practitioners. Ten Steps to a Results-based Monitoring and Evaluation System. The World Bank, Washington DC.

Rogers P (2009) Matching impact evaluation design to the nature of the intervention and the purpose of the evaluation In: Chambers R, Karlan D, Ravallion M, Rogers P

Turrall S, Pasteur K (2006) Pathways for Change: Monitoring and Evaluation. Learning from the Renewable Natural Resources Research Strategy. DFID.

UNDP (2002) Handbook on Monitoring and Evaluating for Results. United Nations Development Programme (UNDP) Evaluation Office, New York.

Woodhill J (2006) Monitoring & evaluation as learning: rethinking the dominant paradigm. Sustaining livelihoods in Sub-Saharan Africa Newsletter, Issue 21, and African Institute for Community Driven Development.