OUTLINE

M&E activities are designed to feed into decision-making needs. Data collected must be shaped, at least in part, by the actual decisions that need to be taken, i.e. the more specific end-use. The more closely data collection does so, the more focused, efficient and effective it is likely to be. Without this focus, one risks collecting a great deal of information that is never used.

This part of the module contains two sessions. The first focuses on the need to ensure a logical flow between the broad purpose of the M&E activity, the more specific decision-makers' questions and eventual questions to guide field data collection. Optional activities allow this to be tested in emergency rapid assessment, programme monitoring, and evaluation contexts. The second session is appropriate where training activities have a more in-depth focus on evaluation. It focuses on linking evaluation criteria and more specific evaluation questions in the same logical flow from purpose of the evaluation down to data collected.

4.5.1

FOCUSING M&E ACTIVITIES ON DECISION-MAKING NEEDS

Several levels of questions will help guide M&E activities. They range from the broadest and overarching ones (e.g. what is happening?) to the most detailed ones (e.g. those appearing in data collection questionnaires). This session focuses on the logical flow that must exist between decision-makers' questions and the eventual data collection for rapid assessment, monitoring and evaluation.

FN	Focusing M&E activities on decision-making needs The facilitator will introduce the hierarchy of M&E questions and then invite participants to work out examples for each stage, first in a very narrowly defined group exercise and then in a more detailed exercise focusing on either rapid assessment, monitoring or evaluation.
CCS	Focusing M & E activities on decision-making needs Multiple purposes of evaluation
OH	Focusing on decision making needs Linking decision makers' questions and information needs
EX	Group exercise instructions - Programme monitoring field visit Group exercise instructions - Rapid assessment

LINKS TO OTHER SESSIONS
When examining the hierarchy of questions, the core content sheet presents types of questions that are analysed in more detail elsewhere in this package.

• SESSION 6.X – RESEARCH QUESTIONS This session introduces the criteria that should guide the choice of questions for data analysis.
• SESSION 5.X – QUESTIONNAIRE
• SESSION 5.X – INTERVIEWS
  These two sessions contain tips on formulating very specific data collection questions to be used in things like questionnaires or interviews.

4.5.2

EVALUATION CRITERIA

This session presents the standard OECD/DAC evaluation criteria (also UNICEF evaluation criteria) as well as more recently defined criteria developed for the evaluation of humanitarian programmes. It demonstrates how they can help in identifying evaluation questions. The case study allows participants to consider some of the issues particular to evaluation of humanitarian programmes.

FN	Linking evaluation criteria to evaluation questions The facilitator will briefly introduce evaluation criteria (What are they? What is their purpose?) and invite participants to identify evaluation questions linked to them through a VIPP card exercise built around a case study of a sample programme.
CCS	Linking evaluation criteria to evaluation questions Linking evaluation criteria to evaluation questions: additional criteria for humanitarian programmes Applying evaluation criteria in humanitarian crisis -- balancing contextual factors
OH	-
EX	Case study: kids in arms

REFERENCE MATERIALS

Gosling, L. and Edwards, M. Toolkits: A practical guide to assessment, monitoring, review and evaluation. Save The Children Development Manual No. 5, 1995.

Habicht, JP, Victoria, CG, Vaughan, JP. Linking Evaluation Needs to Design Choices. UNICEF Staff Working Papers, Evaluation and Research Series No. EVL-97-003.

Patton, M.Q., (1978). Utilization Focused Evaluation. Sage Publications. Inc. Thousand Oaks, CA.

Rossi, P.H., Freeman H.E., Lipsey, M.W. (1999). .Evaluations: A systematic approach 6th edition. Sage Publications. Thousand Oaks, CA. Pp 79-117
This chapter focuses on identifying evaluation questions. It provides some general guidance on what makes a good evaluation question. It then looks at how to determine the evaluation focus, through involvement of key stakeholders and through the use of programme theory to formulate more specific questions. The texts concludes with a discussion of how to organise and prioritise the eventual selection of evaluation questions.

Stufflebeam (2000). Guidelines for developing evaluation checklists. The Checklists Development Checklist (CDC). http://www.wmich.edu/evalctr/checklists/guidelines_cdc.pdf

UNICEF, New York. A UNICEF Guide for Monitoring and Evaluation: Making a Difference? Evaluation Office, 1991.

Worthen, B. R. and Sanders, J. Programme Evaluation: Alternative Approaches and Practical Guidelines, 2nd ed., White Plains, NY: Longman Inc., 1997.

Evaluation criteria

Hallam, A. (1998). Evaluating Humanitarian Assistance Programmes in Complex Emergencies. ODI, London.
This Good Practice Review is the written output of an OECD/DAC project initiated to identify and disseminate best practice in the evaluation of humanitarian assistance programmes. The study seeks to improve the consistency and quality of evaluation methodologies, and enhance the accountability function of evaluation, contribute to institutionalising the lessons learned, and identify better methods for monitoring performance of humanitarian aid operations.

OECD-DAC (1991). The DAC Principles for the Evaluation of Development Assistance. OECD, Paris.

OECD (1999). Guidance for Evaluating Humanitarian Assistance in Complex Emergencies. OECD, Paris. http://www1.oecd.org/dac/htm/pubs/p_eval.htm