IV-a Evaluation Design and Approaches, Mixed Methods for Evaluation

Description of workshop content

For a successful evaluation process, the commissioners and the evaluators must reach an agreement on the foreseen use of evaluation results and on a feasible and effective evaluation plan. This interactive workshop will lead the participants through the key stages of preparation of an evaluation plan for several real-life interventions. The participants are recommended to bring and to explain a Theory of Change of concrete projects or programs to be evaluated by them (or they can select from the cases prepared by the lecturer). As designing an evaluation is a complex process, we will go through all the interrelated steps needed for the evaluations proposed by the participants:

  1. Understanding the context and revising the logic model of the intervention to be evaluated
  2. Deciding on the general evaluation approach (specifying the main purpose of evaluation)
  3. Setting the key evaluation questions (descriptive, normative and cause-effect), and identifying appropriate indicators (baseline, target)
  4. Identifying the design and type of evaluation (experimental, quasi-experimental, non-experimental)
  5. Identifying appropriate data sources (ensuring triangulation)
  6. Proposing appropriate mix of data collection methods/instruments (quantitative, qualitative and mixed methods, ensuring triangulation and participatory approaches)
  7. Deciding on sufficient scope of data (census or sample, randomized and non-random sampling, determining the sample size)
  8. Data analysis (analyzing quantitative and qualitative data, using nominal, ordinal or ratio data, independent and dependent variables)
  9. Finalizing the evaluation design matrix (the main output of the workshop)
  10. Estimating time and budget needs (considering both reliability of evaluation results and feasibility)
  11. Presenting the evaluation (including feedback from the lecturer and other participants to the presented evaluation design matrix)

The workshop will combine summarizing lectures, brief plenary tests, group exercises and work in the small groups as well as sharing of practical experience among participants, including concrete tips & tricks.

Worjshop objectives

The participants will be able:

  1. to prepare feasible evaluation plan, considering real life opportunities and constraints;
  2. to justify the proposed evaluation plan in discussion with the peers; and
  3. to work in the team, with a potential to establish new professional networks across borders and across sectors.

Target group

Evaluators (from diverse sectors), evaluation managers, commissioners.

Level

Fundamentals level – the participants should be familiar with basic evaluation terminology and concepts. Although the key evaluation aspects will be summarized in brief recapitulation lectures and exercises, the participants will work in small groups on concrete tasks.

Prerequisites

The participants should understand the formulation of intervention logic and if possible they should bring their own examples of Theory of Change (or Logical Framework Matrix) from real-life development interventions. As an alternative option, the participants can select an example for the group work from the case studies prepared by the lecturer. Completion of the IPDET Core Course or of a previous IPDET workshop on (Re-)construction of Program Theory will be an asset.

Practical experience from evaluations (or implementation) of development projects or programs will also be an advantage as well as the grassroots knowledge of emerging development issues.

Instructors

Daniel Svoboda