I-d Rigorous Impact Evaluation - how, why and what for

Description of workshop content

Impact evaluation is the type of evaluation that aims at identifying the causal independent effect of an intervention on its target population. For some, the so-called gold standard of evaluation methodology entails achieving or emulating experimental conditions in order to make causal inferences. Although the term rigorous is a contested and politically charged term, in this workshop we critically engage in a methodological and disciplinary discussion of existing methodologies and their utility to inform policy. The workshop has a theoretical-conceptual component and a practical one. The first part of the workshop provides an overview of experimental and quasi-experimental strategies for impact evaluation, clarifies the conditions and assumptions of commonly used identification strategies and reflects upon their scope and limitations, from the perspective of an informed and sophisticated user of these tools (emerging evaluators, evaluation commissioners, evaluation specialists, decision-makers).

The second part of the workshop will engage in collaborative deliberation on the methodological quality, the pertinence and the utility of an impact evaluation in the development field. The main point of this workshop is to make sense with nuance of the validity and the utility of several impact evaluation methodologies and further reflect on the role of impact evaluation in the development field.

Workshop objectives

The participants will:

  1. Know the main features of experimental evaluations (randomized control trials RCT)
  2. Learn key concepts for impact evaluation methodology (e.g. internal vs. external validity, causes and consequences of bias, etc.)
  3. Know the features of common quasiexperimental strategies (Propensity Score Matching, Diff in Diff, Regression Discontinuity)
  4. Be able to understand the differences and relative strengths of these methodologies.
  5. Be able to use this methodological knowledge to assess impact evaluation proposals, impact evaluation reports and critically analyze the validity, scope and utility of impact evaluation findings.

Recommended for

The workshop is designed for evaluators, commissioners and evaluation specialists that may not have practical experience in performing impact evaluations using experimental and quasi-experimental methods, but do need to have a strong domain of the methods used for impact evaluation in order to commission, assess and utilize impact evaluation reports, and to be able to validate, interpret and follow up findings for decision-making and learning purposes.

Level

The workshop is high-intermediate level. Participants should have some familiarity with the evaluation field (i.e. types of evaluation, principles of evaluation, uses of evaluation) and with social science research methodology. But no advanced training in statistics or statistical software is required.

Prerequisites

Although no statistical package-software will be used and no advanced training in statistical methods is required, participants should preferably have basic knowledge of probability in order to understand the statistical underpinnings of experimental and quasi-experimental strategies. The workshop will introduce all the required concepts and knowledge to be able to identify, analyze and critically assess different impact evaluation strategies. However, participants with some background would find it easier to navigate all this material in a short time span. Participants with no background are advised to review basic reading material before the workshop.

Instructors

Claudia Maldonado