Skip to content

101 on Meta-Evaluation, or: how to evaluate Evaluations – Approaches, Methods and Findings

Workshop
Program
2020
Part of
Session Four
October 19 - 23
Level
Intermediate
Recommended for
Comissioners, Evaluators, Policy Makers

description

Do evaluations always provide the ‘right’ information? I mean, are they valid, reliable, correct in a sense? Can we believe their findings? And after all, what is the merit of our work to society, or at least for the people who read our reports?
Not only evaluators contemplate about such questions from time to time. More and more commissioners and political decision-makers want to know if they can trust evaluations and what they can actually learn from them. Moreover, the added value of such exercises for providing the empirical basis for decision making remain often obscure. Hence it is no wonder that the demand for meta-evaluations rises.

We all know what evaluations are good for and how we should implement them, don’t we? There are numerous textbooks, training programs, podcasts, which provide methodological and practical guidance. Complying with such standards and following scientific codes of conduct should warrant valid, reliable and useful findings. – But can we prove it? How do we actually find out about the methodological quality of an evaluation, the validity and reliability of its findings, and eventually the usefulness of its conclusions and recommendations? Basically, how can we distinguish a good one from a bad one?

These are exactly the questions we want to answer in this workshop. We discuss criteria that help us decide if we can trust an evaluation, how we can assess its methodological quality, check if it was appropriately prepared and implemented, and eventually of use. We further jointly elaborate a meta-evaluation approach from the analysis grid to suitable methods for content analysis. Going beyond the evaluation report, we dive into the potentials of meta-evaluation as well as its limitations.

So, if you want to know how to evaluate an evaluation, come in and find out. Bring along – actually please send it a few days in advance – an evaluation you always wanted to know how good it is. I promise you, afterwards you will know (better).

objectives

The participants

  • are able to assess the quality of an evaluation according to common standards,
  • know which criteria should be used therefore and how they can be applied.
  • They evaluate evaluations in practice and thereby learn about the added value of a systematic approach to it.
  • Finally, the participants will gather an overview about practical tools and applications for meta-evaluations.

recommended for

This workshop is particularly relevant for commissioners of evaluations who want to learn about the quality of the evaluations they receive or who want to develop their own quality assurance framework for evaluations. It is, however, likewise interesting for evaluation professionals who want to learn according to which quality standards their work may be assessed if it becomes subject to a meta-evaluation. Eventually, the workshop may be suitable to everyone who reads evaluation reports and contemplates about their trustworthiness.

level

This is an intermediate level workshop.

prerequisites

Participant need to have a basic understanding about evaluation as a management tool. While it is not necessary to have conducted an evaluation, ideally, the participants should have read a number of evaluation reports before and know about their strengths and weaknesses.