Each year more and more evaluations are commissioned worldwide. But do the commissioners really know what they are doing? Do they ask the right questions, engage the right people and then use the results in their work?
For this to happen and for the potential of evaluations to be used to the fullest possible extent, Commissioners also need to know something about evaluation. Commissioners do not conduct evaluations themselves, so they do not have to be able to master all the tasks involved in an evaluation. Nevertheless, some things they need to know about evaluation. Commissioners play a crucial role in an evaluation. They set the course for an evaluation and determine largely whether an evaluation is successful and whether its results are used. They therefore bear a great responsibility. This workshop aims to help them (a) to be aware of their central tasks in an evaluation and (b) to be able to cope with these tasks professionally.
First, it should be made clear who is actually meant by the term Commissioner and what this means for an evaluation. Next, the evaluation should be considered in its social and organizational context. The evaluation boom that we have been observing is due above all to a “new” way of political and administrative management. Without the idea of “evidence based policy” and without the concepts of New Public Management, outcome and results based management, this boom would never have taken place. This correlation is explained in order to recognize the importance and potential of evaluation and to be able to distinguish evaluation from other concepts such as quality management, audit or controlling.
Then we plunge into the evaluation process and learn from practical examples which tasks the Commissioner has to perform and which he/she should leave to the evaluator. Specifically, the goal is to ensure overall stakeholder participation, define roles and develop a comprehensive communication strategy from the planning of an evaluation to the use of the evaluation results. On the other hand, the Commissioner sets the central course: He/she formulates the key evaluation questions and outlines the scope of the evaluation. In doing so, he/she must take great care to ensure that the personnel and financial resources available are appropriate to the defined tasks and that these can also be completed within the planned time frame. In order to avoid any misunderstandings about the tasks to be performed by the evaluator, the Terms of Reference for the assignment must be carefully and precisely formulated. The tender process must be regulated in a clear, transparent and comprehensible manner. Finally, the Commissioner must be able to select the tender that offers the best value for money. After the commission has been granted, final questions can be clarified in a kick-off workshop so that the evaluation can meet the expectations associated with it.
Now it’s the evaluators’ turn! If necessary, the Commissioner can be informed about the progress of the evaluation and he/she can intervene to ensure quality. At the latest, when the evaluation report is available, the Commissioner need to assess it professionally and decide whether the tasks have been completed to his/her satisfaction. With a management response and a targeted dissemination strategy, he/she ensures that as many stakeholders as possible learn something from the evaluation and benefit from it – and that the resources invested were used optimally.
- Know the importance of New Public Management/Outcome Management and their instruments for evidence-informed decision-making
- Be able to plan, manage and quality assure an evaluation
- Are capable to implement a communication and dissemination strategy to assure the best possible use of the evaluation results.
Clients, sponsors, users and commissioners of all kinds of evaluations (ex-ante, ongoing, ex-post) on all levels (i.e. project, program, strategies, policies, institutions), like parliamentarians, executive staff, project and program managers, M&E officers from governmental and non-governmental organizations, bi- und multilateral development organizations.
This is a workshop on beginners’ level. Basic knowledge about evaluation is helpful but not required.
In preparation of the workshop, participants will be asked to read chapter 2 “An Introduction to Evaluation”, chapter 3 “Competing and complementary approaches to evaluation”, in “A practitioner Handbook on Evaluation by Reinhard Stockmann, Edward Elder, 2011, as well as chapter 4 “Evaluation Process” in “Functions, Methods and Concepts in Evaluation Research” by Reinhard Stockmann and Wolfgang Meyer, MacMillan 2013. The chapters will be made accessible on the internal platform of the IPDET website.