Global Outreach: BETTER POLICYMAKING THROUGH EVALUATIONS – Interim Report

Session 1: Use of evaluations for political steering and participation of society

18th October 2021

The focus of the first session was to what extent evaluations can support the work of Parliamentarians and Parliament (Research) Staff and what makes high quality evaluations. In his short input Reinhard Stockmann raised the attention of the audience to the main roles of parliamentarians: Making Laws & Policies, Approving Budgets, and Oversight & Accountability. To fulfil these tasks, Parliamentarians and Parliament (Research) Staff are highly reliant on objective and evidence-based information to strengthen their arguments in political debates and work. This can be obtained through evaluations – but not all evaluations are the same! Therefore, criteria were presented to distinguish between bad and good evaluations.To illustrate how and to what extent evaluation is institutionalized in different countries experts from Uganda, Sri Lanka and Kyrgyz Republic were invited to a panel discussion, moderated by Reinhard Stockmann: Whereas Uganda already counts with a national evaluation policy, which also clearly defines the role of the parliament, Sri Lanka has a regulatory system for evaluation for more than 20 years, however, this is – still – not accompanied with a national evaluation policy and therefore only some parliamentarians are aware of evaluation – especially as trainings on evaluation is missing. Eventually, Kyrgyz Republic adopted a law on M&E in 2014, and also built and implemented a strategy for evaluation for state programs and laws, to further raise awareness for and knowledge about evaluation among parliamentarians.

“I think, the role that parliamentarians play in contemporary society is a really important one. They are not only key actors in the policy making process, but they are enormously influential people in relation to how they help shape culture and the ways in which evidence gets used.” – Dugan Fraser, GEI

“It is our mission of parliament keep the government accountable and efficient.” – Natalia Nikitenko, GPFE

Session 2: Introduction to fundamentals of M&E

26th – 28th October 2021

Based on the introductory session on 18 October the event series continued, to give a further as well as deeper insight into the fundamentals of M&E. The sessions were guided by Reinhard Stockmann and consisted of several theoretical inputs, Q&A-sessions and intensive group works, in which participants were able to deepen their new knowledge on project case studies. To guarantee a denser learning atmosphere, these sessions were open to a selected group of participants only.

In the first meeting of this session the learning group dedicated to M&E basics, which encompassed questions on the definition of the evaluation object, analytical perspectives of evaluation, but also for which purpose, by whom and how an evaluation is to be conducted. Further to this, the second meeting focused on the planning of evaluations. Participants were able to learn about implementing actors, design, methods but also financial resources needed, to actually implement evaluations. Finally, in the third meeting, the participants learnt about standards, results and utility. Standards for instance ensure and develop the quality of evaluations and with it also their final results and practical utility – the latter is from special importance, as utility is one of the core values of evaluations!

“Policy decisions should be based on objective evidence and facts, and no longer on common sense, ideology, tradition, religious beliefs etc. Political goals are best served, when valid and reliable evidence is used comprehensively to inform decisions.“ (…) Parliamentarians who know how to use such evaluation results can convince others with arguments, that are perceived as objective and credible.” – Reinhard Stockmann, CEval

In a very final group work, participants were asked to formulate concrete evaluation questions, taking into account their own and individual case study, the benefits of evaluation in the different program phases but also all other aspects of the content taught previously. All this with the objective, to train and internalize one of the most important tasks for Parliamentarians and Parliament (Research) Staff with view on evaluation: how to formulate adequate evaluation questions, and therefore, how to ‘think evaluative’.

Session 3: Challenges in evaluating SDGs

2nd November 2021

What are the conceptual challenges, in case progress in implementing the SDGs is not only to be monitored but also to be evaluated? In his theoretical input, Wolfgang Meyer draw the attention to three main challenges in the framework of sustainable development – System Integration, System Change and Inclusiveness – and how National Monitoring and Evaluation Systems might accompany the resolution of these challenges.

“We have to recognize that all data is local, so you have to collect all kind of data on a local level, because it is the way it happens. Therefore, we have always the problem to aggregate this information towards a national level. (…) Investing in the abilities for M&E on the local level means also investing in improving the quality of the national statistics.” – Wolfgang Meyer, CEval

Besides that, the participants were able to get deeper insights into country experiences. In Sri Lanka for example, after having three different bodies to take SDGs into account for development programmes – an own ministry, parliamentary committee, as well as a president task force – an independent parliament evaluation commission was established to coordinate all SDG connected development activities.

In the Philippines, M&E activities had been very fragmented, conducted in different government agencies without being connected to each other, and there were no evaluations standards or criteria on the use of evaluations. However, in 2015, when the SDGs were adopted, a memorandum on adopting a national evaluation policy framework was implemented to oversee and coordinate the implementation of the SDGs as well as to mandate all national government agencies to formulate an evaluation agenda.

“The sectoral view allowed us to understand the contribution of each sector to the national priorities as well as the SDG commitments. (…) The multi stakeholder partnerships allow all the actors to leverage their strengths and supplement capacities while working towards the desired goals.” – Shweta Sharma, NITI Aayog

India counts with two main monitoring mechanisms to assess the implementation status of SDGs: One is the SDG India index focusing on periodic monitoring, the other is the Voluntary National Review (VNR), conducted in 2017 and 2020. But they also face some central challenges: limited capacity in the field of M&E, availability of high-quality data, and inclusion of marginalized communities.

Costa Rica was the first country in the world which admitted itself to the SDGs. To assure that SDGs are embedded in planning instruments, a multisectoral and multistakeholder governance system, composed by a high-level council, which is led by the president of the republic and coordinated by the Technical Secretarial of the National Commission of the SDGs, was implemented. Additionally, an advisory committee exists to accompany this process. A further milestone in this process was the implementation of the SDGs into the national development and public investment plan. Important is also, that SDGs are a base criterion for public interventions in the national evaluation agenda.

“One of the most complex and most ambitious things we want to do is to preserve the spirit of an integrated development approach and how that reflects upon the evaluations we do and also our monitoring instruments.” – María del Pilar Garrido Gonzalo, Goverment of the Republic of Costa Rica

Session 4: Closing ceremony

11th November 2021

The event series ended with a closing ceremony, in which the following key take-home-messages were formulated by Reinhard Stockmann:

The information, facts and evidence obtained from evaluations provide Parliamentarians with evidence-based arguments, that they can use for:

Overall, to

• place their own political decisions on a rational more objective, less vulnerable, basis;
• present themselves as well-informed politicians who competently represent the interests of their voters and should therefore be re-elected!

We as IPDET, APEA and GPFE are convinced of the fact that evaluation enables a better world through better policy.
Therefore, personal capacity building in the use of evaluation is necessary to generate evidence for policy making within parliaments and its staff.

In addition to these take-home-messages of the event, 63 participants from 15 countries were officially awarded for taking part in this event series. Furthermore, the participants had the opportunity to establish contacts with each other in a virtual speed networking session.

“As countries recover from the aftermath of the Covid-19 crisis, parliaments and parliamentarians play a key role in shaping and delivering evidence-based policies that can turn around the economic and social effects of the pandemic and can propel the country towards full implementation of the SDGs.” – Marco Segone, UNFPA