The Tracer Study, conducted in 2021/22, covering the years 2014-2020, provides insights about the impact IPDET is making.
The Tracer Study was meant to represent equal number of program years to represent both Canada and Switzerland as venues of the IPDET programs: 2014 to 2016 were three On-Site Program years conducted in Canada. 2018 and 2019 were both conducted On-Site in Switzerland while 2020 was offered only online, the base of operation and management was in Switzerland. This equals to a total number of individuals attending IPDET between 2014-2020 at least one time of 1,018.
The Tracer Study followed an ex-post facto research design. This means that only data from persons who participated in IPDET (and organizations, which sent participants) in former years, were collected.
A multi-step mixed-methods approach, using qualitative and quantitative data collection and analysis methods, was applied for this Tracer Study.
This involved at first level a review of IPDET’s publications and evaluation reports of the years 2014 till 2020, as well as a review of related literature of tracer studies.
Afterwards an alumni database covering the years 2014-2020 was compiled using participants background data received from Canada and merged with the existing database in Switzerland. This database served as the basis for the selection of alumni and organizations for the interviews as well as for the online survey.
In a further step, semi-structured interviews with alumni served mainly as exploratory to support the development of the online survey. After the survey was developed and programmed into the software SoSciSurvey, it underwent several testing. Response from these testing was used to refine and finalize the online survey. The survey was then published, and invitations were sent out on 18 May 2022 and the survey was administered until 6 July 2022.
The population for the online survey came from the alumni database from 2014 to 2020. Invitation to respond to the survey was sent out to 908 participants, however 137 mails failed to deliver; 771 people represent therefore the total population for the online survey. A total of 228 alumni responded to the survey. In the end, 221 cases were included in the analysis, as seven cases only finished one page of the survey, with a response rate of 28.7%.
For the data analysis, qualitative content analysis and descriptive statistics were mainly employed. Data sources and methods used for data analysis are outlined in the analysis grid. Furthermore, quantitative data were triangulated with qualitative data whenever possible.
The overall objectives of the Tracer Study were to provide the IPDET management with information on the achievement of its goals, formulated in its Program Theory. Additionally, it was supposed to support a development process, as it promotes internal and external communication on the results and with this, in the end, further improvement. Lastly, it aimed to create transparency on the programs implementation and be accountable towards its stakeholders.
Based on these objectives, IPDET defined the following categories guiding the Tracer Study: demographics of alumni and organizations, increased knowledge and skills and commitment to produce and use evaluative evidence, and grown community that is actively sharing learning and collaborating. The timeline defined was 2014-2020, to represent equal number of program years to represent both Canada and Switzerland as venues of the IPDET programs. For the implementation a multi-step mixed-methods approach, using qualitative and quantitative data collection and analysis methods, was applied for this Tracer Study.
Result 1: IPDET is reaching a diverse target audience in the international community | Results of the demographic analysis of the alumni database and survey respondents have shown that IPDET attracts and reaches a diverse target audience in terms of gender balance, regional distribution, income classification and type of organization.
- In terms of gender distribution, the participation trend from 2014 to 2020 showcased a distribution that favors participation from women with an average of 58% female participation.
- Most participants at the time of their IPDET attendance were coming – based on work country – from Europe and Central Asia (29%), Sub-Saharan Africa (23%) and North America (15%). Comparing the regional distribution between the two venues revealed that proximity to the venue seems to play a role in the regional distribution of participants: participation from Europe and Central Asia significantly increased in 2018 and 2019 compared to 2014, 2015 and 2016. Also, though smallest in percentage, participants working in the Middle East and North African region doubled 2018 to 2019 from an average of 2.7% to 5.4%. On the other hand, participants from the North American region have significantly reduced from an average of 21.1% (2014-2016) to 12.4% (2018 to 2019).
- An average of 45% were awarded an IPDET scholarship between 2014 and 2020. The availability of IPDET scholarship helped facilitate participation to IPDET trainings for participants coming from low, lower-middle-, and upper-middle-income countries. Furthermore, the online format offered by IPDET since 2020 seems to have helped increase participation and accessibility for participants coming from low- and lower-middle-income countries.
“Without scholarship I would have probably participated only on the Core Course. With the scholarship I was able to add one week workshop to my training, which was very good.” – Participant from 2019
- In terms of type of organization, about a third of alumni were working in government organizations during their IPDET participation. The most striking difference between the two venues is reduction to more than half in the number of participants working in a government agency. The reduction of government agency participants also means increase of participation from other types of organization from 2018 to 2020, namely from the UN (specialized agencies), NGO and foundation, other international/regional or multilateral organizations, and from the World Bank Group.
- Majority of alumni are still currently working in the same type of organization they were working at when they participated at IPDET. However, it is worth noting that there has been a 5% increase of self-employed among the survey respondents.
- While IPDET is reaching a diverse target it seems to be difficult so far to convince most sending organization to send more than one participant or attract past participants to attend other (more advanced) workshops in succeeding years.
“What distinguishes the IPDET 2014 training program is the face-to-face coexistence with the various participants from all over the world. The discussions that took place in the evenings on the university campus after the formal training in the halls had a very prominent role in exchanging knowledge and exchanging cultures, and this in turn developed the personality of the evaluators to understand the reality of the evaluation in other countries around the world. […] Eternal friendships are built from coexistence face to face with each other. […] To this day, we exchange experience and knowledge in the field of evaluation.”– Participant from 2014
- IPDET should consider designing appropriate follow-on courses, both, online and on-site to encourage alumni to enroll again in IPDET courses.
- This goes hand-in-hand with the identification and testing of different course formats for different audiences, e. g. shorter courses, hybrid-formats, flipped classroom, etc.
- Moreover, IPDET should take further advantage of online workshops as it offers the important opportunity to target individuals, more specifically those who are normally unable to attend the On-site Program, to enroll.
- To further strengthen the position of IPDET as a high-quality training provider, IPDET should investigate offering micro-credentials or other types of university certification.
Read the executive summary and the complete Tracer Study 2014-2020 here: