I-a Qualitative Data Collection Methods

Description of workshop contents

The main goal of this workshop is to provide theoretical and practical knowledge about the qualitative methodologies that are mostly used in policy and program evaluation today. Frameworks, concepts and methodologies will be presented from a problem-based-learning and mixed-method perspective, by bringing specific cases from Brazil and worldwide.

The workshop will take place in a 2,5 day slot and will be structured in 3 sections:

  1. Introduction to qualitative evaluation - 4 hours What does qualitative evaluation mean? What are the main goals of qualitative evaluation and why is it important? Main ways of doing qualitative evaluation Application of qualitative evaluation (Case: National Program for Rural Women Documentation, Brazil)
  2. Qualitative methods - 12 hours Direct observation as evidence (Case: Using Participant Observation to Improve Police Effectiveness) Ethnography: advantages and disadvantages for impact evaluation (Case: The Buffalo Creek Desaster) Case study: sampling, designing, planning and organizing Semistructured interviews: relevance in mixed method research Focus groups: history, approaches and applications (group activity)
  3. Transforming qualitative data into evidence and results - 4 hours Analyzing data and interpreting results (group activity) Qualitative data reporting: effectiveness and communication challenges (group activity)

Each section will always open up with a provoking question and will end up with a group activity.

In the first section, goals, types and uses of qualitative evaluation will be introduced. “Qualitative evaluation” will be defined in relation to “quantitative evaluation”, keeping in mind that whenever an evaluation process is initiated, the methodological sequence may develop in multiple ways, depending on the life cicle of the program or policy and the main question and/or problem posed for the evaluation. Finally, a case will be discussed in order to introduce participants into program and policy evaluation practical approaches.

In the second section, basic qualitative methods and data collection techniques will be introduced. This section should provide conceptual notions and practical knowledge for the use of research instruments in policy and program evaluation processes. In order to facilitate the learning process, common sense will be shaked, given that some methods cannot be easily classified. For instance, is direct observation a qualitative or a quantitative method? Can the narrative of an event provide insightful numerical data? The analysis of two cases will help to imagine how the instruments presented might work out in paticular evaluation contexts.

In the third section, participants will be confronted with the various possibilities for the interpretation of qualitative data, from logical inference, to meaning attribution or argument formulation. The interpretation of phenomena, events or conflictive issues requires the evaluator’s judgement. This section also will deal with writing a final evaluation report and how this may be a definitive step to make the research results understandable to different stakeholders. A strong emphasis will be put on the need for gaining competence in writing and communicating evaluation results. Examples of qualitative evaluation reports will be analyzed in small groups.

During the workshop, basic resources will be recommended, such as bibliography, software for qualitative data collection and analysis, relevant online resources, among others.

Workshop Objectives

The participants will be able to design and plan an evaluation project using qualitative methods.

  • know basic frameworks, concepts and methods used in qualitative evaluation today;
  • be able to decide what qualitative method(s) may be relevant in particular policy and program evaluation contexts;
  • get a sense of the possibilities and limitations of qualitative evaluation for the production of evidence and results.

Recommended for

Evaluators (as in the Program), applied social scientists, policy makers at the beginning of their careers, NGO practitioners.

Level

This workshop will have an introductory character.

Prerequisites

Social science research experience; program and policy evaluation basic knowledge.

Instructors

Rocio Lorenzo