I-a: Quantitative Data Collection: Planning and Implementing Surveys

description

Besides interviews, observations and group consensus techniques such as Focus Group Discussions, many evaluations rely on more or less standardized large-scale data collections, for instance of beneficiaries, implementing partners or other key stakeholder groups. Such type of data collections, also known as “surveys”, are of particular benefit when data needs to be collected from a larger number of respondents, when data needs to be collected in an efficient manner and when the required information is already known by the researcher. The key survey instrument is the questionnaire, which should make the respondents providing that information. While conducting a survey appears to be a trivial task, given the numerous practical guidelines and technical aids available today, experience shows that much can go wrong in practice. In fact, nothing causes more frustration among, particularly younger, colleagues than a survey going haywire, not yielding the required data, providing wrong results. I guess we all went down that route before …

… from: “Why on earth did no one answer my questionnaire? I wrote such a nice invitation letter.”

… over: “These results are too good to be true. Did anyone check where that data is coming from?”

… to: “OMG! Where the hack is my data set?! It was still there yesterday when I shut down my computer.”

Having to deal with a low return rate, biased data and technical issues are just a few things among many others that can cause trouble when planning and implementing a survey.
In this two-day introductory workshop, we will deal with the manyfold challenges one has to master in order to obtain valid and reliable survey results, and discuss the methodological basics that need to be considered along the way. The workshop will be structured according to the typical steps of a survey from setting its scope to assuring its correct technical implementation. Key topics that will be dealt with comprise the following:

• Survey prerequisites: When to decide for a survey (or not), methodological and technical requirements, budgeting and staffing
• Designing a survey: Selecting the survey instrument (online, face-to-face, self- or other-administered), developing survey items, choosing the right type of question and measurement scale, formatting a questionnaire, dealing with qualitative and quantitative data in a survey
• Sampling: Probability and non-probability sampling, stratification and clustering, determining the sample size
• Managing a survey: Pre-testing a questionnaire, inviting participants, sending reminders, cleaning and managing data, preparing the data analysis

This is a workshop, not just another method course. So, do not expect to lean back and let the trainer do the talking. While he will do so for a while, much emphasis will be put on a hands-on approach where the participants are actively engaged during group works and plenary discussions. Each topic will be introduced with a presentation, followed by a practical exercise in which the participants apply their freshly gained knowledge with real-world examples. Finally, each working group will have the opportunity to share and discuss its results with all participants.

objectives

The participants …

… know how to set up and implement a survey,

… know how to design a questionnaire,

… know how to arrive with valid and reliable survey results, and

… are able to deal with typical challenges that can arise during a survey.

recommended for

This workshop is suitable for anyone who plans to conduct a survey or who was, is or will be involved in any type of empirical research where data is being collected by means of a survey. It is, however, also relevant for commissioners of evaluations to be able to assess if a survey was implemented correctly and if survey results can be trusted.

level

This workshop is at intermediate level, meaning that it is suitable for participants with some previous working experience (e. g., as an evaluation team member or monitoring officer). Having already conducted a survey is an asset but not necessarily required.

prerequisites

• Computer literacy, experience with office applications
• General knowledge in empirical methods (e.g. knowing the difference between qualitative and quantitative data), no advanced knowledge needed though
• Some research experience, having worked with primary data