II-c An Introduction to Applied Text Analytics in Evaluation Practice

Description of workshop content

Quantitative data analysis has traditionally relied on computers and statistical software Over the past 10 years, there has been an explosion of available data, and data analytics, machine learning and artificial intelligence have become important fields that now automate data analysis.

Approximately 80% of the available data is qualitative text, and qualitative data analysis techniques have traditionally relied on researchers reading, coding and distilling meaning from interviews, transcripts and documents, some using CAQDAS (Computer Assisted Qualitative Data Analysis Software).

Natural Language Processing and Text Analytics is a new field in machine learning and artificial intelligence that aims to make computers better at analyzing and understanding text data. This workshop will provide an introduction and overview of this field, and its (potential) applications for evaluators.

We will introduce participants to the basic concepts, techniques and approaches that natural language processing and text analytics use to get text ready for analysis, to apply a range of machine learning algorithms and analysis tools, and to extract meaning from text and documents.

Participants will be guided and supported to explore and try-out some of these tools: automated document summarization, automatic qualitative theme discovery and clustering using unsupervised machine learning (topic modelling) and sentiment analysis.

The presenters will facilitate a group discussion on how these tools and approaches can be applied to their work situation and broader evaluation practice. The workshop will close with a discussion of potential future developments in text analytics.

Workshop objectives

After the workshop, the participants will:

  1. Have a basic understanding of natural language processing, and how to prepare text and documents for analysis and current constraints of practice.
  2. Have an understanding of concepts and approaches in text analytics and machine learning applied to text, including automated document summarization, topic modelling for theme/topic discovery and sentiment analysis.
  3. Be able to describe the place of text analytics in evaluation practice, and in their own work environment.
  4. Have hands-on experience with a few tools that support or provide text analytics.

Recommended for

Evaluators who are interested in additional tools that can support analysis of text documents, especially when a big corpus of documents needs to be analyzed and summarized in short time frame. Anyone interested in how machine learning and artificial intelligence techniques are applied to text.


Participants should be comfortable with computers and software. Some very basic (and guided) coding may be required. The instructors aim to make this workshop accessible to all comers, but participants should be comfortable trying new analysis techniques that require interpretation.


No prerequisite knowledge is required. Experience and/or interest in machine learning, artificial intelligence, text analytics or coding will be helpful but is not required.

Participants need a laptop that can install new software (not administrator restricted).


Kerry Bruce
Joris Vandelanotte