Trainer’s outline

Data Analysis Qualitative

Approach

• What is Qualitative Data?
• Mapping Data
• Analysing Data
• Data Conclusion and Verification
• Presenting qualitative data

What is Qualitative Data?

For most evaluations it is advisable to use a mix of quantitative and qualitative data.  This session focuses on analysing qualitative data.  Qualitative data tends to be descriptive data rather than numerical data.  Examples are focus groups, interviews, journals and case studies.  These methods are used to add meaning and interpretation to survey results, which can often tell you ‘WHAT’ respondents think  but offer limited information on ‘WHY’ respondents think this. 
Qualitative data contain large amounts of text that needs to be reduced and organised to derive meaning.  The meaningfulness of the data is determined by the goals and objectives of the project being evaluated and the evaluation questions, which have been set.

Mapping Data

This is the process of organising the data.  At the macro level data might be organised according to which stakeholders have supplied it.  For example, interview data from stakeholders with a managing function may be separated from interview data from stakeholders with a delivery function. 
Data may also be organised according to evaluation questions or responses dealing with specific items.  For example, if one of the evaluation questions is “How effective is partnership working on the project?” then data related to partnership working would be grouped together.  This might include interview questions related to communication, information sharing, roles and responsibilities and management.

Analysing Data

The process of analysing qualitative data begins at the same time as data collection.  The evaluator starts to look for patterns or common themes. 

Potential exercises

(a choice depending on time and audience)

Hand out 1 Exercise: Response to sentence completion exercise at launch dinner
Hand out 2    Kidney Machine debate

Stage 1: Data reduction
The data needs to be relevant to answer the evaluation questions.  Relevant data should be gathered together.  Data should be included not just for content but also for the intensity with which different issues are expressed.  The frequency of expression of different issues should be recorded.  But it is important not to “flatten” the data by reducing it to survey-type responses.
Ask questions
What is the overall picture this data is giving?
What are the key themes?
What does it mean?

Stage 2: Data Display
This is the process of organising data to enable conclusion drawing. This can be done using a matrix.

Evaluation Question: How effective is partnership working?

Indicators: Communication, attendance etc. … and to this - context

Interview Question: What do you think of partnership meetings?

Hand out 3

Respondent Group	Comments	Why?	Suggestions
Core Management Team
Project deliverers – large/ public
Project deliverers – small/ community
Evaluator’s observation at meetings

A pattern of cross-group differences can be discerned by comparing the responses.  The data entered in the boxes above needs to be grouped or themed through the process of data reduction.
Qualitative analysis is a process of comparing and contrasting across instances to establish significant patterns, then further questioning and refinement of those patterns as part of an ongoing analytic process.

Stage 3: Conclusion Drawing and Verification
Conclusion drawing is the process of stepping back to consider what the analysed data mean and their implication for the evaluation question at hand.  Validity concerns whether the conclusions being drawn from the data are credible, defensible and able to withstand alternative explanations.  The evaluator can form hypotheses about the project but needs to look for alternative explanations.
Tactics for generating meaning include:

• Noting patterns and themes
• Clustering cases
• Making contrasts and comparisons
• Partitioning variables

Any single perspective is partial and relevant to the respondent’s experiences and social position.
In qualitative analysis, deviant instances or cases that do not appear to fit the pattern are seen as a challenge to further elaboration and verification of an evolving conclusion.

Quality in Qualitative Analysis

• Does the analysis flow well and make sense in relation to the evaluation objectives and the data that was presented?
• Is the analysis interesting, informative, provocative?
• Does the analyst explain how and why he or she drew certain conclusions, or on what basis other possible interpretations were excluded?

It is often useful to bring in stakeholders at the point of translating analytic conclusions to recommendations for action.