Trainer’s outline

Data Analysis Quantitative

Approach

• Review performance indicators
• How they may be measured
• Go through the types of questions that quantitative analysis answers
• Deal with causation, dependent, mediating and independent variables
• Overview the role of statistics

Measuring systems and Performance Indicators

Performance indicators define the type of quantitative evidence that will be needed to answer evaluation questions.
Questions asked through quantitative indicators:

• Are program implementation objectives being attained? If not, why not?
• What types of things were barriers to or facilitated attaining program implementation objectives?
• Are participant outcome objectives being attained? If not, why not?
• What types of things were barriers to or facilitated attaining participant outcome objectives?

Hand out 1 raises the points.

Achievement of objectives

In evaluation, indicators relate to success criteria of the programme.    Indicators apply to quantitative measurement.  For qualitative measurement the term, “descriptor” is used.  Performance indicators relate to the effectiveness and efficiency of outputs, results and impacts.  These indicators are expressed as numbers or percentages.  For example the actual numbers achieving qualifications can be divided by the number expected to provide an indicator of effectiveness. 
Examples of performance indicators are:

• Number of beneficiaries achieving a qualification
• Number of beneficiaries entering employment
• Number of beneficiaries entering further training
• Percentage of beneficiaries who are women, ethnic minorities, disabled, etc.
• Number of employees receiving more than XX hours of training
• Number of businesses assisted
• Number of new business plans generated
• Percentage of businesses with increased sales
• Number of new jobs created
• Percentage of beneficiaries who are satisfied or very satisfied with product or services
• Number of partners involved in new networks
• Average number of hours of advice and guidance delivered to beneficiaries
• Number of jobs saved
• Number of childcare places offered
• Rate of completion of the project – completers/starters
• Number of beneficiaries who have progressed their situation after taking part in the project
• Percentage of beneficiaries hired after placements

Basis of the quantitative analysis

The indicators need to be set according to what the information needs of the stakeholders are.  Often stakeholders will want to make decisions about further development of the project and need evaluation data to help them do this. 
The system of indicators will reflect the objectives of the project.  Evaluators need to check what sources will provide the data. 
If indicators have been set at the start of a project, the project’s monitoring systems need to be compatible with the data that needs to be collected.
Dependent variables are your measures of the knowledge, attitude, or behaviour that you expect will change as a result of your program.
Independent variables refer to your program interventions or elements. For example, the time of data collection (before and after program participation), the level of services or training, or the duration of services may be your independent variables.
Mediating or conditioning variables are those that may affect the relationship between the independent variable and the dependent variable. These are factors such as the participant's gender, socio-economic status, age, race, or ethnicity.

Numbers and statistics

Once collected figures need to be analysed.  Often this is simple comparison of one set of figures against another.
Sometimes figures show clear differences.  One group may have got jobs twice as frequently as another group.  Some thing may be three times as costly to deliver as something else.
For one to be sure that these differences mean something, it is important to ensure that:
The data relate to real activities or results
The numbers involved are big enough and representative of the whole group
Beyond this statistical tests provide results, which demonstrate ‘significant results’ i.e. they are not likely to occur by accident.
This is not a statistics training course.  There are simple computer packages or outside agencies which can perform these statistical analysis.