Types of evaluation

Approach

• Progress through main types of evaluation
• Indicate who is involved and why
• How the stakeholders make a difference
• Return to the necessity to structure and organise

Types of Evaluation - The Big Six!

1. Product evaluation - Fitness-for-purpose of `things’
2. Personnel evaluation - Selection, recruitment, performance appraisal
3. Performance evaluation - Assessment and examination esp. in education
4. Programme and Programme evaluation
   
5. Project evaluation - for collection of projects
6. Proposal evaluation - Applications especially for funding
7. Policy evaluation - Also strategies, plans and other intangibles

Who’s who in the evaluation process?

Accountability model	Developmental model
Paymasters	Policy makers
Promoters	Providers
Participants	Practitioners

There is a near, but not exact, match between the two models. For example, policy makers can also be paymasters but their functionality is different or they may actually be two different organisations. ‘Providers’ will almost certainly include the project promoters but will also include any agency to which they sub contract any part of the delivery.

Accountability model

• Tends to be retrospective not prospective
• The paymasters are concerned that the project has been cost effective, cost efficient and valid.
• The promoters are responsible for its delivery to participants and accountable to the paymasters.
• The response and satisfaction and improved performance of participants as the end recipients provide the evidence to justify the investment.

Developmental model.

• Tends to be prospective rather than retrospective
• Concerned with the learning output, which may be expressed as policy recommendations, lessons learned, key issues for the future, an agenda for action etc. These may be targeted at policy makers, at promoters (at an organisational or institutional level) and at the community of future practitioners.
• Participants, in the sense of direct beneficiaries, trainees, students, or other end users have been removed simply because, in summative terms, the lessons learned are unlikely to impact on that group at that particular time but will benefit future participants.

Who will be involved?

In both cases it is a useful exercise for project staff and evaluators to identify clearly who is potentially going to be involved in the evaluation process and to tag these individuals or groups to one of the groups mentioned above.  There may well be more than one group of people in each category.
`Paymasters’ may include the sponsoring institution, one or more external funding agencies, an executive body or an intermediary organisation.
 `Providers’ could include internal project staff, trainers, external consultants, sub contractors, - each of whom has a partial accountability for the delivery.
‘Participants’ defines all those who may be direct beneficiaries or end users of the particular project and may be individuals or groups. The parameter is that this group is not directly responsible or managerially or financially accountable for the project -although they may have personal or collective responsibility for some aspects of it (for example, students being ‘responsible’ for their own learning).
‘Policy makers’ or policy influencers may operate at a transnational, national, regional, local or institutional level.
‘Promoters’ implies sponsoring bodies or agencies, who may be involved in the future in similar activities.
 Practitioners embraces the wider professional community of those directly involved with the delivery of work in the field.
For each model, in a total systems evaluation environment, all of the groups need to provide data and receive feedback from each of the groups. Pragmatics usually make this impossible. It is, therefore, essential at an early stage that evaluators and project managers identify the key groups in each category and clarify the key information providers and key information recipients.
Failure to do so can result in ambiguity about both the organisational and structural ‘level’ of the evaluation (i.e. the level of detail or desegregation) and also the form and content of the evaluation deliverables.

Progress through contents of handout to show different roles
Hand out 1 Consumers vs Stakeholders
Show a special case for empowerment
Hand out 2 Basic Principles of Empowerment Evaluation
Go through the ‘success’ concept to show how it affects the whole process