Benchmarking

Up one level

Summary of contents:

Trainers' Outline Benchmarking

References and Hand-Outs Benchmarking

Exercise – The Driving Test Benchmarking

Ethics

Trainer’s outline

Benchmarking Evaluation Systems

Approach

What is benchmarking?
How can this be applied to evaluation?
What are the components of a good evaluation system?
Use of exercise to bring these components to life.
Ethical considerations

What is benchmarking

The purpose of benchmarking is to improve products and processes to better meet customer needs. The linkage of the process to customer needs is critical to effective benchmarking. In
Benchmarking is a part of the Total Quality Management system, and it relates well to other TQM initiatives. It has six steps (keyed to the Deming Cycle of continuous improvement):

Plan (Plan)
Research (Plan)
Observe (Do)
Analyse (Do)
Adapt (Check)
Improve (Act)

To practice benchmarking there must be a thorough understanding of:

The critical business processes,
The critical success factors for fulfilling customer needs, and
The best measurements will provide this information.

Hand out 1 Quality Management Principles

Hand out 2 EFQM progress

Application of benchmarking to evaluation

The crucial criteria for meeting customers needs include those of processes, which can be controlled in different situations. They are robust. In summary these resolve to the following.

Reliability
Validity
Credibility
Accountability
Replicability
Transferability
Practicability

Reliability

The evaluation system should be reliable enough to overcome external irrelevant variables.
There are three types of reliability:

Intratester – the type of data collected is not dependant on the mood of the evaluator
Intertester – the type of data collected is not dependant on which evaluator is collecting the data
Extratester – external circumstances such as whether or not it’s raining do not effect the data collected

Validity

The evaluation system should be appropriate for the evaluation to be undertaken effectively.
There are three types of validity:

Face – there is congruity between the methods used and the purpose of the evaluation. The methods reflect the atmosphere of the project.
Content – what is being evaluated is what the project is about
Predictive – it is possible to generalise from the sample or period of the project being evaluated to make predications.

Balance

There is a balance that has to be obtained between reliability and validity. A method may be highly reliable but have low validity because it is not appropriate to the issue being evaluated.

Credibility

The evaluation system is believable. It uses tried and tested approaches and methods.

Accountability

Results generated by the evaluation system can be justified and defended.

Replicability

If the evaluation were to be repeated, it would come up with the same results.

Transferability

The evaluation system could be transferred to other contexts or projects.

Practicability

The evaluation system must be usable and offer methods, which are possible to undertake in a given situation.

Trainers' Outline

References and Hand-Outs

This area contains links to external references and resources, as well as downloadable handouts

Exercise – The Driving Test