Aratohu: Te arotake kaupapa mahitahi Guidance: Evaluating collaborative initiatives

Te Kawa Mataaho Public Service Commission would like to acknowledge the work of Isabelle Collins, Andrea Knox and Viv Smith of the Social Policy Evaluation and Research Unit (Families Commission) who originally developed this material for us in 2017.

Evaluation is the systematic determination of value. Evaluation looks at an activity and asks:

• what changes is it making?
• how valuable are those changes?

Evaluation is not performance monitoring but it uses performance monitoring data. While monitoring measures change over time, evaluation goes further, looking at why changes occur, whether changes are attributable to the activity, and the value of the changes.

In relation to system architecture and design, evaluation can look at:

• what contribution a structural change or joint initiative made to target outcomes and unintended outcomes
• the value for money, cost-effectiveness, or cost-benefit that has been realised from the structural change or joint initiative
• how, and how well the structural change or joint initiative has been implemented.

With this knowledge, we are better able to identify and prioritise next steps, and to make further improvements.

Evaluation should be designed into the structural change or joint working initiative at the outset.

It can be tempting for agencies to delay evaluation in favour of getting on with implementation, especially if the new initiative is urgent, but this can be a false economy. What you learn from the evaluation’s set-up and early findings will help to improve the initiative’s design, and to identify and address problems that arise. An early start to the evaluation will also show what data should be gathered as the initiative progresses, to show effectiveness.

There are some principles of evaluation quality and ethics.

• The Evaluation Standards for Aotearoa New Zealand describe principles for conducting evaluation with integrity (Superu & ANZEA, 2016).
• The Australasian Evaluation Society Guidelines for the Ethical Conduct of Evaluations cover ethical behaviour and decision-making in evaluation (Australasian Evaluation Society, 2013).

When beginning an evaluation, you should investigate the ethics requirements at your agency. Evaluations that will collect information about individual people will probably need ethics approval. Even if approval is not required, you should still think about how to manage any ethical issues that are likely to arise. There are a number of committees responsible for oversight of these issues, such as the Health and Disability Ethics Committees.

Home — Health and Disability Ethics Committees

This glossary uses a mix of legal definition, conventional interpretation and additional description to help you understand system design and architecture terms.