We have used a framework outlined in the Collective Impact Forum’s Guide to Evaluating Collective Impact to evaluate the board model. This framework suggests that in the early years of a collective impact initiative, it is appropriate to focus the evaluation on:
- Social, political and economic context
- Design and implementation including:
- Common agenda: whether all participants have a shared vision for change, including a common understanding of
- Common agenda: whether all participants have a shared vision for change, including a common understanding of the problem and a joint approach to solving it through agreed upon actions
- Continuous communication: whether there is consistent and open communication between the multiple players, to build trust, assure mutual objectives, and create common motivation
- Backbone function: whether the initiative is supported by dedicated staff with specific skills to coordinate participating organisations and agencies
- Mutually reinforcing activities: whether participant activities are coordinated through a mutually reinforcing plan of action
- Shared measurement system: whether data is collected and results measured consistency across all participants, to ensure that efforts remain aligned and participants hold each other accountable.
- Learning culture – i.e. whether parties know how to learn and freely share what they know and are willing to change based on the acquisition of new knowledge
- Capacity – i.e. the financial resources, staff, knowledge, skills, expertise, and infrastructure necessary for the initiative to work as planned
The framework further suggests that once the initiative is more established and in its middle years, evaluation can focus on intermediate outcomes, which it describes as shifts in the way people, organisations, and institutions function and interact. These shifts include both behavioural changes such as professional practices and/or individual behaviour, and systems changes, such as funding flows, cultural norms, and public policy.
It is only during the initiative’s ‘late years’ that progress can begin to be measured against the initiatives’ ultimate goals (see Figure from the Guide to Evaluating Collective Impact).
What we have considered in developing this case study:
- the relationships between the relevant agencies and between board members (e.g. the quality of communication, and the sense of a common purpose)
- how well the board is supported (e.g is the supporting unit working as it should, and is the servicing department providing adequate support)
- how well the work programme is coordinated and driven (e.g. is the support unit getting the information it needs from agencies, and is it supporting the agencies in their cross-agency work)
- how well systems have been set up to support the work (e.g. are finance and resourcing systems working effectively).