Using Evidence for Impact in the NZ Social Sector

Speaker: Clare Ward, Superu Chief Executive

View the presentation from the seminar: Click here to see the Prezi slides

Key points from Clare’s presentation

We are all here to make a difference for New Zealand’s vulnerable children and families. We all have our parts to play in a wider system of action and decision-making that aims to improve social outcomes. Some of us generate evidence about social sector issues, others of us deliver services, and yet others of us develop policy or make funding or investment decisions.

Social investment focuses on:

• Outcomes and investment for results with the individual, family or community at the centre
• Prevention and early intervention for measurably better results in the longer term
• Flexible and responsive to need, and in many cases enable a differentiated response
• Creating a system that learns over time about what works, selects successful approaches and winds down the approaches that fail to achieve good results
• Aligning institutional settings such as accountability, funding, legislation, performance measures, targets, as well as governance with the outcomes sought and the desired way of working
• Data futures partnership - an independent group helping lead the development of the data-use system in New Zealand.

Superu’s purpose is to:

• Generate evidence that helps us understand complex social issues and what works to address them
• Make evidence available and translating it so that it is useful to decision-makers
• Build the capability of the social sector system to use evidence in decision-making

Using evidence for impact involves a progression of different users in the social sector working in a cycle, Superu is:

• Studying the evidence needs of different user’s - policy-makers, funders, and service providers (or NGO’s)
• Mapping their evidence-needs journeys –understanding where their key decision points are and how can these be supported?

To date, Superu has worked with funders in the social sector to understand their environment and their evidence use journey. We found that for funders, evidence is being used for 3 key functions: evidence as intelligence, evidence for assessment, evidence to demonstrate impact.

- View the ‘Funder’s evidence map’ (coming soon to the Superu website)

Superu supporting ‘Evidence as Intelligence’

­ Helping decision-makers understand different populations and their needs as well as complex social issues:

• Research from Growing Up in New Zealand data: alcohol and pregnancy – understanding the NZ context
• Youth Mental Health Project evaluation
• Family and whanau wellbeing research programme
• Commissioning research for decision-makers, for example:
  • Institute of Environmental Science and Research (ESR) to developed a proof of concept systems approach to measuring the effectiveness of the ‘whole-of-system’ response to prevent family violence
  • Ministerial fund commissioned research and evaluation in relation to questions posed by Social Sector Portfolio Ministers:
    • Neighbourhood Social Mix and Outcomes for Social Housing Tenants: Rapid Review

­ Developing research infrastructure and approaches that enable these sorts of questions to be explored:

• Growing Up in New Zealand – a longitudinal study. The Crown funding of the core GUINZ study and data collection is managed by Superu.

Superu supporting ‘Evidence as Assessment’

- Deciding whether to invest or not and on what basis

Resources and tools to support evaluation practices and processes:

1. Quick guide evaluation standards summary tool
2. Evidence checklist to assess funding applications
3. Evaluation guide for funders - how to work with providers to develop useful evaluation
4. Evaluation planning funding applicants
5. Finding and appraising evidence for what works

Superu supporting ‘Evidence as Assessment’

- Demonstrating impact and making change based on what was learnt and then sharing this with others

Superu is helping to build and facilitate a culture of learning within the sector through:

­ Making research more accessible:

• The Hub - a data base containing government-funded social science evidence research and evaluation - http://thehub.superu.govt.nz/.
• The publishing protocol sets out good practice for publishing social science outputs
• Seminars and conferences

­

Growing the evidence base about what works in key areas:

• Evaluation of the Prime Minister’s Youth Mental Health Project
• Development of evaluation strategies and frameworks, for example the Tamaki Regeneration for Treasury and the Tamaki Redevelopment Company; and for the Children’s Action Plan for the Children’s Directorate
• Enabling social services providers to participate in funded external evaluations, to find out what works in delivering improved outcomes for priority groups in priority locations. Superu is currently working on an ‘NGO Fund’

­

Making research easier to understand by synthesising knowledge into attractive easy-to-read products:

• Our What Works series synthesises what we do and don’t know about a specific social sector topic, for example:
  • What works: Improving outcomes for children of gang-involved parents
  • Our In Focus series, which is designed to inform and stimulate debate on specific social issues faced by New Zealanders, for example:
    • In Focus: Family resilience
    • Our At a Glance series use infographics to illustrate research findings or key information about a priority topic, for example:
      • At a Glance: Changes in alcohol consumption during pregnancy.

Clare’s reflections on using evidence in the NZ social sector:

1. We know more than we use – we need to be better at accumulating and sharing knowledge
2. What do we mean by client-centred and what does this mean for the system?
3. We need to focus on existing spend as well as new money
4. Culture and capability are critical
5. Take a broader view of evidence
6. We are a bit ad-hoc - more structured processes could be useful
7. Pay more attention to implementation
8. The problem-solving model of research into decision-making is optimistic

A ‘Standard of Evidence’ is an important component of social investment, and Superu is currently exploring International examples and considering how New Zealand’s standard of evidence should incorporate both Māori and western approaches.

Social investment requires us all to use evidence. Superu has a key role to provide strong leadership and lead this culture change.