From an office in Wellington, a company co-founded by a former prime minister is using big data to work out the social value of charitable interventions.
Time is on the x-axis; on the y-axis is wellbeing.
ImpactLab chief executive Maria English is showing me the graph to help explain the concept of social value, something her company seeks to quantify in dollar terms for charities to show the impact of their interventions.
Life is inherently messy, but as English puts it: “There’s a strong empirical basis for understanding some elements of what drives vulnerability.”
Having a parent in prison or a mother with no high school qualification, for instance, are among the factors linked to a young person's increased chance of adverse outcomes later in life.
The graph shows this kind of knowledge as two hypothetical life paths: one where someone doesn’t come into contact with a service offered by a charity, perhaps giving them housing, and another where they do. The difference between them is the social value created by the intervention.
Through its product GoodMeasure, ImpactLab offers a social return on investment (SROI) figure for each of the charitable programmes it assesses. So far, more than 150 groups have gone through the rigorous six-month process. The cost? About $25,000.
“Basically, what social return on investment says is for every dollar we put into this programme what’s the estimated value returned to society in terms of better lives,” English says.
Avoided government costs, changes in income and more intangible factors, like better health and social relationships, are all taken into account. English concedes there are things ImpactLab can’t measure, but says the wealth of longitudinal studies and rich government data in New Zealand means there’s a lot it can.
Is she a believer in big data, I ask?
“I’m a believer in connecting decisions to information about the impact of those decisions, and I think we need to combine different kinds of information, particularly the human stories and the hard data that supports them.”
Social investment
Maria English is the daughter of Bill English, the former prime minister and social investment advocate who co-founded ImpactLab in 2019. A deep thinker, Bill English outlined his views on social value and the importance of ‘feedback thinking’ – the reworking of policies in response to evidence – in a chapter he contributed to a recent book on policymaking.
In it, he talks about the ‘million-dollar kids’, a cohort identified by the previous government who were predicted to require $1 million in government services – including healthcare and justice system costs – from age 7 to 35.
“These numbers showed us a story we didn’t know before,” the former prime minister wrote. “Once aware, how could anyone stand by while these children face endless misery?”
Knowing the cost of inaction, in other words, justifies a tailored approach to improve the lives of these children, one by one.
Maria English also references the million-dollar kids in our interview. It comes up in the context of costs and whether ImpactLab assesses if charities are delivering services efficiently – a big topic in the charitable world and the focus of organisations like GiveWell, a non-profit founded in the United States to assess which charities are saving or improving the most lives per dollar donated.
“The power of the social return on investment approach is it shifts the conversation from a conversation about cost to a conversation about value,” English says.
If two charities are both working with highly vulnerable clients, it’s possible the one offering a more expensive and intensive intervention could be delivering greater value. Charities working with people who have low trust in mainstream social services, for instance, might have to invest heavily upfront to win their confidence.
“We know of programmes that will make 50 calls to a family before they will engage with a nurse,” English says.
"In the cost view of the world, you go ‘that’s just cost’. In the social value view of the world, you go ‘that’s an investment to get engagement to create impact’.”
What’s in it for charities?
Frontline workers at charities assessed by ImpactLab have cried when they’ve been presented with the results quantifying the value of their work, English says.
Beyond self-validation, the primary reason for a charity to do a GoodMeasure analysis is to show the value of its interventions, with ImpactLab's hard data used to complement personal impact stories.
“We do a process alongside the charity to really understand what they do, how they do it, and the impact they see it creating,” Maria says.
“We collect data from the charity, we then link that with public and academic data to create the metrics.”
A 2020 assessment of a Howard League programme helping youth offenders get their driver's licence, for example, found it delivered $4,509 in social value for each successful participant (measured by things like reduced offending and increased employment) on costs of $1,381, meaning every $1 invested in the Howard League delivers $3.26 in social value through the programme.
A GoodMeasure assessment also helps charities identify what’s working and where there’s room for improvement, English says. Undergoing a rigorous assessment might help a charity decide to keep better data in the future, for instance, or reflect on basic questions like who it is they’re trying to serve.
As an example, English mentions a charity ImpactLab worked with that offered a child therapy programme. It found a large number of children it was seeing were neurodiverse. In response, the charity implemented a new training programme to help staff work better with neurodiverse children.
For funders, the motivation to commission a GoodMeasure assessment is more altruistic. They like being able to help charities they support to show their impact and identify opportunities to improve, English says. It also helps them understand the impact their investment has.
Theoretically, funders should also be able to use ImpactLab assessments to work out which charities they should support to deliver the most social value per dollar invested.
ImpactLab, which has the tagline “Do Good, Better”, describes the social return on investment figure it assigns to services as comparable on its website. However, English qualifies this by saying data quality can vary between charities.
The three pools
When ImpactLab does its assessments, it draws on what English calls three pools of knowledge: academic data, including evidence about the effectiveness of different programmes, data and frontline knowledge from the charity being assessed and public data.
A particularly important source is Statistics NZ and its Integrated Data Infrastructure dataset, which has anonymised data on millions of New Zealanders from across government agencies.
ImpactLab's founders, Bill English, philanthropist Fran Wyborn, Kylie Reiri and policy analyst Emily Mason, who worked on social investment during the last National government, share a common interest in how to make investment work better for communities, Maria English says.
“We spend a lot of money with the intention that it has a positive impact on people’s lives, and particularly an impact on the lives of the most vulnerable people in our society,” she says.
“But the question is, ‘does our money have the impact we want?’, and maybe even more importantly, ‘how can we make decisions about where we invest in the future to increase that impact and make more of a difference?’.”