News
Social Impact Bonds: new findings and fresh theory from the Government Outcomes Lab
In the wake of their recent international conference on the topic, Oxford University’s Government Outcomes Lab has published a selection of new content addressing Social Impact Bonds.
Evaluating the evaluation
DWP has just published a quantitative evaluation of the Innovation Fund (IF) pilot projects delivered between April 2012 and November 2015, a £30 million programme aimed to help disadvantaged young people re-engage in education, training and employment. On the surface, this evaluation takes what appears to be a fairly negative view, but the GO Lab analysts suggest there’s more to the story, and turn their attention to the methodology of the study:
“… our response to these findings is to immediately ask ‘what can we learn?’ Why were young people who attended SIB activities less likely to be in education, training or employment? Why didn’t young people achieve more than their NQF 1 qualification? Why were they more likely to be excluded?”
They suggest that the sample and comparator groups in the study may not have been fully comparable, leading to a bias in the results; that the surveys, taking place as they did during the first year of the programme, may have been premature; and that it is unclear whether the comparator group members were engaged with any other educational support programmes, which again may have skewed the results. The results of the qualitative evaluation, by contrast, imply that the IF programme was “a resounding success”, with “targeted numbers of outcomes met or exceeded and investments largely repaid to social investors”. The contrast between the findings of the two evaluations is somewhat paradoxical, with the implication that it may be the evaluations that are flawed rather than the IF programme itself. The scholars conclude:
“We need to be transparent about weaknesses and not see them as a sign of failure as it would be a much greater failure not to learn from our mistakes. For if we do make mistakes, let’s make them original.”
International SIB conference report
Elsewhere, the GO Labs Policy & Engagement Officer Andreea Anastasiu provides an account of the international conference on SIBs hosted at the Blavatnik School of Government in September.
“Bringing together practitioner and researcher voices is rather unusual for (what started as) an academic conference. They see the world through different lenses, they face different challenges, and they use a different type of language to describe their views. However, they both share a deep commitment to improving public services. This shared mission helped fuel two days of rich discussions, thought provoking debates, and perhaps the emergence of a new breed of expert – the ‘pracademic’, as one of our delegates from South Africa described herself.”
She goes on to discuss the potential of SIBs for the reform of public service provision, and to note that despite the coalition of practitioners and academics, there was a crucial voice missing from the conference, namely that of the service users themselves. She concludes that there remains much more to learn from the investigation of SIBs, and reminds readers that the dates for next year’s conference are already in the calendar (5th – 6th September 2019).
The SIB “culture clash”
In a further reflection on the SIB conference, GO Lab research fellow Ruth Dixon suggests that an academic framework known as the Cultural Theory of Risk offers some insights into the differing perspectives and tensions arising from competing public and private organisational values. She advocates considering the typical worldviews of stakeholders in an SIB programme: the commissioner, the service provider, the investor (or intermediary), and the front-line worker. Dr. Dixon concludes:
“… this framework highlights the fact that cooperation is difficult particularly between organisations with different traditions and outlooks. Organisational culture is difficult to alter, at least in the short term, and the interests of stakeholders with different world-views are difficult to align.
Nevertheless, there are huge potential advantages of bringing together organisations with different outlooks both in order to benefit from the insights and expertise that each can provide and to overcome the inherent disadvantages of each cultural type.”
More material on SIBs and other research can be found at the GO Labs website.