Measuring impact can help organisations regain power
Howard Lake | 12 December 2011 | Blogs
Impact measurement is often seen as something that is imposed on charities and voluntary organisations by their funders or state commissioners. But while organisations are often required by such stakeholders to evaluate their work, this is not the whole story.
Our research at the Third Sector Research Centre has shown that measuring impact can be part of an entrepreneurial process aimed at gaining control over how the organisation is assessed and viewed by the public as well as funding stakeholders.
Our research looked at what motivated organisations to carry out impact measurement and found that meeting expectations of funders and commissioners was certainly among the most powerful. Many said that they were required by funders or commissioners to measure their impact, and in this way it was a tool that enabled external stakeholders to wield a certain amount of control over organisations.
But despite this top down control, organisations create room to manoeuvre within the process of measurement itself as well as in how they present its results. Any process of impact measurement, whatever tools are used, involves decisions about what indicators to use, what data to collect and how, which stakeholders to consult and what information to publish.
Some organisations we spoke to did find the process of impact measurement burdensome, but others found that it helped them to convey messages about their organisation and communicate their achievements. Others had originally embarked on impact measurement due to requirements of funders, but found that the process had been positive, enabling them to better communicate their work to both existing stakeholders and potential new funders and supporters. Some organisations preferred to conduct evaluations themselves as they felt this gave them more control. However, this was weighed against what some people saw as the additional credibility gained by an external impact measurement.
Of course, organisations are also limited by resource considerations, and many of the organisations carrying out external impact measurements were doing so because they had been offered them for free.
Variations in the approaches to evaluations can of course raise questions about the legitimacy of findings. Organisations are keenly aware of this, which was evident in how some reflected on measurement results that seemed ‘too good to be true’ and understood that in order to be credible they would have to adjust and re-assess these results before they were published. It is also worth noting that evaluation reports often contained little information about failure or negative impacts. While it is wrong to see evaluations as neutral reports of ‘reality’ it is equally wrong to draw the conclusion that impact measurement is solely used with the intent to produce the best possible results.
Organisations are crafting how they present social value to others. While they do so with the intent to influence stakeholders in ways that would be to their benefit, they are also concerned that they maintain trust in their results and avoid suspicion of inflated measures.
The flexibility that organisations have in the evaluation process is important. It enables them to regain control and ownership of what is often seen as an imposed activity. Of course, a certain amount of bias is inevitable. In order for impact measurement to be meaningful and useful, it is important to recognise this. Encouraging greater transparency, including regarding negative impacts, procedures and systems of external validation can help ensure that it is not seen as simply a marketing process. Perhaps most importantly, those who read and use the results of evaluations need to understand how they are constructed.
You can download the research report ‘Social impact measurement as an entrepreneurial process’ by Malin Arvidson and Fergus Lyon from the TSRC.
Dr Malin Arvidson is Senior Research Assistant, Third Sector Research Centre, University of Southampton
About Howard Lake
Howard Lake is consultant editor of UK Fundraising. He founded the site in 1994 and successfully sold it in 2022. As director of Giving X Ltd he is exploring growing giving on a massive scale.
He is the founder of Fundraising Camp and co-founder of GoodJobs.