Meaningful and measurable: developing approaches to the analysis and use of information on personal outcomes within health and social care

"Policy and practice across the UK and beyond is committed to improving the lives of people who use services and unpaid carers. Over the past five to ten years there has been a particular concern to shift health and social care systems away from an exclusive focus on their own inputs, processes and outputs to give a more prominent focus to personal outcomes for people using services and unpaid carers. In this context the term personal outcomes is used to refer to both the impact and end results of services and supports on a persons life and more simply, what matters to people.

The proposed project builds on a seven year programme of knowledge exchange and service improvement that has been led by two of the applicants (Dr Ailsa Cook and Dr Emma Miller) and funded by the Joint Improvement Team in Scotland. This programme, known as Talking Points, has involved work with more than 130 organisations to support the development of outcomes focussed practice. A key finding emerging from the Talking Points programme is that limited capacity and skills in the analysis of qualitative and quantitative personal outcomes information within health and social care organisations constitutes a significant barrier to effective outcomes focussed project. Furthermore, this issue is exacerbated by the predominance of performance cultures that prioritise consistency, comparability and measurability of information over meaning.

Over the past year 15 partner organisations have worked together to develop this proposal for funding from the ESRC. The partner organisations have all been involved in the Talking Points programme and include the following academic, practice and national stakeholder partners:
University of Edinburgh (host institution), University of Strathclyde, University of Swansea
Angus Council, Bridgend County Borough Council, City of Edinburgh Council, East Renfrewshire Health and Care Partnership, Moray Health and Social Care Partnership, Penumbra, Stirling Council, VOCAL (Voice of Carers Across Lothian)
Joint Improvement Team, Community Care Benchmarking Network, The ALLIANCE, Social Services Inspection Agency.

The proposed project involves both collaborative action research and knowledge exchange elements. Specifically the academic team will support each of the practice partners to engage in an action research project focussed on the analysis and use of personal outcomes data routinely collected through assessment and review processes within their organisations. Co-ordination and collaboration between these projects will be primarily achieved through the convention of three 'data retreats' two day workshops where project partners work together to develop capacity and skills, exchange knowledge and reflect on the process.

The contribution of national stakeholder partners will ensure that the findings of this project are effectively disseminated to and implemented within a wider constituency of health and social care organisations. This will be achieved through support for a range of knowledge exchange processes, including events, dissemination of materials and staff time to work with other organisations not directly involved in the project to implement the findings in their own practice. Engagement with these partners, all of whom are actively involved in shaping policy, will ensure that the project findings influence national policy, in particular in relation to the development of national outcome performance reporting frameworks.

In these ways it is anticipated that the proposed project will bring significant benefits not only to partner and provider organisations, but to the wider health and social care system. The short term impacts of the project will be systematically evaluated through a process of contribution analysis. Findings from this contribution analysis and the project as a whole will be widely disseminated, including to academic audiences."

"Measurement of personal outcomes:
The extent to which it is it feasible to 'measure' personal outcomes is contested and subject to varying conceptualisations, which emerged as barriers to progress. In response and through collective exploration, the academic team produced briefings, since incorporated as a practical guide:
o The pressure to measure and its impact on understandings about qualitative data
o The need to reconcile different conceptualisations of personal outcomes
o Thinking about categorisation of outcomes and avoiding the risk of categorizing too soon
o Choice of scale measure in accordance with the conceptualisation of personal outcomes, with practice, and acceptability to people using the service

An agreed conclusion was that quantitative data about personal outcomes needs to be used with caveats and not in isolation from qualitative and contextual information.

Recording personal outcomes:
The importance of focusing on practitioner recording skills to generate good quality information and decision-making at individual and collective levels, was already known. However, as this issue re-emerged as a concern for all eight project partners, and featured in all of their action research projects, efforts were invested in exploring the epistemological and practical barriers involved, and in developing common criteria to support good quality narrative recording. This work has been published as a standalone practice guide, incorporating examples which cover a range of barriers identified by practitioners and representing health, social work and third sectors.

Use of qualitative data in practice settings:
Recognising the limits of quantitative outcomes data, most project partners viewed qualitative data as important for decision-making. It was however apparent that understandings of qualitative data remain limited in service settings. First, there is a widespread tendency to equate 'qualitative data' with 'story', resulting in potential misuse of individual stories in informing decision-making. Second, when working with qualitative data:
• Quantitative data sampling assumptions and criteria are carried forward, resulting in inappropriate use of large, 'random' samples;
• Uncertainty about the conditions under which qualitative findings might be applied more broadly results in myths about 'how many' cases are needed, and a lack of attention to selection criteria.
The researchers worked with partners to progress qualitative data analysis in service settings, resulting in briefings addressing the above themes and an overview of different approaches. The resulting guide presents a novel approach through tackling common misunderstandings in service settings.
Use of data about personal outcomes
It was already known that significant effort must be invested in recording practice to generate good quality data. This project confirmed the importance of the feedback loop within the organisation, in using personal outcomes data for practice development purposes, with partners identifying methods for doing this. Importantly, collaborative and appreciative approaches were found to be more effective, in contrast to the more prescriptive, top down approaches currently employed for data capture. While subject to continuing analysis, implications emerge for how the data in turn inform bottom up approaches to performance management, with novel ideas emerging about links to service planning and commissioning, representing a rich seam for exploration through future research."