Transformation Directorate

User-centred service design via free text analytics

Online cancer education platform for healthcare professionals

Problem to be solved

The NHS Long Term Plan sets out a clear course to delivering more person-centred care. ‘What matters to someone is not just what the matter is with someone. Individuals’ values and preferences differ and ensuring choice and sharing control can improve care outcomes. The London Borough of Islington wanted to investigate how it could advance this work by tapping into the intelligence existing in free text data that is produced by care staff every day.

The project 

Social workers and care workers generate large quantities of free text data from their written records of visits to clients. As opposed to structured data such as a person’s name, age or address, free text data entry has no field or format constraints and consists of prose-style notes recorded by professionals on their visits to and conversations with individuals. Whilst this kind of data has traditionally been difficult to analyse at scale, it provides a rich source of information on what individuals say they need and want.

The council identified 3,000 residents aged 65 and above who were long-term users of adult social care, and whose free text data it wanted to analyse and link with other datasets. It  conducted an extensive engagement exercise to inform these residents about the project and to tell them how their data would be used. A resident information leaflet has been designed to inform all people using adult social care services about the project and to give them the option to opt out, all as required by the General Data Protection Regulation (GDPR)

With funding awarded through an NHS Digital Social Care Pathfinders Programme grant and working together with NEL Commissioning Support Unit, the council put together an application to the Confidential Advisory Group (CAG) to be able to process free text data and link to GP and hospital data. This application has been approved.

The council is now collating, pseudonymising and processing this free text data together with other adult social care data. By November 2020, the plan is to hand over this curated data to a team at the London School of Economics who will apply Natural Language Processing techniques to draw out meaningful patterns from the data. This free text analysis will focus on gaps in service provision and where services could be better designed around people’s needs.

Armed with this advanced analysis, the council and its partners anticipate being able to work in more integrated ways and deliver more effective outcomes set out by the Better Care Fund. In practice, this will mean:

  • a better understanding of opportunities for preventing ill-health and social isolation
  • shifting hospital care into the community
  • addressing delayed transfers of care from hospital back into the community 

Future plans involve rolling out this project across the North Central London Clinical Commissioning Group (CCG) footprint.

Lessons learned

The biggest challenge has been sharing data between organisations across health and care. Securing widespread agreement on what can be shared and what can’t be shared has been time-consuming. It’s been hard work putting in place the right mechanism for de-identifying data before it’s shared, that enables it to be linked to records in other datasets. To overcome this challenge, the project team focused on getting the right people in the room at the right time, not making assumptions about how things should be done, encouraging open conversations, and then finalising details with explicit agreements.

To find out more about this project, contact Mahnaz Shaukat on email:

Online cancer education platform for healthcare professionals

Explore Further