Transformation Directorate

CDU data science team

Owner

Funded by NHS England, and carried out by Nottinghamshire Healthcare NHS Trust. The project is licensed under the MIT licence, which allows for free use of the software for modification, distribution, private use and commercial use.

Background

Patient experience is fundamental to healthcare, and data on the patient experience must be collected and communicated effectively. Whilst all trusts collect Friends and Family Test data, many do not have enough staff time to fully analyse and classify each comment and item of feedback received under an appropriate theme.

Situation

Covid-19 has continued to have an impact on face to face services, shifting a large amount of delivery online, and increasing data processing needs. The backlog from Covid-19 also means that those receiving patient data struggle for time to trawl through big spreadsheets, and need data analysis tools to use their data effectively.

Aspiration

  • Stop relevant text being hidden in high volumes of information, or needing vast amounts of time to find and analyse.
  • Classify text-based feedback from Friends and Family data, making it more useful and accessible under appropriate themes, and more easy to explore by its urgency.

Solution and impact

The Clinical Development Unit (CDU) Data Science Team in the Nottinghamshire Healthcare Trust have produced a machine learning tool to label patient feedback automatically, helping staff who currently classify data ‘by hand’. It includes a dashboard that explores the data once classified, including comments, to provide appropriate and useful analysis. This is intended to support human insight, placing each item of feedback in context, not replace it.

The CDU are keeping the project free and open source to encourage a collaborative and innovative culture of openness and transparency. This is especially important around patient feedback, not just because it makes our process more robust, and opens the process by which we improve to inspection, but because it actively demonstrates how the NHS values and learns from that feedback.

We've got to learn the skills of open and they are considerable, but it’s worth it and it makes you a better programmer. That’s why I always say, My code is better open, not because it’s open – just because it has to be better.

Chris Beeley, Senior Data Scientist with The CDU Data Science Team out of Nottinghamshire's Foundation Trust.

Open source working also promotes replication, reproducibility and further development of the code. The algorithm and accompanying dashboard are already being hosted for five trusts, with recruitment of more trusts ongoing.  The tool can be run anywhere: being open sourced means it can be freely reused across sites, allowing trusts to choose pre-existing tools instead of commissioning new ones.

The tool’s component pieces (i.e., specific parts of the classification and dashboard process) are also being supplied individually, letting others pull apart the CDU’s method, learn from it, improve or change it, and put it back together again for other, different applications. This approach to modular, open source code, means that it can be reused by developers working across healthcare or government, avoiding duplication of work and reducing costs. Publishing source code under an open licence also means less chance of getting locked-in to working with a single supplier.

Functionality

The CDU is currently working in partnership with multiple NHS trusts (who hold patient feedback text) to:

  • Learn from already classified feedback to understand how different kinds of comments fall into different themes
  • Classify new items of feedback according to those themes, making the process faster and more efficient
  • Supply feedback relating to specific themes to staff more quickly

Capabilities

  • With further modification this tool could be used to classify and display themes for any text-based data in healthcare.

Scope

The tool is designed to be used primarily by service managers and those who are concerned with looking at feedback about clinical services, but it could be used anywhere, including by the public where feedback about a service is published.

Key learning points

  • Friends and Family test data is collected using a wide variety of systems and producing a robust data analysis is challenging when done manually
  • Engaging users within the NHS can also be challenging because of operational pressures. Engagement should be sought as early as possible
  • The transparency of open source allows the creation of an open community, removing a bunker style mentality, and allowing developers to share both tools and lessons learnt.
  • It can be useful to advocate for dedicated project support and resource to keep organisations involved
  • The ability to reuse is critical to adoption

Digital equalities

  • Collaboration improves project output and allows for more diversity in design: "I want them to read the code and say, this is not right, you're systematically discriminating against such and such a group by doing this, I absolutely want that." says Chris Beeley.
  • Algorithmic analysis can create error, but also helps information to not get lost in a huge volume of data. Where smaller groups struggle to be heard, good classification helps identify consistent patterns in feedback.

Give us feedback

Open Source Digital Playbook feedback survey

Page last updated: September 2022