CDU data science team
Owner
Funded by NHS England, and carried out by Nottinghamshire Healthcare NHS Trust. The project is licensed under the MIT licence, which allows for free use of the software for modification, distribution, private use and commercial use.
Background
Patient experience is fundamental to healthcare, and data on the patient experience must be collected and communicated effectively. Whilst all trusts collect Friends and Family Test data, many do not have enough staff time to fully analyse and classify each comment and item of feedback received under an appropriate theme.
Situation
Covid-19 has continued to have an impact on face to face services, shifting a large amount of delivery online, and increasing data processing needs. The backlog from Covid-19 also means that those receiving patient data struggle for time to trawl through big spreadsheets, and need data analysis tools to use their data effectively.
Aspiration
- Stop relevant text being hidden in high volumes of information, or needing vast amounts of time to find and analyse.
- Classify text-based feedback from Friends and Family data, making it more useful and accessible under appropriate themes, and more easy to explore by its urgency.
Solution and impact
The Clinical Development Unit (CDU) Data Science Team in the Nottinghamshire Healthcare Trust have produced a machine learning tool to label patient feedback automatically, helping staff who currently classify data ‘by hand’. It includes a dashboard that explores the data once classified, including comments, to provide appropriate and useful analysis. This is intended to support human insight, placing each item of feedback in context, not replace it.
The CDU are keeping the project free and open source to encourage a collaborative and innovative culture of openness and transparency. This is especially important around patient feedback, not just because it makes our process more robust, and opens the process by which we improve to inspection, but because it actively demonstrates how the NHS values and learns from that feedback.
We've got to learn the skills of open and they are considerable, but it’s worth it and it makes you a better programmer. That’s why I always say, My code is better open, not because it’s open – just because it has to be better.
Chris Beeley, Senior Data Scientist with The CDU Data Science Team out of Nottinghamshire's Foundation Trust.
Open source working also promotes replication, reproducibility and further development of the code. The algorithm and accompanying dashboard are already being hosted for five trusts, with recruitment of more trusts ongoing. The tool can be run anywhere: being open sourced means it can be freely reused across sites, allowing trusts to choose pre-existing tools instead of commissioning new ones.
The tool’s component pieces (i.e., specific parts of the classification and dashboard process) are also being supplied individually, letting others pull apart the CDU’s method, learn from it, improve or change it, and put it back together again for other, different applications. This approach to modular, open source code, means that it can be reused by developers working across healthcare or government, avoiding duplication of work and reducing costs. Publishing source code under an open licence also means less chance of getting locked-in to working with a single supplier.
Functionality
The CDU is currently working in partnership with multiple NHS trusts (who hold patient feedback text) to:
- Learn from already classified feedback to understand how different kinds of comments fall into different themes
- Classify new items of feedback according to those themes, making the process faster and more efficient
- Supply feedback relating to specific themes to staff more quickly
Capabilities
- With further modification this tool could be used to classify and display themes for any text-based data in healthcare.
Scope
The tool is designed to be used primarily by service managers and those who are concerned with looking at feedback about clinical services, but it could be used anywhere, including by the public where feedback about a service is published.
Key learning points
- Friends and Family test data is collected using a wide variety of systems and producing a robust data analysis is challenging when done manually
- Engaging users within the NHS can also be challenging because of operational pressures. Engagement should be sought as early as possible
- The transparency of open source allows the creation of an open community, removing a bunker style mentality, and allowing developers to share both tools and lessons learnt.
- It can be useful to advocate for dedicated project support and resource to keep organisations involved
- The ability to reuse is critical to adoption
Digital equalities
- Collaboration improves project output and allows for more diversity in design: "I want them to read the code and say, this is not right, you're systematically discriminating against such and such a group by doing this, I absolutely want that." says Chris Beeley.
- Algorithmic analysis can create error, but also helps information to not get lost in a huge volume of data. Where smaller groups struggle to be heard, good classification helps identify consistent patterns in feedback.
Give us feedback
Disclaimer
These case studies summarise user and patient experiences with digital solutions along the relevant care pathway. Unless expressly stated otherwise, the apps and digital tools referenced are not supplied, distributed or endorsed by NHS England or the Department of Health and Social Care and such parties are not liable for any injury, loss or damage arising from their use.
All playbook case studies have either passed, or are currently undergoing the Digital Technology Assessment Criteria (DTAC) assessment.
Please note the full legal disclaimer: NHS England playbook disclaimer
Page last updated: September 2022