Improving data reporting in adult social care
Enhancing social care data reporting accuracy and efficiency with machine learning at Skills for Care
Challenge
We seek opportunities to partner with organisations delivering truly impactful work. Skills for Care is the leading source of adult social care workforce intelligence in England. They work tirelessly to provide the government with up-to-date metrics on the state of England’s adult social care sector. This includes how many care workers there are and how they’re distributed throughout the country. A task that’s made all the harder when you realise there is no legal requirement for any privately owned care home to report on these statistics.
As you might imagine, this is a mammoth task. When we first landed, the analytics group was performing the heroic feat of collating, cleaning and analysing years of data. From tens of thousands of facilities through a semi-manual and time-consuming process. The process was taking them months to complete and they knew they weren’t utilising their data to its full potential. They knew technologies existed that could help, but implementing them required skills and experience they didn’t (yet) have. We were eager to help them with that.
Our approach
To deliver the best value, we set out to build a solution that was both easy to maintain and powerful enough to handle their vast data collection. We also needed to make sure this was something they could continue developing once we’d left. Making upskilling the Skills for Care analysts from near-zero coding experience vital to success. Once these things were in place, it was clear that the deep sector knowledge the analysts held could be put to work – improving their reporting.
Our data engineers wasted no time building a data pipeline. One that would automatically store, version, clean, join and transform – making sure all data was ready for analysis. The Skills for Care analysts leapt at the opportunity to develop new skills. They quickly became valuable partners in the development process by identifying opportunities for improvement and contributing significantly to the codebase.
We demonstrated the importance of test-driven development, the power of CI/CD and the necessity of generating effective, usable documentation throughout.
By the end of the project, the team had turned a task that usually took about a month to complete into an automated process that runs every 2 weeks and takes about 20 minutes.
While this was happening, our data scientist was interrogating their data. Uncovering trends and developing new processes designed to extract as much insight as possible. They held regular showcase sessions. Presenting new analyses to shine fresh light on the trends that Skills for Care knew to be true but had never had the data to demonstrate. By working closely with their analysts we developed new machine-learning models that enable much more granular reporting.
To make sure this bump in accuracy remains (knowing the efficacy of all models decays over time), we built monitoring steps into the pipelines. As soon as model accuracy metrics fall outside of a set range, analysts are notified and re-training jobs are triggered. Deploying a model and simply assuming it will continue to be as performant as the day you deployed it is never enough.
Results
The impact of this work cannot be understated. Through this partnership, Skills for Care has gained a wealth of new development and data science technical ability. They now possess a suite of bespoke, fully integrated AI models that deliver data-driven insights all the way to the highest levels of government. This transformation has allowed them to rethink their data collection process. And has resulted in a massive increase in data quality – compounding their improved reporting capability.
Skills for Care now perform complex analysis using version-controlled Python scripts and notebooks.
This allows them to access a whole world of open-source analysis tools and frees up their time to perform other aspects of their job.
Most importantly, this work will enable Skills for Care to produce their most accurate reports to date – starting from November this year. These enhanced reports will empower Ministers and MPs to deliver data-driven decisions about the care sector. Leveraging modern data principles and AI, Skills for Care has significantly improved the quality and impact of their reporting.
Case studies
From feedback to forecast at the Met Office
Boosting the Met Office’s insights from their users feedback with a modern analytics platform.
Streamlining data flows to improve data-driven insights in adult social care
Uncovering data-driven insights using AWS to help Skills for Care plan, monitor and fund adult social care more quickly and accurately.
