Bringing costs into focus: data-driven insights at the Met Office

Challenge

Many of us in the UK use the Met Office website and apps to check weather forecasts – often planning our day’s activities around it. From a technical side, this requires a lot of infrastructure under the bonnet. All of which help the website and apps to run, pull in the correct data, scale when it gets busy and send notifications to the public.

The Met Office wanted better visibility of its costs for these activities across all different accounts – in one easy to see place. This would help teams to understand how much each feature costs and where to make changes with data-driven insights backing them up. This is called financial operations, or FinOps. Working closely with the Met Office we set out to make its cost information visible and accessible with a new data platform. One where everyone can see it, ask questions about it and work together to save money.

Two people stand by a river, closely examining a smartphone in their hands as they engage in conversation.

Our approach

The app and services run on Amazon Web Services (AWS) using a Pay as you Go pricing model. This means information is in multiple AWS accounts that not everyone who needs cost information has access to. The process of gathering that data can also be time consuming. For the Met Office, having good visibility and tracking of costs can be a challenge.

We didn’t want to reinvent the wheel. As AWS was already providing all cost information, we simply wanted to work out how we could pull all of that information into one central location that made sense to everyone – from top level product owners to the developers working on individual system components. We set out to present the data in a bespoke way that worked for the Met Office. A way that allowed teams to answer the question ‘How much does this product/feature etc cost?‘ within a matter of seconds.

To keep costs low we took advantage of existing AWS solutions. We also used existing offerings within the Met Office, such as its business intelligence platform, as a way to visualise information.

Spending 3 days with the team we understood what they really needed from a data platform. This meant better visibility on how they use data, how to make savings and a clearer understanding of costs. We created a lightweight, low cost solution to get the data, processed it and then put it on their visualisation platform to get feedback. This allowed us to react quickly to changing business needs.

Results we achieved together

Creating stories out of data

As an example we can use the new platform to see that July running costs are higher, why is that? We suspect it’s because people are off on holiday and checking for weather updates – now the team can test this hypothesis with existing data to see exactly how traffic impacts cost.

This platform is a way to empower the team to ask specific questions in relation to cause and effect. Specifically around weather and human behaviour, with data to back up their assumptions. Teams can break down how they view costs by business area, services and different parts of functionality. This means they can clearly see where to make savings and challenge the idea that more users always means higher costs. An example might be seeing cost spikes around the time push notifications are rolling out. By putting all parts of a story together, the Met Office can review when and how they send these out. Now, cost becomes everybody’s responsibility and we begin to see a culture change.

A split image featuring two collaborative work scenes for data driven insights

Changes to new and existing features

The new data platform also means teams have better control of cost estimates for new website and app features – thereby reducing risk. By collecting data over time, teams can predict how much a new feature will cost or what savings they’ll make by changing the way they do things . This allows the Met Office to be more flexible in any changes it makes to the apps and website – helping to really measure value for money.

By making data visible, the Met Office can now switch off features it doesn’t need. Since February, the Met Office has reduced the cost of non product environments by 54% as a result of seeing cost information. And product costs are down for its main service by 20% since February. The great news is that the Met Office has had to change very little to their ways of working to achieve this. These savings come down to everyone being able to see the cost information through the platform, giving teams the ability to make simple and speedy changes.

Keeping all data within the Met Office’s existing systems was a key achievement of this project. While on paper the platform is simple – it does exactly what the Met Office wanted and needed it to.

This project on understanding costs was cost neutral for the Met Office, thanks to savings made from our previous work with user feedback and forecasting accuracy. We’re proud to be a part of helping the Met Office build a bigger data picture.