Software delivery performance – Measuring DevOps

INDUSTRY

Wagering / Gaming

CUSTOMER OVERVIEW

Our client, an ASX100 company, is one of the largest publicly listed gaming and wagering organisations globally.

As one of the few integrated gambling and entertainment companies in the world, their iconic brands and stable of subscription television channels ignite passion and excitement in millions of customers each year.

Case Study

BUSINESS CHALLENGE

Our customer has made significant investments in project delivery teams to evolve their DevOps maturity. As a result, the management team wanted to measure the impact of these new practices as well as identify the best area to focus on next.

A popular way to gauge software delivery performance is to look at the metrics covered in the “Accelerate” school of thought. This has the potential to become the base for measuring in-house achievements as well as highlighting possible growing points in the future. However, with multiple development teams working across different projects it can be difficult to track all of these metrics. Even more so, if the software is delivered across multiple platforms.

Case Study

SOLUTION

The origins of the solution was a continuous integration and delivery (CI/CD) dashboard proof-of-concept (POC), built within the Platform Engineering team and including members of Cevo.

This early solution utilised Elasticsearch, Logstash and Grafana dashboards to centralise all pipelines across the organisation into a single accessible location and collect key metrics around builds, passes/failures, duration, etc.

After showcasing the POC to a wider internal audience, the value of central, meaningful performance data was realised (e.g. key build/test/deploy flaws were immediately visualised, such as failing component tests consuming up to 800% more resources and time). 

This triggered the organisation to engage Cevo to assist in delivering the POC to production – a “performance hub” to monitor and analyse the development efforts and operational stability of their systems.

In collaboration with our Business Intelligence sister company Exposé, a bespoke data model was designed which collects data from project delivery, change management, and incident management systems. This data model was developed using the Power BI platform since it best suits the organisation’s in-house skill sets.

The BI platform does not necessarily replace the operational reporting of each individual system but rather brings the separate data sets together to add organisational context to them as a whole. By keeping the details behind the dashboard, it allows the users to easily link back to the source systems for each record.

The four key metrics quantified were: avg lead time, deployment frequency, change failure rate, and mean time to recovery. Each data point is aligned to a team and a date which empowers the client to analyse their software delivery performance from a structural perspective (team, division, etc.) or chronologically.

Case Study_1

BENEFITS

Defining the key metrics and curating the important data from development platforms allows our customer to have an up to date overview of the software delivery and its impact on the organisation’s systems.

Specific business benefits of the project have included:

  • Quantified and measured software delivery performance.
  • Comprehensive overview of all the processes related to software delivery.
  • Ability to monitor the impact of changes after the fact.
  • Individual delivery platforms do not necessarily have to be integrated with every other system in the process. The BI platform becomes the hub of contextual information.
  • Users can drill into detailed records and link back to the source system to view full details within the source systems’ context.
  • An interactive visual analytic that can be used by management or team members alike.
  • Triggered notifications for critical thresholds.