Reducing Risk and Cost of Failure Using the Potato Cannon Pattern for Australia’s largest gaming company
The opportunity at a glance
Automation reduced what was a 4-month manual process to under a day’s work, reducing risk and accelerating efficiency for an ASX listed gaming and wagering company.
Our customer, an ASX100 listed company, is one of the largest publicly listed gaming and wagering organisations globally.
As one of the few integrated gambling and entertainment companies in the world, their iconic brands and stable of subscription television channels ignite passion and excitement in millions of customers each year.
The customer was, as is common, very focused on their waterfall approach to the delivery of software. Across multiple delivery teams, the expectation was to build and deliver new features frequently through a primarily manual process. A significant challenge with this software delivery method is that functionality is grouped together and released in large batches with the intention of mitigating the risks of change by reducing its frequency. In a typical waterfall pipeline the following workflow generally occurs (or a variation of):
Our customer’s software build and test process is highly customised for each component; builds are performed on independent physical systems, each configured differently, and copied manually from dev, to test, and then onward towards production. This introduces a high degree of variability in delivery times, adds system library-specific version dependencies on built artifacts, reduces visibility into the current state and history of each artifact, and promotes key-person risks thanks to the specific domain knowledge required to administer each build system.
As a result, they wanted to reduce the variability in the software artifact generation process, in order to achieve more predictable delivery lead times, improve software quality, improve confidence in the released software, and mitigate the risk of specific physical systems and key-person domain knowledge.
A management decision was made to hire consultants within the delivery teams, bringing in specific skill sets in a co-sourced delivery model. During this exercise, the Cevo technologists on site identified a pattern in the process which lead to an increased risk of failure causing an increased cost of failure.
In the current state, at each step in the process a separate team was expected to test the code, with any issues bringing an increase in the average time to fix as it moved through the chain. This increased cost had heavily influenced the decision to combine and more carefully plan releases as frequent releases became a less commercially viable option. This is a common occurrence in organisations which is often overlooked. The Cevo team saw this as a valuable opportunity to introduce technology and process changes which would allow our customer to release software more frequently with reduced risk and at a significantly reduced cost.
A solution was identified and tested to automate and standardise the build process. This would allow failure to be identified in the first step, reducing the amount of time required to fix it. Once identified and tested, an automated process based on a pattern known as Potato Cannon, would access the changes in the code repository, test them and deploy to the relevant servers for further testing.
The results are significant. The Potato Cannon pattern, which can be applied across any organisation with the same challenges, reduced the previously manual 4-month process (developing the code through to deployment to a testing environment) to under a day’s work. The idea behind the process was to shift as much of the test planning and documentation to the start of the process to reduce the cost of failure. It was built in a way that makes all parts of the system replicable to mitigate the risk of redundancy.
The Potato Cannon pattern has been implemented in one area of the business and has proved to be extremely successful in raising efficiency and reducing potential costs of projects allowing the business to implement changes more frequently. It can be reused across all delivery teams to achieve the same result.