MYOB is a leading provider of online business management solutions. MYOB makes business life easier for approximately 1.2 million businesses across Australia and New Zealand by simplifying accounting, payroll, tax, practice management, CRM, job costing, inventory and more.
Cashbook is a ledger product for accountants built for and running on AWS. Cashbook is integrated with MYOB’s practice management suites and has turned out to be one of MYOB’s most profitable applications. It was developed as a stepping stone for customers as they move to MYOB’s cloud-based SaaS solution, and as such the lifespan was only ever seen as short term. Due to an increase in use and migration de-prioritisation, the deprecation and retirement date for Cashbook keeps getting delayed.
Over time, some of the system dependencies and functions became out-dated, creating risks as well as the potential to render the deployment pipeline non-functional. MYOB determined that the risks required mitigation.
MYOB also wanted a clearer picture of how long Cashbook would survive under a forecast usage increase in the short to medium term.
The above mentioned challenges were deemed as needing to be solved urgently – however internal teams had other existing priorities and as such there were no available resources with the required skill sets.
MYOB engaged Cevo’s AWS and cloud engineering skills and experience to alleviate the capacity gap, and assist in meeting their objectives.
The engagement consisted of three parts:
Part 1: Upgrade Buildkite Configuration:
Cashbook was an early adopter of Buildkite, when the orchestration function was moved from Jenkins. Since this migration, the MYOB standard Buildkite configuration has changed, and as a result Cashbook’s Buildkite configuration needed updating.
Cevo worked with the Cashbook team to uplift the existing Buildkite pipelines to align with the current standard configuration. This required a process of review and validation with incremental changes to ensure that the new pipelines not only aligned with the updated standard, but continued to deliver the outcome as expected by the Cashbook team.
Part 2: Retire ageing Jenkins instance:
Although build and deployment orchestration was moved from Jenkins to Buildkite, Jenkins was still running a number of scheduled or manually triggered jobs, some of which were crucial to other parts of MYOB. The Jenkins server was suffering periodic outages, was in need of upgrading, and had become out-dated as the solution for its current function, and so these jobs needed to be migrated elsewhere.
A design decision was made not to rely on any third party tool for the replacement Jenkins jobs, so Buildkite was not used as the job scheduler. Most of the jobs were migrated from Ruby rake jobs to Node.js Lambda functions, packaged using the Serverless Framework. ECR Lifecycle Policies were also used. As some of the lambda functions required access to the RDS database and S3, the AWS networking needed to be updated to accommodate this. The VPC CIDR range was fully allocated, so this was increased by adding a secondary CIDR range. Public and private subnets were added to support the VPC lambda functions.
Part 3: Revisit the existing load testing and load-based failure thresholds:
The aim of this work was to give MYOB confidence that Cashbook would remain functional without any re-architecting until after the probable shut down date, or alternatively provide justification to spend time extending the life of the product through architectural improvements.
As a result of undertaking this work, MYOB were able to realise a number of benefits across the three parts of the project.
Part 1: The Buildkite Configuration was successfully updated to the MYOB standard. A welcome but unexpected side effect of this was reduced build times for some pipeline stages by around 70% due to a combination of more appropriate AWS build resources, and Docker build fine tuning.
Part 2: Jenkins was successfully retired, replaced with a more modern and robust set of solutions. The retirement of this solution allowed the Cashbook team to reduce the operational overhead of managing multiple different divergent technology stacks – and spend more of their time focused on delivering customer value. By retiring the Jenkins services it is projected that the team would save hours of lost productivity each week maintaining the legacy tooling.
Part 3: The maximum load that was tested means MYOB now have confidence in the overall capacity of their Cashbook platform in a consistent and repeatable way. Much time was being spent debating architecture changes to support a clear load – all based on missing and inconsistent metrics. By increasing the stability and confidence of the load test platform they are now able to have clearly informed discussions with confidence.