Automated machine learning for Australian financial services company
By leveraging Machine Learning Ops, our customer was able to improve their fraud detection model to identify more sophisticated fraud cases.
Fraud detection model
Leveraging machine learning ops
MLOps
Solution
Financial Services
Industry
Our customer is a large Australian financial services company that provides the Australian and New Zealand market with finance in the form of credit cards, car loans, insurance and personal loans.
Business challenge
Our customer has an effective platform to detect fraudulent transactions; this system is currently implemented through a set of complex and hard to manage rules. As the bad guys get faster, the good guys need to continue to hone and evolve their approaches to detecting this malicious behaviour.
While the existing fraud detection platform was identifying most of the fraud cases, they wanted to explore the possibility of using a machine learning approach to create highly predictive models to detect additional fraud. Much press has been given to Machine Learning (ML) and its ability to be applied to complex data sets to detect patterns that even the most complex rules engines are unable to find.
It was recognised that the current team managing the fraud solution would require some support and upskilling in this area, and so Cevo and CorticAi were engaged to provide experience in navigating this emerging space.
In addition to the above, as COVID-19 impacted the market place, it was evident that a shift in business priority was also required. The team that had been looking at fraud was tasked to develop a model that is capable of predicting customers who will apply for hardship in order to assist them ahead of time.
Solution
Cevo and CorticAi’s roles were to help build an automated machine learning pipeline that created the ability to rapidly and accurately generate predictive models. This is known as MLOps – DevOps applied to machine learning. Cevo’s expertise in developing automated delivery pipelines for infrastructure and applications was leveraged to apply these same techniques to the Machine Learning domain.
The modern nature of ML tooling is strongly API driven, and therefore the same “as code” practices applied to applications and infrastructure are able to be extended into ML. Cevo has a strong history in delivering efficiency in the development and deployment of solutions, and so was the next natural step to apply these tools and techniques to ML
The pipeline utilised many AWS services as well as an automated machine learning platform developed by CorticAi called AugustAi to train hundreds to thousands of models in a couple of hours. AugustAi automates much of the tedious work that a data scientist would do such as feature selection and data cleansing. It will build hundreds of variations of a model with different algorithms and parameters and pick the model that is most predictive for the given data.
An automated feature engineering solution was developed which would take data and generate features. These features were used when training the model. The pipeline will then train models using AugustAi. After a model is trained, it will publish the model to Sagemaker Endpoints for hosting.
Sagemaker endpoints is a model hosting service on AWS. It supports the ability to have multiple versions of a model behind a single endpoint. This allows for any new models to be assessed alongside the current model.
Fraud Detection
Whilst our customer already possesses a highly capable system, the primary use case for this project was to implement an MLOps approach of automation to train a model capable of detecting fraudulent transactions. At a click of a button, hundreds of models could be trained and assessed within an hour using millions of rows of data. This massive increase in efficiency allows for more comprehensive models to be created and validated.
Using the platform, a model was generated in three hours capable of detecting substantial amounts of additional fraud a month.
COVID-19 Machine Learning
The idea to use machine learning to help support customers applying for hardship came about halfway through the project. From inception to a deployed model took four days. Not only was this incredibly fast, but this new model was much more likely to predict hardship than the existing approach, allowing the organisation to better assist their customers.
This is an impressive capability proving that applying MLOps principles enables rapid agile delivery of machine learning-based outcomes for organisations.
Outcomes
The platform took 6 months of development with which many models were generated for different business needs at a click of a button. By investing in MLOps, our customer was able to pivot a solution to the market in less than a week and proves that MLOps is another valuable investment in an organisation’s technology journey.
By applying these MLOps concepts to this project, the organisation has been able to reduce the time taken to train and deploy models from multiple months to a number of days.
Of significant note, a model capable of detecting substantial amounts of additional fraud a month was generated in just 3 hours. This is an extraordinary example offering business value with only a few hours of model training. With further data ingestion and feature engineering automation, the detection rate will increase.
Similarly, the newly deployed model for predicting customer hardship is much more effective than the current approach, and significantly improves customer care.
Through machine learning automation, teams can have greater confidence that the changes they are making have increased quality – and due to the reduced cycle times greater experimentation and market responsiveness can be achieved.
Enjoyed this customer story?
Share it with your network!
You may also like
