Dashboards-as-Code for financial services company
The opportunity at a glance
Leveraging automation for build, test and deployment of dashboards to provide actionable insights made ready to go.
Our customer is a large Australian financial services company that provides the Australian and New Zealand market with finance in the form of credit cards, car loans, insurance and personal loans.
The company had identified the need for a real-time view on the operational status and security posture of their Amazon Web Services (AWS) based cloud-native platform. The company has chosen Sumo Logic as its log aggregation and data analytics toolchain for all of this infrastructure.
The cloud-native platform relies on an efficient and intricate collection of cloud-based assets, and has implemented activity logging across all accounts by default. Sumo Logic does not provide an API for dashboard development and deployment, so the challenge became how could the company automate this?
Cevo were engaged to accelerate the integration of the Sumo Logic analytics platform with Cloud Security Tools and AWS Logs, in order to analyse and alert on events of interest as driven by project or tenant requirements by use case or security control definition.
1. Reverse engineering of the Sumo Logic front-end website.
Sumo Logic doesn’t have an API to deploy the dashboards and panels. – creating summary and drill down dashboards and panels is a time consuming ‘clicks ops’ task. To save countless hours, for every use case and security control, automation was used to create summary and drill down dashboards.
Reverse engineering of the Sumo Logic front-end was performed by analysing the http requests and responses resulting from manually generating dashboards and panels. From this information, the length and verbose json payload that is required to be built was determined. The json data was broken down into each individual dashboard control and turned into python objects which could then be reused to piece together custom panels and dashboards on the fly.
2. Developing a python application to automatically create a series of relevant dashboards with a base query as configuration.
A python application was written using the data structures discovered through reverse engineering the front end requests and responses, and was based around a simple yaml configuration. The end-user need only write a base query in Sumo Logic to return all results and add this to the yaml configuration, select which panels they would like on the dashboard, and then save and commit the configuration.
As an example, an end-user would add six lines to a configuration file if they wanted to visualize the number of times a ‘root’ login occurred within all of the accounts:
3. Deploying the generated artefacts to a 3rd party that does not have an API.
On commit, the python application takes the configuration reads in it, and commences a payload build based on the panels and dashboards selected. The authentication method used to authenticate with Sumo Logic was reversed engineered. Using that method and a service account credentials saved within AWS SSM Parameter Store, a http post is sent with the dashboard payload defined within the build pipeline.
4. Visualisations Generated
The dashboards generated and deployed are ready for viewing within seconds, saving countless man hours. This now allows the company to showcase these dashboards on screens as an information radiator for the current security posture of their AWS accounts.
Automating the building, testing and deployment of dashboards using “dashboards-as-code” has enabled expanding use cases and security controls quickly and efficiently for the cloud-native platform.
In addition, our customer can now:
- Visualise issues in real-time
Sumo Logic provides the live representation on what is happening on each use case or security control.
- Proactively remediate problems
By scheduling real-time analytics queries in Sumo Logic, notifications are sent to the company’s Slack security channel, alerting all subscribed users of the event of interest.
- Reduced friction around adding further security automation
By writing a base query, the solution will automatically build and deploy visualisations to the Sumo Logic service and configure scheduled queries to alert on events of interest via the company’s security alerts Slack channel. The solution provides Developers a clear and simple path of developing new use cases and security controls.
- Lowered on-going maintenance costs associated with detailed dashboard reporting
Historically, dashboards were created manually. Now a base query is written and committed to source where automation builds and deploys a series of visualisations.
- Infinite reusability.
Reuse of code to generate other interesting visualisations using the same configuration techniques.