A modern data-as-a-service solution for Australia’s leading retailer
By utilising AWS serverless and automation technologies, we delivered a complete real-time loyalty rewards solutions to millions of David Jones’ customers.
Digital wallets solution
For millions of David Jones customers
Retail
Industry
David Jones and the Country Road Group (DJ/CRG) is a multi-national portfolio of premium brands, providing everything for a modern lifestyle. They’re one of Australia’s largest specialty fashion retailers with a market-leading position in the mid to upper tier of specialty fashion. DJ/CRG comprises the five iconic brands Country Road, Mimco, Trenery, Witchery and the flagship department store David Jones.
Business challenge
The DJ/CRG had built a simple initial ‘Digital Wallet’ solution to show membership loyalty cards with Apple Wallet, Google Wallet, and Stocard, but they needed to improve their user experience across the five brands to give customers real-time rewards updates, connect them with promotional campaigns, and enable the use of geo-location services.
This improved experience required amalgamating data from multiple sources systems and the ability to operate in near-real-time.
Historically, DJ/CRG’s software was built with traditional, monolithic on-premises applications driven by batch style processing. Adding to and modifying these systems typically involved multi-year project lifecycles. This legacy approach offered stability historically, but cost significantly more time and money; DJ/CRG now required a dramatically different delivery methodology.
Cevo partnered with DJ/CRG to design and deliver a near-real-time solution, with the goal of being ready to launch for both the reopening of retail following a lengthy COVID lockdown and the start of the traditionally busy Christmas season in just six months, more than twice as fast as previous projects.
Solution
In early 2021, Cevo previously provided the core expertise in the Data as a Service team who have responsibility for the serverless data lake implementation. Leveraging that data knowledge, and working in close collaboration with the DJ/CRG architects and business domain experts, Cevo designed a cloud-native solution of event-driven microservices with real-time messaging. Upstream data was sourced from DJ/CRG’s existing transaction and customer relationship management systems.
Cevo’s approach built on Agile principles, delivering value from day one with a comprehensive automated unit, integration and end to end testing, and an automated delivery pipeline.
The ‘everything as code’ approach enabled engineers to spin up as many copies of the production environment as required to troubleshoot, experiment, tune, and improve in parallel.
A modern, event-driven serverless microservices approach combined with an agile and DevOps mindset yielded multiple benefits for the build and delivery of this project, including a shorter time to value, the ability to validate assumptions early, and to pivot design, implementation, and deployment options as new services and features became available from the ever-evolving AWS platform.
AWS Foundations
The use of serverless technologies means that the ongoing operational overhead of the project is nearly removed; there are no operating systems to patch, no drives to fill up, and since the solution is entirely described and deployed as code, no manual tweaks and fiddles to make once in production. The use of Amazon CloudWatch for logs metrics, and AWS X-Ray for distributed application monitoring, means that a holistic observability capability was delivered as part of the solution, equal in every environment.
The end-to-end latency of the system under production load measured at sub-second values for common events like changing a customer name, or low single-digit seconds for onboarding a new customer.
The upgrade brought significant value in terms of ease of use for DJ/CRG customers. They can manage their personal details displayed on the digital wallet, which previously had been static (imagine changing your name but having to show your old name on your digital wallet pass) and unfriendly (customers had to log in to a non-mobile-friendly website to check their reward balance, or check their email to find offers available to them). The upgrade allows for offer and reward notifications to be sent, customer can quickly access information regarding their membership; rapidly identify their rewards balance, or a new rewards tier gained; and DJ/CRG are now able to target at the individual customer level to offer specific custom rewards or offers.
In addition, DJ/CRG’s previous loyalty and reward mobile application didn’t retain data. Cevo’s solution also provided integration back to the existing SalesForce Marketing Cloud (SFMC) instance, allowing brand and loyalty management to track customer engagement, digital wallet uptake, and the impact of the solution on offer and reward awareness and consumption.
The Digital Wallets application is built entirely on AWS serverless technologies. By utilising the on-demand nature of serverless technologies, DJ/CRG can now seamlessly use as much or as little capacity as required, while tracking operational costs as closely to actual demand as possible.
The serverless design utilised Amazon AppFlow to transfer low-latency from the upstream Salesforce CRM, and Amazon Athena for data lake queries, combined with Amazon EventBridge for event transport, AWS Lambda for microservices, and supporting capabilities from Amazon SQS for worker orchestration, Amazon DynamoDB for fast NoSQL data storage of ready-to-roll customer data, and AWS Step Functions for orchestrating more complex workflows. Amazon Kinesis was used to save a long-term history of events to AWS S3 for later integration back to the data lake for analytics. AWS Secrets Manager was used to hold connection credentials for external services, ensuring they never needed to be committed to the source code repository.
Application Modernisation
The team used AWS CloudFormation to describe, deploy, and manage the lifecycle of the solution components into the DJ/CRG AWS Cloud environment. This allowed building multiple copies of the application entirely from source code, rather than relying on shared development and UAT environments, and greatly reduced development and test environment contention.
The convenience of this was not lost on engineers, who could create as many instances of the application as required, enabling contention-free development and allowing the team to scale up and down as required. Additionally, by creating multiple parallel environments, automated tests were run alongside the development activities with zero impact on other activities. This zero impact meant that automated testing was embraced by the whole team as a valuable contributor to the productivity, rather than a nice-to-have. Over 90% of bugs were found and fixed during development, with only trivial business logic adjustments requiring adaptation once in production.
Onboarding time for new team members was effectively reduced to zero, as new engineers were able to deploy a new and separate copy of the application in minutes.
The deployment to production was identical to the development and testing environments, leveraging the inherent fast-feedback cycle. This ensured any deployment issues were found as soon as possible, rather than at the expensive end of the change curve in UAT or production.
By utilising Lambda, EventBridge and SQS as the core compute and messaging process, changes that once required large scale rewrites in a monolithic system were bound to the lambdas that created them, drastically limiting refactoring to only the relevant locations.
Delivery Engineering
From the beginning, the team used a CI/CD ‘always green pipeline’ approach for continuous delivery. The process, starting at the existing DJ/CRG Bitbucket Git repository that triggered a BuildKite pipeline, ran all unit, integration, functional and end-to-end tests, and used CloudFormation to deploy the entire solution stack. Having the pipeline build a re-deployable artifact meant that promoting proven-working code to production was straightforward.
Once business stakeholders were satisfied with the state of the release, a single click change would deploy the exact same configuration into production. By delivering into the production environment many times a day from the first sprint of the project, the team was able to see the application in action with real data before the public release. This continuous delivery model meant that the go-live issues were resolved as soon as they were discovered, and the project was able to reach the public on time and with maximum effectiveness.
“Cevo’s engineering expertise and customer focus enabled a rapid and valuable response to our request for a “knowledge transfer” Redshift workshop. During our “Data as a Service” project there had been no scope to demonstrate sourcing business-aligned datasets from the system-aligned data landing in the Data Lake. Cevo’s Redshift workshop not only illustrated that this could be achieved relatively easily but also demonstrated several different approaches.
Ali Amin
IT Architect
Outcomes
From the first discussion to going live, the Digital Wallets project took a small team almost exactly six months to build and deliver. Cevo’s encouragement of the use of agile methodologies, a DevOps mindset, and AWS services allowed the business to adapt to changing demands, pivot to new approaches, and reprioritise key deliverables throughout the project, while still delivering on time and on budget.
By utilising AWS serverless and automation technologies, the team retained agility and velocity during the project and delivered a complete real-time loyalty solution to millions of DJ/CRG’s loyal customers.
The automated testing and delivery mean that future evolutionary changes to the application can be made with the confidence that existing features will not be compromised.
By entirely avoiding on-premise servers and traditional VMs the normal ongoing operational overhead is automated away.
“We have worked with Cevo on several projects over the last 10 months, and enjoy a great working relationship – their AWS skills are a constant contributing factor towards achieving our business goals.”
Ali Amin
IT Architect
Enjoyed this customer story?
Share it with your network!
You may also like
