Given the speed and pace of new releases coming out of the AWS services factory, one could be forgiven for struggling to keep up. With over 1,900 new releases in 2018, it’s important to cast one eye back and take the time to review the new, in order to continue to move forward and innovate. With all these new releases it’s often easy to overlook Cost Management updates, so I thought I’d take the time to point a spotlight on some recent Cost Management releases.
Amazon Macie
Let’s start with a recent release, “Enhanced Amazon Macie with substantially reduced pricing”. Well, actually that appears to be an understatement, as the Macie team have fundamentally changed the pricing structure offering savings of up to 80-90%, and have added a treasure trove of new features and enhancements.
If you are not familiar with Macie, it is a managed data classification service which scans through your AWS S3 hosted data to identify and classify sensitive data through the use of Machine Learning. When you consider the voluminous torrent of data that an organisation can move through their S3 buckets and the different formats that data can take, you can begin to understand the complexity of building a system which automates the detection and classification of data in motion. Luckily and thanks to AWS, you can lean on Amazon Macie.
Macie Pricing Summary
Previously, in order to estimate monthly Macie pricing, now referred to as Macie Classic, you would have to factor in the following dimensions:
- Amount of S3 content classified (GB)
- The estimated number of CloudTrail events expected
- Extended data retention period.
CloudTrail events and the extended data (metadata) created and retained, are used to support Macie Classic’s S3 anomaly detection feature.
Amazon Macie has simplified its pricing model to which is now calculated based on:
- Amount of S3 content classified (GB)
- Number of buckets selected for classification.
Additionally they have significantly reduced the cost to serve per GB of data classified, which I have detailed below.
Important Things to Know about Pricing and Macie
- Both pricing models provide detection of the first GB of sensitive data for free, however Macie Classic charges $5/GB thereafter. In contrast, Amazon Macie provides tiered pricing starting at $1.25 for the next 50TB, $0.63/GB for the next 450TB after that and $0.31/GB for anything over 500TB per month.
- You may be asking, as I did, how does Amazon Macie provide S3 Anomaly detection given that the pricing model has dropped CloudTrail event estimates and extended data retention? The Macie Classic FAQ provides the answer indicating that S3 anomaly is no longer a feature of the Amazon Macie service, which is now supported as a beta feature in GuardDuty. This further reduces the cost of Macie with s3 anomaly detection costs presumably wrapped up into existing CloudTrail Event Analysis costs.
- As indicated above, Amazon Macie has enhanced it’s offering to include Multi-account support via integration with AWS Organisations, offers full API coverage, expanded sensitive data discovery with updated ML models and now supports customer-defined sensitive data types using regular expressions.
- Macie Classic can no longer be turned on for new accounts, however if you are an existing Macie Classic customer, AWS provides a migration path to the new Macie service. Also important to note that during the migration export process, you would incur charges for both Macie Classic and the new Macie service.
Trusted Advisor Enhancements
Trusted Advisor is a long standing AWS service which performs checks against customer resources and provides recommendations across Cost, Reliability, Performance, Security and AWS limits.
As old hands at wrangling Trusted Advisor metrics would attest, the number one Trusted Advisor feature request has been the ability to aggregate metrics and recommendations across all AWS accounts and present it in a single pane. It has taken 7 long years since AWS released Trusted Advisor in 2013 for that wish to be granted. As of May the 4th, 2020, Business and Enterprise Support customers can leverage the AWS Systems Manager Explorer service to get an aggregated view of Trusted Advisor metrics and recommendations across all accounts in an organisation.
Four days later on the May the 7th, AWS also released a further 5 Cost checks to help customers optimise their purchasing options across EC2, Elasticache, RDS, ElasticSearch and Redshift. These are captured below and detailed on AWS’ announcement page. https://aws.amazon.com/about-aws/whats-new/2020/05/aws-trusted-advisor-adds-5-cost-optimization-checks/
Simultaneously, AWS released a Trusted Advisor Explorer solution via the AWS Solutions library. The Trusted Advisor Explorer solution enables customers to deploy resources via a CloudFormation template and implement a service that ingests AWS Trusted Advisor cost optimisation recommendations and uses the Tag resource groups service tag editor data to enrich Trusted Advisor data. Amazon Athena and Amazon QuickSite are then used to query and visualise AWS Trusted Advisor Cost Optimisation recommendations over time. AWS has estimated running costs at $5 USD/month for around 200 accounts based on the North Virginia region pricing. The key benefit of this solution is the ability to visualise cost optimisation recommendations over time at tag level granularity which in turn allows you to highlight teams that have effectively implemented them.
New EC2 Instance Types
In the past, the introduction of new and shiny compute generally resulted in asking for additional capital outlay and investment with the promise of better features and performance. AWS has turned this paradigm on its head, as every new EC2 announcements now typically comes with better performance and features and……better price efficiencies.
Tip – Every time you hear “new EC2 Instance release”, think “opportunity to optimise on costs”.
On June the 4th, AWS announced the general availability of the C5a AMD instances (AWS’ CPU Optimised EC2 instance family) which features second generation AMD EPYC processors. AMD pricing presents the best pricing per vCPU in the entire AWS portfolio, and the new C5a instances represent a 10% lower cost over similar instances. As with the M and R family AMD releases, AWS has also released disk and bare metal variants including C5ad (local NVMe, Instance Storage) and C5an.metal (100Gb network bandwidth) and C5adn.metal.
Great!, I hear you say. But how can I benefit from new instance types quickly and efficiently? Rehosting applications is hard work.
Well I’m glad you asked and here is good best practice that you can use as a starting point which that has proven to work for our customers.
- Baseline current EC2 instance performance for comparison purposes. Having a good monitoring capability in place, simplifies this activity.
- Bake a new AMI or build a new server instance using the new EC2 instance for initial use in a non-prod environment.
- Run simulated performance tests to gauge performance against the known baseline.
- Create an artifact which you can promote with confidence to production. This would take the form of an AMI (Amazon Machine Image) in the AWS ecosystem.
- Introduce the new instance into production, if possible via a blue / green deployment process, or by manually cutting over portions of your fleet as confidence increases over time.
- Continue to monitor performance and cost efficiency which is one of the key outcomes of this activity.
In general, the ability to automate the above activities will provide you with agility and speed required to react to new AWS EC2 instance announcements, and to implement these in a repeatable way.
At Cevo, we help customers effectively adopt cloud services to help them drive value out of their technology investments.
If you would like to know more about how Cevo can help you with your Cost Management requirements, please contact us to arrange a discussion.
