AWS cost optimisation refers to the practice of managing and reducing the expenses associated with using Amazon Web Services (AWS) cloud services. It involves implementing strategies, best practices, and tools to optimise the usage of AWS resources while minimising costs.
The goal of AWS cost optimisation is to ensure that organisations are getting the most value out of their cloud investments by aligning their cloud usage with business needs and avoiding unnecessary expenditures.
Cost optimisation plays a central role within the FinOps framework and is a key consideration in AWS Well-Architected practices. Let’s explore how cost optimisation is integrated into both approaches.
FinOps and Cost Optimisation
- FinOps, derived from merging “Finance” and “DevOps,” represents a managerial approach adopted by organisations to enhance the financial efficiency of their cloud computing infrastructure. This practice encourages the distribution of financial accountability for cloud expenses among departments such as information technology (IT), DevOps, and other collaborative teams. The aim is to enhance decision-making procedures and propel higher business value outcomes.
- Cost optimisation is a fundamental pillar of FinOps, focusing on aligning cloud spending with business objectives, maximising cost efficiency, and ensuring optimal resource utilisation.
- FinOps establishes processes, best practices, and governance frameworks to monitor, analyse, and control cloud costs effectively. It promotes accountability, transparency, and collaboration to achieve cost optimisation goals.
AWS Well-Architected and Cost Optimisation
- The AWS Well-Architected Framework provides a set of best practices to design, deploy, and operate workloads on AWS with a focus on security, reliability, performance efficiency, operational excellence, and cost optimisation.
- Cost optimisation is one of the six pillars of AWS Well-Architected Framework. It ensures that workloads are architected and operated in a cost-effective manner by providing guidance on optimisation principles and techniques, such as right-sizing, selecting appropriate pricing models, utilising managed services, and optimising data transfer and storage costs.
- It encourages organisations to evaluate cost implications at each stage of the workload lifecycle and provides tools like AWS Cost Explorer, AWS Trusted Advisor, and AWS Budgets for cost analysis and optimisation.
By incorporating cost optimisation within the FinOps framework and following the recommendations of the AWS Well-Architected Framework, organisations can ensure that their AWS workloads are cost-effective, scalable, and aligned with their business objectives.
Cost optimisation lifecycle
Cost optimisation is an ongoing process that involves various stages and activities throughout the lifecycle of an organisation’s cloud infrastructure.
The above cycle shows various stages like Analysis, Cost Optimisation, Cloud Governance, and Cost and Usage monitoring. As you can see, each stage involves various activities. Initially, the process may appear too long, but with repeated iterations and practice, it becomes more seamless. Embracing a continuous cost optimisation practice is akin to developing a muscle. Once integrated into your business operations, the more frequently the process is repeated over time, the more it becomes a standard practice, seamlessly woven into everyday operations.
Some of the reasons for high cloud costs
There are several factors that contribute to high cloud costs, and when we examine them closely, we can identify the following key themes
- Unused instances: The appeal of pay-as-you-go cloud computing is its fast and easy provisioning, allowing developers to handle traffic spikes and launch new projects with agility. However, this seamless provisioning process can lead to instances and resources being forgotten, resulting in unnecessary costs.
- Incorrect instance sizing: Rightsizing is crucial for optimising cloud instances, ensuring they have the appropriate compute resources. Overallocation leads to inefficient resource utilisation and unnecessary expenses, while under allocation results in downtime and performance issues.
- Ungoverned autoscaling: Autoscaling is a valuable feature that allows organisations to dynamically adjust resources based on traffic and consumption forecasts. However, when auto-scaling is not properly governed, it can lead to over-provisioning and increased cloud spending.
- Inefficient storage selection: Choosing the wrong type of storage for different workloads can significantly impact costs. Organisations may overspend by using high-performance storage when lower-tier options would suffice, or they may underestimate their storage needs and end up paying for additional capacity or data transfer costs.
- Limited cloud visibility: Lack of visibility into cloud infrastructure and processes makes it challenging for IT teams to optimise performance and monitor costs. This issue is exacerbated by containers, which present a cost visibility challenge due to their multi-tenant and ephemeral nature.
The aforementioned scenarios represent common causes of high costs. To gain a more comprehensive understanding and detailed insights into the specific services that contribute to these costs, organisations can leverage the tools offered by AWS. These tools provide valuable information and analysis to help identify the areas driving high expenses. We will talk about these tools later in the article.
Key pillars of AWS Cost Optimisation
The key pillars of AWS cost optimisation provide insights and practical tips to help organisations unlock efficiency and maximise savings in their AWS deployments.
- Rightsizing and instance optimisation: By rightsizing instances, organisations can eliminate unnecessary costs and achieve optimal performance. It involves selecting the right instance types and sizes that align with your workload requirements, avoiding both overprovisioning and underutilisation.
Example: By analysing resource utilisation data, an organisation identifies that a particular EC2 instance type is consistently underutilised, with CPU utilisation averaging at only 10%. They decide to rightsize the instance to a lower-cost, appropriately sized instance type that still meets the workload requirements. This adjustment reduces costs by eliminating unnecessary over-provisioning. - Pricing models and purchasing options: This pillar includes analysing and selecting the appropriate pricing models, such as on-demand, reserved instances (RIs), and spot instances, based on workload characteristics and usage patterns. Optimising costs through RIs and savings plans offers substantial savings for long-term, predictable workloads. Additionally, leveraging spot instances for non-critical, flexible workloads can provide significant cost benefits.
Example: A company can identify a set of mission-critical instances that have consistent, predictable usage. They can decide to purchase RIs to take advantage of the cost savings offered by upfront commitments. By utilising RIs instead of solely relying on on-demand instances, they achieve significant cost reductions, saving up to 50% or more on their compute costs. - Elasticity and scalability: The ability to scale resources up or down based on demand ensures that organisations pay only for what they need. AWS Auto Scaling, along with the use of managed services such as Amazon RDS or AWS Lambda, allows for automatic adjustment of resources to match workload requirements, eliminate wasted resources and improve cost efficiency.
Example: An e-commerce platform experiences significant traffic spikes during seasonal sales events. By implementing AWS Auto Scaling, they dynamically adjust the number of EC2 instances based on demand. During peak periods, additional instances are provisioned to handle the increased traffic, ensuring optimal performance and customer experience. When traffic subsides, the platform scales down, avoiding unnecessary costs for idle resources. - Storage optimisation: Techniques such as data lifecycle management, utilising different storage classes like Amazon S3 storage tiers, and leveraging features like Amazon EBS snapshots or lifecycle policies can help reduce storage costs. Intelligent use of AWS data transfer options and content delivery networks (CDNs) further optimises costs associated with data transfer.
Example: A media streaming service analyses their data access patterns and classifies their data based on frequency of access. They utilise Amazon S3’s intelligent tiering feature to automatically move infrequently accessed data to a lower-cost storage class while keeping frequently accessed data in a higher-performance tier. This optimisation strategy reduces storage costs without compromising data availability or performance. - Data transfer cost reduction: Data transfer costs between AWS services and regions can accumulate rapidly if not managed efficiently. Strategies include leveraging AWS Direct Connect for reduced data transfer costs, utilising content delivery networks (CDNs) to offload data delivery, and implementing efficient data transfer protocols. Careful consideration of data transfer patterns and optimising data flows can lead to substantial cost savings.
Example: A multinational organisation with data centres in different AWS regions implements AWS Direct Connect, establishing private network connections between their on-premises infrastructure and AWS. By bypassing the public internet, they significantly reduce data transfer costs between their data centres and AWS services, optimising costs and improving network performance.
AWS Cost Optimisation tools
AWS Cost Explorer: Cost Explorer provides a comprehensive view of cost and usage data, enabling organisations to analyse their AWS spending patterns. It offers intuitive visualisations, customisable reports, and cost forecasting capabilities, allowing users to identify cost drivers, track spending trends, and make informed decisions for cost optimisation. By visualising cost and usage data, you can identify specific services that incur significant costs. Armed with this insight, you can take necessary actions to lower the cost.
AWS Scheduler(EventBridge): The Scheduler can be used to schedule the start and stop times of EC2 instances based on predefined rules. By automatically shutting down non-production instances during off-hours, weekends, or scheduled maintenance windows, organisations can save significantly on compute costs. By leveraging the scheduler, you can eliminate costs associated with idle instances, optimising their compute expenses without impacting operational requirements.
AWS CloudWatch: CloudWatch provides monitoring and observability capabilities for AWS resources and applications. By utilising CloudWatch metrics, businesses can gain insights into resource utilisation, identify potential cost-saving opportunities, and set up custom alarms to trigger actions based on predefined thresholds. CloudWatch empowers organisations to optimise costs by monitoring and adjusting resource utilisation in real-time. Example: A company utilises AWS CloudWatch to monitor their application performance and resource utilisation. By setting up custom CloudWatch alarms and metrics, you can receive real-time notifications when resource utilisation exceeds predefined thresholds. This allows you to proactively adjust resources, optimise costs, and maintain optimal performance levels.
AWS Budgets: Budgets enables organisations to set custom cost and usage budgets for AWS services. It helps establish cost control measures by providing notifications and alerts when actual or forecasted costs exceed defined thresholds. By proactively monitoring costs and staying within budgetary limits, businesses can optimise spending, adjust resource allocations, and make informed decisions to maximise their cost efficiency. Example: A startup sets up a budget in AWS Budgets to closely monitor their monthly spending. They configure alerts to notify them when their costs exceed a predetermined threshold. As they receive real-time notifications, they can proactively optimise costs, adjust resource allocations, and make informed decisions to stay within their budgetary limits.
AWS Trusted Advisor: Trusted Advisor is a valuable tool that provides real-time guidance to optimise costs, enhance security, improve performance, and ensure high availability. Within the cost optimisation category, Trusted Advisor offers recommendations on areas such as idle resources, underutilised EC2 instances, unused Elastic IP addresses, and potential savings through the use of Reserved Instances. By leveraging these recommendations, organisations can identify cost optimisation opportunities and take appropriate actions. Example: An e-commerce platform leverages AWS Trusted Advisor to identify idle resources within their infrastructure. The tool recommends terminating these resources to eliminate unnecessary costs. By acting on these recommendations, they effectively optimise their infrastructure and reduce costs associated with underutilised resources.
You can create reports to aggregate the check results for all member accounts in your organisation as well using Trusted Advisor which gives cost saving opportunities across all the members accounts.
AWS Cost & Usage Report: Cost & Usage Report (aka CUR) provides granular details about costs and resource usage. Organisations can access this report to analyse and allocate costs at a more granular level, enabling better cost management and optimisation. It provides detailed insights into resource utilisation, pricing, and usage patterns, allowing businesses to identify optimisation opportunities, allocate costs accurately, and make data-driven decisions.
AWS CloudTrail: CloudTrail enables governance, compliance, operational auditing, and risk auditing of your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. You can complement cost and usage data by enabling CloudTrail Insights to detect unusual activity in the logs with just a few clicks. CloudTrail Insights analyses historical API calls, identifies usage patterns, and generates Insight events for unusual activity. Based on the usage patterns and activities you can take necessary actions to optimise cost by modifying/delete the resources.
Conclusion
In the ever-evolving AWS landscape, cost optimisation remains a pivotal and ongoing aspect of proficient cloud financial management. To navigate this dynamic environment successfully, it’s imperative for companies to not only comprehend the pillars of AWS cost optimisation but also familiarise themselves with the latest AWS capabilities. By doing so, and by implementing best practices and leveraging the available tools and techniques, organisations can achieve substantial cost savings while maximising efficiency and resource utilisation. With a comprehensive cost optimisation strategy in place, businesses can unlock the true potential of AWS and attain their desired business outcomes.
