AWS Cost Management: The Basics

As your AWS cloud infrastructure grows, so do your costs—but they don’t have to grow at the same rate.

Follow the best practices for AWS cost management and deliver dramatic increases in cloud resources growth while keeping costs stable.

The math is straightforward. Sound cloud cost management techniques can save you at least 30% year-over-year. At Apptio, we’ve seen many customers invest those savings right back into their AWS cloud.  In the first year, you’re getting 130% as much cloud for the same cost. By the second year, you’re getting 169% as much cloud, and 220% as much cloud by the third, all for the same amount of investment. Win-win-win.

Cost savings will please corporate finance, but what about the investment opportunities you can seize with extra budget. Bets on machine learning, exploring new product features, improving customer experience: cost savings fuel digital transformation.

The possibilities are endless, and it all starts with AWS cost management.

 

Step #1: Visibility into your cloud costs

When you first start out in the cloud, cost usually isn’t on the forefront of your mind. You’re more likely focused on taking advantage of the scalability and flexibility

But it doesn’t take long to realize how important costs are, especially if you want to expand your cloud to roll out another app, launch in another region, or migrate part of your data center.

It only takes one cost spike to show you just how fast costs can grow, and how important it is to manage them.

The key to managing these AWS cloud costs is visibility.

Why do you need cloud cost visibility?

Managing cloud spend is the top priority for enterprises running workloads and applications in the cloud, but it can be complex.

An entry in an AWS CUR file for a single instance for a single hour can easily take up 150+ lines, which means your billing file can easily have hundreds of millions of lines.

The sheer mass of data makes it very hard to make usage predictions, build cloud strategies, or even understand what’s going on. When you can do all three, you know you have enough visibility.

Visibility also means different things to different people. Your Finance team might want an accurate breakdown of spend, while an IT director may want to proactively catch abnormal trends and correct them before they become problems.

Cloud cost visibility means getting stakeholders the data they need to make data-driven decisions about your cloud footprint— no matter what team they’re on.

Get started with AWS cloud cost visibility

There are several pillars in the cloud cost visibility system:

• Company culture
• Data
• Tagging
• Team views
• Anomaly detection
• Budgets and forecasts.

To get a more detailed view, check out this 40 point checklist that cover these pillars and some key cost visibility tasks to get started.

Resource Tagging and AWS cost visibility

Tagging is one of the single most proactive things you can do to increase cloud visibility and the effectiveness of your AWS cost management.

Tags and labels tie your cloud resources to your company structure or development architecture. They’re the vital link between your cloud resources and your business, so getting them set up right is crucial.

Tagging should always be used to answer a specific question. For example, a common use case is an environment tag, answering the question, “What resources are being used by staging, dev, and test environments?”

The right taxonomy builds the right reports to answer business questions and allow informed decisions. Involve stakeholders from Finance, Operations and Engineering as everyone wants insights on different things (e.g., a finance view split by cost center, an engineering view of instances by role).

Consider building in automation of tagging using tools like Puppet or Chef and flag untagged resources to ensure maximum coverage. Higher the tag coverage delivers better cost visibility.

AWS cost visibility core points

Cloud cost visibility must be useful. True visibility enables and empowers you to act: Visibility without a call to action is merely trivia, not insight.

Full visibility is the basis for defensible strategic decisions, but visibility must be adaptable. No two cloud architectures are the same — even within the same company. Knowing what the whole company is doing won’t be as helpful to a single product feature team, just as only having visibility into that one feature won’t help executives on a company-wide scale.

Full visibility allows you to zoom into or out of the data and gives you the flexibility to create custom views.  People get all the data they need and only the data they need.

Cloud cost visibility must be complete. Seeing 60% of the data means you’re only seeing 60% of the picture, and that means your decisions can only be 60% accurate — at best.

The only way to have full confidence in decisions about your cloud is if you have 100% visibility into your cloud spend, including past, present and projected use.

 

DOWNLOAD:  AWS Tagging Strategy Best Practices

 

Step #2: Chargebacks and Allocations

Once you have visibility into your AWS cloud costs, look to correctly allocate spend across the business.

Tags should, minimally, identify consumers of cloud services. When they do, tags support defensible chargeback. When they don’t, chargeback is a challenge.  Shared IT services, not wholly attributed to one business unit, need an allocation method (e.g., consumption, headcount, even-spread) that simple tagging doesn’t capture.

Tagging accuracy feeds chargeback defensibility. Knowing your tagging model is, say, only 80% accurate means that only 80% of costs can be allocated back to consumers. For the rest, you are left to eyeball it with a less-defensible allocation method.

It gets more complicated when you throw in discounts, credits, amortization and custom surcharges into the mix. These aren’t reflected in AWS billing data, yet are critical for allocations and chargebacks.

This is where the difference between reporting and analytics comes in. Reporting spits data out in the same form it’s received, limiting the ways it can be viewed — and the insights you can get.

Analysis finds relationships between the data, allowing you to view the data several different ways to gain valuable, actionable insights.

Get a handle on chargebacks

Chargebacks are an essential part of IT operations — and they’re even more important in the cloud.

AWS makes it easy for teams to spin up the resources they need instead of going through the arduous purchasing process of physical data centers. This also makes it easier for them to rack up large AWS bills.

Accurate chargeback keeps these costs under control by tying them to the correct department. It also helps make sure you know exactly how your resources are being used with your company.

Getting the insights you need with AWS cost allocation

After you’ve segmented your costs across accounts and tags, generate spending reports specific to the needs of individuals within your company.

Operation teams may want AWS cost allocation reports by department split by linked department accounts, while Engineering Leads may want to filter their costs by resource or role.

And odds are good that the Finance team will want to do all that and more.

The sooner a structure is set, the sooner AWS cost management becomes achievable.  A tagging and taxonomy structure gives you the data segmentation you need.

The right AWS cost management tools are essential for building out your allocation and chargeback system. These tools automatically sort your billing data to fit your internal system, taking care of time-consuming data-processing so you can focus on making actionable insights.

Gaining confidence in your cloud cost decisions

Visibility into your AWS cloud costs and correct allocation systems feeds confident decision making.

Obviously, this is easier than it sounds, but data science and machine learning are here to help.

Future budgets and forecasts are more accurate, and you can make valuable cost-savings investments that will pay off in big ways, such as purchasing AWS RIs.

When preparing forecasts, involve key stakeholders in the process—there will always be new or unexpected costs that can’t be predicted, like changes in AWS services or new applications.

Build a strong foundation for future spend so those items have as little negative impact as possible.

 

Step #3: AWS optimization & automation

Visibility, chargeback, and allocations can give you an incredible amount of insight to help lower costs. But the third step, optimization and automation, is where you really start lowering costs and increasing your AWS cloud efficiency.

The biggest strengths of AWS is a good news/bad news scenario. The sheer variety of options available to build your cloud infrastructure provides endless opportunities for waste: It’s alarmingly easy to build resource waste into your system.

Optimization and automation eliminates waste without negatively impacting performance.

In essence, you’re able to get the same cloud power for less.

Rightsizing is the core of AWS cost optimization

Rightsizing is when you match up the right AWS resource with your actual AWS use. It’s all too common to spin up General Use instances like M5s when testing something, but once you’re using that feature more, a more specialized instance might be better.

With thousands of instances in large architectures, this can be hard to keep track of.

Rightsizing strives to keep track of your usage, then use machine learning to compare options and find your ideal solutions.

Rightsizing also extends beyond choosing the instance type. Depending on your use, you might be able to save massive amounts of money (as much as 60%) by utilizing RIs. RI opportunities aren’t always easy to see since they involve combining past usage with predicted future usage, but they have massive Rightsizing potential.

Beware of the pitfalls of “wrong-sizing” your cloud resources

There’s a lot of potential for rightsizing your resources, but you do have to approach it with care to avoid potential pitfalls.

• Using recommendations that don’t take cost spikes into account can negatively affect your website or app. They might work most of the time, but your system won’t be able handle spikes, leading to delays at best and system failure at worst. Take any recommendation that doesn’t account for these spikes with caution.
• Consider all your EC2 options rather than sticking to the common types. There are a whole host of combinations with new options released regularly, and you need to make sure you have checked all the potential options, including convertibles and size flexibility versions.
• Rightsizing goes beyond EC2, so don’t stop there. Optimization can be applied across other resources that may have hidden cost waste, including databases and storage.
• Don’t just set it and forget it. Rightsizing is not a one time exercise, and should be done regularly to maintain the optimization benefits. To help with this, there are tools to give you confidence in taking the right decision.
• Hesitation can cost you dearly. Your cloud is always going to be complex, and your business’ requirements might shift unexpectedly. There are always lower-commitment actions you can take, such as choosing convertible RIs or only doing basic automation.

DOWNLOAD: The Complete Handbook for AWS Reserved Instances.

 

Get started with AWS cost management today

There’s a lot of parts to AWS cost management, but fortunately, you don’t have to do everything by yourself or by building your own tools.

Going through all of these resources will help give you a solid foundation of AWS Cost Management. AWS and the cloud industry is always changing, so make sure you check out our Resources section for the latest and greatest.

The next step is to take action, and that’s where an AWS cloud cost management platform comes in. Apptio is an AWS Advanced Technology Partner and an AWS Cloud Management Tool Competency Partner for Resource and Cost Optimization.

If you’re ready to take your AWS cost management to the next level, then start a free trial today.

We’re also available through the AWS Marketplace.