Cloud Cost Optimization #1: Visibility

In the on-prem world, budgets and costs are something that managers and finance deal with in long, multi-year cycles. It’s a world built around big capital expenditures and overprovisioned capacity.

The world of the cloud is very different. It’s a far more dynamic world based on consumption — you pay for what you use. Application and operations teams make decisions every day that immediately affect the cost and size of your cloud architecture.

Applying outdated financial processes to the cloud results in wasted resources and money. Companies migrating to the cloud need to transition from the slow, static on-prem financial model to the real-time model of the cloud. They need to involve engineers, managers and finance in a continuous optimization process.

That process is FinOps. FinOps is a combination of systems, best practices and culture. FinOps increases an organization’s ability to understand cloud costs and make tradeoffs.

FinOps has three phases: Inform, Optimize, and Operate. Let’s look at some of the goals for the Inform phase:

• Getting visibility into your IT spend
• Instituting granular cost allocation
• Creating team-level budgets & tracking

You can accomplish those goals, no matter what cloud provider you use, by:

1. Creating a FinOps team
2. Putting a realistic cost allocation structure in place
3. Tagging your resources

Create a FinOps Team

A FinOps team is a small, interdisciplinary group of people. Together, the team members represent Finance, Engineering (both Development and Operations) and Line of Business (LoB).

The FinOps team meets and discusses choices about cloud infrastructure. The team’s goal
is to make sure everyone understands the interplay between the actual infrastructure, the
infrastructure costs and the business goals. People from Finance can take on a financial
planning and advisory role. LoB can give its perspective on what exactly should be
optimized in terms of cost, speed and quality. Development and Operations can contribute
by explaining what cloud resources they need to build the applications and features that
management has identified as adding value to the business.

Working together, the FinOps team can decide on the cost allocation structure and the tagging policies that will give detailed visibility into the company’s use of the cloud.

Put a Cost Allocation Structure in Place

Anyone trying to gain visibility into their cloud spend needs to make sure that all of the company accounts are known and included in the accounting model. You want to create a cost allocation structure that matches the way you think about your business. If your company is structured around autonomous business units, your structure should reflect that. If it’s a DevOps organization, you might want to create a structure based on projects, since each team manages its application end to end.

Using automation is a big help. Amazon, Microsoft Azure and Google Compute Platform (GCP) all have tools that help you create and manage your hierarchy.

AWS

Accounts are fundamental to an AWS cost hierarchy. AWS Organizations is a policy-based management tool that allows you to automate account creation and to group accounts. You can apply and manage permissions to those accounts, as well as enable consolidated billing and reporting for your organization. Through integrations with other AWS services, you can use AWS Organizations to define central configurations and resource sharing across accounts in your organization.

GCP

Projects are the core organizing entity for Google Cloud. When you have a service such as Compute Engine or Kubernetes or Cloud SQL or Storage, you’ll see them creating resources in the context of projects. Projects are containers for resources and for describe the settings, permission and other metadata around your workloads.

Use Cloud Deployment Manager to create projects. With Deployment Manager, you create a configuration file that describes a set of GCP resources that you want to deploy together.

Use Cloud Identity and Access Management (Cloud IAM) to associate billing accounts to projects. Cloud IAM lets you control who (users) has what access (roles) to which resources by setting Cloud IAM policies. You can set a Cloud IAM policy (roles) at the organization level, the folder level, the project level or (in some cases) on the service-level resource.

Microsoft Azure

Many Azure users find that they have an ever-increasing number of subscriptions to manage. You can use Azure Cost Management to create management groups, which are a great way to organize those subscriptions. With Azure Cost Management, you can apply role-based access control (RBAC), tagging policies, cost analysis and budgets at any scope. Management groups are a shared context across Azure. That means you can also access all the additional management and security services.

Tag Your Resources

A good cost allocation structure doesn’t, by itself, give you the granular information you need to really understand your cloud spend. For that, you need tags. Tags add context for cost analysis.

A tag is a label that you assign to a resource. Each tag consists of a key and an optional value, both of which you define. Here are a couple examples.

Owner = QA
Stack = Test

Here, the keys are Owner and Stack. The values are QA and Test.

As of today, Amazon allows up to 50 tags per resource. GCP, which uses the term label instead of tag, allows 64. On Azure, you can have up to 15. No matter what cloud provider you use, you should make sure your tags answer these core four questions:

• What cost center is the resource tied to?
• Which application is the resource tied to?
• Which individual or team is the resource tied to?
• When do we expect the product to end?

Creating tags (or labels) programmatically is the best way to ensure that resources are always tagged and that they’re tagged according to the policies defined by the FinOps team.

In Amazon, you can implement a programmatic tagging solution via script, API or CloudFormation templates. For GCP, use the Resource Manager API. With Azure, Azure Cost Management is used to create tagging policies. For all three cloud providers, you can also use third-party products such as Ansible, Chef or Puppet to tag resources.

Deliver Specific Views to Specific Budget Holders

Once the FinOps team has a cost hierarchy and tagging policy in place, it’s now possible to deliver specific reporting views for specific budget holders. Successful companies use cloud cost management platforms to turn the flood of billing information into simple, actionable charts. These allow Finance, Operations and Engineering teams to quickly understand what they’re spending on a daily basis, while flagging opportunities to optimize their cloud cost and usage to align with a healthy business bottom line.

Take Control of Your Cloud Spend

Cloudability is a cloud financial management platform designed for FinOps. Using data science, machine learning and automation, Cloudability enables IT, finance and business teams to continually optimize cloud consumption to improve the unit economics of cloud.

Get a free trial to see how we can give you complete cloud cost visibility. And you can find out more about FinOps and the FinOps Foundation at FinOps.org.