Cloud Cost Optimization #2: Automation

Once you have a cost structure and tagging policy in place, you’ll begin getting the information you need from your bill to start saving money. Almost everyone has idle or partially idle resources in their cloud infrastructure. Eliminating them or shutting them down when they aren’t actively being used are two of the easiest ways to either reduce cloud spend or allocate the resources somewhere in your cost hierarchy where they’ll be useful.

If you’re a small company that uses only a small number of cloud resources, you might find that you can perform these activities manually but, as you scale up, it’s simply not practical. Instead, use automation. Automation eliminates a lot of human error and puts repeatable, documented procedures in place.

For example, you might find 15 EC2 instances on AWS that are still running, but always idle. Eliminating them is a quick, easy way to save money and you can automate the process of scanning for idle resources and deleting them.

You might automatically shut down instances over weekends and holidays if they aren’t being used, such as testing or production instances. Or you might automate looking for leftover unused storage services like unattached EBS volumes. The important thing is that automation can help you accomplish these tasks at scale.

Of course, before you start eliminating resources or shutting them down, particularly in production, you’ll probably want to notify the owners. You can do this automatically as well by setting up a notification system that alerts the owners if they’re about to lose a resource. You could also have a workflow that requires approvals before a resource disappears.

AWS, Microsoft Azure and Google Compute Platform (GCP) all have ways to automatically manage resources. Let’s briefly look at each one.

AWS

The AWS Instance Scheduler enables you to configure custom start and stop schedules for EC2 instances. (You can use it for their RDS service as well.) If, for example, you have instances you want to shut down after business hours or on weekends, the Instance Scheduler may be for you.

The scheduler uses an Amazon CloudWatch event to trigger an AWS Lambda function that checks the current state of each appropriately tagged instance against the targeted state defined in an associated schedule in Amazon DynamoDB. Then, the Lambda function applies the applicable start or stop action to the instance.You’ll need to fill in a CloudFormation template to launch the Instance Scheduler stack. You can also use DynamoDB to configure periods and schedules.

GCP

In GCP, to automate resource management, you write a Cloud Function that uses the Compute Engine API. You can use Node.js, Python, or Go. That function automates the Cloud Scheduler to perform a task, such as shutting down and starting VMs, on a regular basis. Cloud Scheduler is a fully managed cron job scheduler and allows you to schedule virtually any job. Cloud Scheduler has a GUI you can use to set up a recurring schedule.

Microsoft Azure

On Azure, you can use Azure Automation to perform repetitive tasks such as shutting down and starting up VMs on a schedule. Azure Automation relies on runbooks, which are scripts. The easiest way to write runbooks is with Windows PowerShell. There is also a gallery of already written runbooks that perform all sorts of useful functions. Use the Schedules feature of Azure Automation to create a schedule, link it to the runbook and set the times when you want the tasks to occur.

Apptio Cloudability and Automation

As you’ve noticed, you need a broad set of technical skills to use automation on any of the cloud providers. For example, on AWS, you’ll need to understand DynamoDB, Lambda, CloudWatch custom metrics, and Cloudformation templates, including how to do Python scripting and CloudFormation coding.

If you’re on AWS, you’ll find that Apptio Cloudability’s automation feature will make your life much easier. For example, you can use the GUI to schedule cleanup of unused EBS volumes. All you have to do is choose the accounts and tags you’d like to scan for unattached volumes and configure how long an unattached volume should sit before Cloudability generates a snapshot and deletes the volume.

You can just as easily scale autoscaling groups up and down on schedule and stop and start EC2 and RDS instances.

If you’re interested in learning more about how Apptio Cloudability can help you get control of your cloud spend, then sign up for a free trial!