![ApptioCareers-Employee[1]](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyMjUgMjc1IiB3aWR0aD0iMjI1IiBoZWlnaHQ9IjI3NSIgZGF0YS11PSJodHRwcyUzQSUyRiUyRnd3dy5hcHB0aW8uY29tJTJGd3AtY29udGVudCUyRnVwbG9hZHMlMkZlbGVtZW50b3IlMkZ0aHVtYnMlMkZBcHB0aW9DYXJlZXJzLUVtcGxveWVlMS1xZzVuenB5MHdoaWJtbzZvdmJqNGM3MHh0aTUzdmo1OHM0endwajh0dHMucG5nIiBkYXRhLXc9IjIyNSIgZGF0YS1oPSIyNzUiIGRhdGEtYmlwPSIiPjwvc3ZnPg== "ApptioCareers-Employee[1]")
IBM ApptioAWS S3 costs reflect the type of storage you use and how you use it. Get the most from your S3 spend by knowing your options and picking the most cost-effective storage options. IBM Cloudability customers consistently spend around 75% and 15% of their S3 spend on storage and API requests (i.e., add, editing, deleting, and pulling data from storage). In this blog post, we will focus on those two drivers of S3 spend and how you can optimize them.
What is AWS S3?
Amazon Simple Storage Service (Amazon S3), the first AWS service launched in 2006, is an object storage service committed to scalability, availability, security, and performance. Object storage manages data as objects rather than as part of a file hierarchy (i.e., file storage) or blocks within sectors and tracks (i.e., block storage). Object storage is often used for unstructured data (e.g., images, videos, log files, archives).
Most organizations don’t want to be in the business of creating their own IT infrastructure; it’s an operational headache they can do without. AWS offers up websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics as a utility. Use it when you want. Outsource the upkeep and innovation to the service provider. Pay for what you use. Simple.
Amazon S3 provides management features to organize data and configure access controls to meet your specific business, organizational, and compliance requirements.
It’s as bespoke as you need it to be.
AWS S3 storage classes
S3 storage classes serve different use cases. The primary drivers of choice come down to how often you need to access the storage and the level of redundancy you need. These storage classes are not static; they can be updated manually or via a lifecycle policy. S3 lifecycle policy allows you to manage the data lifecycle (i.e., data will automatically transfer to a different storage class without any changes to your application). Data is stored in an AWS S3 bucket, and each bucket holds objects. Each bucket can have objects with a mixture of classes.
Storage classes and expected uses are:
- S3 Standard: general-purpose storage of frequently accessed data
- S3 Intelligent-Tiering: unknown or changing access patterns
- S3 Standard-IA: long-term but less frequently accessed data
- S3 One Zone-IA: long-term, less frequently accessed data in one geo
- S3 Glacier Instant Retrieval: long-term archive with fast retrieval
- S3 Glacier Flexible Retrieval: long-term archive without a need for quick retrieval
- S3 Glacier Deep Archive: long-term archive and digital preservation suitable for regulatory compliance
S3 Standard
S3 standard is the AWS default. We see that over 93% of objects belong to this class. It’s designed for data that is accessed frequently.
S3 Standard is the workhorse of Amazon S3. The low latency and high throughput make it an extremely versatile backbone for many applications. However, it is the most expensive storage class per GB stored.
S3 Intelligent-Tiering
S3 Intelligent-Tiering relies on inbuilt monitoring and automation capabilities to move data between a frequent-access tier (FA) and an infrequent-access (IA) tier for cost optimization. Intelligent-Tiering ensures you’re not paying FA storage prices for data that isn’t being accessed regularly — files stored in FA are charged at the S3 Standard rate while files stored in IA are discounted between 40-46%. While there is a monthly monitoring and auto-tiering fee, there are no data retrieval fees, so you don’t have to worry about unexpected bill spikes if a data access pattern changes.
S3 Standard-Infrequent Access (IA)
S3 Standard-IA is best for storing data that is accessed less frequently than data stored in S3 Standard, but that still requires rapid access when needed. It’s ideal for long-term storage or backups, and it’s often used as a data store for disaster recovery. Its storage costs are lower (discounted 40-46%) than S3 Standard, but there are data retrieval charges.
S3 One Zone-Infrequent Access (S3 One Zone-IA)
This Amazon S3 class stores data in a single AWS Availability Zone (AZ). Unlike the other S3 classes, it isn’t designed to be resilient to the physical loss of an AZ due to a major event such as an earthquake or a flood. But if you don’t need the extra protection provided by geographic redundancy, then you can take advantage of prices 20% lower than S3 Standard-IA.
S3 Glacier Instant Retrieval
Amazon S3 Glacier Instant Retrieval delivers the lowest-cost storage for long-lived data that is rarely accessed and requires retrieval in milliseconds.
S3 Glacier Flexible Retrieval
S3 Glacier Flexible Retrieval offers similar capabilities as S3 Glacier Instant Retrieval but is only expected to be accessed one to two times a year and doesn’t need immediate access. S3 Glacier Flexible Retrieval provides a balance between cost and access time.
S3 Glacier Deep Archive
S3 Glacier Deep Archive is the lowest-cost option for S3 and is built for long-term retention and digital preservation of data that won’t be regularly accessed. It can be used for backup and disaster recovery, but one of its strongest use cases is for highly regulated industries that must retain data sets for regulatory compliance.
Comparing AWS S3 Classes
| S3 Standard | S3 Intelligent-Tiering | S3 Standard-IA | S3 One Zone-IA | S3 Glacier Instant Retrieval | S3 Glacier Flexible Retrieval | S3 Glacier Deep Archive | |
|---|---|---|---|---|---|---|---|
| Designed for durability | 99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
99.999999999% (11 9’s) |
| Designed for availability | 99.99% | 99.9% | 99.9% | 99.5% | 99.9% | 99.99% | 99.99% |
| Availability zones | ≥3 | ≥3 | ≥3 | 1 | ≥3 | ≥3 | ≥3 |
| Minimum capacity charge per object | N/A | N/A | 128 KB | 128 KB | 128 KB | 40 KB | 40 KB |
| Minimum storage duration charge | N/A | N/A | 30 days | 30 days | 90 days | 90 days | 180 days |
| Retrieval charge | N/A | N/A | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved |
| First byte latency | milliseconds | milliseconds | milliseconds | milliseconds | milliseconds | minutes or hours | hours |
| Storage type | Object | Object | Object | Object | Object | Object | Object |
| Lifecycle transitions | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Optimize your cloud spend by rightsizing your S3 storage. IBM Cloudability analyses your storage classes and usage to find savings.
AWS S3 pricing
At the unit rate level, S3 pricing looks astonishingly cheap. Don’t be fooled. As with all cloud solutions, you pay for what you use. Nominal unit rates deliver a sizeable bill when you are inattentive to the fully burdened cost of S3.
There’s an inherent contradiction with cloud adoption success. Early cloud adopters weren’t asked to reign in cloud spend because in comparison to the rest of the IT budget, total spend was small. That’s not the case anymore. Today, successful cloud adoption delivers a substantial hit on OpEx spend that invites more scrutiny and oversight. OpEx-focused cloud spend providing agility and cutting the longtail commit of CapEx investments isn’t appreciated by Finance teams seeing month-over-month cloud growth with no respite.
Understand upfront how you will interact with each storage class. S3 Glacier is for long-term, rarely used material (e.g., end-of-lifecycle data). It’s not a good option as a data store for disaster recovery. For that, S3 Intelligent Tiering is a better fit. These aren’t a matter of semantics. Expected usage patterns inform service pricing — that’s how AWS bundles in their profit margin. Prepare to pay over the odds when you provision resources outside those expectations.
Components of S3 pricing
Storage costs
These costs are as you would expect — the cost of storing your data on S3, charged per GB-month.
Bucket and object requests
Unless you are using S3 for archive storage or regulatory compliance, S3 data doesn’t idly sit in storage. Access, edits, scanning: all these actions on your S3 data are quantifiable and, inevitably, billed. Each storage class has different pricing per request, but activities are the same across classes. They are:
- POST creates a new object (i.e., upload of a new file)
- PUT a new object or update of an existing object (i.e., creation or update of a file)
- LIST request for the contents of a given S3 bucket
- GET downloads a file from S3
- DELETE would be the deletion of a file
- S3 Select Data Returned/Data Scanned. S3 Select pulls only the data you need from a storage object. Smaller data pulls improve performance
S3 Storage Management
Keeping on top of your data requires monitoring tools. S3 monetizes those too. They are:
- S3 Inventory of listed objects. Cost: $0.0025 per million objects listed.
- S3 Analytics Storage Class Analysis monitors access to objects. Cost: $0.1 per million objects monitored per month.
- S3 Object Tagging manages and controls access for S3 objects. Cost: $0.01 per 10,000 tags per month.
Data transfer. There are transfer costs associated with how you add or remove data. The main charges are for transferring out to the internet or between AWS regions. Not surprisingly, adding new data to S3 is free. But there are different tiers for the transfer of data once it’s added. Transferring from one S3 geo to another ($0.02/GB) is far cheaper than taking your AWS data and shifting to another provider (e.g., $0.09/GB up to 10TB per month). Organizations concerned with vendor lock-in must evaluate data transfer costs as part of their cloud migration TCO.
When time is of the essence, AWS offers a premium data transfer service. It isn’t a given that the transfer acceleration is faster than standard data transfer. Generally, S3 transfer acceleration delivers data speed improvements the further you are away from an S3 region. Check the relative speeds of standard vs. accelerated S3 data transfer here.
Cross-region replication. The replicated copy of storage is based on the destination region, while pricing for requests and inter-region data transfers is based on the source AWS Region.
AWS S3 storage costs
S3 data storage costs vary according to their region. As an example, this table shows the costs of all S3 classes in the US West (Oregon) region:
| S3 Standard – General purpose storage for any type of data, typically used for frequently accessed data | ||
|---|---|---|
| First 50 TB / Month | $0.023 per GB | |
| Next 450 TB / Month | $0.022 per GB | |
| Over 500 TB / Month | $0.021 per GB | |
| S3 Intelligent-Tiering – Automatic cost savings for data with unknown or changing access patterns | ||
| Monitoring and Automation, All Storage / Month (Objects > 128 KB) | $0.0025 per 1,000 objects | |
| Frequent Access Tier, First 50 TB / Month | $0.023 per GB | |
| Frequent Access Tier, Next 450 TB / Month | $0.022 per GB | |
| Frequent Access Tier, Over 500 TB / Month | $0.021 per GB | |
| Infrequent Access Tier, All Storage / Month | $0.0125 per GB | |
| Archive Instant Access Tier, All Storage / Month | $0.004 per GB | |
| S3 Intelligent – Tiering – Optional asynchronous Archive Access tiers | ||
| Archive Access Tier, All Storage / Month | $0.0036 per GB | |
| Deep Archive Access Tier, All Storage / Month | $0.00099 per GB | |
| S3 Standard – Infrequent Access – For long-lived but infrequently accessed data that needs millisecond access | ||
| All Storage / Month | $0.0125 per GB | |
| S3 One Zone – Infrequent Access – For re-creatable infrequently accessed data that needs millisecond access | ||
| All Storage / Month | $0.01 per GB | |
| S3 Glacier Instant Retrieval – For long-lived archive data accessed once a quarter with instant retrieval in milliseconds | ||
| All Storage / Month | $0.004 per GB | |
| S3 Glacier Flexible Retrieval (Formerly S3 Glacier) – For long-term backups and archives with retrieval option from 1 minute to 12 hours | ||
| All Storage / Month | $0.0036 per GB | |
| S3 Glacier Deep Archive – For long-term data archiving that is accessed once or twice in a year and can be restored within 12 hours | ||
| All Storage / Month | $0.00099 per GB | |
Sample pricing table for S3 requests with S3 Standard in US West (Oregon). Check out the full pricing page for other classes and regions.
The key to managing your storage costs is to pair the right data with the right storage class based on how the data is used. Sort your data by how frequently it needs to be accessed to figure out whether you need S3 Standard or not. Once you’ve ruled out S3 Standard, decide whether to stick with S3 Standard-IA or leverage savings from One Zone-IA and Glacier. If the data can be easily re-created or has redundancy, then S3 One Zone-IA can be a good choice. If the data’s archive or long-term storage, go with Glacier. Focus on how you’re using the data now rather than years in the future. Unsure? S3 Intelligent-Tiering can help by governing your data’s placement in Standard or IA storage.
Amazon S3 request costs
Requests include PUT, COPY, POST, LIST, GET, and SELECT. The PUT, COPY, POST, and LIST requests, which are for uploading objects, cost more than the GET and SELECT requests, which are for downloading objects. Like most AWS costs, these will vary by region.
| Pricing | |
|---|---|
| PUT, COPY, POST, or LIST requests | $0.005 per 1,000 requests |
| GET, SELECT, and all other requests | $0.0004 per 1,000 requests |
| Lifecycle Transition requests into Standard – Infrequent Access or One Zone – Infrequent Access | $0.01 per 1,000 requests |
Sample pricing table for S3 requests with S3 Standard in US West (Oregon). Check out the full pricing page for other classes and regions.
AWS S3 storage lifecycle management
AWS has several tools that let you manage your storage. They include AWS Object Lifecycle Management, inventory, analytics, and object tagging. The last three have costs associated with them.
Using lifecycle management to reduce costs
AWS Object Lifecycle Management lets you programmatically configure rules for data deletion or migration between types of S3 storage to help lower long-term storage costs. For example, active data should remain in S3 Standard storage. However, if some data begins to be accessed less frequently, you can automatically migrate that data over to S3 Standard-IA for a cheaper storage rate. S3 Intelligent-Tiering incorporates a similar functionality in a ready-made package.
Object Lifecycle actions are sorted into Transaction Actions (moving data) and Expiration Actions (deleting data). Expiration Actions are free, and while there are no actual costs for using AWS Object Lifecycle Management, there are costs for Transaction Actions, such as transferring data between S3 Standard-IA and Glacier.
Lifecycle management tip: Glacier requests
You can add lifecycle management rules to tell Amazon S3 to transition objects to another S3 class, including Glacier. If you plan to do this frequently, remember that some extra storage is needed to store metadata. Also, remember that there are fees for data retrieval, the retrieval speed, the archive request, and a few other options. Take a look at the AWS documentation “Transitioning objects using Amazon S3 lifecycle” for more info.
Getting a handle on your AWS S3 costs
As you can see, there are a lot of moving parts to Amazon S3, all of which can unexpectedly drive up your costs. This is where a culture of cloud cost management and the right cloud cost management platform is vital for an organization.
With a solid cost management culture, every member of your team from executive to developer is empowered to take actions that get you more for your cloud investment. And the right platform, like Cloudability, will make sure they have the data they need to be empowered. IBM Cloudability ingests, normalizes, and structures billing data across the major cloud providers so that you can actively manage spend and consumption to continuously improve the unit economics of running cloud. Cloudability enables technology, finance, and business teams to optimize cloud costs and maximize the value of their public cloud strategy.
Sign up for a free trial find out how Cloudability can give your team the visibility it needs to get more from their cloud.
