If you are new to AWS, you might not be very worried about cost. After all, $0.05 per hour for an EC2 instance sounds pretty good. But, if you have been at it for a while, you may be noticing that the costs start to add up and have the potential to get out of control if you don’t give cost management a little bit of attention. The good news is that there are several tools and approaches you can use to keep things under control with a minimum of effort.
First, pay attention to what you are using. Take the time once you get your app running to make sure your EC2 instance selection is the least expensive type that meets your requirements. And, make good use of autoscale groups and the t2 instance family if the load on your app varies. Rather than using a larger instance type to handle load, tune for the minimum load and configure your autoscale group to grow a cluster of instances to meet the demand. The t2 instance family, in particular, has bursting capability to handle short spikes in demand without even requiring the autoscale group to be scaled up.
Second, make sure that EC2 instances and other resources that you aren’t using are turned off. If you have a 40 hour work week, there are 128 other hours in the week that aren’t work hours. Why leave your dev and test resources running all the time racking up charges unnecessarily? Save some money, and turn them off when not in use. Use tags or another strategy to distinguish prod from non-prod systems, and turn the non-prod systems off when you go home. Or, better yet, put together some simple automation to turn the non-prod systems off automatically when they aren’t in use. You can take an approach for stopping instances marked with a dev/test tag that is similar to this technique for terminating untagged instances: Use AWS Lambda to Terminate Untagged EC2 Instances
Third, use AWS reservations to reduce costs for prod EC2 and RDS instances. I will cover instance reservation strategies in another article, but you can save up to 75% compared to on-demand prices by reserving the instances that you use. It is a no-brainer to reserve instances that need to stay running all of the time. Depending on your situation, it may even make sense to reserve instances for non-prod if they need to stay running for a while. See Getting Started with Reserved Instances for more information.
Finally, try the EC2 spot instance market. For non-prod systems and for batch processes where you can afford to lose an instance with very little notice, spot instances can save you a lot of money. You need to be prepared to lose regular instances unexpectedly anyway, so it isn’t a big stretch to put the same availability concepts to use with spot instances. Spot instances represent excess EC2 capacity that AWS sells to the highest bidder. As long as you are the high bidder, you get to use the instances you request. When you are outbid, the instances are stopped and the resources get redirected. You generally pay only a fraction of the on-demand price, and you only pay for full hours of use. See Amazon EC2 Spot Instances for more information.
Depending on the size of your operation, it may be very reasonable to manage your costs using the tools that AWS provides by default in your account, which now includes Cost Explorer. As you grow, you may find it helpful to supplement that with AWS Trusted Advisor, which comes with the paid support options. There are a number of third-party tools, such as CloudHealth, CloudCheckr, and Cloudability, that provide lots of insights and analytics to assist with cost management. You may find one of those tools to be essential to managing your accounts if you have a large or complex environment.
As for organizing resources to allow cost tracking and charge-back to individual projects, there are a couple of approaches. The most flexible approach is to tag the resources with a project or department name. See Tagging and ARN Path Best Practices for some possible tagging strategies to help with insight into usage and costs. As noted there, there are some limitations to what can be tagged, and there are variations in tag restrictions from one service to another. For that reason, some AWS customers prefer to have multiple AWS accounts linked to a consolidated billing account and assign each project or each department an account to exclusively use. That is a more bullet-proof, if somewhat limiting, approach. It tends to work best if the projects or departments are independent and have limited interaction with each other.
P.S. Don’t overlook the possibility that you don’t need a server at all for some of your computing needs. AWS Lambda, paired with other AWS services such as API Gateway, can simplify your management burden and reduce your costs significantly over traditional architectures.
March 18th, 2016
AWS EC2 Cost Management Tips
By Gregg Gibson
If you are new to AWS, you might not be very worried about cost. After all, $0.05 per hour for an EC2 instance sounds pretty good. But, if you have been at it for a while, you may be noticing that the costs start to add up and have the potential to get out of control if you don’t give cost management a little bit of attention. The good news is that there are several tools and approaches you can use to keep things under control with a minimum of effort.
First, pay attention to what you are using. Take the time once you get your app running to make sure your EC2 instance selection is the least expensive type that meets your requirements. And, make good use of autoscale groups and the t2 instance family if the load on your app varies. Rather than using a larger instance type to handle load, tune for the minimum load and configure your autoscale group to grow a cluster of instances to meet the demand. The t2 instance family, in particular, has bursting capability to handle short spikes in demand without even requiring the autoscale group to be scaled up.
Second, make sure that EC2 instances and other resources that you aren’t using are turned off. If you have a 40 hour work week, there are 128 other hours in the week that aren’t work hours. Why leave your dev and test resources running all the time racking up charges unnecessarily? Save some money, and turn them off when not in use. Use tags or another strategy to distinguish prod from non-prod systems, and turn the non-prod systems off when you go home. Or, better yet, put together some simple automation to turn the non-prod systems off automatically when they aren’t in use. You can take an approach for stopping instances marked with a dev/test tag that is similar to this technique for terminating untagged instances: Use AWS Lambda to Terminate Untagged EC2 Instances
Third, use AWS reservations to reduce costs for prod EC2 and RDS instances. I will cover instance reservation strategies in another article, but you can save up to 75% compared to on-demand prices by reserving the instances that you use. It is a no-brainer to reserve instances that need to stay running all of the time. Depending on your situation, it may even make sense to reserve instances for non-prod if they need to stay running for a while. See Getting Started with Reserved Instances for more information.
Finally, try the EC2 spot instance market. For non-prod systems and for batch processes where you can afford to lose an instance with very little notice, spot instances can save you a lot of money. You need to be prepared to lose regular instances unexpectedly anyway, so it isn’t a big stretch to put the same availability concepts to use with spot instances. Spot instances represent excess EC2 capacity that AWS sells to the highest bidder. As long as you are the high bidder, you get to use the instances you request. When you are outbid, the instances are stopped and the resources get redirected. You generally pay only a fraction of the on-demand price, and you only pay for full hours of use. See Amazon EC2 Spot Instances for more information.
Depending on the size of your operation, it may be very reasonable to manage your costs using the tools that AWS provides by default in your account, which now includes Cost Explorer. As you grow, you may find it helpful to supplement that with AWS Trusted Advisor, which comes with the paid support options. There are a number of third-party tools, such as CloudHealth, CloudCheckr, and Cloudability, that provide lots of insights and analytics to assist with cost management. You may find one of those tools to be essential to managing your accounts if you have a large or complex environment.
As for organizing resources to allow cost tracking and charge-back to individual projects, there are a couple of approaches. The most flexible approach is to tag the resources with a project or department name. See Tagging and ARN Path Best Practices for some possible tagging strategies to help with insight into usage and costs. As noted there, there are some limitations to what can be tagged, and there are variations in tag restrictions from one service to another. For that reason, some AWS customers prefer to have multiple AWS accounts linked to a consolidated billing account and assign each project or each department an account to exclusively use. That is a more bullet-proof, if somewhat limiting, approach. It tends to work best if the projects or departments are independent and have limited interaction with each other.
P.S. Don’t overlook the possibility that you don’t need a server at all for some of your computing needs. AWS Lambda, paired with other AWS services such as API Gateway, can simplify your management burden and reduce your costs significantly over traditional architectures.