Last week a friend asked for a solution to automatically take backups of their EBS-Backed EC2 instances on AWS. The EC2 instances included a mix of Linux and Windows AMIs.This solution uses an AWS Lambda function written in Python that is scheduled using Cloudwatch and the whole solution is deployed to AWS using Terraform.
The following two articles that manually create Lambda functions using the console helped as a starting point for my solution:
- Serverless – Automate AMI creation / deletion using AWS Lambda
- How to Automate AMI Backups & Cleanups, using AWS Lambda (Serverless), with EC2 Tags
This solution (https://github.com/Irtaza/maintain_amis/blob/master/lambda_function.py) uses a single Lambda function that does the following:
- Look for all EC2 instances that have a tag with key Backup and value True
- For each instance that has the Backup tag, look for the tag with key Retention with a integer value that specifies the number of days the backup AMI should be retained for. If a tag doesn’t exist the default value of 7 days is used.
- Create an AMI for each EC2 instance that has the Backup tag with a value of True. The AMI creation process will automatically create snapshots of each instance’s root volume and any other EBS volumes attached to that instance.
- Add a tag with key DeleteOn using the the value of Retention tag to calculate a date value for the DeleteOn tag.
- Look for all AMIs that have a DeleteOn tag with a date value less then the execution date. Then delete all of these AMIs.
- Previous step doesn’t automatically delete the snapshots of EBS volumes. So the next step gets all snapshots linked to the expired AMIs and then deletes them.
The complete Terraform script for deploying the Lambda function is available here on GitHub. I will breakdown the Terraform script in the following sections.
1. Create the Lambda function
The first step is to create the Lambda function.
2. Create IAM execution role
The next step is to create an IAM role that the Lambda function will assume during execution.
3. Create an AWS IAM policy
The lambda function needs access to various AWS resources. This is where an IAM policy comes in handy. The policy will be defined in two steps. First we will create the policy resource:
Then we will use Terraform data source to construct a JSON representation of an IAM policy document, that is referenced in the last line of the script above against the policy key. The policy lists all the actions that the Lambda function will need to perform on various AWS resources.
4. Attach the policy to the role
Now we will attach the policy created in the last step to the IAM role we created in the second step.
5. Create and attach a Cloudwatch rule
The last step is to add a Cloudwatch rule to trigger the Lambda function once a week.
If you don’t know how to deploy this script using Terraform, you should follow the “Getting Started” guide on Terraform’s website.