Cost is a big reason many dev teams are transitioning to serverless. However, there are still some ways costs can creep up on you in serverless apps. The biggest culprit I've found in my own experience is the VPC resource.

Because adding a VPC to a serverless stack is ridiculously easy in Stackery, I've sometimes gotten carried away. I'd deploy a stack with a VPC for testing, then quickly forget about it. Meanwhile, the unused but still very much billed AWS resource would be sitting there, very un-serverlessly, adding to our AWS bill, until the one day at the end of the month when our CTO would politely remind me that my negligence was costing the company upwards of tens of dollars. D'oh!

So I decided this problem of human forgetfulness could be solved with some automation - specifically, a VPC eradicator Lambda function triggered by a cron timer. Here's how I built it.

Search, then destroy

In order to eradicate a VPC, we need to find it first. For that, I created a Lambda function in NodeJS10.x and added the following simple script using the AWS SDK for Javascript:

const aws = require('aws-sdk'); exports.handler = async () => { const ec2 = new aws.EC2(); let findVpcs = await ec2.describeVpcs().promise(); return findVpcs; };

It's a simple start, but running stackery local invoke on this function will only return a VPC if it's in your default region. So, we need to get all available regions and iterate through them. That step looks like this:

const aws = require('aws-sdk'); exports.handler = async () => { const ec2 = new aws.EC2(); const ec2Regions = await ec2.describeRegions().promise(); for (let region of ec2Regions.Regions) { await runEradicator(region.RegionName); } return {}; }; async function runEradicator (region) { const ec2region = new aws.EC2({region: region}); let findVpcs = await ec2region.describeVpcs().promise(); if (findVpcs.Vpcs.length > 0) { await eradicateVpc(findVpcs.Vpcs, region); } else { console.log(`No VPCs found in ${region}, your money is safe for now!`); } };

Now we can list all of our VPCs in every supported region. We can even send a warning email when those VPCs pop up.

But if we want to delete them, well, that's when things get tricky.

Clean all the things?

Despite years of pleading from developers, there is no way to delete all of a VPC's dependencies outside of the AWS Console. In fact, even in the console, one must go region by region to delete each VPC individually. If you try using the AWS CLI or the SDK, you will get the following message:

The vpc 'vpc-09dsf5654123eaa' has dependencies and cannot be deleted.

Unfortunately, there's no easy way to find out what those dependencies are. Through extensive googling and some trial and error, I managed to find a rough order of operations for VPC dependency deletion, and it goes like this:

1. Describe, detach and delete Elastic Network Interfaces (ENIs)
2. Describe, detach and delete Internet Gateways (IGWs)
3. Describe and delete Route Tables
4. Describe and delete Network Access Control lists (ACLs)
5. Describe and delete Security Groups
6. Describe and delete Subnets
7. Finally, delete the VPC

The Lambda function that does all of the above is rather lengthy, so you can view it here. The handler calls a function for each of the above actions.

The good news is, the VPC eradicator stack itself is very simple:

It consists of the Lambda function linked to above, which is doing all of the heavy lifting, and then a cron timer that triggers the function at the same time each day. Here is the stack's template.yaml file:

AWSTemplateFormatVersion: 2010-09-09 Transform: AWS::Serverless-2016-10-31 Resources: eradicateVPC: Type: AWS::Serverless::Function Properties: FunctionName: !Sub ${AWS::StackName}-eradicateVPC Description: !Sub - Stack ${StackTagName} Environment ${EnvironmentTagName} Function ${ResourceName} - ResourceName: eradicateVPC CodeUri: src/eradicateVPC Handler: index.handler Runtime: nodejs10.x MemorySize: 3008 Timeout: 900 Tracing: Active Policies: - AWSXrayWriteOnlyAccess - AWSConfigRole - AmazonVPCFullAccess Events: Timer: Type: Schedule Properties: Schedule: cron(15 16 * * ? *) Metadata: StackeryName: TriggerDailyEradication Parameters: StackTagName: Type: String Description: Stack Name (injected by Stackery at deployment time) EnvironmentTagName: Type: String Description: Environment Name (injected by Stackery at deployment time)

Short and sweet, yet it does the job - mostly.

Unfortunately, there are limitations to the eradicator, in that it works best on default VPCs and won't delete a VPC that's part of an active CloudFormation stack. This is a limitation of the AWS SDK, but the good news is that deleting a stack with a VPC will delete that VPC and its dependencies, and can be done very easily with the following Stackery CLI command:

stackery undeploy -n <the offending stack name> -e <environment name>

Give it a spin

If you want to try it out yourself, here is the AWS VPC Eradicator repository, which I've open-sourced and opened to contributors. You can clone it to your Stackery account and then deploy it into your AWS account with these simple commands:

# create a new stack in your Git account based on the existing stack stackery create -n vpc-eradicator -p github --github-org <your github username> --blueprint-git-url https://github.com/bildungsroman/aws-vpc-eradicator/ # deploy the stack to your AWS account stackery deploy -n vpc-eradicator -e <your enviornment name> -r master --aws-profile <your AWS account profile>

The easiest way to test it out is to create a default VPC in the AWS Console, and after you have deployed the aws-vpc-eradicator stack, use Stackery local invoke in your cloned repo's src/eradicateVPC/ directory to test it out:

stackery local invoke --env-name <your deployed enviornment name> --aws-profile <your AWS account profile> # expected output: 2019-08-26T18:39:48.702Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO No VPCs found in eu-north-1, your money is safe for now! ... 2019-08-26T18:40:00.515Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO No VPCs found in us-east-2, your money is safe for now! 2019-08-26T18:40:00.958Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO No VPCs found in us-west-1, your money is safe for now! 2019-08-26T18:40:01.113Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO Oh noes! 1 VPC discovered in region us-west-2! Running eradicator. 2019-08-26T18:40:01.500Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO Detaching IGW igw-021f76714ca83c550 2019-08-26T18:40:01.644Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO Deleting IGW igw-021f76714ca83c550 ... 2019-08-26T18:40:03.960Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO Deleting subnet subnet-082e57b8ad3a37c10 2019-08-26T18:40:04.124Z 52fdfc07-2182-154f-163f-5f0f9a621d72 INFO vpc-09fd8d149849f8270 in region us-west-2 eradicated! Use that cash for something else!

Then, when you go refresh the AWS Console, you should see this message:

Good luck eradicating those VPCs!