Fargate and Cucumber-js: A Review
Lately, here at Stackery, as we've begun shipping features more rapidly into the product, we've also been shifting some of our focus towards reliability and integration testing in preparation. I decided to try out AWS Fargate for UI integration testing using BDD and Cucumber-js in a day long experimental POC. Cucumber is a behavior driven development testing framework with test cases written in a language called gherkin that focuses specifically on user features. AWS Fargate is a recently released abstraction on top of ECS services that gets rid of managing EC2 instances. These are my conclusions:
1. Fargate is awesome. Why would you not use Fargate?
If you’re configuring a Fargate task via the AWS UI it’s somewhat confusing and clumsy. With Stackery, you can configure Fargate while avoiding the pain of the AWS UI entirely. The communication between AWS Lambda to a Fargate task is the same as it would be for a normal ECS service, so moving existing ECS clusters/services to Fargate is straightforward application logic-wise. Here’s a simplified code snippet, dockerTaskPort refers to the conveniently provided Stackery Port environment variable. See our docs for the Docker Task node for more information.
const repoName = `cross-region-us-east`;
const browserCiRepo = `https://${token}@github.com/sbaum1994/${repoName}.git`;
const dockerCommands = [
`echo 'Running node index.js'`,
`node index.js`
];
const env = {
ENV_VAR: 'value'
};
let dockerCommand = ['/bin/bash', '-c', dockerCommands.join('; ')];
const params = {
taskDefinition: dockerTaskPort.taskDefinitionId,
overrides: {
containerOverrides: [
{
name: '0'
}
]
},
launchType: 'FARGATE'
};
params.networkConfiguration = {
awsvpcConfiguration: {
subnets: dockerTaskPort.vpcSubnets.split(','),
assignPublicIp: (dockerTaskPort.assignPublicIPAddress ? 'ENABLED' : 'DISABLED')
}
};
params.overrides.containerOverrides[0].command = dockerCommand;
params.overrides.containerOverrides[0].environment = Object.keys(env).map((name) => {
return {name, value: env[name]};
});
const ecs = new AWS.ECS({ region: process.env.AWS_REGION });
return ecs.runTask(params)...
It's a nice plus that there are no EC2 configurations to worry about, and it also simplified scaling. In the past we’ve had to use an ECS cluster and service for CI when the integration testing has been too long running for AWS lambda. Here, my Fargate service just scales up and down nicely without having to worry about configuration, bottlenecks or cost.
Here's my UI integration testing set up, triggered by an endpoint that specifies the environment to test.
With Fargate there is still technically an ECS cluster that needs configuring on set up, and when using a load balancer and target group. You are still creating a task definition, containers, and a service. Stackery's UI makes it easy to understand and configure, but if I were doing this on my own I'd still find it a PIA. Furthermore, I could see Fargate not being ideal in some use cases, since you can't select the EC2 instance type.
Stackery UI setting up Fargate:
2. Cucumber is pretty cool too. BDD creates clear tests and transparent reporting.
I really like the abstraction Cucumber provides between the test definitions and underlying assertions/implementations. For this POC I created a simple "login.feature" file as follows:
Feature: Login
In order to use Stackery
As a single user
I want to login to my Stackery account
Background:
Given I've navigated to the Stackery app in the browser
And it has loaded successfully
Scenario: Logging in as a user with a provider set up
Given a test user account exists with a provider
When I login with my username and password
Then I'm taken to the "Stacks" page and see the text "Select a stack"
And I see the "Stackery Stacks" section populated in the page
And I see the "CloudFormation Stacks" section populated in the page
Each step maps to a function that uses Selenium Webdriver on headless chrome under the hood to run the tests. I also pass in configuration that lets the test know what the test account username and password is, what Stackery environment is being tested, and other definitions like the timeout settings. In my pipeline, I also added an S3 bucket to hold the latest Cucumber reporting results for visibility after a test finishes.
Report generated:
Overall, I think this can potentially be a great way to keep track of adding new features while maintaining existing ones / making sure everything is regressively tested on each merge. Furthermore it's clear, organized and user flow oriented, which can work well for a dashboard style app like ours with multiple, repeatable, extensible steps (Create Environment, Deploy a Stack To Environment) etc.
Related posts
