Highlights from the final week of re:Invent

The three weeks of re:invent are finally drawing to a close and it’s been a whirlwind of announcements and recaps of the most unusual year of the century. The major takeaway for me, above all the industry news, is that the work didn’t stop. Innovation continued at full speed despite all our lives being disrupted. From the talks this week it’s clear that serverless is growing at massive speed this year.

Increasing innovation with Serverless applications

David Richarson, VP of serverless from AWS, started the session off by thanking the Serverless Community for all its hard work.

“We launched Lambda six years ago, and we continue to be humbled by how many people have adopted it. Hundreds of thousands of customers have built applications that already drive trillions of invokes per month, and lambda continues to grow at a phenomenal rate. Lambda may have started serverless, but the incredibly generous community of Serverless heroes have turned serverless into a revolution in how to build in the cloud.”

Thanks, David, we appreciate being acknowledged, and I am also grateful for the generosity and collaboration that the Serverless Community provides.

The talk also touched on the Extensions API, which opens up new avenues for growth for Lambda tooling. If you haven’t tried them, extensions let you run processes in a shared space with your Lambda process. For those of us making tooling for Lambda functions, Extensions have the potential to unlock new avenues for governance, measurement, and management of Lambda functions.

Then there was this great anecdote about growing with Lambda Functions:

“Coca Cola is a great customer who had to react quickly to the changes brought on by the COVID pandemic. They wanted to provide a touchless experience for their freestyle drink dispensers. And so they built a new smartphone app that allows customers to order and pay for drinks, without coming into contact with their vending machines. Because they built with Lambda, their team was able to focus on the application, rather than server latency and scalability, since those are built-in with Lambda.

And as a result, they built this new application in just 100 days. And now over 30,000 machines have the touchless capability.”

Next in the talk we heard from Lavanya Raja, Vice President of Software Engineering at Capital One. She described how their call center had found itself nearly overwhelmed early in the quarantine. But with a serverless architecture they managed to implement tons of new features for self-service and get more of their call center working from home in just a matter of weeks.

They architected their call centers with Amazon connect at its core and are using Lambda Functions to orchestrate the call flow.

Finally, we heard from Vishal Parikh, Vice President and Head of Product at Discovery Golf. Launched in 2019, GOLFTV had already launched monolithic code and then shifted to serverless for some of their stack: “This means we can grow and adapt quickly when opportunities arise, swapping things in and out as the business changes. For example, we can scale up automatically for unpredictable traffic spikes, adapt to new data sources quickly, or even add features for new revenue streams faster. We're happy with the tasks that we chose given the increase in engineering velocity, the reduction of cloud costs, and the estimated reduction of cloud cost to implement compared to a non-serverless approach”.

Just look at that flexible architecture!

Again, the takeaway is that work hasn’t stopped. The pressure for innovation has continued to grow and serverless has been the method for enabling that innovation.

Going global in minutes: Multi-Region expansion simplified

This talk was a really nice ‘trifecta’ of components. Using open source, containers, and serverless is bound to catch my interest.

Nima Kaviani from AWS and Brandon Leach of Autodesk showed us how Autodesk leveraged Amazon ECS, AWS Lambda, and OSS tooling such as Spinnaker to enable it to expand into new Regions with minimal operational overhead.

The challenge faced by Autodesk was smooth scaling in a multi-region environment where data sovereignty is an issue, and for example, you need all Canadian users’ data to stay in Canada.

Some great tidbits in the talk, I would have assumed that Terraform and Spinnaker weren’t really things you’d use together, but Brandon said otherwise:“Spinnaker and Terraform are very complimentary. You know, we've utilized Spinnaker’s ability to run Terraform in the context of a deployment pipeline to allow for the management of all service components. This capability allows us to manage all infrastructure related to a service using Spinnaker pipelines. Another step we took was to abstract away the complexity of Spinnaker pipelines terraform accounts in networking by implementing a declarative interface Common Application definition file.”

Adopting open source can be a real power move for the right team. In this case the group ended up adding some features to Spinnaker that they needed, meaning everyone got the benefit of new multi-account tools.

Data-driven vehicle development at Volkswagen Commercial Vehicles

Last but not least, another talk from the automotive sector about managing massive amounts of data using serverless. Volkswagen is gaining insights from terabytes of measurement data from its test fleet as well as from the vehicles of pilot customers. They set up a scalable, serverless data processing pipeline to harvest insights from raw vehicle sensor data using Amazon S3, AWS Lambda, Amazon Elasticsearch Service, and more.The talk was delivered by John Fisher of AWS with Dr. Mohammad Al-Rifai and Dr. Michael Nolting from Volkswagen Commercial Vehicles.

First off I liked this quote describing why Volkswagen was doing all this measurement:

“We do a lot of tests trying to measure the load of different components in different driving situations. We collect a lot of data from our test fleet, we analyze this data, and we make sure that the vehicles will be working as expected in every possible driving situation. We have some customers where we get their consent to install some data loggers in their vehicles and collect a lot of data while they are using it. And then we try to compare the data with the data from our test fleet to make sure that, first of all, we are testing the vehicles in the right way.”

That’s going to result in an absolute sea of data, and while S3 can certainly hold all that data, getting it in was a challenge:

“The challenge at the beginning was that we are dealing with more than 100 terabytes of data that needs to be imported or uploaded and how we solve it is by using a few snowballs.”

I love hearing about the use of AWS Snowball, just the idea of terabytes of data travelling through the US mail.

In the end, the Volkswagen team used Lambda Functions to process the data into usable models, and a SQL storage system for their parameterized data.

“What we have shown you here is how we build this very scalable solution to process terabytes of data on AWS to extract some useful insights, out of the data to have this understanding of the usage of the vehicles by customers to be able to compare it with our testing procedures in our test fleet. So, starting from the vehicle and the data, creating the insights. We can then learn from it, and build better surfaces and better products for our customers, which is a win win situation for all of us.”

While I think it’s neat to see stories of agility, this talk was an illustration of the raw power of AWS. There simply would have been no way to do this level of data processing without a billion-dollar investment before modern cloud tools.

That’s a wrap from me this re:Invent, I have truly enjoyed sharing all these stories of how companies are leveraging the true power of what Serverless unlocks. I am excited for what 2021 holds - the shift toward microservice architecture means we’ll see even more democratization of the tech industry: ten years ago you really had to be an expert to deploy a basic web backend.

Now, even a n00b like me, can deploy an API endpoint that deploys to multiple environments and scales automatically. If you look at the cloud security industry, most of its luminaries are autodidacts. Successful organizations will be ones that can welcome newcomers, see their skills when other people are asking for degrees, and foster skills growth.