The memes around serverless are endless, generally involving how there are no servers, no operations teams, and how unicorns will use their magical horns to solve your application stability problems.
I had a great chat with @ryanycoleman about what #serverless means for #operations, how it frees up ops teams to do more meaningful and creative work, and why #NoOps is neither the goal nor promise of serverless. Give it a listen! https://t.co/U2XN1NhAV4
— Jeremy Daly @ AWS re:Invent 2020 (@jeremy_daly) November 30, 2020
As I said a few weeks ago with Jeremy Daly, serverless isn’t orthogonal to IT Operations. I believe quite the opposite, that serverless is the wave beyond VM configuration management in empowering operations-minded people to reclaim their focus, creativity, and business relevance.
Does the world’s largest serverless provider understand and embrace that concept?
It was pretty fun to watch AWS Serverless VP, David Richardson's, talk Increasing innovation with serverless applications, even if from a neon-stage (that's no Amerstadam sugar beet factory).
David recapped the many serverless announcements over the last few weeks while providing context on how they impact customers, through the lens of three use cases for serverless adoption.
It’s quite nice to see the IT automation focus for serverless!
Lambda’s event-driven architecture and on-demand computing footprint lends itself to IT automation workflows that are reactionary or investigative in nature, like running compliance checks on EC2 instances as they spin up, or diagnosing an outage through data aggregation and analysis.
I saw this happening ad-hoc, usually through bundles of scripts, but even die-hard IT Operations companies like Puppet are responding to the growth of event-driven IT automation with managed services like Relay.
IT Operations is in large part anticipating and reacting to events, which makes event-driven architectures like serverless a great fit for troubleshooting and governance workflows.
The unfortunately named DevOps Guru stole several keynote spotlights and gives me pause on AWS’s seriousness towards Ops. But don’t worry, ML-powered cloud operations won’t eliminate your ops team.
Having worked with Puppet and Splunk on something similar, the value here lies in reducing monotonous work to recognize patterns and run repetitive scripts to troubleshoot problems. Unlike the offering from Puppet, DevOps Guru does not appear to trigger any automation to remediate the identified issues. But, it might help you spend more time fixing problems instead of finding them.
The second major use-case for serverless adoption, in Richardson’s view, is data processing. From media streaming to log analytics to data lakes, AWS managed services excel at ingesting anything you have like an expensive game of hungry hungry hippos.
AWS has made a bunch of small improvements to make Lambda a better fit for these use cases, reducing overall server burden and limiting compute to when it’s needed.
If core compute is the third use-case, it is last but not least. This is the meat of the sandwich and it’s where all of the announcements live.
Stackery got a nice mention as a service you can use to manage your whole serverless lifecycle, including the delivery of AWS Lambda extensions. While initially focused on monitoring providers, provides a way for teams to assert requirements on Lambda functions without requiring development teams to include these requirements into their development process.
Instead of getting involved in every single Lambda function, Operations team can assert observability needs in an Extension and rest assured that every function will include them.
Stackery was also first to support AWS Lambda container image (build) support which sounds super confusing but is actually very helpful for IT Operations and release engineering teams who have invested in robust container delivery workflows but not yet Lambda.
This is not some magic that means you can invoke any container in response to a Lambda event (though ECS works for that!). Developers still need to account for Lambda’s event handlers and expect the assumptions of Lambda’s runtime environment.
But, the tools you use to manage, validate, build, and ship container images can be entirely reused in the software delivery lifecycle for Lambda functions shipped as container images. Stackery will go even further with you, ensuring that AWS ECR is maintained as changes to functions pass through its turn-key CI/CD pipeline.
Release engineering practices built around container images can be reused to validate, build, and deliver Lambda functions which previously required a bespoke toolchain.
David went on to talk about how serverless compute is used to decouple legacy systems in an incremental fashion, which can come from changing the source of data feeding into an application, emitting events on the backside of a legacy application, or refactoring an API one route at a time.
You do not have to jump into serverless head first, with incremental approaches to modernizing your application from every conceivable angle.
Up next, Richardson shared some of the improvements AWS made to help operators increase performance, while ensuring security, at reduced costs.
Lambda is billed at 1ms intervals now, code-signing support is baked-in (including SAM support), EFS volumes can finally be mounted in Lambda, and functions can now have 10GB of memory for those of you running Chrome in Lambda. And for some reason the impressive Aurora Serverless v2 wasn’t even mentioned (or maybe I missed it).
There’s a lot in there to unpack in January, and you can get Stackery will make all of this more accessible with our bidirectional infrastructure-as-code designer that generates best-practices by default.
I wrote operations in this post about as many times as AWS uses the word innovation in their presentations, but I’m walking away from re:Invent with the impression that AWS is serious about both.
How AWS deploys in waves and what to think about when planning your deployments
What to look for at re:Invent for serverless and DevOps professionals