AWS Lambda Dev Workflow

Question

I've been using AWS for a while now but am wondering about how to go about developing with Lambda. I'm a big fan of having server-less functions and letting Amazon handle the maintenance and have been using it for a while. My question: Is there a recommended workflow for version control and development?

I understand there's the ability to publish a new version in Lambda. And that you can point to specific versions in a service that calls it, such as API Gateway. I see API Gateway also has some nice abilities to partition who calls which version. ie Having a test API and also slowly rolling updates to say 10% of production API calls and scaling up slowly.

However, this feels a bit clunky for an actual version control system. Perhaps the functions are coded locally and uploaded using the AWS CLI and then everything is managed through a third party version control system (Github, Bitbucket, etc)? Can I deploy to new or existing versions of the function this way? That way I can maintain a separation of test and production functions.

Development also doesn't feel as nice through the editor in Lambda. Not to mention using custom packages require to upload anyways. Seems local development is the better solution. Trying to understand others workflows so I can improve mine.

How have you approached this issue in your experience?

Answer 1

You can have version control on your lambda using aws CodeCommit (much simpler than using an external git repository system, although you can do either). Here is a tutorial for setting up a CodePipeline for commit/build/deploy stages: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html

This example deploys an EC2 instance, so for the deploy portion for a lambda, see here

If you set up a pipeline you can have an initial commit stage, then a build stage that runs your unit tests and packages the code, and then a deploy stage (and potentially more stages if required). It's a very organized way of deploying lambda changes.

Answer 2

I wrote roughly a dozen lambda functions that trigger based on S3 file write event or time, and make a HTTP req to an API to kickstart data processing jobs.

I don't think there's any gold standard. From my research, there are various approaches and frameworks out there. I decided that I didn't want to depend on any kind of frameworks like Serverless nor Apex because I didn't want to learn how to use those things on top of learning about Lambda. Instead I built out improvements organically based on my needs as I was developing a function.

To answer your question, here's my workflow.

1. Develop locally and git commit changes.
2. Mock test data and test locally using mocha and chai.
3. Run a bash script that creates a zip file compressing files to be deployed to AWS lambda.
4. Upload the zip file to AWS lambda.

Answer 3

I would suggest you to have a look at SAM. SAM is a command line tool and a framework to help you to develop your serverless application. Using SAM, you can test your applications locally before to upload them to the cloud. It also support blue / green deployment and CI/CD workflows, starting automatically from github.

https://github.com/awslabs/aws-sam-cli

Answer 4

I think Serverless Framework + standard Git workflow is your best bet here. Each Git branch is tied to a completely separate environment (or stage). They can also be in different AWS accounts as well .

Your team works in feature branches or uses a PR based workflow and your deploys are associated to Git commits. This saves you the trouble of dealing with Lambda versioning.

So locally you might deploy to a certain set of branches. While for your staging and production environments you might want to connect them through your CI process.

We've written about the Git workflow aspect of this in detail here - https://seed.run/blog/git-workflow-for-serverless-apps