部署到AWS Lambda匹配大小限制和包（例如numpy）不兼容

Question

I have a relatively small script where I use the following dependencies:

firebase-admin
pandas

Nevertheless, when you install these dependencies you get over 60 additional dependencies installed as well, such as numpy , scikit , google_cloud etc.

I encountered many problems during deployment, of which the following were the most striking:

1. You quickly hit the size limit of 250MB. numpy alone is approx 100mb.
2. Some packages with compiled libraries are not compatible, eg numpy , so you need to unwheel them (see link ).

I have spent now maybe two days trying to get around these problems. However, I did not manage to succeed to run my small script at all.

I am therefore wondering:

• Is AWS lambda at all suitable for my use case given the difficulties in deploying? If it is so hard to deploy a simple script with the dependencies listed above?
• What are other good practices to overcome the dependency installation? Is it possible to split up the dependencies from the actual functions?
• What are other better (more simple) ways to deploy to Lambda? In particular, I find it cumbersome to zip every time all my functions and site-packages in order to just to do a small update .

---

Note on the latter: I have tried also the following solutions:

• zappa : Following this tutorial , I tried deploying it using zappa. After encountering and somehow resolving the size limit problem, I had another problem which is that my small script did not had an app function in the format myservice.app. After googling I found that zappa was designed for WSGI apps.
• python-lambda : I also tried this other often recommended alternative. However, I was unable to call the command lambda init after installation. See my question on that issue here .
• using aws sam and layers . I followed the README.me after installing aws-sam-cli and followed all the steps. This resulted that I uploaded a new package on S3, which I used to create a new layer. Nevertheless, I kept receiving the error "numpy not found" despite that I put numpy in my requirements.txt and properly build the site packages (as it was in my SamApp visible).

Answer 1

You can deploy to lambda & surrounding services using serverless framework. I use serverless framework with the serverless-python-requirements plugin which will compile and bundle any needed python requirements. Under the hood it translates into a cloudformation template and deploys your function(s) and dependent resources as a stack into AWS.