AWS Python lambda: pip install requirements.txt with --no-deps

Question

Using AWS CDK v2, I want to deploy two lambda functions . As both depend on pandas , I want to deploy a layer first, which has pandas and its dependencies in requirements.txt located at entry .

One of the lambda functions also depends on pyarrow , which I put into the requirements.txt located at entry of that lambda function. As both pyarrow and pandas depend numpy , I currently end up with numpy and its dependencies being installed both in the layer and that lambda function, which obviously isn't what I want.

In the end, I'm looking for a way to pass --no-deps to the pip installation of requirements.txt of the lambda function that depends on pyarrow . Any ideas how to achieve this?

Answer 1

I don't think this is possible to specify no-deps when using a requirements.txt. CDK also can use pipenv 's Pipfile and Poetry 's poetry.lock . That being said, in my research none of those tools seem to support installing with --no-deps either unless you manually mess with the lock files (not advised or maintainable really).

Looking deeper at the specific problem, I don't see pyarrow listing pandas as a dependency but I do see it installing numpy which is a dependency of pandas

I think your best option presently is to just deal with having numpy in both places or put both pyarrow and pandas in the layer.