AWS Lambda Docker Custom Python Library (Runtime.ImportModuleError)

Question

I'm trying to deploy a custom machine learning library on Lambda using aLambda docker image .

The image is looking approximately as follows:

FROM public.ecr.aws/lambda/python:3.9

CMD mkdir -p /workspace
WORKDIR /workspace

# Copy necessary files (e.g. setup.py and library/)
COPY ...

# Install library and requirements
RUN pip3 install . --target "${LAMBDA_TASK_ROOT}"

# Copy lambda scripts
COPY ... "${LAMBDA_TASK_ROOT}/"

# CMD [ my_script.my_handler ]

Thus, it installs a local python package including dependencies to LAMBDA_TASK_ROOT (/var/task). The CMD (handler) is overriden in AWS, eg preprocessing.lambda_handler .

The container works fine for handlers that DO NOT use the custom python library (on AWS and locally). However, trying to use the custom library on AWS, it fails with Runtime.ImportModuleError claiming that "errorMessage": "Unable to import module 'MY_HANDLER': No module named 'MY_LIBRARY'" .

Everything works when running the container locally with the runtime interface emulator . The file-level permissions should be ok as well.

Is there anything wrong with this approach?

Answer 1

Answering my own question here: There was no problem with the Docker container itself. The problem was that Lambda references docker image versions via their SHA, not the tag , thus update to the tag did not update the container of the functions. To update the container image, you have to use something like

aws lambda update-function-code --function-name $MY_FN --image-uri $MY_URI