AWS Lambda Code Unable to Import Depdendencies from S3 — Runtime.ImportModuleError

Question

I'm trying to deploy a Python lambda function with external dependencies, but I'm hitting an error because it doesn't see my external dependencies. "Unable to import module 'metrics': No module named 'github'"

Here's my deploy script. My python script with the lambda_handler() is metrics.py .

mkdir lambda_code

# populate lambda_code directory with python libraries
pip3 install --quiet -r requirements.txt --target lambda_code/

# compress the lambda_code directory and add metrics.py to the zip
zip -qq -r9 lambda_code.zip lambda_code/

zip -qq -g lambda_code.zip metrics.py

aws s3 cp lambda_code.zip s3://$BUCKET/lambda_code.zip

aws lambda update-function-code --function-name $FUNCTION_NAME --s3-bucket $BUCKET --s3-key lambda_code.zip

Here's the tree of my upacked lamdba_code.zip This is where things aren't working. It doesn't make sense to me why the lambda can't see the github module. I've also tried putting metrics.py directly in the lambda_code directory, but still nothing.

.
├── lambda_code
│   ├── Deprecated-1.2.5.dist-info
│   ├── PyGithub-1.43.7.dist-info
│   ├── PyJWT-1.7.1.dist-info
│   ├── __pycache__
│   ├── bin
│   ├── certifi
│   ├── certifi-2019.3.9.dist-info
│   ├── chardet
│   ├── chardet-3.0.4.dist-info
│   ├── cycler-0.10.0.dist-info
│   ├── cycler.py
│   ├── dateutil
│   ├── deprecated
│   ├── easy_install.py
│   ├── github
│   ├── idna
│   ├── idna-2.8.dist-info
│   ├── jwt
│   ├── kiwisolver-1.1.0.dist-info
│   ├── kiwisolver.cpython-37m-darwin.so
│   ├── matplotlib
│   ├── matplotlib-3.0.3-py3.7-nspkg.pth
│   ├── matplotlib-3.0.3.dist-info
│   ├── mpl_toolkits
│   ├── numpy
│   ├── numpy-1.16.3.dist-info
│   ├── pandas
│   ├── pandas-0.24.2.dist-info
│   ├── pkg_resources
│   ├── pylab.py
│   ├── pyparsing-2.4.0.dist-info
│   ├── pyparsing.py
│   ├── python_dateutil-2.8.0.dist-info
│   ├── pytz
│   ├── pytz-2019.1.dist-info
│   ├── requests
│   ├── requests-2.21.0.dist-info
│   ├── setuptools
│   ├── setuptools-41.0.1.dist-info
│   ├── six-1.12.0.dist-info
│   ├── six.py
│   ├── urllib3
│   ├── urllib3-1.24.3.dist-info
│   ├── wrapt
│   └── wrapt-1.11.1.dist-info
└── metrics.py

Finally, here's the beginning of the lambda code. The error occurs when trying to import github.

"""Obtains total number of releases on Github.com and creates data 
visualizations"""

import datetime
import io
import os
import sys

from base64 import b64decode
from github import Github
import boto3
import matplotlib.pyplot as plt
import pandas as pd

ENCRYPTED = os.environ['github_credentials']
DECRYPTED = 
boto3.client('kms').decrypt(CiphertextBlob=b64decode(ENCRYPTED)) . 
['Plaintext']

def lambda_handler(event, context):

Answer 1

You either need to have your metrics.py and the sub folders of lambda_code into a single folder, or you need to import the modules like, lambda_code.pytz for each module that is zipped.

If you follow your current directory structure, lambda_code becomes a module and all other modules can be referred only using lambda_code. because, folders inside lambda_code becomes sub modules of lambda_code . I would suggest you to copy sub folders of lambda_code to root directory, ie the directory which your metrics.py resides. Then delete the lambda_code folder and zip and upload. This way, you might not need to edit your code.