Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'httplib2'

Question

I am have installed all these libs:

google-api-core 1.22.1
google-api-python-client 1.10.0
google-auth 1.20.1
google-auth-httplib2 0.0.4
google-auth-oauthlib 0.4.1
google-cloud 0.34.0
google-cloud-core 1.4.1
google-cloud-storage 1.31.0
google-crc32c 1.0.0
google-resumable-media 1.0.0
googleapis-common-protos 1.52.0
httplib2 0.18.1

to a folder --> zipped it --> uploaded to AWS lambda's layers. However, I keep on getting [ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'httplib2' even thought when I run my pyhon file locally with such libs being in the enviroment I don't get any error after calling import httplib2 . Note: every other libs works on lambda so my uploading process should be correct.

Answer 1

You can create a Lambda custom layer with your packages.

To check this solution I created such a layer and can confirm that it works .

The technique used includes docker tool described in the recent AWS blog :

• How do I create a Lambda layer using a simulated Lambda environment with Docker?

Thus for this question, I verified it as follows:

1. Create empty folder, eg mylayer .

2. Go to the folder and create requirements.txt file with the content of

google-api-core==1.22.1
google-api-python-client==1.10.0
google-auth==1.20.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.1
google-cloud==0.34.0
google-cloud-core==1.4.1
google-cloud-storage==1.31.0
google-crc32c==1.0.0
google-resumable-media==1.0.0
googleapis-common-protos==1.52.0
httplib2==0.18.1

3. Run the following docker command:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"

4. Create layer as zip:

zip -r -9 mylayer.zip python 

5. Create lambda layer based on mylayer.zip in the AWS Console. Don't forget to specify Compatible runtimes to python3.8 .

6. Test the layer in lambda using the following lambda function:

import httplib2

def lambda_handler(event, context):
    # TODO implement
    print(dir(httplib2))
    return 

The function executes correctly:

['AUTH_SCHEME_CLASSES', 'AUTH_SCHEME_ORDER', 'AllHosts', 'Authentication', 'BasicAuthentication', 'CA_CERTS', 'Credentials', 'DEFAULT_MAX_REDIRECTS', 'DEFAULT_TLS_VERSION', 'DigestAuthentication', 'FailedToDecompressContent', 'FileCache', 'GoogleLoginAuthentication', 'HOP_BY_HOP', 'HTTPConnectionWithTimeout', 'HTTPSConnectionWithTimeout', 'HmacDigestAuthentication', 'Http', 'HttpLib2Error', 'HttpLib2ErrorWithResponse', 'KeyCerts', 'MalformedHeader', 'NORMALIZE_SPACE', 'ProxiesUnavailableError', 'ProxyInfo', 'REDIRECT_CODES', 'RETRIES', 'RedirectLimit', 'RedirectMissingLocation', 'RelativeURIError', 'Response', 'SAFE_METHODS', 'SCHEME_TO_CONNECTION', 'ServerNotFoundError', 'UNQUOTE_PAIRS', 'URI', 'USE_WWW_AUTH_STRICT_PARSING', 'UnimplementedDigestAuthOptionError', 'UnimplementedHmacDigestAuthOptionError', 'WWW_AUTH_RELAXED', 'WWW_AUTH_STRICT', 'WsseAuthentication', '_', '__all__', '__author__', '__builtins__', '__cached__', '__contributors__', '__copyright__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_bind_write_headers', '_build_ssl_context', '_cnonce', '_convert_byte_str', '_decompressContent', '_entry_disposition', '_get_end2end_headers', '_md5', '_normalize_headers', '_parse_cache_control', '_parse_www_authenticate', '_sha', '_updateCache', '_wsse_username_token', 'base64', 'calendar', 'certs', 'copy', 'debuglevel', 'email', 'errno', 'gzip', 'has_timeout', 'header', 'hmac', 'http', 'io', 'iri2uri', 'os', 'parse_uri', 'proxy_info_from_environment', 'proxy_info_from_url', 'random', 're', 're_unsafe', 're_url_scheme', 'safename', 'socket', 'socks', 'ssl', 'sys', 'time', 'urllib', 'urlnorm', 'zlib']