是否可以通过 serverless.yml 向 PATH 环境变量添加路径？

Question

When I create an AWS Lambda Layer, all the contents / modules of my zip file go to /opt/ when the AWS Lambda executes. This easily becomes cumbersome and frustrating because I have to use absolute imports on all my lambdas. Example:

import json
import os
import importlib.util
spec = importlib.util.spec_from_file_location("dynamodb_layer.customer", "/opt/dynamodb_layer/customer.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

def fetch(event, context):

    CustomerManager = module.CustomerManager
    customer_manager = CustomerManager()

    body = customer_manager.list_customers(event["queryStringParameters"]["acquirer"])

    response = {
        "statusCode": 200,
        "headers": {
            "Access-Control-Allow-Origin": "*"
        },
        "body": json.dumps(body)
    }

    return response

So I was wondering, is it possible to add these /opt/paths to the PATH environment variable by beforehand through serverless.yml? In that way, I could just from dynamodb_layer.customer import CustomerManager , instead of that freakish ugliness.

Answer 1

I've Lambda layer for Python3.6 runtime. My my_package.zip structure is:

my_package.zip
 - python
   - lib
     - python3.6
       - site-packages
         - customer

All dependencies are in build folder in project root: eg build/python/lib/python3.6/site-packages/customer

Relevant section of my serverless.yml

layers:
  my_package:
    path: build             
    compatibleRuntimes:     
      - python3.6

In my Lambda I import my package like I would do any other package: import customer

Answer 2

Have you tried setting your PYTHONPATH env var? https://stackoverflow.com/a/5944201/6529424

Have you tried adding to sys.path? https://stackoverflow.com/a/12257807/6529424

Answer 3

In the zip archive, the module needs to be placed the in a python subdirectory so that when it is extracted as a layer in Lambda, it is located in /opt/python . That way you'll be able to directly import your module without the need for importlib .

It's documented here or see this detailed blogpost from an AWS dev evangelist for more.

Answer 4

Setting of PYTHONPATH variable is not required, as long as you place items correctly inside the zip file.

Simple modules, and package directories, these should be placed inside a directory "python", and then the whole python/ placed into the zip file for uploading to AWS as a layer. Don't forget to add the "compatible runtimes" (eg Python 3.6, 3.7, 3.8 ...) settings for the layers.

So as an example:

python/
- my_module.py
- my_package_dir
-- __init__.py
-- package_mod_1.py
-- package_mod_2.py

which then get included in the zip file.

zip -r my_layer_zip.zip python/

The modules can then be imported without any more ado, when accessed as a layer:

....
import my_module
from my_package.package_mod_2 import mod_2_function
....

You can see the package structure from within the lambda if you look at '/opt/python/' which will show my_module.py, my_package/ etc., this is easily tested using the AWS Lambda test function, assuming the layer is attached to the function (or else the code will error)

import json
import os

def lambda_handler(event, context):
    # TODO implement

    dir_list = os.listdir('/opt/python/')

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!'),
        'event': json.dumps(event),
        '/opt/python/': dir_list
    }