gspread-pandas 作为 AWS Lambda 中的层

Question

So I tried to set-up the package in a layer in AWS Lambda to use it later in a function. Unfortunately, I can't seem to make the env variable work or set it properly so it can read the google_secret.json

To prepare the lambda package I:

1. Created an EC2 instance
2. Created a local directory: build/python/lib/python3.7/site-packages
3. Installed all the packages I use in that directory, including gspread_pandas
4. Zipped the python directory
5. Uploaded it to S3 and then used it as a layer

The process worked for all the other packages except for this one, as I didn't have the config pointing to the google_secret.json

As a fix I tried to:

1. Change this path in the config file _default_dir = ( Path(environ.get("$XDG_CONFIG_HOME", Path(environ.get("HOME")) / ".config")) / "gspread_pandas" ) to a path I created in the python directory (that was zipped) where I stored the google_secret.json: "python/config/gspread_pandas/"
2. Create a file google_secret.json in the lambda function, create a environment variable "GSPREAD_PANDAS_CONFIG_DIR" and set it the file's path

None of the above worked, unfortunately.

In both cases, I get the following error

    { "errorMessage": "expected str, bytes or os.PathLike object, not NoneType", 
"errorType": "TypeError", 
"stackTrace": [ " File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n return load_source(name, filename, file)\n", 
" File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n module = _load(spec)\n", 
" File \"<frozen importlib._bootstrap>\", line 696, in _load\n", " File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n", 
" File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n", 
" File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n", 
" File \"/var/task/lambda_function.py\", line 8, in <module>\n import gspread_pandas\n", 
" File \"/opt/python/lib/python3.7/site-packages/gspread_pandas/__init__.py\", line 5, in <module>\n from .client import Client\n", 
" File \"/opt/python/lib/python3.7/site-packages/gspread_pandas/client.py\", line 15, in <module>\n from gspread_pandas.conf import default_scope, get_creds\n", 
" File \"/opt/python/lib/python3.7/site-packages/gspread_pandas/conf.py\", line 25, in <module>\n Path(environ.get(\"$XDG_CONFIG_HOME\", Path(environ.get(\"HOME\")) / \".config\"))\n", 
" File \"/var/lang/lib/python3.7/pathlib.py\", line 1022, in __new__\n self = cls._from_parts(args, init=False)\n", 
" File \"/var/lang/lib/python3.7/pathlib.py\", line 669, in _from_parts\n drv, root, parts = self._parse_args(args)\n", 
" File \"/var/lang/lib/python3.7/pathlib.py\", line 653, in _parse_args\n a = os.fspath(a)\n" ] }

I have to mention that I don't have much experience with coding, so I apologize in advance for the poor explanation.

Any help would be greatly appreciated!

Answer 1

These steps worked out fine for me.

1. Create this folder structure for your layer:

   $ mkdir -p lambda_layers/python/lib/python3.7/site-packages

2. Get the package into the site-packages folder:

   $ pip install gspread-pandas -t lambda_layers/python/lib/python3.7/site-packages/.

3. Create the layer's zip file:

   $ cd lambda_layers

   $ zip -r gspread-pandas_lambda_layer.zip *