numpy error when importing pandas with AWS Lambda

Question

I'm currently have an issue with importing the library pandas to my AWS Lambda Function. I have tried two scenarios.

• Installing pandas directly into one folder with my lambda_function and uploading the zipped file.

• Creating a layer with an uploaded zip file with the following structure:

- python
    - lib
        - python3.8
            - site-packages
                - all the pandas packages here

My lambda_function is just:

import json
import pandas as pd

def lambda_handler(event, context):
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

This is my error:

START RequestId: 9e27641e-587b-4be2-b9be-c9be85007f9e Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'main': Unable to import required dependencies:
numpy: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/var/lang/bin/python3.8"
  * The NumPy version is: "1.21.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

Is there any other approach? I don't want to use Docker for this task. Thanks!

Answer 1

I have solved the issue, thanks to this article:

https://korniichuk.medium.com/lambda-with-pandas-fd81aa2ff25e

In my case, I cannot normally install the libraries through pip, I'm on a windows machine. You must install the linux versions of pandas and numpy. Since I'm on python 3.8 I installed these versions:

• numpy-1.21.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
• pandas-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

After downloading the packages, I replaced the pandas and numpy folders that were originally from the install via pip install pandas . I used my first scenario as showed in my question.

Answer 2

Just to add on, here is a collection of some very useful libraries as layers

AWSome Lambda Layers

Kudos to the contributors!