AWS Lambda - Python - ModuleNotFoundError：沒有名為“pandas”的模塊

Question

無論我嘗試多少種不同的配置，我都會遇到錯誤。

Activating Conda env - wfl
Conda env, wfl, activated
Setting up paths
/app/python
/opt/conda/lib/python3.9
/opt/conda/lib/python39.zip
/opt/conda/lib/python3.9/lib-dynload
/opt/conda/lib/python3.9/site-packages
/app/python/domain
/app/python/persistence
/app/python/processors
/app/python/sharedpython
/app/python/tools
/app/python/tests
Traceback (most recent call last):
File "/app/python/lambdas/test.py", line 3, in <module>
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Activating Conda env - wfl

我有一個 AWS Lambda function、python，它指向 ECR 中的 docker 圖像。 該應用程序使用 conda，這也增加了一點復雜性。 （完整的 requirements.txt 包括需要 conda 的 fbprophet）。 要激活 conda env，我需要入口點是一個包裝器 shell 腳本，docker-entrypoint.sh。 下面顯然是一個精簡版，但應該包含足夠的信息來收集洞察力。 當在本地運行或直接執行 docker 容器時，腳本運行沒有問題，直到它在 AWS Lambda 的上下文中執行時才會拋出這些錯誤......路徑/python 路徑是否有我丟失的東西？ 任何幫助將不勝感激！

注意：我是 Dockerfile/commands/optimizing layers/images 的新手，正如你所說的那樣……任何精簡圖像的見解也會有所幫助。 （還有一個 python/conda 環境/管理的新手......bleh！）

蟒蛇/lambdas/test.py

import json
import time
import pandas as pd

def lambda_handler(event, context):
    print("event: " + json.dumps(event, indent=2))
    print("context: " + json.dumps(context, indent=2))
    x = pd.DataFrame()
    print(x)

print('test.py')
x = pd.DataFrame()
print(x)
time.sleep(200)  # used to allow time to search file contents of container

Dockerfile

# Base Image
####
FROM ubuntu:20.04 AS core
# Upgrade installed packages
RUN apt update -y  &&  apt upgrade -y && \
    apt install -y --no-install-recommends python3.8 unixodbc gnupg2 curl ca-certificates python3-distutils unixodbc-dev && \
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1

# Add SQL ODBC Driver 17 for Ubuntu 20.04 -- https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15#ubuntu17
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
    curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
    apt update && \
    ACCEPT_EULA=Y apt install -y --no-install-recommends --allow-unauthenticated msodbcsql17 && \
    ACCEPT_EULA=Y apt install -y --no-install-recommends --allow-unauthenticated mssql-tools && \
    echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile && \
    echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc

RUN apt clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && \
    apt remove -y --purge curl gnupg2

# Set entrypoint
ENTRYPOINT [ "/bin/bash" ]


####
# Developer / Compiler Image
####
FROM core AS dev
# Upgrade installed packages
RUN apt update -y  &&  apt upgrade -y && \
    apt install -y --no-install-recommends curl python3.8-dev git python3-pip unixodbc-dev build-essential

# Install miniconda
RUN apt install wget
ENV CONDA_DIR /opt/conda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda
# Put conda in path so we can use conda activate
ENV PATH=$CONDA_DIR/bin:$PATH

RUN conda init bash && \
    conda create --name wfl && \
    activate wfl

# Install Python dependencies
COPY requirements.txt .

# Install pip and dependencies
RUN python3 -m pip install -U pip setuptools wheel && \
    python3 -m pip install --user -r requirements.txt && \
    rm -f requirements.txt

RUN apt clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && \
    apt remove -y --purge git curl python3.8-dev python3-pip unixodbc-dev build-essential

# Set entrypoint
ENTRYPOINT [ "/bin/bash" ]


####
# Runtime Image
####

# This is the third and final image; it copies the compiled
# binary over but starts from the core ubuntu:20.04 image.
FROM core AS runtime

COPY --from=dev /root/.local /root/.local
# Make sure scripts in .local are usable:
ENV PATH=/root/.local/bin:$PATH
ENV CONDA_DIR /opt/conda
ENV PYTHON_APP_DIR /app/python
# copy conda to be available for env activation
COPY --from=dev $CONDA_DIR $CONDA_DIR
ENV PATH=$CONDA_DIR"/bin:$PATH"
RUN pip install pandas

# cleanup any non application files
RUN rm -rf ./python/tests
# Load application files
COPY ./python $PYTHON_APP_DIR
COPY ./docker-entrypoint.sh $PYTHON_APP_DIR

# Set work directory
WORKDIR $PYTHON_APP_DIR
ENV PYTHONPATH $PYTHON_APP_DIR

# Set entrypoint
ENTRYPOINT ["/app/python/docker-entrypoint.sh"]
CMD ["lambdas/test.py"]

setup_paths.py

import os
import sys

PROJECTS_ROOT = os.path.dirname(os.path.abspath(__file__))

def add_package_to_sys_path(base, package_relative_path):
    package_path = os.path.join(base, package_relative_path)
    if package_path not in sys.path:
        sys.path.append(package_path)


add_package_to_sys_path(PROJECTS_ROOT, 'lambdas')
add_package_to_sys_path(PROJECTS_ROOT, 'domain')
add_package_to_sys_path(PROJECTS_ROOT, 'persistence')
add_package_to_sys_path(PROJECTS_ROOT, 'processors')
add_package_to_sys_path(PROJECTS_ROOT, 'sharedpython')
add_package_to_sys_path(PROJECTS_ROOT, 'tools')
add_package_to_sys_path(PROJECTS_ROOT, 'tests')

for p in sys.path: print(p)

docker-entrypoint.sh

#!/bin/bash
set -e

echo 'Activating Conda env - wfl'
activate wfl
echo 'Conda env, wfl, activated'

python setup_paths.py
python "$@"

要求.txt

pandas==1.2.4

項目結構

Answer 1

有 2 種可能性 A. 您可能沒有激活您的虛擬環境，所以請確保
B.你還沒有安裝pandas所以寫pip安裝pandas