使用 package 用 nvcc 编译并安装 setup.py 和远程解释器

Question

我使用我的 docker 容器的远程解释器。

Dockerfile 内容为：

FROM nvidia/cuda:8.0-cudnn7-devel-ubuntu16.04

RUN apt-get update && apt-get install -y \
    rsync \
    htop \
    git \
    openssh-server \
    nano \
    cmake \
    python-opencv \
    python2.7 \
    unzip


RUN apt-get -qq -y install curl bzip2 && \
    curl -sSL https://repo.continuum.io/miniconda/Miniconda2-4.6.14-Linux-x86_64.sh -o /tmp/miniconda.sh && \
    bash /tmp/miniconda.sh -bfp /usr/local && \
    rm -rf /tmp/miniconda.sh && \
    conda install -y python=2 && \
    conda update conda && \
    apt-get -qq -y autoremove && \
    apt-get autoclean && \
    rm -rf /var/lib/apt/lists/* /var/log/dpkg.log && \
    conda clean --all --yes
    
#RUN . usr/local/etc/profile.d/conda.sh #didnt work

#RUN conda create -n pwcnet_test python=2.7
#RUN chmod u+r+x usr/local/etc/profile.d/conda.sh
#RUN conda env list
#RUN usr/local/etc/profile.d/conda.sh activate pwcnet_test


# install pytorch and other dependencies
#RUN pip install http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
#RUN pip install torchvision visdom dominate opencv-python cffi
# install external packages 
ADD torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
RUN pip install torch-0.2.0.post3-cp27-cp27mu-manylinux1_x86_64.whl
#RUN conda install pytorch=0.2.0 cuda80 -c soumith
RUN pip install torchvision==0.2.2 visdom==0.1.8.8 dominate==2.3.5 opencv-python==4.1.0.25 cffi==1.12.2

ADD external_packages external_packages
RUN bash /external_packages/correlation-pytorch-master/make_cuda.sh
#RUN mkdir /usr/local/lib/python2.7/site-packages/correlation
#RUN cp /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.egg /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.zip
#RUN unzip /usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.zip -d /usr/local/lib/python2.7/site-packages/correlation


ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility

make_cuda.sh 的内容是：

#!/usr/bin/env bash

CUDA_PATH=/usr/local/cuda-8.0

#cd correlation-pytorch/correlation_package/src
echo "Compiling correlation layer kernels by nvcc..."

# TODO (JEB): Check which arches we need
nvcc -c -o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr_cuda_kernel.cu.o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr_cuda_kernel.cu -x cu -Xcompiler -fPIC -arch=sm_52
nvcc -c -o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr1d_cuda_kernel.cu.o /external_packages/correlation-pytorch-master/correlation-pytorch/correlation_package/src/corr1d_cuda_kernel.cu -x cu -Xcompiler -fPIC -arch=sm_52

#cd ../../
python /external_packages/correlation-pytorch-master/correlation-pytorch/setup.py build install

图像创建成功，解释器运行良好（在运行和调试中）。

如果我运行pip list ，我可以看到 package correlation-package 0.1出现在已安装模块的列表中。

如果我查看我看到的packages-site内容和 egg 文件： correlation_package-0.1-py2.7-linux-x86_64.egg 。

我还可以注意到，如果我运行 'python -m site'，在 output 中会出现'/usr/local/lib/python2.7/site-packages/correlation_package-0.1-py2.7-linux-x86_64.egg' 。

但是，如果我尝试imp.find_module('correlation-package')[1] ，它会写信给我：

Traceback (most recent call last):
  File "<input>", line 1, in <module>
ImportError: No module named correlation-package

如果我尝试从模块中简单地导入 class，它也无法识别： from correlation_package.modules.corr import Correlation返回：

usr/local/bin/python2.7 /opt/.pycharm_helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 172.17.0.1 --port 45203 --file /opt/project/models/PWCNet.py 
Connected to pydev debugger (build 222.3739.30)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
Traceback (most recent call last):
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1496, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/project/models/PWCNet.py", line 13, in <module>
    from correlation_package.modules.corr import Correlation
ImportError: No module named correlation_package.modules.corr
python-BaseException

当然它不会在文件本身中看到它：

您可以在 Dockerfile 中的注释命令中看到我尝试解压缩模块的 .egg 存档中的文件并使用imp.load_source将其导入，如下所示：

Correlation = imp.load_source('Correlation', '/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py')

但这以错误结束：

raceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 7, in <module>
    __bootstrap__()
  File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 6, in __bootstrap__
    imp.load_dynamic(__name__,__file__)
ImportError: /usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.so: undefined symbol: __gxx_personality_v0

如果我从 python 控制台运行此命令，并且在 pycharm 的调试器中调试此行时弹出不同的错误：

/usr/local/bin/python2.7 /opt/.pycharm_helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 172.17.0.1 --port 41629 --file /opt/project/models/PWCNet.py 
Connected to pydev debugger (build 222.3739.30)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/vista/.cache/JetBrains/PyCharm2022.2/remote_sources/-609766439/-183260405/correlation/correlation_package/_ext/corr/_corr.py (will have no effect)
Traceback (most recent call last):
  File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1496, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/project/models/PWCNet.py", line 17, in <module>
    Correlation = imp.load_source('Correlation', '/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py')
  File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 7, in <module>
    __bootstrap__()
  File "/usr/local/lib/python2.7/site-packages/correlation/correlation_package/_ext/corr/_corr.py", line 6, in __bootstrap__
    imp.load_dynamic(__name__,__file__)
ImportError: dynamic module does not define init function (initCorrelation)
python-BaseException

因为corr.py文件的内容如下：

def __bootstrap__():
    global __bootstrap__, __loader__, __file__
    import sys, pkg_resources, imp
    __file__ = pkg_resources.resource_filename(__name__, '_corr.so')
    __loader__ = None; del __bootstrap__, __loader__
    imp.load_dynamic(__name__,__file__)
__bootstrap__()

解压后的鸡蛋/correlation/correlation_package/_ext/corr/的内容是：

_corr.py
_corr.pyc
_corr.so

此load_source操作的调试显示bootstrap imp.load_dynamic中的 imp.load_dynamic 运行如下：

导入 docker 容器环境中的 package 的正确方法是什么？

提前致谢。