尝试使用Torch / PyTorch创建Docker映像时出现MemoryError

Question

我正在按照本教程为Flask应用程序创建Docker映像。 该应用程序取决于Torch / PyTorch。 因此，我的requirements.txt文件如下所示。

flask
flask-cors
pytorch
torchvision
pandas

然后，我的Dockerfile如下所示。

FROM ubuntu:latest
LABEL My Company "info@mycompany.com"
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
COPY . /app
WORKDIR /app
RUN pip install -vvv --no-cache-dir -r requirements.txt
ENTRYPOINT ["python3"]
CMD ["app.py"]

当我输入命令docker build -t flask-sample-one:latest . 我在Mac上收到以下错误消息（当pip尝试安装割炬时发生）。

Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 882, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python2.7/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/filewrapper.py", line 63, in read
    self._close()
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/filewrapper.py", line 50, in _close
    self.__callback(self.__buf.getvalue())
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/controller.py", line 275, in cache_response
    self.serializer.dumps(request, response, body=body),
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 55, in dumps
    "body": _b64_encode_bytes(body),
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 12, in _b64_encode_bytes
    return base64.b64encode(b).decode("ascii")
MemoryError
The command '/bin/sh -c pip install -vvv -r requirements.txt' returned a non-zero code: 2

我不确定MemoryError来自何处。 我试图在具有16 GB RAM的Macbook Pro上构建docker映像。 关于正在发生的事情或如何使用Python创建docker映像的任何想法吗？

在AWS EC2实例上，出现以下错误。

Running setup.py install for pytorch: started
    Running command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile
    Traceback (most recent call last):
      File "", line 1, in 
      File "/tmp/pip-build-SeBh33/pytorch/setup.py", line 13, in 
        raise Exception(message)
    Exception: You should install pytorch from http://pytorch.org
    Running setup.py install for pytorch: finished with status 'error'
Cleaning up...
  Removing source in /tmp/pip-build-SeBh33/pytorch
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-SeBh33/pytorch/
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 360, in run
    prefix=options.prefix_path,
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 878, in install
    spinner=spinner,
  File "/usr/lib/python2.7/dist-packages/pip/utils/__init__.py", line 725, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-SeBh33/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-5jxyuS-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-SeBh33/pytorch/
The command '/bin/sh -c pip install -vvv --no-cache-dir -r requirements.txt' returned a non-zero code: 1

有什么想法吗？ 我只是在尝试构建具有少量依赖关系的docker映像，以便可以从flask应用程序提供PyTorch模型。 我不确定是否有关于此操作的教程（目前尚未找到）。

Answer 1

如果有人感兴趣，我可以按如下方法将PyTorch安装到PyTorch容器中。

我修改了requirements.txt ，使其如下所示。

flask
flask-cors
pandas

然后，我将Dockerfile修改为如下所示。 这很奇怪，并且不遵循使用pip安装Python软件包的常规方法，但是可以使用。 请注意，我必须先安装Python 3，再安装PyTorch wheel，最后再安装torchvision的pip安装。

FROM ubuntu:latest
LABEL My Company "info@mycompany.com"
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
RUN apt-get install -y python3.6
RUN apt-get install -y python3-pip
COPY . /app
WORKDIR /app
RUN pip3 install -vvv --no-cache-dir -r requirements.txt
RUN pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp36-cp36m-linux_x86_64.whl
RUN pip3 install torchvision
ENTRYPOINT ["python3"]
CMD ["app.py"]