[英]Using custom dependencies in Google Cloud ML Engine
I am trying to use a python package that is not listed in PyPI with the Google Cloud ML Engine. 我正在尝试将PyPI中未列出的python 包与Google Cloud ML Engine一起使用。 This package has itself dependencies that even though are listed in PyPI are not installed by default in the ML engine environment, namely the Cython package. 该软件包本身具有依赖性,即使在PyPI中列出了,默认情况下也不会在ML引擎环境中安装Cython软件包。
Looking at the documentation it is not really clear how to proceed in this case, I have tried packaging this package in a .tar.gz
file and passing it under the --packages
argument, but I got the following error: 在文档中查看并不清楚在这种情况下如何继续,我曾尝试将此程序包打包到.tar.gz
文件中,并将其传递到--packages
参数下,但是出现以下错误:
File "<string>", line 1, in <module> IOError: [Errno 2] No such file or directory: '/tmp/pip-jnm3Ml-build/setup.py'
After I tried using a setup.py
file and packaging my code but google cloud ml engine is not able to find the package in dependency_links
在我尝试使用setup.py
文件并将代码打包后,但Google Cloud ml引擎无法在dependency_links
找到该软件包
Here is my current setup.py
: 这是我当前的setup.py
:
from setuptools import find_packages, setup
required_packages = ['cython', 'numpy', 'tensorflow', 'scipy', 'cython']
dependency_links = ['git+https://github.com/lucasb-eyer/pydensecrf.git']
setup(name='trainer',
version='0.1',
packages=['trainer'],
install_requires=required_packages,
dependency_links=dependency_links,
include_package_data=True,
description='description')
I would like to avoid doing this by trial and error since sending jobs to the cloud costs money even if they fail immediately. 我想避免通过反复试验来做到这一点,因为即使将作业立即失败,将作业发送到云也会花费金钱。
Thanks in advance. 提前致谢。
To do this, you will need to add the Cython to the list of required packages in your setup.py
. 为此,您需要将Cython添加到setup.py
中所需软件包的列表中。 Instructions can be found here . 说明可以在这里找到。
Here is a sample setup.py
, that would reside in the parent directory of the directory you pass as --package-path
to gcloud
. 这是一个示例setup.py
,它将驻留在您作为--package-path
传递给gcloud
的目录的父目录中。
from setuptools import find_packages
from setuptools import setup
REQUIRED_PACKAGES = ['Cython>=0.26']
setup(
name='trainer',
version='0.1',
install_requires=REQUIRED_PACKAGES,
packages=find_packages(),
include_package_data=True,
description='My trainer application package.'
)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.