[英]How to pass parameters to a training script in Azure Machine Learning service?
I am trying to submit an experiment in Azure Machine Learning service locally on an Azure VM using a ScriptRunConfig
object in my workspace ws
, as in 我正在尝试使用工作区
ws
的ScriptRunConfig
对象在Azure VM 本地在Azure Machine Learning服务中提交实验,如下所示:
from azureml.core import ScriptRunConfig
from azureml.core.runconfig import RunConfiguration
from azureml.core import Experiment
experiment = Experiment(ws, name='test')
run_local = RunConfiguration()
script_params = {
'--data-folder': './data',
'--training-data': 'train.csv'
}
src = ScriptRunConfig(source_directory = './source_dir',
script = 'train.py',
run_config = run_local,
arguments = script_params)
run = experiment.submit(src)
However, this fails with 但是,这失败了
ExperimentExecutionException: { "error_details": { "correlation": { "operation": "bb12f5b8bd78084b9b34f088a1d77224", "request": "iGfp+sjC34Q=" }, "error": { "code": "UserError", "message": "Failed to deserialize run definition"
ExperimentExecutionException:{“ error_details”:{“ correlation”:{“ operation”:“ bb12f5b8bd78084b9b34f088a1d77224”,“ request”:“ iGfp + sjC34Q =”},“ error”:{“ code”:“ UserError”,“ message”: “无法反序列化运行定义”
Worse, if I set my data folder to use a datastore (which likely I will need to) 更糟糕的是,如果我将数据文件夹设置为使用数据存储区(可能需要使用它)
script_params = {
'--data-folder': ds.path('mydatastoredir').as_mount(),
'--training-data': 'train.csv'
}
the error is 错误是
UserErrorException: Dictionary with non-native python type values are not supported in runconfigs.
UserErrorException:runconfigs不支持具有非本地python类型值的字典。
{'--data-folder': $AZUREML_DATAREFERENCE_d93269a580ec4ecf97be428cd2fe79, '--training-data': 'train.csv'}{'-数据文件夹':$ AZUREML_DATAREFERENCE_d93269a580ec4ecf97be428cd2fe79,'--training-data':'train.csv'}
I don't quite understand how I should pass my script_params
parameters to my train.py
( the documentation of ScriptRunConfig
doesn't include a lot of details on this unfortunately). 我不太了解如何将我的
script_params
参数传递给train.py
( ScriptRunConfig
是, ScriptRunConfig
的文档中并未包含很多详细信息)。
Does anybody know how to properly create src
in these two cases? 在这两种情况下,有人知道如何正确创建
src
吗?
The correct way of passing arguments to the ScriptRunConfig and RunConfig is as a list of strings according to https://docs.microsoft.com/nb-no/python/api/azureml-core/azureml.core.runconfiguration?view=azure-ml-py . 将参数传递给ScriptRunConfig和RunConfig的正确方法是根据https://docs.microsoft.com/nb-no/python/api/azureml-core/azureml.core.runconfiguration?view=azure作为字符串列表-ml-py 。
Modified and working code would be as follows. 修改后的工作代码如下。
from azureml.core import ScriptRunConfig
from azureml.core.runconfig import RunConfiguration
from azureml.core import Experiment
experiment = Experiment(ws, name='test')
run_local = RunConfiguration()
script_params = [
'--data-folder',
'./data',
'--training-data',
'train.csv'
]
src = ScriptRunConfig(source_directory = './source_dir',
script = 'train.py',
run_config = run_local,
arguments = script_params)
run = experiment.submit(src)
In the end I abandoned ScriptRunConfig
and used Estimator
as follows to pass script_params
(after having provisioned a compute target): 最后,我放弃了
ScriptRunConfig
并按如下方式使用Estimator
传递script_params
(在script_params
了计算目标之后):
estimator = Estimator(source_directory='./mysourcedir',
script_params=script_params,
compute_target='cluster',
entry_script='train.py',
conda_packages = ["pandas"],
pip_packages = ["git+https://github.com/..."],
use_docker=True,
custom_docker_image='<mydockeraccount>/<mydockerimage>')
This also allowed me to install my pip_packages
dependency by putting on https://hub.docker.com/ a custom_docker_image
Docker image created from a Dockerfile like: 这还允许我通过将https://hub.docker.com/上的
custom_docker_image
Docker映像安装到pip_packages
安装我的pip_packages
依赖项,例如:
FROM continuumio/miniconda
RUN apt-get update
RUN apt-get install git gcc g++ -y
(it worked!) (有效!)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.