無法為 MLFlow 設置 sftp 工件存儲

Question

我想設置一個帶有外部指標和工件存儲的跟蹤 MLFlow 服務器。 我在 docker 網絡中有以下 docker 容器：mlflow-server、postgres、sftp-mlflow 和 python-client。 我能夠設置 postgres 並將其連接到 mlflow-server 和客戶端：

mlflow server --backend-store-uri postgresql://postgres:<pass>@mlflow_db:5432/mlflow_db --default-artifact-root sftp://sftp:<pass>@sftp-mlflow:22 -h 0.0.0.0 -p 8000

但是，我對工件存儲無能為力。 嘗試了以下 sftp 圖像

也遵循了這個指南。 但是工件存儲仍然不起作用=（

在我的客戶端，我有

import mlflow
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

remote_server_uri = "http://mlflow-server:8000" # set to your server URI
mlflow.set_tracking_uri(remote_server_uri)

# plotting
fig.savefig("test.png")

ARTIFACT_URI = "sftp://sftp:<pass>@sftp-mlflow:22"
EXPERIMENT_NAME = "test"
mlflow.create_experiment(EXPERIMENT_NAME, artifact_location=ARTIFACT_URI)
mlflow.set_experiment(EXPERIMENT_NAME)
with mlflow.start_run():
    mlflow.log_param("a", 1)
    mlflow.log_metric("b", 2)
    mlflow.log_artifact('test.png')

運行此代碼時，我得到：

2020/08/06 01:05:19 ERROR mlflow.utils.rest_utils: API request to http://mlflow-server:8000/api/2.0/mlflow/experiments/create failed with code 500 != 200, retrying up to 0 more times. API response body: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/run.py", line 24, in <module>
    mlflow.create_experiment(EXPERIMENT_NAME, artifact_location=ARTIFACT_URI)
  File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/fluent.py", line 357, in create_experiment
    return MlflowClient().create_experiment(name, artifact_location)
  File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/client.py", line 164, in create_experiment
    return self._tracking_client.create_experiment(name, artifact_location)
  File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 126, in create_experiment
    return self.store.create_experiment(
  File "/usr/local/lib/python3.8/site-packages/mlflow/store/tracking/rest_store.py", line 54, in create_experiment
    response_proto = self._call_endpoint(CreateExperiment, req_body)
  File "/usr/local/lib/python3.8/site-packages/mlflow/store/tracking/rest_store.py", line 32, in _call_endpoint
    return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)
  File "/usr/local/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 142, in call_endpoint
    response = http_request(
  File "/usr/local/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 86, in http_request
    raise MlflowException("API request to %s failed to return code 200 after %s tries" %
mlflow.exceptions.MlflowException: API request to http://mlflow-server:8000/api/2.0/mlflow/experiments/create failed to return code 200 after 3 tries

我可以使用 sftp 從 mlflow-server 容器和 python 客戶端連接到 sftp 存儲： sftp -P 22 sftp@sftp-mlflow

Answer 1

在您的 mlflow-server 中，您已經定義了默認的工件存儲，所以我認為沒有必要在客戶端指定它。 您是否已經嘗試過mlflow.create_experiment(EXPERIMENT_NAME) ？

另一個注意事項：您是如何在您的 sftp 服務器上指定用戶的（命令 arguments、環境變量或文件）？ MLflow 文檔建議不要使用 user:pass，而是...

將客戶端配置為無需密碼即可通過 SSH 登錄 SFTP 服務器（例如公鑰、ssh_config 中的身份文件等）

https://www.mlflow.org/docs/latest/tracking.html#sftp-server

無法為 MLFlow 設置 sftp 工件存儲

問題描述

1 個解決方案

解決方案1
0 2020-09-16 12:58:04

無法為 MLFlow 設置 sftp 工件存儲

問題描述

1 個解決方案

解決方案1 0 2020-09-16 12:58:04

解決方案1
0 2020-09-16 12:58:04