How to set output directory for Google's Machine Learning task?

Question

I using this python code to create Machine Learning task:

credentials = GoogleCredentials.get_application_default()
ml = discovery.build('ml','v1beta1', credentials=credentials)
projectID = 'projects/{}'.format('testtf')
jobDict = { 'jobId': 'test_job_2', 'trainingInput': { 'scaleTier': 'BASIC', 'packageUris': [ 'gs://testtf-ml/cloudmldist/1479282298/trainer-0.0.0.tar.gz' ], 'pythonModule': 'trainer.task', 'region': 'us-central1' } }
request = ml.projects().jobs().create(parent = projectID, body = jobDict)
response = request.execute()

It creates the task and it successfully executes, but there is no any output on Google Cloud Storage.

I trying to repeat this https://cloud.google.com/ml/docs/quickstarts/training tutorial, but from the code and not console.

There is such commands to put task from console:

JOB_NAME=mnist_${USER}_$(date +%Y%m%d_%H%M%S)

PROJECT_ID=`gcloud config list project --format "value(core.project)"`
TRAIN_BUCKET=gs://${PROJECT_ID}-ml
TRAIN_PATH=${TRAIN_BUCKET}/${JOB_NAME}
gsutil rm -rf ${TRAIN_PATH}

gcloud beta ml jobs submit training ${JOB_NAME} \
  --package-path=trainer \
  --module-name=trainer.task \
  --staging-bucket="${TRAIN_BUCKET}" \
  --region=us-central1 \
  -- \
  --train_dir="${TRAIN_PATH}/train"

The last line specifies output directory. I don't understand how I can specify it from the code. I using this method: https://cloud.google.com/ml/reference/rest/v1beta1/projects.jobs/create

And this is input fields descriptions: https://cloud.google.com/ml/reference/rest/v1beta1/projects.jobs

The only suspicious field I see there is "args": [string] for TrainingInput , but I don't understand how to use it. I tried to specify them this way:

jobDict = { 'jobId': 'test_job_2', 'trainingInput': { 'scaleTier': 'BASIC', 'packageUris': [ 'gs://testtf-ml/cloudmldist/1479282298/trainer-0.0.0.tar.gz' ], 'pythonModule': 'trainer.task', 'args': [ 'train_dir="gs://testtf-ml/test_output"' ], 'region': 'us-central1' } }

But such dir wasn't created. Which way I can specify output folder for training task? I use this example: cloudml-samples-master\\mnist\\trainable

Answer 1

Output directory should be set same way as in command line arguments:

jobDict = { 'jobId': 'test_job_2', 'trainingInput': { 'scaleTier': 'BASIC', 'packageUris': [ 'gs://testtf-ml/cloudmldist/1479282298/trainer-0.0.0.tar.gz' ], 'pythonModule': 'trainer.task', 'args': [ '--train_dir=gs://testtf-ml/test_output' ], 'region': 'us-central1' } }

without additional quotation marks for path and -- in the beginning.