I have created a cloudbuild.json for kubeflow pipeline deployment. but it is giving error saying file is not present

Question

This is my cloudbuild.json

{
    "steps": [
        {
            "name": "gcr.io/cloud-builders/docker",
            "args": [
                "build",
                "-t",
                "trainer_image",
                "."
            ],
            "dir": "./trainer_image/"
        },
        {
            "name": "gcr.io/cloud-builders/docker",
            "args": [
                "build",
                "-t",
                "base_image",
                "."
            ],
            "dir": "./base_image/"
        },
        {
            "name": "gcr.io/dmgcp-pkg-internal-poc-oct-04/kfp-cli",
            "args": [
                "dsl-compile --py covertype_training_pipeline.py --output covertype_training_pipeline.yaml"
            ],
            "env": [
                "BASE_IMAGE=gcr.io/dmgcp-pkg-internal-poc-oct-04/base_image:test",
                "TRAINER_IMAGE=gcr.io/dmgcp-pkg-internal-poc-oct-04/trainer_image:test",
                "RUNTIME_VERSION=1.15",
                "PYTHON_VERSION=3.7",
                "COMPONENT_URL_SEARCH_PREFIX=https://raw.githubusercontent.com/kubeflow/pipelines/0.2.5/components/gcp/",
                "USE_KFP_SA=False"
            ],
            "dir": "./pipeline/"
        },
        {
            "name": "gcr.io/dmgcp-pkg-internal-poc-oct-04/kfp-cli",
            "args": [
                "-c",
                "kfp --endpoint 66df1d31e46e6510-dot-us-central2.pipelines.googleusercontent.com/ pipeline upload -p credit_fraud_training_test covertype_training_pipeline.yaml"
            ],
            "dir": "./pipeline/"
        }
    ],
    "images": [
        "gcr.io/dmgcp-pkg-internal-poc-oct-04/trainer_image:test",
        "gcr.io/dmgcp-pkg-internal-poc-oct-04/base_image:test"
    ]
}

The issue is in step2: here is the error that i got "Step #2: gcr.io/dmgcp-pkg-internal-poc-oct-04/kfp-cli:latest Step #2: /bin/bash: dsl-compile --py covertype_training_pipeline.py --output covertype_training_pipeline.yaml: No such file or directory Finished Step #2 " i am running this command to run the pipeline "!gcloud builds submit . --config cloudbuild.json"

hers are my folder structure enter image description here enter image description here

Answer 1

What's in your Dockerfile?

"dsl-compile --py covertype_training_pipeline.py --output covertype_training_pipeline.yaml"

Are you sure this works as a single string?

"dir": "./pipeline/"

Are you sure this relative path is correct?

PS You can create self-compiling pipelines which takes dsl-compile out of the picture.