在 Google Cloud Build 中使用 gcloud compute scp 时出错

Question

I've been trying to run a gcloud compute scp command using Google Cloud Build trigger. I want my code to be copied to the respective directories in listed servers.

However, it's kinda strange that when I do a gcloud compute scp * username@instance-name:/home/directory --zone=us-central1-a --recurse in Cloud Shell, it works.

But when I write the equivalent in the cloudbuild.yaml file, it fails with error: *: No such file or directory .

Here are the contents for my cloudbuild.yaml :

- name: 'gcr.io/cloud-builders/gcloud'
  args: ['compute' , 'scp' , '*', 'username@instance-name:/home/directory' , '--zone=us-central1-a', '--recurse']

I've also tried using the remote-builder and that fails as well with error:

ssh: connect to host <ip_address> port 22: Connection timed out
lost connection
ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1].

I've also tried other combinations by replacing the source * with /workspace/* , ./* etc. but none of them worked. Executing with source as /workspace/ works but that copies the entire workspace directory instead of the contents of the directory as desired.

I've tried running the following set as well but that failed too.

cloudbuild.yaml :

- name: 'ubuntu'
  args: ['bash', '-c' , 'scp', '*', 'username@instance-name:/home/directory']

Error:

Already have image: ubuntu
scp: command not found

Again, all I want is to use Cloud Build to copy my code over to the VMs. Please help.

Answer 1

For the 'gcr.io/cloud-builders/gcloud' container image option:

The asterisk, "*", is interpreted by bash and expanded before passing the referents to the command, and, as gcloud in this container image is used directly and not through bash as an intermediary, "*" is not expanded, but passed literally to the gcloud command, which doesn't find any file named "*". If you need to copy all the code (directories recursively and their files) in the repository, use the same command but writing the root symbol, "/" (which, by the way, is a literal -not expandable- to bash ), instead of "*".

For the OS container image option with bash:

The container image must have scp installed by default to work that way, and the "ubuntu" image doesn't have it by default. You could either use an image with the openssh-client package installed by default, or install it in place and then run the command, something like this (for the step syntax, check this ):

steps:
- name: 'ubuntu'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    sudo apt update && sudo apt install openssh-client -y && scp * username@instance-name:/home/directory

Answer 2

I had the same issue. I have ended up creating two steps instead:

  - name: 'google/cloud-sdk:slim'
    args:
      - compute
      - scp
      - '--recurse'
      - '--compress'
      - /workspace/dist
      - 'build@instance-1:/var/www/'
      - '--zone'
      - us-west2-a
    id: copy-file-to-server
    entrypoint: gcloud
  - name: 'google/cloud-sdk:slim'
    args:
      - compute
      - ssh
      - '--zone'
      - us-west2-a
      - build@instance-1
      - '--'
      - 'mv /var/www/dist /var/www/$_SHORT_SHA '
    id: rename-dist
    entrypoint: gcloud