How does the Shell executor run scripts?

Question

We've recently moved to Gitlab and have started using pipelines. We've set up a build server (an Ubuntu 16.04 instance) and installed a runner that uses a Shell executor but I'm unsure of how it actually executes the scripts defined in the .gitlab-ci.yml file. Consider the following snippet of code:

script:
    - sh authenticate.sh $DEPLOY_KEY
    - cd MAIN && sh deploy.sh && cd .. 
    - sh deploy_service.sh MATCHMAKING
    - sh deauthenticate.sh

I was under the impression that it will just pipe these commands to Bash, and hence I was expecting the default Bash behaviour. What happens, however, is that the deploy.sh fails because of an ssh error ; Bash then continues to execute deploy_service.sh (which is expected behaviour) however this fails with a can't open deploy_service.sh error and the job terminates without Bash executing the last statement.

From what I understand, Bash will only abort on error if you do a set -e first and hence I was expecting all the statements to be executed. I've tried adding the set -e as the first statement but this makes no difference whatsoever - it doesn't terminate on the first ssh error.

I've added the exact output from Gitlab below:

Without set -e

$ cd MAIN && sh deploy.sh && cd ..
deploy.sh: 72: deploy.sh: Bad substitution
Building JS bundles locally...

> better-npm-run build

running better-npm-run in x
Executing script: build

to be executed: node ./bin/build 
-> building js bundle...
-> minifying js bundle...
Uploading JS bundles to server temp folder...
COMMENCING RESTART. 5,4,3,2,1...
ssh: Could not resolve hostname $: Name or service not known
$ sh deploy_service.sh MATCHMAKING
sh: 0: Can't open deploy_service.sh
ERROR: Job failed: exit status 1

With set -e

$ set -e
$ cd MAIN && sh deploy.sh && cd ..
deploy.sh: 72: deploy.sh: Bad substitution
Building JS bundles locally...

> better-npm-run build

running better-npm-run in x
Executing script: build

to be executed: node ./bin/build 
-> building js bundle...
-> minifying js bundle...
Uploading JS bundles to server temp folder...
COMMENCING RESTART. 5,4,3,2,1...
ssh: Could not resolve hostname $: Name or service not known
$ sh deploy_service.sh MATCHMAKING
sh: 0: Can't open deploy_service.sh
ERROR: Job failed: exit status 1

Why is it, without set -e , terminating on error (also, why is it terminating on the second error only and not the ssh error)? Any insights would be greatly appreciated.

Answer 1

Gitlab script block is actuality an array of shell scripts. https://docs.gitlab.com/ee/ci/yaml/#script Failure in each element of array will fail a whole array. To workaround put your script block in some script.sh file

like

script:
    - ./script.sh

Answer 2

I don't think your sh deploy.sh is generating a non-zero exit code.

You are using set -e to tell the current process to exit if a command exits with a non-zero return code, but you are creating a sub-process to run the shell script.

Here's a simple example script that I've called deploy.sh :

#!/bin/bash
echo "First."
echox "Error"
echo "Second"

If I run the script, you can see how the error is not handled:

$ sh deploy.sh 
First.
deploy.sh: line 5: echox: command not found
Second

If I run set -e first, you will see it has no effect.

$ set -e
$ sh deploy.sh 
First.
deploy.sh: line 5: echox: command not found
Second

Now, I add -e to the /bin/bash shebang:

#!/bin/bash -e
echo "First."
echox "Error"
echo "Second"

When I run the script with sh the -e still takes no effect.

$ sh ./deploy.sh 
First.
./deploy.sh: line 3: echox: command not found
Second

When this script is run directly using bash, the -e takes effect.

$ ./deploy.sh 
First.
./deploy.sh: line 3: echox: command not found

To fix your issue I believe you need to:

• Add -e to the script shebang line ( #!/bin/bash -e )
• Call the script direct from bash using ./deploy.sh and not run the script through sh .

Bear in mind that if deploy.sh does fail then the cd .. will not run ( && means run the next command if the preceding one succeeded), which would mean you were in the wrong directory to run the deploy_service.sh . You would be better with cd MAIN; sh deploy.sh; cd .. cd MAIN; sh deploy.sh; cd .. cd MAIN; sh deploy.sh; cd .. , but I suggest replacing your call to deploy.sh with simpler alternative:

script:
  - sh authenticate.sh $DEPLOY_KEY
  - (cd MAIN && sh deploy.sh) 
  - sh deploy_service.sh MATCHMAKING
  - sh deauthenticate.sh

This is not wildly different, but will result in the cd MAIN && sh deploy.sh to be run in a sub-process (that's what the brackets do), which means that the current directory of the overall script is not affected. Think of it like "spawn a sub-process, and in the sub-process change directory and run that script", and when the sub-process finishes you end up where you started.

As other users have commented, you're actually running your scripts in sh, not bash, so all round this might be better:

script:
  - ./authenticate.sh $DEPLOY_KEY
  - (cd MAIN && ./deploy.sh) 
  - ./deploy_service.sh MATCHMAKING
  - ./deauthenticate.sh