GitLab CI stuck on running NodeJS server

Question

I'm trying to use GitLab CI to build, test and deploy an Express app on a server (the Runner is running with the shell executor). However, the test:async and deploy_staging jobs do not terminate. But when checking the terminal inside GitLab, the Express server does indeed start. What gives ?

stages:
  - build
  - test
  - deploy

### Jobs ###

build:
  stage: build
  script:
    - npm install -q
    - npm run build
    - knex migrate:latest
    - knex seed:run
  artifacts:
    paths:
      - build/
      - node_modules/
  tags:
    - database
    - build

test:lint:
  stage: test
  script:
    - npm run lint
  tags:
    - lint

# Run the Express server
test:async:
  stage: test
  script:
   - npm start &
   - curl http://localhost:3000
  tags:
   - server

deploy_staging:
  stage: deploy
  script:
    - npm start
  environment:
    name: staging
    url: my_url_here
  tags:
    - deployment

The npm start is just node build/bundle.js . The build script is using Webpack.

Answer 1

You are starting a background job during your test phase which never terminates - therefore the job runs forever.

The idea of the GitLab CI jobs are shortly-running tasks - like compiling, executing unit tests or gathering information such as code coverage - which are executed in a predefined order. In your case, the order is build -> test -> deploy ; since the test job doesn't finish, deploy isn't even executed.

Depending on your environment, you will have to create a different job for deploying your node app. For example, you can push the build output to a remote server using a tool like scp or upload it to AWS; after that, you reference the final URL in the url: field in your .gitlab-ci.yml .

Answer 2

Note: solution works fine when using a gitlab runner with shell executor

Generally in Gitlab CI we run ordered jobs with specific tasks that should be executed one after the end of the other.

So for the job build we have the npm install -q command that runs and terminates with an exit status (0 exit status if the command was succesful), then runs the next command npm run build and so on until the job is terminated.

For the test job we have npm start & process that keeps running so the job wont be able to terminate.

The problem is that sometimes we need to have some process that need to run in background or having some process that keeps living between tasks. For example in some kind of test we need to keep the server running, something like that:

test:
  stage: test
  script:
   - npm start
   - npm test

in this case npm test will never start because npm statrt keeps running without terminating.

The solution is to use before_script where we run a shell script that keeps npm start process running then we call after_script to kill that npm start process

so on our .gitlab-ci.yml we write

test:
  stage: test
  before_script:
    - ./serverstart.sh
  script:
   - npm test
  after_script:
    - kill -9 $(ps aux | grep '\snode\s' | awk '{print $2}')

and on the serverstart.sh

# !/bin/bash

# start the server and send the console and error logs on nodeserver.log
npm start > nodeserver.log 2>&1 &

# keep waiting until the server is started 
# (in this case wait for mongodb://localhost:27017/app-test to be logged)
while ! grep -q "mongodb://localhost:27017/app-test" nodeserver.log
do
  sleep .1
done
echo -e "server has started\n"
exit 0

thanks to that serverstart.sh script is terminated while keeping npm start process alive and help us by the way move to the job where we have npm test .

npm test terminates and pass to after script where we kill all nodejs process.