gitlab-ci中的增量更新

Question

Community!

Company I am working for is moving towards git and Gitlab-CI. We have pretty large codebase - it takes ~10min to download sources, ~20 more minutes to download supplemental stuff from nexus, and then ~50min to build. So it is absolutely necessary to have incremental builds - thus we can get sources and build only updated files in a few minutes, rather than >1h.

Unfortunately GitLab's CI default strategies (particularly fetch) do no work for incremental build as they have hardcoded 'git clean -ffdx' which removes all untracked files, so it requires a full rebuild. I ended up developing my own update sequence:

• git init
• git fetch --tags $CI_REPOSITORY_URL +refs/heads/ :refs/remotes/origin/
• git config remote.origin.url $CI_REPOSITORY_URL
• git checkout -B $CI_BRANCH_NAME origin/$CI_BRANCH_NAME

This script seems to be working on a fresh computer (acts as clone), as well updating existing repo. At the same time I am not sure that it is working properly, as it works almost silently.

Question 1: Is this correct sequence to checkout to an arbitrary branch? Question 2: Does it require a git merge or git pull command after it? Question 3: Is there a way to get a list of changed files with this particular update (something like git pull does). This was extremely handy on our previous CI system to troubleshoot build issues Question 4 (relates to #3): Is there a way to get a list of commits in the current update? This is irrelevant if moving between far branches, but could be very helpful when updating the same branch

Answer 1

You should configure the cache property on your CI jobs for the untracked files that support incremental builds. The paths you list will be cached between jobs, so that you can do things like incremental builds.

As a quick start, you can cache all untracked files by simply untracked to true.

ci_job:
  cache:
    untracked: true