[英]Gitlab CI/CD cache expires and therefor build fails
I got AWS CDK application in typescript and pretty simple gitlab CI/CD pipeline with 2 stages, which takes care of the deployment:我在 typescript 和非常简单的 gitlab CI/CD 管道中获得了 AWS CDK 应用程序,它有 2 个阶段,负责部署:
image: node:latest
stages:
- dependencies
- deploy
dependencies:
stage: dependencies
only:
refs:
- master
changes:
- package-lock.json
script:
- npm install
- rm -rf node_modules/sharp
- SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install --arch=x64 --platform=linux --libc=glibc sharp
cache:
key:
files:
- package-lock.json
paths:
- node_modules
policy: push
deploy:
stage: deploy
only:
- master
script:
- npm run deploy
cache:
key:
files:
- package-lock.json
paths:
- node_modules
policy: pull
npm run deploy
is just a wrapper for the cdk command. npm run deploy
只是 cdk 命令的包装器。
But for some reason, sometimes it happens, that the cache of the node_modules (probably) expires - simply deploy
stage is not able to fetch for it and therefore the deploy
stage fails:但出于某种原因,有时会发生 node_modules 的缓存(可能)过期 - 只是deploy
阶段无法获取它,因此deploy
阶段失败:
Restoring cache
Checking cache for ***-protected...
WARNING: file does not exist
Failed to extract cache
I checked that the cache name is the same as the one built previously in the last pipeline run with dependencies
stage.我检查了缓存名称是否与之前在dependencies
阶段运行的最后一个管道中构建的缓存名称相同。
I suppose it happens, as often times this CI/CD is not running even for multiple weeks, since I contribute to that repo rarely.我想它会发生,因为这个 CI/CD 经常几个星期都没有运行,因为我很少为那个回购做出贡献。 I was trying to search for the root causes but failed miserably.我试图寻找根本原因,但惨遭失败。 I pretty much understand that cache can expire after some times(30 days from what I found by default), but I would expect CI/CD to recover from that by running the dependencies
stage despite the fact package-lock.json
wasn't updated.我非常理解缓存会在一段时间后过期(从我默认发现的 30 天开始),但我希望 CI/CD 通过运行dependencies
阶段从中恢复,尽管package-lock.json
没有更新.
So my question is simply "What am I missing? Is my understanding of caching in Gitlab's CI/CD completely wrong? Do I have to turn on some feature switcher?"所以我的问题很简单“我错过了什么?我对 Gitlab 的 CI/CD 中的缓存的理解完全错误吗?我必须打开一些功能切换器吗?”
Basically my ultimate goal is to skip the building of the node_modules part as often as possible, but not failing on the non-existent cache even if I don't run the pipeline for multiple months.基本上我的最终目标是尽可能多地跳过 node_modules 部分的构建,但即使我几个月没有运行管道也不会在不存在的缓存上失败。
A cache is only a performance optimization, but is not guaranteed to always work .缓存只是一种性能优化,但不能保证始终有效。 Your expectation that the cache might be expired is most likely correct, and thus you'll need to have a fallback in your deploy script.您对缓存可能过期的预期很可能是正确的,因此您需要在部署脚本中进行回退。
One thing you could do is that you change your dependencies job to:您可以做的一件事是将依赖项作业更改为:
Eg something like this:例如这样的事情:
dependencies:
stage: dependencies
only:
refs:
- master
changes:
- package-lock.json
script:
- |
if [[ -d node_modules ]]; then
exit 0
fi
- npm install
- rm -rf node_modules/sharp
- SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install --arch=x64 --platform=linux --libc=glibc sharp
cache:
key:
files:
- package-lock.json
paths:
- node_modules
See also this related question .另请参阅此相关问题。
If you want to avoid spinning up unnecessary jobs, then you could also consider to merge the dependencies & deploy jobs, and take a similar approach as above in the combined job.如果您想避免旋转不必要的作业,那么您还可以考虑合并依赖项和部署作业,并在组合作业中采用与上述类似的方法。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.