简体繁体 English

Pip 在 gitlab-ci 中太慢了

[英]Pip too slow in gitlab-ci

原文 2020-12-10 12:43:29 2 1 python/ pip/ gitlab

Since yesterday my gitlab-ci is too slow to download pip packages like:从昨天开始，我的 gitlab-ci 下载 pip 包的速度太慢，例如：

more_itertools more_itertools
importlib_metadata导入库元数据
amqp amqp
kombu海带
jmespath jmespath
idna伊娜

Example:例子：

And in the end I get this error: ERROR: Job failed: execution took longer than 1h0m0s seconds最后我得到这个错误： ERROR: Job failed: execution took longer than 1h0m0s seconds

Does anyone have any ideas?有没有人有任何想法？

1 个解决方案

Installing packages like this (whether it's OS packages though apt-get install... or pip, or anything else) is generally against best practices for CI/CD jobs because every job that runs will have to do the same thing, costing a lot of time as you run more pipelines.安装这样的软件包（无论是通过apt-get install...还是 pip 或其他任何方式安装的操作系统软件包）通常违反 CI/CD 作业的最佳实践，因为运行的每个作业都必须做同样的事情，成本很高运行更多管道时的时间。 The alternative is to search for an existing image that has everything you need (possible but not likely with more dependencies), split up your job into pieces that might be solved by an image with just one or two dependencies, or create a custom docker image to use in your jobs.另一种方法是搜索具有您需要的一切的现有图像（可能但不太可能具有更多依赖项），将您的工作拆分为可能由仅具有一个或两个依赖项的图像解决的部分，或者创建自定义 docker 图像在您的工作中使用。

Since you didn't post the full job output or your .gitlab-ci.yml file, I'm just assuming you're using the Docker executor for your runners, but if you're using shell , another solution is to install all the dependencies you need on the host running gitlab-runner as the shell executor will have access to them. Since you didn't post the full job output or your .gitlab-ci.yml file, I'm just assuming you're using the Docker executor for your runners, but if you're using shell , another solution is to install all您在运行gitlab-runner作为shell执行程序的主机上所需的依赖项将可以访问它们。

I just answered another question about creating a custom docker image for Gitlab CI that you can reference here: "Unable to locate package git" when running GitLab CI/CD pipeline I just answered another question about creating a custom docker image for Gitlab CI that you can reference here: "Unable to locate package git" when running GitLab CI/CD pipeline

Once you've moved your package dependencies to a docker image, your job will pull in the already-built image and skip to the meat of your job.将 package 依赖项移动到 docker 映像后，您的工作将拉入已构建的映像并跳至工作的核心。