[英]Pending status of pods when running Gitlab CI runner on Kubernetes
I am currently trying to use a Kubernetes cluster for the Gitlab CI.我目前正在尝试将 Kubernetes 集群用于 Gitlab CI。 While following the not so good docs ( https://docs.gitlab.com/runner/install/kubernetes.html ), what I did was manually register a runner with the token from Gitlab CI section so I could get another token and use it in the ConfigMap I use for the deployment.在遵循不太好的文档( https://docs.gitlab.com/runner/install/kubernetes.html )的同时,我所做的是使用 Gitlab CI 部分的令牌手动注册一个运行器,以便我可以获得另一个令牌并使用它在我用于部署的 ConfigMap 中。
-ConfigMap -ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: gitlab-runner
namespace: gitlab
data:
config.toml: |
concurrent = 4
[[runners]]
name = "Kubernetes Runner"
url = "https://url/ci"
token = "TOKEN"
executor = "kubernetes"
[runners.kubernetes]
namespace = "gitlab"
-Deployment -部署
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: gitlab-runner
namespace: gitlab
spec:
replicas: 4
selector:
matchLabels:
name: gitlab-runner
template:
metadata:
labels:
name: gitlab-runner
spec:
containers:
- args:
- run
image: gitlab/gitlab-runner:latest
imagePullPolicy: Always
name: gitlab-runner
volumeMounts:
- mountPath: /etc/gitlab-runner
name: config
restartPolicy: Always
volumes:
- configMap:
name: gitlab-runner
name: config
With these two I get to see the runner in the Gitlab Runner section but whenever I start a job, the new created pods stay in pending status.通过这两个,我可以在 Gitlab Runner 部分看到运行器,但是每当我开始工作时,新创建的 pod 都会保持挂起状态。
I would like to fix it but all I know is that the nodes and pods get these events:我想修复它,但我只知道节点和 pod 会收到这些事件:
-Pods: -豆荚:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
35s 4s 7 {default-scheduler } Warning FailedScheduling No nodes are available that match all of the following predicates:: MatchNodeSelector (2).
-Nodes: -节点:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
4d 31s 6887 {kubelet gitlab-ci-hc6k3ffax54o-master-0} Warning FailedNodeAllocatableEnforcement Failed to update Node Allocatable Limits "": failed to set supported cgroup subsystems for cgroup : Failed to set config for supported subsystems : failed to write 3783761920 to memory.limit_in_bytes: write /rootfs/sys/fs/cgroup/memory/memory.limit_in_bytes: invalid argument
Any idea of why this is happening?知道为什么会这样吗?
EDIT: kubectl describe added:编辑:kubectl describe 添加:
Name: runner-45384765-project-1570-concurrent-00mb7r
Namespace: gitlab
Node: /
Labels: <none>
Status: Pending
IP:
Controllers: <none>
Containers:
build:
Image: blablabla:latest
Port:
Command:
sh
-c
if [ -x /usr/local/bin/bash ]; then
exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
exec /usr/bin/bash
elif [ -x /bin/bash ]; then
exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
exec /usr/bin/sh
elif [ -x /bin/sh ]; then
exec /bin/sh
else
echo shell not found
exit 1
fi
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-1qm5n (ro)
/vcs from repo (rw)
Environment Variables:
CI_PROJECT_DIR: blablabla
CI_SERVER: yes
CI_SERVER_TLS_CA_FILE: -----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
CI: true
GITLAB_CI: true
CI_SERVER_NAME: GitLab
CI_SERVER_VERSION: 9.5.5-ee
CI_SERVER_REVISION: cfe2d5c
CI_JOB_ID: 5625
CI_JOB_NAME: pylint
CI_JOB_STAGE: build
CI_COMMIT_SHA: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_COMMIT_REF_NAME: master
CI_COMMIT_REF_SLUG: master
CI_REGISTRY_USER: gitlab-ci-token
CI_BUILD_ID: 5625
CI_BUILD_REF: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_BUILD_BEFORE_SHA: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_BUILD_REF_NAME: master
CI_BUILD_REF_SLUG: master
CI_BUILD_NAME: pylint
CI_BUILD_STAGE: build
CI_PROJECT_ID: 1570
CI_PROJECT_NAME: blablabla
CI_PROJECT_PATH: blablabla
CI_PROJECT_PATH_SLUG: blablabla
CI_PROJECT_NAMESPACE: vcs
CI_PROJECT_URL: https://blablabla
CI_PIPELINE_ID: 2574
CI_CONFIG_PATH: .gitlab-ci.yml
CI_PIPELINE_SOURCE: push
CI_RUNNER_ID: 111
CI_RUNNER_DESCRIPTION: testing on kubernetes
CI_RUNNER_TAGS: docker-image-build
CI_REGISTRY: blablabla
CI_REGISTRY_IMAGE: blablabla
PYLINTHOME: ./pylint-home
GITLAB_USER_ID: 2277
GITLAB_USER_EMAIL: blablabla
helper:
Image: gitlab/gitlab-runner-helper:x86_64-a9a76a50
Port:
Command:
sh
-c
if [ -x /usr/local/bin/bash ]; then
exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
exec /usr/bin/bash
elif [ -x /bin/bash ]; then
exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
exec /usr/bin/sh
elif [ -x /bin/sh ]; then
exec /bin/sh
else
echo shell not found
exit 1
fi
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-1qm5n (ro)
/vcs from repo (rw)
Environment Variables:
CI_PROJECT_DIR: blablabla
CI_SERVER: yes
CI_SERVER_TLS_CA_FILE: -----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
CI: true
GITLAB_CI: true
CI_SERVER_NAME: GitLab
CI_SERVER_VERSION: 9.5.5-ee
CI_SERVER_REVISION: cfe2d5c
CI_JOB_ID: 5625
CI_JOB_NAME: pylint
CI_JOB_STAGE: build
CI_COMMIT_SHA: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_COMMIT_REF_NAME: master
CI_COMMIT_REF_SLUG: master
CI_REGISTRY_USER: gitlab-ci-token
CI_BUILD_ID: 5625
CI_BUILD_REF: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_BUILD_BEFORE_SHA: ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
CI_BUILD_REF_NAME: master
CI_BUILD_REF_SLUG: master
CI_BUILD_NAME: pylint
CI_BUILD_STAGE: build
CI_PROJECT_ID: 1570
CI_PROJECT_NAME: blablabla
CI_PROJECT_PATH: blablabla
CI_PROJECT_PATH_SLUG: blablabla
CI_PROJECT_NAMESPACE: vcs
CI_PROJECT_URL: blablabla
CI_PIPELINE_ID: 2574
CI_CONFIG_PATH: .gitlab-ci.yml
CI_PIPELINE_SOURCE: push
CI_RUNNER_ID: 111
CI_RUNNER_DESCRIPTION: testing on kubernetes
CI_RUNNER_TAGS: docker-image-build
CI_REGISTRY: blablabla
CI_REGISTRY_IMAGE: blablabla
PYLINTHOME: ./pylint-home
GITLAB_USER_ID: 2277
GITLAB_USER_EMAIL: blablabla
Conditions:
Type Status
PodScheduled False
Volumes:
repo:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
default-token-1qm5n:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-1qm5n
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
39s 8s 7 {default-scheduler } Warning FailedScheduling No nodes are available that match all of the following predicates:: MatchNodeSelector (2).
@djuarez As long the deployment selector matches the pods label in the template section, and in this case from what I can see that is the case: @djuarez 只要部署选择器与模板部分中的 pods 标签匹配,在这种情况下,我所看到的就是这种情况:
selector:
matchLabels:
name: gitlab-runner
template:
metadata:
labels:
name: gitlab-runner
it should not be a problem;这应该不是问题; provided the correct API is used which in this case apiVersion: extensions/v1beta1
is also correct.如果使用了正确的 API,在这种情况下apiVersion: extensions/v1beta1
也是正确的。 The describe
output shows MatchNodeSelector
which has nothing to do with the Deployment selector. describe
输出显示了与部署选择器无关的MatchNodeSelector
。 My guess is the full Deployment config is not being shown here and something else is wrong like trying to deploy the pods to specific nodes via nodeSeletor
that do not have the requested label in the nodeSelector condition.我的猜测是这里没有显示完整的部署配置,还有其他错误,比如尝试通过nodeSeletor
将 pod 部署到特定节点,而 nodeSelector 条件中没有请求的标签。
Because of:因为:
selector:
matchLabels:
name: gitlab-runner
No pod is able to retrieve a job with that label.没有 Pod 能够检索带有该标签的作业。
Removing the selector would be enough, there is no need for it.删除选择器就足够了,没有必要。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.