Gitlab 使用 Kube.netes 代理和私有容器注冊表的 CI/CD
[英]Gitlab CI/CD using Kubernetes agent and the private container registry
問題描述
我會先說我對如何管理 Kuberentes 集群知之甚少，所以我可能在做一些愚蠢的事情。 開始：
我使用 GCP 的 Autopilot 模式創建了一個 Kube.netes 集群，並且我（認為我）使用“Infrastructure->Kube.netes Clusters”菜單（它顯示為在線）將集群注冊到我的 Gitlab 存儲庫。
使用 Gitlab 的 CI/CD，我有一個build階段，它將圖像推送到 repo 的容器注冊表（我看到圖像確實存在）。
然后我有一個帶有以下腳本的deploy階段：（為了隱私審查了一些數據）
kubectl config use-context <my_username>/<my_reponame>:<my_reponame>
kubectl delete secret regcred --ignore-not-found
kubectl create secret docker-registry regcred --docker-server="$CI_REGISTRY" --docker-username="$CI_REGISTRY_USER" --docker-password="$CI_REGISTRY_PASSWORD"
kubectl delete pod <my_reponame> --ignore-not-found
kubectl apply -f kube.yaml
我的kube.yaml包含以下內容：
apiVersion: v1
kind: Pod
metadata:
name: <my_reponame>
spec:
containers:
- name: <my_reponame>
image: registry.gitlab.com/<my_username>/<my_reponame>:latest
imagePullSecrets:
- name: regcred
我還有一個.gitlab/agents/<my_reponame>/config.yaml文件，其中包含以下內容：
ci_access:
projects:
- id: <my_username>/<my_reponame>
這是構建階段的 output：
Running with gitlab-runner 14.8.0~beta.44.g57df0d52 (57df0d52)
on blue-2.shared.runners-manager.gitlab.com/default XxUrkriX
Preparing the "docker+machine" executor
00:12
Using Docker executor with image bitnami/kubectl:latest ...
Pulling docker image bitnami/kubectl:latest ...
Using docker image sha256:208d070e071a0165e48ad7bf20b30c054328bcaaad76b0c53a9270a5e8627480 for bitnami/kubectl:latest with digest bitnami/kubectl@sha256:51eb9cb7d811e74bba30f97700cb433424d4025aabe70e8ff80e1289a964ab9c ...
Preparing environment
00:01
Running on runner-xxurkrix-project-32365648-concurrent-0 via runner-xxurkrix-shared-1646113353-c926e727...
Getting source from Git repository
00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/<my_username>/<my_reponame>/.git/
Created fresh repository.
Checking out 23743f89 as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:13
Using docker image sha256:208d070e071a0165e48ad7bf20b30c054328bcaaad76b0c53a9270a5e8627480 for bitnami/kubectl:latest with digest bitnami/kubectl@sha256:51eb9cb7d811e74bba30f97700cb433424d4025aabe70e8ff80e1289a964ab9c ...
$ kubectl config use-context <my_username>/<my_reponame>:<my_reponame>
Switched to context "<my_username>/<my_reponame>:<my_reponame>".
$ kubectl delete secret regcred --ignore-not-found
secret "regcred" deleted
$ kubectl create secret docker-registry regcred --docker-server="$CI_REGISTRY" --docker-username="$CI_REGISTRY_USER" --docker-password="$CI_REGISTRY_PASSWORD"
secret/regcred created
$ kubectl delete pod <my_reponame> --ignore-not-found
$ kubectl apply -f kube.yaml
Warning: Autopilot set default resource requests for Pod default/<my_reponame>, as resource requests were not specified. See http://g.co/gke/autopilot-defaults.
pod/<my_reponame> created
Cleaning up project directory and file based variables
00:00
Job succeeded
但是，部署到集群實際上並沒有起作用。 這是在 gcloud shell 中運行的一些數據
<my_username>@cloudshell:~ (<my_reponame>-342620)$ kubectl get pods
NAME READY STATUS RESTARTS AGE
<my_reponame> 0/1 ImagePullBackOff 0 13h
<my_reponame>-69775f4b-cft8w 0/1 ImagePullBackOff 0 13h
kubectl get pod <my_reponame> -o yaml`
Name: <my_reponame>
Namespace: default
Priority: 0
Node: gk3-<my_reponame>-nap-73l1ao51-a8e3ee2d-wjd8/10.132.0.5
Start Time: Tue, 01 Mar 2022 19:11:18 +0000
Labels: <none>
Annotations: autopilot.gke.io/resource-adjustment:
{"input":{"containers":[{"name":"<my_reponame>"}]},"output":{"containers":[{"limits":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"2Gi"},"requ...
seccomp.security.alpha.kubernetes.io/pod: runtime/default
Status: Pending
IP: 10.34.0.77
IPs:
IP: 10.34.0.77
Containers:
<my_reponame>:
Container ID:
Image: registry.gitlab.com/<my_username>/<my_reponame>:latest
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Limits:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
Requests:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-96wsd (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-api-access-96wsd:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 17s gke.io/optimize-utilization-scheduler Successfully assigned default/<my_reponame> to gk3-<my_reponame>-nap-73l1ao51-a8e3ee2d-wjd8
Warning Failed 14s kubelet Failed to pull image "registry.gitlab.com/<my_username>/<my_reponame>:latest": rpc error: code = Unknown desc = failed to pull and unpack image "registry.gitlab.com/<my_username>/<my_reponame>:latest": failed to copy: httpReaderSeeker: failed open: failed to authorize: failed to fetch oauth token: unexpected status: 401 Unauthorized
Normal BackOff 14s kubelet Back-off pulling image "registry.gitlab.com/<my_username>/<my_reponame>:latest"
Warning Failed 14s kubelet Error: ImagePullBackOff
Normal Pulling 2s (x2 over 16s) kubelet Pulling image "registry.gitlab.com/<my_username>/<my_reponame>:latest"
Warning Failed 1s (x2 over 14s) kubelet Error: ErrImagePull
Warning Failed 1s kubelet Failed to pull image "registry.gitlab.com/<my_username>/<my_reponame>:latest": rpc error: code = Unknown desc = failed to pull and unpack image "registry.gitlab.com/<my_username>/<my_reponame>:latest": failed to resolve reference "registry.gitlab.com/<my_username>/<my_reponame>:latest": failed to authorize: failed to fetch oauth token: unexpected status: 401 Unauthorized
kubectl get pod <my_reponame> -o yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
autopilot.gke.io/resource-adjustment: '{"input":{"containers":[{"name":"<my_reponame>"}]},"output":{"containers":[{"limits":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"2Gi"},"requests":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"2Gi"},"name":"<my_reponame>"}]},"modified":true}'
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"<my_reponame>","namespace":"default"},"spec":{"containers":[{"image":"registry.gitlab.com/<my_username>/<my_reponame>:latest","name":"<my_reponame>"}],"imagePullSecrets":[{"name":"regcred"}]}}
seccomp.security.alpha.kubernetes.io/pod: runtime/default
creationTimestamp: "2022-03-01T19:11:18Z"
name: <my_reponame>
namespace: default
resourceVersion: "1299911"
uid: d8a636ee-1dfc-40e9-9ba4-26b5674d3259
spec:
containers:
- image: registry.gitlab.com/<my_username>/<my_reponame>:latest
imagePullPolicy: Always
name: <my_reponame>
resources:
limits:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
requests:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
securityContext:
capabilities:
drop:
- NET_RAW
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-96wsd
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
imagePullSecrets:
- name: regcred
nodeName: gk3-<my_reponame>-nap-73l1ao51-a8e3ee2d-wjd8
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: gke.io/optimize-utilization-scheduler
securityContext:
seccompProfile:
type: RuntimeDefault
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-96wsd
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2022-03-01T19:11:18Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2022-03-01T19:11:18Z"
message: 'containers with unready status: [<my_reponame>]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2022-03-01T19:11:18Z"
message: 'containers with unready status: [<my_reponame>]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2022-03-01T19:11:18Z"
status: "True"
type: PodScheduled
containerStatuses:
- image: registry.gitlab.com/<my_username>/<my_reponame>:latest
imageID: ""
lastState: {}
name: <my_reponame>
ready: false
restartCount: 0
started: false
state:
waiting:
message: 'rpc error: code = Unknown desc = failed to pull and unpack image
"registry.gitlab.com/<my_username>/<my_reponame>:latest": failed to resolve reference
"registry.gitlab.com/<my_username>/<my_reponame>:latest": failed to authorize: failed
to fetch oauth token: unexpected status: 401 Unauthorized'
reason: ErrImagePull
hostIP: 10.132.0.5
phase: Pending
podIP: 10.34.0.77
podIPs:
- ip: 10.34.0.77
qosClass: Guaranteed
startTime: "2022-03-01T19:11:18Z"
從最后的命令日志來看，它似乎無法從容器注冊表中提取 - 我假設我傳遞了錯誤的憑據，但我找不到有關如何實現此目的的任何示例或文檔。
如有必要，我當然願意提供任何信息。
提前致謝：）
1 個解決方案
解決方案1
0 2022-03-28 22:11:19
嘗試https://chris-vermeulen.com/using-gitlab-registry-with-kube.netes/ ，您的集群無法從注冊表中提取，因為它沒有訪問權限。
CI/CD Gitlab 與 Harbor Registry
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.
相關問題
CI/CD Gitlab 與 Harbor Registry
對GitLab CI使用私有Docker注冊表
Gitlab CI/CD 與 JHipster 和 Docker 注冊表
將 kube.netes 連接到 GitLab Container Registry
帶有 GitLab、docker 和 docker 私有注冊表的 CD
Gitlab CI/CD 部署 Django Docker 容器
如何在Gitlab CI / CD中啟動Docker容器
GitLab CI/CD - nginx 容器上的供應商目錄
在 Gitlab CI/CD 中運行 Docker 容器時出現問題
GitLab CI 從私有注冊表中提取 docker
相關標簽