从另一个吊舱更改 kubernetes stroge class 安装值

Question

I want to publish sonarqube with kubernetes. I did successfully with official packages. But i want to use some plugins old version and some custom plugins. In local with docker-compose files, i created a fly-away container that fills the plugins directory(/opt/sonarqube/extensions/plugins) with plugins. And use that volume with sonarqube container. As a conclusion: Sonarqube extensions volume directory is created (or filled) from different container(do the job and die).

I want to use the same path with kubernetes but couldn't do that. My flyaway container didn't fill the path.

My kubernetes deployments files:

1-) sonar-pvc-extensions.yml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: claim-sonar-extensions
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi

2-) sonarqube-deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sonarqube
spec:
  replicas: 1
  selector:
    matchLabels:
      name: sonarqube
  template:
    metadata:
      name: sonarqube
      labels:
        name: sonarqube
    spec:
      containers:
        - image: sonarqube:7.9.1-community
          name: sonarqube
          env:
            - name: SONARQUBE_JDBC_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-pwd
                  key: password
            - name: SONARQUBE_JDBC_URL
              value: jdbc:postgresql://sonar-postgres:5432/sonar
          ports:
            - containerPort: 9000
              name: sonarqube
          volumeMounts:               
            - name: data-sonar-extensions
              mountPath: /opt/sonarqube/extensions/plugins
          resources:
            requests:
              memory: 2000Mi
            limits:
              memory: 2000Mi
      volumes:
        - name: data-sonar-extensions
          persistentVolumeClaim:
            claimName: claim-sonar-extensions
      initContainers:
        - name: sysctl
          image: busybox
          imagePullPolicy: IfNotPresent
          command: ['sysctl', '-w', 'vm.max_map_count=262144']
          securityContext:
            privileged: true

3-)Sample plugins Dockerfile

FROM alpine:3.4

RUN apk --no-cache add --repository http://dl-cdn.alpinelinux.org/alpine/edge/community wget ca-certificates

ENV PLUGINS_DIR /opt/sonarqube/extensions/plugins

WORKDIR $PLUGINS_DIR
RUN wget https://github.com/gabrie-allaigre/sonar-gitlab-plugin/releases/download/4.1.0-SNAPSHOT/sonar-gitlab-plugin-4.1.0-SNAPSHOT.jar
RUN wget https://binaries.sonarsource.com/Distribution/sonar-java-plugin/sonar-java-plugin-6.3.0.21585.jar
RUN wget https://github.com/SonarSource/sonar-php/releases/download/3.4.0.5461/sonar-php-plugin-3.4.0.5461.jar
ENV JAVASCRIPT_VERSION 2.20.0.4207

VOLUME $PLUGINS_DIR

CMD ls -asl $PLUGINS_DIR

I tried that approach with sonar-plugin-deployment.yml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sonarqube-plugin
spec:
  replicas: 1
  selector:
    matchLabels:
      name: sonarqube-plugin
  template:
    metadata:
      name: sonarqube-plugin
      labels:
        name: sonarqube-plugin
    spec:
      containers:
        - image: my-kubernetes-registry/plugins
          name: sonarqube-plugins
          volumeMounts:
            # This name must match the volumes.name below.
            - name: data-sonar-extensions
              mountPath: /opt/sonarqube/extensions/plugins
      volumes:
        - name: data-sonar-extensions
          persistentVolumeClaim:
            claimName: claim-sonar-extensions

But didn't successed. And break something. This time my plugins directory became empty:

sonarqube@sonarqube-85b98d9845-l2sql:/opt/sonarqube/extensions/plugins$ ls -al
total 24
drwxr-xr-x 3 root      root       4096 May 30 16:19 .
drwxr-xr-x 1 sonarqube sonarqube  4096 May 30 16:39 ..
drwx------ 2 root      root      16384 May 30 16:19 lost+found

I am not using persistent volume. PVC is looking to stroge class. So i cant use accessModes as ReadWriteMany.

As a results: I want to change a stroge path with a fly-away container and use that path inside an app.

I am noob to kubernetes if you suggest a different approach i will be apreciated.

Answer 1

Are you sure, you want to use the same directory for container volume mounts and to store the plugins. I believe this is causing a conflict. Can you try changing the volume mount directory (in deployment container spec)?

Answer 2

Check Init Containers should suit your need.

You can populate volume with data using init container and when it's done you can run your app on this data.

Answer 3

When we mount Docker's WORKDIR in Kubernetes, Kubernetes cleans up or override the directory. In such cases, we have to set a different directory from the mount path as WORKDIR in Docker and later in Kubernetes we have to move our contents to mounted directory.

Dockerfile, here I configured WORKDIR as /opt/sonarqube/plugins

FROM alpine:3.11

RUN apk --no-cache add --repository http://dl-cdn.alpinelinux.org/alpine/edge/community wget ca-certificates

WORKDIR /opt/sonarqube/plugins

RUN wget https://github.com/gabrie-allaigre/sonar-gitlab-plugin/releases/download/4.1.0-SNAPSHOT/sonar-gitlab-plugin-4.1.0-SNAPSHOT.jar &&\
    wget https://binaries.sonarsource.com/Distribution/sonar-java-plugin/sonar-java-plugin-6.3.0.21585.jar &&\
    wget https://github.com/SonarSource/sonar-php/releases/download/3.4.0.5461/sonar-php-plugin-3.4.0.5461.jar

sonar.yml, Here I have used postStart lifecycle hook to copy the plugins to mount path. It copies the external plugins to the mount path before pod starts.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: sonarqube
  name: sonarqube
spec:
  containers:
  - image: harik8/demo:latest
    name: sonarqube
    resources: {}
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh","-c", "mv /opt/sonarqube/plugins/*.jar /opt/sonarqube/extensions/plugins"]
    command: ["cat"]
    tty: true
    volumeMounts:
    - name: data-sonar-extensions
      mountPath: /opt/sonarqube/extensions/plugins
  dnsPolicy: ClusterFirst
  restartPolicy: Never
  volumes:
  - name: data-sonar-extensions
    emptyDir: {}
status: {}

Once pod starts, exec and list /opt/sonarqube/extensions/plugins directory,

$ kubectl exec -it sonarqube sh
/opt/sonarqube/extensions/plugins # pwd
/opt/sonarqube/extensions/plugins
/opt/sonarqube/extensions/plugins # ls -la
total 33320
drwxrwxrwx    2 root     root          4096 May 31 20:59 .
drwxr-xr-x    3 root     root          4096 May 31 20:59 ..
-rw-r--r--    1 root     root      10280677 Mar 28  2019 sonar-gitlab-plugin-4.1.0-SNAPSHOT.jar
-rw-r--r--    1 root     root      18712457 Apr  8 13:26 sonar-java-plugin-6.3.0.21585.jar
-rw-r--r--    1 root     root       5114341 May 11 15:24 sonar-php-plugin-3.4.0.5461.jar
/opt/sonarqube/extensions/plugins