How to run Jenkins with Docker on Kubernetes?

Question

I'm attempting to execute a Jenkins & Docker CLI container on Kubernetes. Here are my steps:

I create the pod using:

kubectl --kubeconfig my-kubeconfig.yml run my-jenkins-pod --image=trion/jenkins-docker-client --restart=Never

Which creates a pod-based on the image https://hub.docker.com/r/trion/jenkins-docker-client

I create the deployment using:

kubectl --kubeconfig my-kubeconfig.yml apply -f /kub/kube

/kub/kube contains jenkins-deployment-yaml which I have configured as:

apiVersion: v1
kind: Service
metadata:
  name: my-jenkins-pod
spec:
  ports:
    - protocol: "TCP"
      port: 50000
      targetPort: 5001
  selector:
    app: my-jenkins-pod
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-jenkins-pod
spec:
  selector:
    matchLabels:
      app: my-jenkins-pod
  replicas: 1
  template:
    metadata:
      labels:
        app: my-jenkins-pod
    spec:
      containers:
        - name: ml-services
          image: trion/jenkins-docker-client
          ports:
            - containerPort: 5001

To access the Jenkins container I expose the IP using:

kubectl --kubeconfig my-kubeconfig.yml expose deployment my-jenkins-pod --type=LoadBalancer --name=my-jenkins-pod-public

To return the IP of the Jenkins and Docker image I use:

kubectl --kubeconfig my-kubeconfig.yml get services my-jenkins-pod-public

Which returns:

NAME                    TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
my-jenkins-pod-public   LoadBalancer   9.5.52.28   161.61.222.16   5001:30878/TCP   10m

To test I open the URL at location:

http://161.61.222.16:5001/

Which returns:

This page isn’t working
161.61.222.16 didn’t send any data.
ERR_EMPTY_RESPONSE

It seems the service has started but the port mappings are incorrect?

The log of the pod my-jenkins-pod contains:

Running from: /usr/share/jenkins/jenkins.war webroot: EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:42.899+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized @274ms to org.eclipse.jetty.util.log.JavaUtilLog 2021-04-03 11:15:43.012+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file 2021-04-03 11:15:44.369+0000 [id=1] WARNING oejshandler.ContextHandler#setContextPath: Empty contextPath 2021-04-03 11:15:44.416+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.39.v20210325; built: 2021-03-25T14:42:11.471Z; git: 9fc7ca5a922f2a37b84ec9dbc26a5168cee7e667; jvm 1.8.0_282-b08 2021-04-03 11:15:44.653+0000 [id=1] INFO oejwStandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 2021-04-03 11:15:44.695+0000 [id=1] INFO oejssDefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0 2021-04-03 11:15:44.695+0000 [id=1] INFO oejssDefaultSessionIdManager#doStart: No SessionScavenger set, using defaults 2021-04-03 11:15:44.696+0000 [id=1] INFO oejserver.session.HouseKeeper#startScavenging: node0 Scavenging every 660000ms 2021-04-03 11:15:45.081+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:45.203+0000 [id=1] INFO oejshandler.ContextHandler#doStart: Started w.@24f43aa3{Jenkins v2.286,/,fi le:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war} 2021-04-03 11:15:45.241+0000 [id=1] INFO oejserver.AbstractConnector#doStart: Started ServerConnector@4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} 2021-04-03 11:15:45.241+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: Started @2616ms 2021-04-03 11:15:45.245+0000 [id=21] INFO winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled 2021-04-03 11:15:46.479+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Started initialization 2021-04-03 11:15:46.507+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Listed all plugins 2021-04-03 11:15:47.654+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all plugins 2021-04-03 11:15:47.660+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Started all plugins 2021-04-03 11:15:47.68 0+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions 2021-04-03 11:15:48.620+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: System config loaded 2021-04-03 11:15:48.621+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: System config adapted 2021-04-03 11:15:48.621+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Loaded all jobs 2021-04-03 11:15:48.622+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Configuration for all jobs updated 2021-04-03 11:15:48.704+0000 [id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Started Download metadata 2021-04-03 11:15:48.722+0000 [id=40] INFO hudson.util.Retrier#start: Attempt #1 to do the action check updates server 2021-04-03 11:15:49.340+0000 [id=26] INFO jenkins.install.SetupWizard #init:

---

---

************************************************************* Jenkins initial setup is required. An admin user has been created and a password generated. Please use the following password to proceed to installation: ab5dbf74145c405fb5a33456d4b97436 This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

---

---

************************************************************* 2021-04-03 11:16:08.107+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Completed initialization 2021-04-03 11:16:08.115+0000 [id=20] INFO hudson.WebAppMain$3#run: Jenkins is fully up and running 2021-04-03 11:16:08.331+0000 [id=40] INFO hmDownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller 2021-04-03 11:16:08.332+0000 [id=40] INFO hudson.util.Retrier#start: Performed the action check updates server successfully at the attempt #1 2021-04-03 11:16:08.334+0000 [id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 19,626 ms

Is Jenkins server is started at port 8080? because of this log message:

11:15:45.241+0000 [id=1] INFO oejserver.AbstractConnector#doStart: Started ServerConnector@4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} 2021-04-03

I tried changing jenkins-deployment-yaml to point at port 8080 instead of 50000 , resulting in the updated jenkins-deployment-yaml :

apiVersion: v1
kind: Service
metadata:
  name: my-jenkins-pod
spec:
  ports:
    - protocol: "TCP"
      port: 8080

But the same error is returned when I attempt to access http://161.61.222.16:5001/

Are my port mappings incorrect? Is this a correct method of adding an existing docker container that is available on the docker hub to a Kubernetes cluster?

Update:

The result of command kubectl describe services my-jenkins-pod-public is:

Name:                     my-jenkins-pod-public
Namespace:                default
Labels:                   <none>
Annotations:              kubernetes.digitalocean.com/load-balancer-id: d46ae9ae-6e8a-4fd8-aa58-43c08310059a
Selector:                 app=my-jenkins-pod
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.245.152.228
IPs:                      10.245.152.228
LoadBalancer Ingress:     161.61.222.16
Port:                     <unset>  5001/TCP
TargetPort:               5001/TCP
NodePort:                 <unset>  30878/TCP
Endpoints:                10.214.12.12:5001
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Trying to access http://161.61.222.16:30878/ via browser returns:

This site can't be reached159.65.211.46 refused to connect. Try:

Checking the connection Checking the proxy and the firewall ERR_CONNECTION_REFUSED

Trying to access http://161.61.222.16:5001/ via browser returns:

This page isn't working 161.61.222.16 didn't send any data. ERR_EMPTY_RESPONSE

Seems the port 5001 is exposed/accessible but is not sending any data.

I also tried accessing 10.214.12.12 on ports 5001 & 30878 but both requests time out.

Answer 1

You need to use http://161.61.222.16:30878/ from outside of the host which is running containers on. Port 5001 is just accessible inside the cluster with internal IP (9.5.52.28 is in your case). Whenever you expose your deployment, automatically (also you can define manually) one of the NodePort (by default between 30000 - 32767)assign to the service for external request.

For service details, you need to run the below command. The command output will give you NodePort and another details.

kubectl describe services my-service

Please check related kubernetes documentation

Also you have configured service with port 5001 but Jenkins working with 8080 as far as I see in logs. Try to change target port of service to 8080 from 5001