How To Set up Jenkins On Kubernetes Cluster.

How To Set up Jenkins On Kubernetes Cluster.

Hosting Jenkins on a Kubernetes cluster is beneficial for Kubernetes-based DevOps and dynamic container-based scalable Jenkins agents.

what is Jenkins in DevOps?

An open-source automation server that enables developers around the world to reliably build, test, and deploy their software.  Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.

According to the official site
Jenkins for Kubernetes Cluster
Jenkins Logo

Why Choose Jenkins for your DevOps?

Continuous Integration and Continuous Delivery

As an extensible automation server, Jenkins can be used as a simple CI server or turned into the continuous delivery hub for any project.

Easy installation

Jenkins is a self-contained Java-based program, ready to run out-of-the-box, with packages for Windows, Linux, macOS and other Unix-like operating systems.

Easy configuration

Jenkins can be easily set up and configured via its web interface, which includes on-the-fly error checks and built-in help.

Extensible

Jenkins can be extended via its plugin architecture, providing nearly infinite possibilities for what Jenkins can do.

Distributed

Jenkins can easily distribute work across multiple machines, helping drive builds, tests and deployments across multiple platforms faster.

Plugins

With hundreds of plugins in the Update Center, Jenkins integrates with practically every tool in the continuous integration and continuous delivery toolchain.

Kubernetes Jenkins Deployment.

Lets get started with deploying Jenkins on Kubernetes.

Create a Namespace

Create a Namespace for Jenkins. It is good to categorize all the devops tools as a separate namespace from other applications.

kubectl create namespace devops-tools
Create a service account with Kubernetes admin permissions.

Create a serviceAccount.yaml file and copy the following admin service account manifest.

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-admin
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-admin
  namespace: devops-tools

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins-admin
subjects:
- kind: ServiceAccount
  name: jenkins-admin
  namespace: devops-tools

The serviceAccount.yaml creates a jenkins-admin clusterRole, jenkins-admin ServiceAccount and binds the clusterRole to the service account.

The jenkins-admin cluster role has all the permissions to manage the cluster components. You can also restrict access by specifying individual resource actions.

Now create the service account using kubectl.

Create a volume.yaml and copy the following persistent volume manifest.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv-volume
  labels:
    type: local
spec:
  storageClassName: local-storage
  claimRef:
    name: jenkins-pv-claim
    namespace: devops-tools
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - worker-node01

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
  namespace: devops-tools
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

Important: Replace worker-node01 with any one of your cluster worker nodes hostname.

Authors Note

You can get the worker node hostname using the kubectl.

kubectl get nodes
Create local persistent volume for persistent Jenkins data on Pod restarts.

To create the volume using kubectl

kubectl create -f volume.yaml
Create a deployment YAML and deploy it.

Create a Deployment file named deployment.yaml and copy the following deployment manifest.

Here we are using the latest Jenkins LTS docker image from the Docker hub.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  namespace: devops-tools
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins-server
  template:
    metadata:
      labels:
        app: jenkins-server
    spec:
      securityContext:
            fsGroup: 1000 
            runAsUser: 1000
      serviceAccountName: jenkins-admin
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts
          resources:
            limits:
              memory: "2Gi"
              cpu: "1000m"
            requests:
              memory: "500Mi"
              cpu: "500m"
          ports:
            - name: httpport
              containerPort: 8080
            - name: jnlpport
              containerPort: 50000
          livenessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 90
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 5
          readinessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
          volumeMounts:
            - name: jenkins-data
              mountPath: /var/jenkins_home         
      volumes:
        - name: jenkins-data
          persistentVolumeClaim:
              claimName: jenkins-pv-claim

In this Jenkins Kubernetes deployment we have used the following.

  1. securityContext for Jenkins pod to be able to write to the local persistent volume.
  2. Liveliness and readiness probe.
  3. Local persistent volume based on local storage class that holds the Jenkins data path /var/jenkins_home
Create a service YAML and deploy it.

Create a service.yaml and copy the following service manifest.

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: devops-tools
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path:   /
      prometheus.io/port:   '8080'
spec:
  selector: 
    app: jenkins-server
  type: NodePort  
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 32000

We have created a deployment. However, it is not accessible to the outside world. For accessing the Jenkins deployment from the outside world, we should create a service and map it to the deployment.

Note: Here, we are using the type as NodePort which will expose Jenkins on all kubernetes node IPs on port 32000. If you have an ingress setup, you can create an ingress rule to access Jenkins. Also, you can expose the Jenkins service as a Loadbalancer if you are running the cluster on AWS, Google, or Azure cloud.

Create the Jenkins service using kubectl.

kubectl apply -f service.yaml
Access the Jenkins application on a Node Port.

Now if you browse to any one of the Node IPs on port 32000, you will be able to access the Jenkins dashboard.

http://<node-ip>:32000

Jenkins will ask for the initial Admin password when you access the dashbaord for the first time.

You can get that from the pod logs either from the kubernetes dashboard or  CLI. You can get the pod details using the following CLI command.

kubectl get pods --namespace=devops-tools

And with the pod name, you can get the logs as shown below. replace the pod name with your pod name.

kubectl logs jenkins-deployment-2539456353-j00w5 --namespace=jenkins

Alternatively, you can run the exec command to get the password directly from the location as show below.

kubectl exec -it jenkins-559d8cd85c-cfcgk cat  /var/jenkins_home/secrets/initialAdminPassword -n devops-tools

Once you enter the password you can proceed installing the suggested plugin and creating a admin user. All these steps are self-explanatory from the Jenkins dashboard.

So, This was Annotating Kubernetes Services For Humans Also, Check New OpenSource Alternative to Datadog?

If you need help with your Website Development or Digital Marketing, discover our Services.

Amit Chaudhary

SRE at Calibo. Helping OpenSource Community. Co-founder hyCorve limited. Certified Checkbox Unchecker. Connecting bare metal to cloud.

All author posts
Write a comment