Make K3s cluster on your home-lab

Make K3s cluster on your home-lab

Introduction

K3s is a lightweight fully production-ready Kubernetes distribution. K3s is designed from the ground up while keeping key components in mind like the ease of installation, lower memory overhead, and a single binary that is less than 100MB in size. K3s packs runtime such as Containers, CNI ( Container network interface) with Flannel, CoreDNS, Metrics server, Traefik for ingress, Kube-route, Helm-controller, and much more in a single Binary file.

K3s simplifies many daunting K8s such as managing TLS certificates of Kubernetes components, connections between worker and server nodes. Not everyone has a Physical server in their home to play with hence K3s makes perfect sense here as it needs minimum resources to run cluster as it’s designed to be deployed on edge devices that are oftentimes not highly configured.

So if you want to deploy a Lightweight Kubernetes Cluster within 5 min along with ease of installation, less memory footprint, and all in one binary which is less than 100MB? Don’t worry here we will tell you just that.

Prerequisites

  • Physical or virtual machines where you will deploy your cluster. It is recommended to either just use one node or three nodes or more to make a cluster as 2 node clusters have a performance penalty.
  • Having a domain/DNS is preferred as it makes things easier else you will be using the host’s file (we will show both ways).
  • Working internet connection to get necessary scripts and binaries.
Hosts

Here I have 3 virtual machines that I will be using k3sa, k3sb, and k3sc. Where k3sa will be master hence it has 4 CPU cores and 8GB ram, while k3sb and k3sc will be worker nodes with 2 CPU cores and 4GB ram. Just to prove the Flexibility of k3s ill be running master on Centos 7 and worker on RHEL 8, Though I will recommend you to stick to one.

Let’s get started

First thing would be to log in to your matching (root is preferred as it will make things easy).

Login in node 1

Then to make things easy we will disable firewall.(I’ll later make a dedicated guide on setting up firewall for k3s)

It can be easily done by single command. For Centos or RHEL use systemctl disable --now firewalld and for Ubuntu use sudo ufw disable

Disable firewall

Now we need to set dns of hostfile (if you are making single node cluster then you can skip this step), and it can be done by adding entries in your DNS (here you can see entries in my Cloudflare ).

DNS

or you need to add these in /etc/hosts

sudo tee -a /etc/hosts<<EOF
192.168.0.30 k3sa k3sa.tros.tech
192.168.0.31 k3sb k3sb.tros.tech
192.168.0.32 k3sc k3sc.tros.tech
EOF

you will see output like this (Remember to change the hostname and ip range according to your setup).

address in hosts

Now comes the fun part of installing K3S and the real fun part is it can be done with single command and it will install all dependencies and start k3s.

curl -sfL https://get.k3s.io | sh -  
Initializing k3s

If you were making a single node cluster you are pretty much done at this point.

Now we will add two more worker nodes. For that we will now need to get node token from the first node and we can get it by following command.

cat /var/lib/rancher/k3s/server/node-token
Node Token

Now login to another machines and disable firewall (with same command as listed above). Then you need to create two environment variables namely k3s_token and k3s_url

where K3S_TOKEN will have node token that got from previous location, and K3S_URL will have master node address on port 6443 as shown in the image below.

Worker node

Now just run curl -sfL https://get.k3s.io | K3S_URL=${k3s_url} K3S_TOKEN=${k3s_token} sh - as k3s is smart enough to find the env variable to install itself as agent.

Agent installed on worker nodes

Now you can run kubectl get nodes to see if all nodes are connected

3 node cluster ready

And you have a cluster Ready 🙂

Troubleshooting

Sometimes it can happen that your worker node is not connected to the master node, it can because when the firewall is not disabled or not properly configured. This can also happen when you haven’t set env variables properly.

uninstalling k3s

K3s is super easy to remove you just have to run /usr/local/bin/k3s-uninstall.sh on the master node and /usr/local/bin/k3s-agent-uninstall.sh on worker node and k3s will completely uninstall.

Wrapping up

This was the guide to help you make a k3s cluster at home. K3s is a good choice for the home lab as it’s very resource-efficient and you can also run the workload on the master node. It is easy to install, and maintain.

Amit Chaudhary

SRE at Calibo. Helping OpenSource Community. Co-founder hyCorve limited. Certified Checkbox Unchecker. Connecting bare metal to cloud.

All author posts
Write a comment