kubernetes homelab: Installing rancher and creating k8s cluster

We have a small server and we’re going to set up some VMs in KVM for a Kubernetes homelab. I am planing to use OpenVSwitch for my virtual network, set a proxy and a DNS, and finally, install Rancher, and create a cluster with Calico. 

Over this post I am sharing my notes about installing rancher and settings up a kubernetes cluster with two controllers and two workers.

In the post, I will bring some details regarding:

  • Install rancher in a containers for testing purposes
  • What you need to get started and create your first kubernetes cluster
  • How to clean your nodes up if you mess this up and start over.

Install Rancher in a container for testing purposes

Rancher for production should be install in a cluster of three nodes. However, for testing purposes you have the option to install it in a container with self-signed certificate. We used the latter in this case. I assuming you have docker already installed in your server. Also, you need to be sure ports 80 and 443 are not been used for other application. Let’s create our container.

sudo docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher:latest

You should see something like this:

# sudo docker ps
CONTAINER ID        IMAGE                    COMMAND             CREATED             STATUS              PORTS                                      NAMES
a3c4930525f4        rancher/rancher:latest   "entrypoint.sh"     2 minutes ago       Up 2 minutes        0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   sad_poitras

Now, you should go for your first backup. You need to stop the container. Create and copy the volumes of your container to a tar file. Like this:

sudo docker stop sad_poitras
sudo docker create --volumes-from sad_poitras --name rancher-data-20200816 rancher/rancher:latest
sudo docker run  --volumes-from rancher-data-20200816 -v $PWD:/backup:z busybox tar pzcvf /backup/rancher-data-backup-2.4-20200816.tar.gz /var/lib/rancher
sudo docker start sad_poitras

If you need to restore, you can do the following:

sudo docker stop sad_poitras
sudo docker run  --volumes-from sad_poitras -v $PWD:/backup \
busybox sh -c "rm /var/lib/rancher/* -rf  && \
tar pzxvf /backup/rancher-data-backup-2.4-20200816.tar.gz"
sudo docker start sad_poitras

Get start with rancher

Now it’s time to get into you rancher portal. Just go to your server port 443. Remember the certificates are self-signed, then your browser will give you a security heads-up. Just accept the risks and go ahead. You should see something like this:

Installing rancher and creating k8s cluster: welcome page

Se the password and you’re done to create your first cluster.

Cleaning up all K8s cluster nodes

If this is your first install and you haven’t created any cluster yet, then skip this section. If you need to recreate your cluster. Then, you need to properly clean everything up before to start. It’s important when you recreate your cluster, to not use the same cluster name.  Even after cleaning, something was wrong when I used the same name and I had to go back again to. reate a new one with a different name. Maybe there’s a way to avoid this issue. I’ll let you to figure this out by yourself.

First of all, start cleaning up all containers, images and volumes running this command in every-node.

docker rm -f $(docker ps -qa)
docker rmi -f $(docker images -q)
docker volume rm $(docker volume ls -q)
for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
rm -rf /etc/ceph \
       /etc/cni \
       /etc/kubernetes \
       /opt/cni \
       /opt/rke \
       /run/secrets/kubernetes.io \
       /run/calico \
       /run/flannel \
       /var/lib/calico \
       /var/lib/etcd \
       /var/lib/cni \
       /var/lib/kubelet \
       /var/lib/rancher/rke/log \
       /var/log/containers \
       /var/log/kube-audit \
       /var/log/pods \
       /var/run/calico

And then, after that, restart all nodes to flush out any iptables rule. And we are ready to go.

Let’s create our first cluster

Here you have a high level view of the topology. Everything is one Centos server configured with KVM instances and linux/ovs bridges.

Installing rancher and creating k8s cluster: high level topology view

Back to the rancher portal:

  • Hit “Cluster” and the “Add Cluster”

Installing rancher and creating k8s cluster: add cluster

  • Then now you need to choose “From existing nodes (Custom)”, because in this case we have our KVM instances previously created. 

Installing rancher and creating k8s cluster: select customer nodes

  • Set the name of your cluster

Installing rancher and creating k8s cluster: name your cluster

  • Select your network provider. In this case I will use Calico.

Installing rancher and creating k8s cluster: select network provider

  • Now you can hit the next, unless you want to do something else (i.e. add labels)

We are going now to install the control servers. Those servers also will run etcd. For high availability  purposes, I should set 3 controllers. However, in those case I will only set two, because I don’t have much capacity in the host. But I didn’t want to have just one, to have the experience of settings more than one controller. Rancher gives us a really easy way to set those controller just copying the docker command to install the agent with some options:

  • Next screen unselect “worker” and select “etcd” and “control plane”

Installing rancher and creating k8s cluster: installing controller

  • Copy and run this command directly in your VM assigned as controller. IN my case, it’s control1.
  • Wait until the server is completely registered, that process can take several minutes.
  • Then, when its done, copy the same command to the second controller.
  • Wait some minutes for the registration to finish and the server active.
  • Then, unselect “etcd” and “control plane” and select worker.
  • Copy that command to all workers. You don’t need to wait to anyone to finish to stat with the next one.

It’s everything goes ok, you should see all nodes in the cluster like this:

Installing rancher and creating k8s cluster: nodes status after installation

And we are done. You can go directly to the cluster and run any kubelet command or use the rancher catalog of apps to do your first deployment.

Don’t forget to comment and share. See ya!

One Comment Add yours

Leave a Reply

Your email address will not be published. Required fields are marked *