One-node k8s setup with kubeadm and Calico

Calico is a pure layer-3 network solution to interconnect workloads as containers or virtual servers. I relies on bird (BGP opensource project), iptables and the Linux IP stack to work. Brings also additional security features like micro-segmentation. Thanks to BGP is very scalable and also it’s tightly integrated to Kubernetes as CNI plugin.

Now we’ll try the same setup, using kubeadm. One-node k8s setup with kubeadm and Calico. kubeadm is an open source basic tool to build kubernetes clusters. A more manual process and we’ll do one node kubenertes cluster

Prepare your node

Check first if you have all requirements of your server in the official documentation.

In my case, i am using a 8GB memory, 4 vCPUs and 25GB of disk KVM instance with Centos 7.8. No swap disk. Meeting all the requirements. More details of the configuration in my XML file uploaded in github.

I’ve run this script to prepare and install the require packages for Kubernetes 1.21.2: kubelet, kubeadm and kubectl. I am using this release, because normally in the Telco industry versions must be managed carefully. Other thing, in my case, I am using a Proxy to download packages or images. unset those in your profile to avoid issues.

#!/bin/sh
## Remove unset proxy settings in ENV


# Setup kubelet repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
# exclude=kube*
EOF

# Set SELinux in permissive mode (effectively disabling it)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# Enable iptables bridge routing
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

#Install kubelet, kubeadm and kubectl to version 1.21.2

yum -y clean all
yum -y install kubelet-1.21.2 kubeadm-1.21.2 kubectl-1.21.2
yum -y clean all
yum -y install kubelet-1.21.2 kubeadm-1.21.2 kubectl-1.21.2
# Maybe those next args are deprecated, but I have to set them to mae kubelet work. Got error message regarding the cgroups definition.
echo 'KUBELET_EXTRA_ARGS=--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice' > /etc/sysconfig/kubelet
systemctl restart kubelet
systemctl enable --now kubelet
systemctl status kubelet

You may want to check if those packages were actually installed

# yum info kubeadm-1.21.2 kubelet-1.21.2 kubectl-1.21.2 | grep "Name\|Version\|Size"
Name        : kubeadm
Version     : 1.21.2
Size        : 43 M
Name        : kubectl
Version     : 1.21.2
Size        : 44 M
Name        : kubelet
Version     : 1.21.2
Size        : 113 M

Initialize your node with kubeadm

Now it’s time to initialize your node. Set the –pod-network-cidr argument to the network segment you will use with Calico. It must not overlap with any other network address space in your LAN.

kubeadm init --pod-network-cidr=10.140.0.0/16

If you get the message of “Your Kubernetes control-plane has initialized successfully!” your are on the right path.

It’s time to set your kubectl:

# overwrite your previous config
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should see something like this:

[root@k8s-node01 ~]# kubectl get nodes
NAME         STATUS     ROLES    AGE    VERSION
k8s-node01   NotReady   master   2m6s   v1.21.2

[root@k8s-node01 ~]# journalctl -n 1 -u kubelet
-- Logs begin at Wed 2020-10-14 16:31:40 UTC, end at Wed 2020-10-14 18:18:37 UTC. --
Oct 14 18:18:37 k8s-node01 kubelet[8352]: E1014 18:18:37.304363    8352 kubelet.go:2184] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uni

Because cni plugin is not yet installed. It’s normal you get this message. It means it’s time to proceed installing your network solution: Calico.

Calico installation

Let’s start downloading the required files for resource and operator definitions

# Using proxy in my case, remove or modify that argument 
curl --proxy http://192.168.1.130:3128 https://docs.projectcalico.org/manifests/tigera-operator.yaml -o tigera-operator.yaml
curl --proxy http://192.168.1.130:3128  https://docs.projectcalico.org/manifests/custom-resources.yaml -o custom-resources.yaml

You must change the CIDR in the resources definition to match the one you have entered when you initialized your node

# This section includes base Calico installation configuration.
# For more information, see: https://docs.projectcalico.org/v3.16/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  # Configures Calico networking.
  calicoNetwork:
    # Note: The ipPools section cannot be modified post-install.
    ipPools:
    - blockSize: 26
      cidr: 10.140.0.0/16
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()

Now, create the operator and resources:

kubectl create -f tigera-operator.yaml
kubectl create -f custom-resources.yaml

Wait about 2 or 3 minutes for the pods to be in “running” status. You should see something like this:

[root@k8s-node01 ~]# kubectl get pods --all-namespaces
NAMESPACE         NAME                                       READY   STATUS    RESTARTS   AGE
calico-system     calico-kube-controllers-6b4855598b-lwc98   1/1     Running   0          2m22s
calico-system     calico-node-xsrkd                          1/1     Running   0          2m22s
calico-system     calico-typha-565684b598-95ldb              1/1     Running   0          2m23s
kube-system       coredns-6955765f44-n5nkr                   1/1     Running   0          5m34s
kube-system       coredns-6955765f44-vlkv9                   1/1     Running   0          5m34s
kube-system       etcd-k8s-node01                            1/1     Running   0          5m26s
kube-system       kube-apiserver-k8s-node01                  1/1     Running   0          5m26s
kube-system       kube-controller-manager-k8s-node01         1/1     Running   0          5m26s
kube-system       kube-proxy-pv8qf                           1/1     Running   0          5m35s
kube-system       kube-scheduler-k8s-node01                  1/1     Running   0          5m26s
tigera-operator   tigera-operator-5b466c74ff-9xlx5           1/1     Running   0          2m42s
[root@k8s-node01 ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
k8s-node01   Ready    master   5m58s   v1.21.2

And finally, you will need you need to untaint your node. because this is a controller, by default, no pods can be scheduled. Then, we need to change this condition.

kubectl taint nodes --all node-role.kubernetes.io/master-

And see something like this:

[root@k8s-node01 ~]# kubectl taint nodes --all node-role.kubernetes.io/master-
node/k8s-node01 untainted

Time to test

I’ll create a simple pod based in my hello-node image and test it

#hello-node-pod.yaml
# use :set paste in vi to set off autoindent. depends in your .vimrc

apiVersion: v1
kind: Pod
metadata:
    name: hello-node-app
    labels:
         app: hello-node-app
         type: front-end
spec:
    containers:
       - name: hello-node-app
         image: pinrojas/hello-node:v1

If you check the pod info. You should get something like this:

[root@k8s-node01 ~]# kubectl describe pod hello-node-app | grep podIP
Annotations:  cni.projectcalico.org/podIP: 10.140.85.197/32
              cni.projectcalico.org/podIPs: 10.140.85.197/32

Because it’s a pure layer-3 solution. And it’s not using overlay. You should be able to get access directly to the pod IP. You can test the Pod doing this:

[root@k8s-node01 ~]# curl http://10.140.85.197:8080
Hello World version ONE! Host/Pod: hello-node-app

And we’re done. See ya!

Source Info

Install kubernetes cluster with kubeadmfrom kubernetes.io documentation

Install calico manually after kubeadm from projectcalico.com documentation