We have a small server and we’re going to set up some VMs in KVM for a Kubernetes homelab. I am planing to use OpenVSwitch for my virtual network, set a proxy and a DNS, and finally, install Rancher, and create a cluster with Calico.
Over this post I am sharing my notes about installing KVM and creating the VM instances ( or KVM domains) and connecting them to a virtual network.
In the post, I will bring some details regarding:
- How to install KVM
- How to create VMs or KVM domains from a Centos7 cloud image
- Configure those VMs and have them ready to start installing docker and K8s
This is one of a serie of posts with the developing of this lab. here you have the list of all:
I will create 4 instances:
- 2 for k8s controllers. Yeah! I know 3 instances are the best for HA, but this is a lab and I just want to test to have more than one controller to check how to deal with something like with Rancher.
- 2 for workers
Install required packages
Install KVM and guestfish:
sudo yum -y install qemu-kvm libvirt bridge-utils libguestfs-tools libvirt-python genisoimage
Download last Centos QCOW2 minimal image
curl -s http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 -o CentOS-7-x86_64-GenericCloud.qcow2
Modify root password in the QCOW2 image
First of all, let’s set a password for root (i.e. “newpass”):
openssl passwd -1 newpass $1$7e5esg3P$iXizEINjtaCMJOnxeRGWg.
And then let’s get into the image using guestfish and add this password to the /etc/shadow file and user root. We’ll use vi to change the file.
sudo LIBGUESTFS_BACKEND=direct guestfish --rw -a CentOS-7-x86_64-GenericCloud.qcow2 ><fs> run ><fs> list-filesystems /dev/sda1: xfs ><fs> mount /dev/sda1 / ><fs> vi /etc/shadow ><fs> exit
The top part of the former /etc/shadow file:
root:!!:18353:0:99999:7::: bin:*:18353:0:99999:7::: daemon:*:18353:0:99999:7:::
The file now:
root:$1$7e5esg3P$iXizEINjtaCMJOnxeRGWg:18353:0:99999:7::: bin:*:18353:0:99999:7::: daemon:*:18353:0:99999:7:::
Resize QCOW2 image
Now, we’ll resize the QCOW2 image to get more disk space for our pods (i.e. 20G more).
# cp CentOS-7-x86_64-GenericCloud.qcow2 centos7.qcow2
# qemu-img info centos7.qcow2
image: centos7.qcow2
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 819M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
# qemu-img resize centos7.qcow2 +20G
Now we need to resize the underlying filesystems using “virt-resize“.
qemu-img resize centos7.qcow2 +20G
Note that “virt-resize” CANNOT resize disk images in-place. So we need to make a backup copy and use it as input and use the original qcow as output. See example below.
First, we make a backup copy of the disk as shown below.
cp centos7.qcow2 centos7-orig.qcow2
NOTE: In this example /dev/sda1 is not the /boot partition. So be careful you are growing the correct partitions on your qcow.
# sudo LIBGUESTFS_BACKEND=direct virt-resize --expand /dev/sda1 centos7-orig.qcow2 centos7.qcow2 [ 0.0] Examining centos7-orig.qcow2 ********** Summary of changes: virt-resize: warning: unknown/unavailable method for expanding the xfs filesystem on /dev/sda1 /dev/sda1: This partition will be resized from 8.0G to 28.0G. ********** [ 3.8] Setting up initial partition table on centos7.qcow2 [ 4.2] Copying /dev/sda1 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
Resize operation completed with no errors. Before deleting the old disk, carefully check that the resized disk boots and works correctly.
# qemu-img info centos7.qcow2
image: centos7.qcow2
file format: qcow2
virtual size: 28G (30064771072 bytes)
disk size: 821M
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
Important: Remember to use “xfs_growfs /” in the VM, when you connect to it, to expand the root file system later.
Finally, we verify that the filesystems have grown.
# sudo LIBGUESTFS_BACKEND=direct virt-filesystems --long -h --all -a centos7.qcow2 Name Type VFS Label MBR Size Parent /dev/sda1 filesystem xfs - - 28G - /dev/sda1 partition - - 83 28G /dev/sda /dev/sda device - - - 28G -
Create and configure KVM instance
Kubernetes homelab with KVM and OpenVSwitch: High level design
Create qcow2 images for all instances in your lab
sudo mkdir /home/qemu
for instance in {"control1","control2","worker1","worker2"} ; do \
sudo cp centos7.qcow2 /home/qemu/virt-images/k8s-$instance.qcow2; \
done
Change ownership and permissions
sudo chown qemu:qemu -R /home/qemu
I am installing 2 kubernetes master servers and 2 kubernetes worker servers with 8GB memory each using the following commands:
for instance in {"control1","control2","worker1","worker2"} ; do \
sudo virt-install --name k8s-$instance --memory 8192 --vcpus 4 --disk /home/qemu/virt-images/k8s-$instance.qcow2 --network=bridge:br_int,model=virtio,virtualport_type=openvswitch --import --os-variant rhel7 --noautoconsole; \
done
Let’s create now the ifcfg files to configure the network
Let’s create now the ifcfg files to configure the network cat > ifcfg-eth0-k8s-worker1 << _EOF_ DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no TYPE=Ethernet PREFIX=24 BOOTPROTO=static IPADDR=10.10.10.21 GATEWAY=10.10.10.1 _EOF_ cat > ifcfg-eth0-k8s-worker2 << _EOF_ DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no TYPE=Ethernet PREFIX=24 BOOTPROTO=static IPADDR=10.10.10.22 GATEWAY=10.10.10.1 _EOF_ cat > ifcfg-eth0-k8s-control1 << _EOF_ DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no TYPE=Ethernet PREFIX=24 BOOTPROTO=static IPADDR=10.10.10.11 GATEWAY=10.10.10.1 _EOF_ cat > ifcfg-eth0-k8s-control2 << _EOF_ DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no TYPE=Ethernet PREFIX=24 BOOTPROTO=static IPADDR=10.10.10.12 GATEWAY=10.10.10.1 _EOF_
Now, let’s copy those ifcfg file to the QCOW2 images thru guestfish
for instance in {"control1","control2","worker1","worker2"} ; do \
sudo guestfish -a /home/qemu/virt-images/k8s-$instance.qcow2 -i upload ifcfg-eth0-$instance /etc/sysconfig/network-scripts/ifcfg-eth0; \
done
Destroy instances a recreate instances with the new network configuration:
for i in `sudo virsh list --all| awk -F ' ' 'FNR>2{print $2}' | sed '/^$/d'` ; do sudo virsh destroy $i; done
for i in `sudo virsh list --all| awk -F ' ' 'FNR>2{print $2}' | sed '/^$/d'` ; do sudo virsh start $i; done
And we are done!
Checking instances and configurations in our Kubernetes homelab
Let’s check our bridge in openvswitch bridge and ports:
# sudo ovs-vsctl show
14b3eb6f-13e7-4dff-9508-6f6688bc2408
Bridge br_ext
Port "enp0s20f0"
Interface "enp0s20f0"
Port "tap_2"
Interface "tap_2"
type: internal
Port br_ext
Interface br_ext
type: internal
Bridge br_int
Port "vnet2"
Interface "vnet2"
Port "vnet3"
Interface "vnet3"
Port br_int
Interface br_int
type: internal
Port tap_test
Interface tap_test
type: internal
Port "vnet1"
Interface "vnet1"
Port "vnet0"
Interface "vnet0"
Port "tap_1"
Interface "tap_1"
type: internal
ovs_version: "2.5.10"
Check the interface name in the instance:
# sudo virsh dumpxml k8s-control1 | grep "vnet"
<target dev='vnet0'/>
Hope this Kubernetes homelab with KVM and OpenVSwitch, could be a good help. Please, share or comment.
See ya!
Information source:
How to Resize a qcow2 Image and Filesystem with Virt-Resize by C. Paquin
3 thoughts on “Kubernetes homelab: Prepping cluster with KVM and OpenVSwitch”