kubernetes homelab: Setting up a Proxy and DNS for docker and centos packages

sudo yum -y install squid
sudo systemctl start squid
sudo systemctl  enable squid

Now you can check if the port TCP 3128 is listening

# sudo lsof -i -P -n | grep LISTEN | grep 3128
squid     18477    squid   11u  IPv6 743499      0t0  TCP *:3128 (LISTEN)

If this is ok, then we need to check if our private networks are allowed to use the proxy. You just need to check the configuration at /etc/squid/squid.conf and uncomment your network space. Also check ports 80 and 443 are allowed to use.

acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
#acl localnet src fc00::/7       # RFC 4193 local private network range
#acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http

And this is it. Now we go to the DNS

Setting up a Proxy and DNS for docker and centos packages: high level topology

Install bind and set your local domain

First, check your DNS ports are free to use. My case I had to disable dnsmasq, because I got the following:

#sudo lsof -i -P -n | grep LISTEN | grep 53
dnsmasq   1441   nobody    5u  IPv4  22219      0t0  TCP *:53 (LISTEN)
dnsmasq   1441   nobody    7u  IPv6  22221      0t0  TCP *:53 (LISTEN)

Then I have disabled dnsmasq and install bind:

sudo systemctl stop dnsmasq
sudo systemctl disable dnsmasq
yum -y install bind bind-utils

I have configured the local domain and the dns service:

# cat /etc/named.conf 
options {
    directory "/var/cache/bind";

    recursion yes;
    allow-recursion { 10.10.10.0/24; };
    listen-on { any; };

    forwarders {
            8.8.8.8;
            8.8.4.4;
    };
};

zone "lab.home" {
    type master;
    file "/etc/named/zones/db.lab.home";
};

And the the local zone called lab.home

#cat /etc/named/zones/db.lab.home
$TTL    604800
@       IN      SOA     ns1.lab.home. root.lab.home. (
                  3       ; Serial
             604800     ; Refresh
              86400     ; Retry
            2419200     ; Expire
             604800 )   ; Negative Cache TTL
;
; name servers - NS records
     IN      NS      ns1.lab.home.

; name servers - A records
ns1.lab.home.          IN      A      192.168.1.130

control1.lab.home.        IN      A      10.10.10.11
control2.lab.home.        IN      A      10.10.10.12
worker1.lab.home.        IN      A      10.10.10.21
worker2.lab.home.        IN      A      10.10.10.22

And finally, you have to enable the service and you are done with the DNS

sudo systemctl enable named
sudo systemctl start named

Configuring VMs to use DNS and Proxy

Now, we have to configure every VM to use this proxy and DNS. Let’s start with the DNS, changing the /etc/resolv.conf and /etc/hosts.

#cat /etc/resolv.conf
nameserver 192.168.1.130


# cat /etc/hosts
10.10.10.11 control1 control1.lab.home
10.10.10.12 control2 control2.lab.home
10.10.10.21 worker1 worker1.lab.home
10.10.10.22 worker2 worker2.lab.home

Also, I recommend use “hostnamectl set-hostname <<name>>” command in each node.

Let’s start settting the proxy. Set the profile to use env variables:

# cat .bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin
HTTP_PROXY=http://192.168.1.130:3128
HTTPS_PROXY=http://192.168.1.130:3128

export PATH

And also the yum.conf file:

# cat /etc/yum.conf 
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
proxy=http://192.168.1.130:3128

Install docker and enable the service. 

sudo yum install docker 
sudo systemctl enable docker
sudo systemctl start docker

Now, create docker service folder and add the proxy settings like this:

sudo mkdir -p /etc/systemd/system/docker.service.d
sudo cat > /etc/systemd/system/docker.service.d/http-proxy.conf << _EOF_
[Service]
Environment="HTTP_PROXY=http://192.168.1.130:3128"
Environment="HTTPS_PROXY=http://192.168.1.130:3128"
_EOF_

And reload the service:

sudo systemctl daemon-reload
sudo systemctl restart docker

Now, you are done and you can test with any docker image.

Setting up a Proxy and DNS for docker and centos packages: setting up docker 

You have the option the create all those files locally and use this script to set everything up:

#!/bin/sh
# I did it in a hurry, then, you're welcome to optimize it

# stop all instances
for i in `virsh list --all| awk -F ' ' 'FNR>2{print $2}' | sed '/^$/d'` ; do virsh destroy $i; done
 
# change configuration files for DNS and adding ssh keys
for instance in {"control1","control2","worker1","worker2"} ; do \
guestfish -a /home/qemu/virt-images/k8s-$instance.qcow2 -i upload resolv.conf /etc/resolv.conf; \
guestfish -a /home/qemu/virt-images/k8s-$instance.qcow2 -i upload hosts /etc/hosts; \
guestfish -a /home/qemu/virt-images/k8s-$instance.qcow2 -i upload authorized_keys /root/.ssh/authorized_keys; \
done
 
# starting instances again 
for i in `virsh list --all| awk -F ' ' 'FNR>2{print $2}' | sed '/^$/d'` ; do virsh start $i; done

# sleeping for 2 minutes
sleep 120

# add instances to known_hosts setting up proxy settings 
for instance in {"control1","control2","worker1","worker2"} ; do \
ssh-keyscan -H $instance >> ~/.ssh/known_hosts; \
scp yum.conf root@$instance:/etc/yum.conf; \
scp bash_profile root@$instance:/root/.bash_profile; \  
done
 
# Installing docker in all servers
for instance in {"control1","control2","worker1","worker2"} ; do \
ssh root@$instance 'yum -y install docker' ; \ 
done

# sleeping for 10 seconds
sleep 10

# Creating folder and adding file http proxy conf file to docker service
for instance in {"control1","control2","worker1","worker2"} ; do \
ssh root@$instance 'mkdir -p /etc/systemd/system/docker.service.d' ; \ 
scp http-proxy.conf root@$instance:/etc/systemd/system/docker.service.d/http-proxy.conf; \
ssh root@$instance 'sudo systemctl daemon-reload' ; \ 
ssh root@$instance 'sudo systemctl restart docker' ; \ 
done

Now everything is up to install rancher and create our first cluster. See you in the next post.