Prepare, Install & configure of Kubernetes (K3S) on Raspberry Pi 4B 8GB Cluster with Raspbian OS 64 BIT

“Failure is only the opportunity to begin again, this time more intelligently.” ~ Henry Ford

The real reason to put the above quote and picture at the beginning of this blog is because I tried at least 20-30 times failed attempts to configure the cluster until I achieved the victory 🙂

It all started during the COVID-19 Lockdown days to do something new and equip knowledge about the unfamiliar to me. I picked up Raspberry Pi 4B 8GB X 3 and setting up a K8S miniature version Rancher K3S running with Raspbian OS 64 bit. But it wasn’t an easy deal as I thought…

After going through 30+ iterations (2-weekends) here is the guide to prepare, install and configure Raspberry Pi 4B cluster.

Image 1: Raspberry Pi Logo – Addictive Isn’t it?

Image 2: A Simple Architecture Diagram showing the connectivity from the Pi’s to the internet.

Image 3: The 3 musketeers (Pi’s) are connected together to a POE (Power Over Ethernet) switch. My mate told me that those cables look like noodles 🙂

Hardware Requirements

1. Raspberry Pi 4B 8GB X 3
2. Raspberry Pi POE HAT X 3
3. Cat-6 Ethernet Cables X 4
4. POE Switch (Gigabit preferred) X 1
5. SSD Card Class 10 128 GB (32 GB is enough) X 3
6. Micro HDMI to HDMI Converter with HDMI Cable X 1 each
7. 4 layer Raspberry Pi 4 Stack Case X 1 (Optional)
8. Keyboard, Mice and Monitor for initial configuration (Optional)
   • Total Cores = 12 Total Memory = 24 GB Total Disk = 384 GB!!

Step by Step Illustration

1. Download the beta version of Raspbian OS 64 bit save it on your computer
2. Download the Raspberry Pi Imager and install on your computer
3. Flash the SD card with the downloaded image X 3 times 🙂
4. Add the text cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory at the end of the first line in the file cmdline.txt {make sure its only single line in the file}
5. Eject the SD Card and insert into the SD Card slot of Pi master and power on the Pi by connecting the ethernet cable
6. Wait for it to boot the GUI (Yes, preinstalled – can’t help it). Before proceeding with Initial Pi Setup Wizard, Open terminal and type command ip a
7. Note down the MAC, assign IP address in the Manual DHCP Reservation list in the router {Most of the new router support this feature} X 3 times
8. Now proceed with wizard
   • Change the default password
   • Select the WiFi Network and enter credentials
   • Allow to update the packages and complete wizard
9. Edit /etc/hosts file and update the host entries
10. Set the firewall to use iptables-legacy mode
11. Edit /etc/sysctl.conf file and enable IPv4 and IPv6 forwarding and disable RP Filter
12. Enable Promiscuous mode on both eth0 and wlan0
13. Git clone the repo https://github.com/rancher/k3s-ansible and follow the instructions to automate the install and configuration of Kubernetes miniature version K3S
14. Alternatively refer to Grégoire Jeanmart’s blog series with the detailed explanation and screenshots.
15. Install Metallb Load Balancer and get an exposed IP/IP’s to install many cloud native apps and practice, practice & practice 🙂

Screenshots of the above steps

Image 4: Click on the Operating System and select custom image and point to Raspbian OS 64 bit.

Image 5: Select SDXC Card – 128 GB and click on WRITE

• 

• 

Image 6 & 7: After image is written to the SD card then Imager verifies the flashed image.

Image 8: Re-Insert the SD card and open the file cmdline.txt and add cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory at the end of the line.

Image 9: MAC Addresses of Pi’s with Manual DHCP IP Reservation specified on the router

Image 10: Run the command grep mem /proc/cgroups | awk ‘{ print$4 }’ – Output should be 1. Else edit the /boot/firmware/cmdline.txt and add the text.

• 

• 

• 

• 

• 

• 

Image 11-16: Inital Pi Setup Wizard – Set Password, Configure Wireless, update software and restart the Pi’s

Image 17: Update hostname by the command hostnamectl set-hostname <name> and check the DHCP IP addresses allocated by the WiFi Router. Make sure the IP’s are assigned properly to Master, Node 1 and 2.

Image 18: Change iptables to legacy mode with below commands and edit the /etc/sysctl.conf file

sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

Image 19: Enable IP forwarding (set to 1) and disable reverse path filter (set to 0)

Image 20: Enable PROMISCUOUS mode on both physical interfaces eth0 and wlan0 using below commands

sudo ip link set wlan0 promisc on
sudo ip link set eth0 promisc on

***EDIT***
To set the promiscuous mode on every time Pi's boot. Use the below service file.

Create file on master:
nano /etc/systemd/system/promisc.service
[Unit]
Description=Control promiscuous mode for interface %i
After=network.target
[Service]
Type=oneshot
ExecStart=/sbin/ip link set wlan0;eth0 promisc on
ExecStop=/sbin/ip link set wlan0;eth0 promisc off
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target

Copy to rest of the nodes:
rsync --rsync-path="sudo rsync" /etc/systemd/system/promisc.service pi@192.168.15.18:/etc/systemd/system/promisc.service

Enable the service:
sudo systemctl enable promisc.service
sudo systemctl start promisc.service

• 

• 

Image 20 & 21: Generate SSH key pair and Copy Key Pair to all the nodes

Follow the instructions of Step 13 (Automated – Preferred) or Step 14 (Manual). K3S cluster build should complete without much hassles.

Image 22: K3S cluster nodes and list of entities of all the namespaces.

Image 23: Nginx deployed and exposed via Metallb provided IP. This IP was further exposed to internet through Virtual Servers configuration on the Home Router having Static IP.

If you enjoyed this post, I’d be very grateful if you’d help it spread by emailing it to a friend, or sharing it on Twitter or Facebook. Thank you!

What am I missing here? Let me know in the comments and I’ll add it in!

Advertisement