How to Run Kubernetes on Raspberry Pi 4 - Ubuntu Server

In this post I just want to write or kind of document how I managed to run Kubernetes on Ubuntu Server image of Raspberry Pi. Just for a short background, I got two Raspberry Pi 4 Model B 4GB as a birthday Gift from my wife and since the first night I started to play with them.

Finding an Image

As you may know, you have to find an image that suits you. In general, Raspbian is the official Operating System of Raspberry Pi but somehow it was not suitable for me since I was looking for something more like server operating system. Thus, I decided to go with Ubuntu Server Image. Creating Installation Media was pretty neat for me since I just followed the documention Raspberry Pi 4 Model B 4GB

Since I have two Raspberry Pi I have done the same thing twice!

Home Network

In General you can have dedicated switch for your Raspberry Pi and connect them into this Switch. However, in my case, I just directly connected them into my router. My home network subnet is 192.168.0.50/32. Take a note of your home address since you will need it for your configuration.

First Boot

YaY! First boot and now I have both my devices up and running. But hey! Wait! I have no Monitor and due to some reason I have no direct access to my router to find the IP Address of the devices. Yes! Right! Too early for googling for a solution but I have no idea so I started to search and I figure out that I can use Address Resolution Protocol aka arp to see the IP Addresses of my local network:

$ arp -a

Running above told me that following IPs are my Raspberry Pi:

192.168.0.50
192.168.0.51

Now I can SSH into my machines and do the initial setup like a normal Ubuntu Server.

Installing Docker

Docker installation is straight forward and you need to run following:

$ sudo apt install docker.io
$ sudo usermod -a -G docker $USER

Enabling CGroup Memory

One of the biggest challenge for me was to enabling CGroup Memory in Ubuntu image of Raspberry Pi. Unlike other normal places for enabling it, in Raspberry Pi Ubuntu’s image you need to append /boot/firmware/nobtcmd.txt and add following:

ipv6.disable=1 cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1

Note :

I have disabled IPv6 as well to make the setup easier.

Ensure iptables tooling does not use the nftables backend

First, enable iptables Filtering on Bridge Devices:

sudo sysctl net.bridge.bridge-nf-call-iptables=1

Apart from that, as per mentioned in documentation of kubernetes , you need to switch the iptables tooling to ‘legacy’ mode to avoid duplicated firewall rules and breaks kube-proxy.

sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy

Installing Kubernetes

First lets add the Kubernetes package:

$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
$ sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y

Now is the time to install kubeadm:

$ sudo apt-get install -y kubeadm

Configuring kubeadm

First of all we have to pull image k8s.gcr.io/kube-apiserver:v1.16.3 :

$ kubeadm config images pull

When we need to configure second node?

You have to run all above steps on both Raspberry Pi.

Node 1:

When you initializes a Kubernetes control-plane node, you need to consider assigning a cidr for your pod by --pod-network-cidr. It specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.

$ sudo kubeadm init --pod-network-cidr=10.244.0.0/16

Note :

Do not use your home’s subnet for --pod-network-cidr because later on when you setting up a flannel, you will end up with having a conflict between flannel IP Range and your home’s network. In my case, I initially used home subnet and then I ended up with having flannel IP Address as 192.168.0.1 which is my Router IP Address. It seems flannel is taking a control of the network by picking the first IP Address from the range.

Once you finish initializing your first node you will get instructed to configure $HOME/.kube/config as a regular user:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

As well as you will get the message like following to see how you can join this cluster:

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.50:6443 --token <token here> --discovery-token-ca-cert-hash sha256:<hash value here>

But hey! Wait we have one more step to do.

Deploy a pod network to the cluster.

As you can see in the message after initialization, You should now deploy a pod network to the cluster:

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Above gives you more options but I have already selected Flannel . Thus, I applied it by following:

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Above deployment will be deploy network configuration pods into your kube-system namespace and if you run following you will se the pods:

$ kubectl --namespace=kube-system get pods -o wide

Node 2:

Now it’s time to join the first node! You just need to copy and paste from the note after initialization on first node:

$ kubeadm join 192.168.0.50:6443 --token <token here> --discovery-token-ca-cert-hash sha256:<hash value here>

Now if you run following in your first node you will se the details of your both nodes:

kubectl get nodes -o wide

Hope this blog helps you to configure your cluster easier. I will try to write more.

Allow/Deny Access to Certain IP Address in Tomcat

Tomcat is one of the most powerful webserver for implementing Java applications. As an administrator, you may need to restrict access to your webserver and practically you have got lots of options to achieve this requirement. However, One of the easiest ways to allow or deny access to Tomcat is via Remote Address filter of Tomcat valves component. You can achieve it by adding following component to your server.xml file:

<Valve className="org.apache.catalina.valves.RemoteAddrValve" Attribute" />

According to Apache Tomcat documentation, Remote Address has following attributes:

  • allow
  • deny
  • denyStatus
  • addConnectorPort
  • invalidAuthenticationWhenDeny
Allow Certain IP Addresses:

For example, if you want to allow IP x.x.x.x , y.y.y.y and subnet of z.z.z.* you need to add following:

<Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="x.x.x.x,y.y.y.y,z.z.z.*" />

Above configuration forces Tomcat to allow clients with these specific IP addresses only.

Allow localhost to access via default port while other addresses are accessible via 1234:
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
   addConnectorPort="true"
   allow="127\.\d+\.\d+\.\d+;\d*|::1;\d*|0:0:0:0:0:0:0:1;\d*|.*;1234"/>

With above configuration, localhost (127.0.0.1) is able to access tomcat via default connector port while all other users are accessing tomcat via port 1234.

In this example, addConnectorPort configured as true and it means, tomcat compares allowed IP addresses against IP:PORT where IP is the client IP and PORT is the Tomcat connector port.

Note:

  • If you are using DHCP, this configuration may not be suitable for you since your IP Address may change via DHCP.
  • If your application is integrated with other applications, you need to ensure that IP Address of those applications is listed in Tomcat. Otherwise, they are not able to communicate.
  • You are able to add Regex instead of individually adding IP Addresses

TDA and Common Error for Thread Dumps with Java 8

The external thread dumps are the best option if you want to investigate performance degradation of Java applications. Almost all geeks out there who are dealing with Thread Dumps are familiar with awesome tools called TDA. However, once in a while when you are generating your thread dumps for applications with Java 8, you will encounter to following error in console:

Exception in thread "AWT-EventQueue-0" java.lang.NumberFormatException: For input string: "5 os_prio=0"
  at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  at java.lang.Long.parseLong(Long.java:589)
  at java.lang.Long.<init>(Long.java:965)
  at com.pironet.tda.utils.ThreadsTableModel.getValueAt(ThreadsTableModel.java:80)
  at com.pironet.tda.utils.TableSorter.getValueAt(TableSorter.java:285)
 ....

TDA team investigated this issue and they belive it’s related to fact that an os_prio culmn exist between prio and tid and they fixed it here. However, if you are not willing to apply the fix or changing your TDA version, you can clean up your Thread dumps with following regex:

For Linux:

sed -i 's/prio=[0-9]\{1,2\} os_prio=-\?[0-9]\{1,2\}/prio=5/g' <filename>

For Mac:

sed -i '' -E 's/prio=[0-9]{1,2} os_prio=-?[0-9]{1,2}/prio=5/g' <filename>

Credit of above solution goes to Atlassian Team (here).

TDA is available in GitHub

Basic of Regex

The first topic that I’ve chosen to export it from my mind is Regex. While Regex is handy and you can pick it up quickly, it causes tons of frustration. You can use it everywhere when you want to search for a specific part of your file, no matter if it’s a Sublime Text Editor or a grep command in Linux.

In this article, I show some basic examples of Regex with grep in a Linux Terminal. For those who may not know what’s grep for: grep is a single command in Linux for searching a text data with a specific patterns and it uses as following:

grep [options] PATTERN [file]

If you want to know available options of grep, you can easily go through the grep –help (in Ubuntu) since it contains all options that you have for using this command. Anyway, let’s go back to the topic:

We use Regext for finding the pattern that we want. However, Regex uses special expressions in combination with any of the following:

Literal

Any character used in a search or matching expression

Metachracter

One or more special characters with special meaning

Escape Sequence

Use of metachracters as a literal

Most difficult part of using Regex is how to build a pattern for finding what we want! Following are some basic options of Regex:

[ ]     Matches anything inside brackets. It can be either number or letters but.
        Be aware that, it's case sensitive.

-    This option creates a range. For example, 1-9 means between 1 to 9 OR a-z means between a to z.
    Be aware that, it's case sensitive and you shouldn't use space between.

.    Find any character in its position.

*    Matches any character zero or more times

^    Negates a search when used inside brackets.
    The caret is used outside brackets to find only lines that begin with a given string.

( )    Combine multiple patterns.

|    Find left or right values. Mostly used with () option.

$    Find line base on their ending character. It's Similar to caret outside brackets.

Well, we’re the generation of UI so it’s mind blowing to go through all of them without any example. Let’s assume that we have following text file:

Here is my sample text file
This text file contains 10 lines

Let's search for my name "saleh"
Love my wife
love my life

I born in 1987
November is the best month

1987 + 11 + 18 = 2016

WoW, my birthday equation is the year that I posted this!

Now we want to find:

All lines that start with either L or l

saleh@Saleh:~/Personal Project$ grep ^[Ll]. saleh.txt 

Result:

Let's search for my name "saleh"
Love my wife
love my life

Find saleh:

saleh@Saleh:~/Personal Project$ grep saleh saleh.txt

Result:

Let's search for my name "saleh"

Find all Lines start with L

saleh@Saleh:~/Personal Project$ grep ^L saleh.txt 

Result:

Let's search for my name "saleh"
Love my wife

Hope she visits this site and see this

Find Numbers Only

saleh@Saleh:~/Personal Project$ grep -v [A-Za-z] saleh.txt 

Result:

1987 + 11 + 18 = 2016

-v prints non-matching lines and it’s one of the grep options

Find any lines that has number

saleh@Saleh:~/Personal Project$ grep [0-9] saleh.txt 

Result:

This text file contains 10 lines
I born on 1987
1987 + 11 + 18 = 2016

You can always use the combination of above for building your search patterns. However, don’t forget that prior to going through the patterns you need to know what you want to find. Hope it helps.