<p><a href="https://people.centos.org/pgreco/">Install CentOS</a> <a href="https://www.raspberrypi.org/software/">or Debian</a> on a Raspberry Pi. I'm using CentOS, but I'll admit that Debian is the easier option by far.</p>
<p>Do this on 3 separate Pi units, each with the same specs.</p>
<h2 id="set-hostnames">Set hostnames</h2>
<p>You must have unique hostnames for each Pi. Without unique hostnames, your cluster cannot function.</p>
<p>There are several "kinds" of hostnames, so to avoid confusion I change all of them.</p>
<p>I use a simple naming scheme: <code>k</code> for "kubernetes" + an integer, starting at 100 + <code>c</code> for "cluster":</p>
<pre class="bash"><code>$ sudo hostname k100c
$ sudo sysctl kernel.hostname=k100c
$ sudo hostnamectl set-hostname k100c
$ sudo reboot
</code></pre>
<p>Do this for each Pi. At a minimum, you end up with Pi computers named k100c, k101c, and k102c.</p>
<h2 id="set-verbose-prompts">Set verbose prompts</h2>
<p>When working with many different hosts, it's helpful to have a very verbose prompt as a constant reminder of which host you're connected to. Add this to the <code>~/.bashrc</code> of each Pi:</p>
<pre><code>export PS1='\[&#x005C;033[1;32m\]\! \d \t \h:\w \n% \[&#x005C;033[00m\]' 
</code></pre>
<h2 id="install-a-pi-finder-script">Install a Pi finder script</h2>
<p>Install an LED blinker so you can find a specific Pi when you need one. This brilliant script is by Chris Collins for his article <a href="https://opensource.com/article/20/6/find-raspberry-pi">Use this script to find a Raspberry Pi on your network</a>, which explains how to run it.</p>
<pre class="bash"><code>#!/bin/bash

set -o errexit
set -o nounset

trap quit INT TERM

COUNT=0
LED=&quot;/sys/class/leds/led0&quot;

if ! [ $(id -u) = 0 ]; then
   echo &quot;Must be run as root.&quot;
   exit 1
fi

if [[ ! -d $LED ]]
then
  echo &quot;Could not find an LED at ${LED}&quot;
  echo &quot;Perhaps try &#39;/sys/class/leds/ACT&#39;?&quot;
  exit 1
fi

function quit() {
  echo mmc0 &gt;&quot;${LED}/trigger&quot;
}

echo -n &quot;Blinking Raspberry Pi&#39;s LED - press CTRL-C to quit&quot;
echo none &gt;&quot;${LED}/trigger&quot;

while true
do
  let &quot;COUNT=COUNT+1&quot;
  if [[ $COUNT -lt 30 ]]
  then
    echo 1 &gt;&quot;${LED}/brightness&quot;
    sleep 1
    echo 0 &gt;&quot;${LED}/brightness&quot;
    sleep 1
  else
    quit
    break
  fi
done
</code></pre>
<h2 id="install-k3s-on-your-control-plane">Install K3s on your control plane</h2>
<p>K3s is Kubernetes for IoT and Edge computing. It's the easiest, cleanest, and most serious method of getting Kubernetes on an ARM device. You can try other solutions (Microk8s, Minikube, OXD, and so on), but the best support comes from k3s.</p>
<p>First, you must install k3s on one Pi. You can use any of your Pi units for this, but I use host k100c because it's the first in the sequence, so it feels logical.</p>
<pre class="bash"><code>[k100c]$ curl -sfL https://get.k3s.io -o install_k3s.sh
[k100c]$ chmod 700 install_k3s.sh
</code></pre>
<p>Read the script to ensure that it seems to do what you expect, and then:</p>
<pre class="bash"><code>[k100c]$ ./install_k3s.sh
</code></pre>
<p>After installation, you're prompted to add some arguments to your bootloader. Open <code>/boot/cmdline.txt</code> in a text editor and add <code>cgroup_memory=1 cgroup_enable=memory</code> to the end of it.</p>
<pre class="text"><code>console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p3 rootfstype=ext4 elevator=deadline rootwait cgroup_memory=1 cgroup_enable=memory
</code></pre>
<p>Reboot:</p>
<pre class="bash"><code>[k100c]$ sudo reboot
</code></pre>
<p>Once the Pi is back up, verify that your node is ready:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl get node
NAME    STATUS  ROLES                  AGE
k100c   Ready   control-plane,master   42s
</code></pre>
<p>This Pi is the "control plane", meaning it's the Pi that you use to administer your cluster.</p>
<h2 id="get-the-node-token">Get the node token</h2>
<p>Obtain the control plane's node token. Thanks to k3s, this is autogenerated for you. If you not using k3s, then you must generate your own with the command <code>kubeadm token generate</code>.</p>
<p>Assuming you're using k3s:</p>
<pre class="bash"><code>$ MYTOKEN=$(sudo cat /var/lib/rancher/k3s/server/node-token)
$ echo $MYTOKEN
K76351a1c2497d907ba7a156028567e0ccc26b82d2174161c564152ab3add6cc3fb::server:808771e4e695e3e3465ed9a14a0581da
</code></pre>
<h2 id="add-your-control-plane-hostname-to-your-hosts-file">Add your control plane hostname to your hosts file</h2>
<p>If you know how to manage local DNS settings, then you can use a DNS service to identify the hosts in your cluster. Otherwise, the easy way to make your nodes know how to find your control plane is to add the control plane's hostname and IP address to the <code>/etc/hosts</code> file on each node. This also assumes that your control plane has a static local IP address. For example, this is the host file of k101c and k102c:</p>
<pre class="text"><code>127.0.0.1  localhost.localdomain localhost
::1        localhost6.localdomain6 localhost6

10.0.1.100 k100c
</code></pre>
<p>Verify that each host can find the control plane. For example:</p>
<pre class="bash"><code>[k101c]$ ping -c 1 k100c || echo &quot;fail&quot;
[k101c]
</code></pre>
<h2 id="add-nodes-to-your-cluster">Add nodes to your cluster</h2>
<p>Now you can add the other Pi computers to your cluster. On each Pi you want to turn into a computer node, install k3s with the control plane and token as environment variables. On my second Pi, for instance, I run this command:</p>
<pre class="bash"><code>[k101c]$ curl -sfL https://get.k3s.io | K3S_URL=https://k100c:6443 K3S_TOKEN=&quot;${MYTOKEN}&quot; sh -
</code></pre>
<p>On my third and final Pi, I run the same command:</p>
<pre class="bash"><code>[k102c]$ curl -sfL https://get.k3s.io | K3S_URL=https://k100c:6443 K3S_TOKEN=&quot;${MYTOKEN}&quot; sh -
</code></pre>
<h2 id="verify-your-cluster">Verify your cluster</h2>
<p>On your control plane, verify that all nodes are active:</p>
<pre class="bash"><code>% k3s kubectl get nodes
NAME        STATUS     ROLES                  AGE     VERSION
k100c       Ready      control-plane,master   2d23h   v1.21.4+k3s1
k102c       Ready      &lt;none&gt;                 21h     v1.21.4+k3s1
k101c       Ready      &lt;none&gt;                 20h     v1.21.4+k3s1
</code></pre>
<p>It can take a few minutes for the control plane to discover all nodes, so wait a little while and try the command again if you don't see all nodes right away.</p>
<p>You now have a Kubernetes cluster running. It isn't doing anything yet, but it's a functional Kubernetes cluster. That means you have a tiny Pi-based cloud entirely at your disposal. You can use it to learn about Kubernetes, cloud architecture, cloud-native development, and so on.</p>
<h2 id="create-a-deployment-and-some-pods">Create a deployment and some pods</h2>
<p>Now that you have a Kubernetes cluster running, you can start running applications in containers. That's what Kubernetes does: it orchestrates and manages containers. You've may have heard of containers. I did an episode about Docker containers in episode 1522 of HPR, you can go listen to that if you need to catch up. I've also done an episode on LXC in episode 371 of my own show, GNU World Order.</p>
<p>There's a sequence to launching containers within Kubernetes, a specific order you need to follow, because there are lots of moving parts and those parts have to reference each other. Generally, the hierarchy is this:</p>
<ul>
<li>namespaces are the "project spaces" of kubernetes. I cover this in great detail in my GNU World Order episode 13x39.</li>
<li>create a deployment that manage pods.</li>
<li>pods are groups of containers. it helps your cluster scale on demand.</li>
<li>services are front-ends to deployments. A deployment can be running quietly in the background and it'll never see the light of day without a service pointing to it.</li>
<li>traffic, or exposure. A service is only available to your cluster until you expose it to the outside world with an external IP address.</li>
</ul>
<p>First, create a namespace for your test application to use.</p>
<pre class="bash"><code>[k100c]$ k3s kubectl create namespace ktest
</code></pre>
<p>The Kubernetes project provides an <a href="https://k8s.io/examples/application/deployment.yaml">example Nginx deployment definition</a>. Read through it to get an idea of what it does. It looks something like this:</p>
<pre class="yaml"><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
</code></pre>
<p>This creates metadata named <code>nginx-deployment</code>. It also creates a label called <code>app</code>, and sets it to <code>nginx</code>. This metadata is used as <em>selectors</em> for pods and services later.</p>
<p>For now, create a deployment using the example:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl --namespace ktest \
create -f https://k8s.io/examples/application/deployment.yaml
</code></pre>
<p>Confirm that the deployment has generated and started new pods:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl --namespace ktest get all
3s kubectl --namespace ktest get all
NAME                                  READY
pod/nginx-deployment-66b[...]   1/1   Running
pod/nginx-deployment-66b[...]   1/1   Running

NAME                               READY
deployment.apps/nginx-deployment   2/2

NAME
replicaset.apps/nginx-deployment-66b6c48dd5
</code></pre>
<p>See the pods labelled with <code>app: nginx</code>:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl --namespace ktest \
get pods -l app=nginx
NAME                                READY STATUS
nginx-deployment-66b6c48dd5-9vgg8   1/1   Running
nginx-deployment-66b6c48dd5-prgrf   1/1   Running
nginx-deployment-66b6c48dd5-cqpgf   1/1   Running
</code></pre>
<h2 id="create-a-service">Create a service</h2>
<p>Now you must connect the Nginx instance with a Kubernetes Service.</p>
<p>The <code>selector</code> element is set to <code>nginx</code> to match pods running the <code>nginx</code> application. Without this selector, there would be nothing to correlate your service with the pods running the application you want to serve.</p>
<pre class="bash"><code>[k100c]$ cat &lt;&lt; EOF | k3s kubectl \
--namespace ktest create -f -
apiVersion: v1
kind: Service
metadata:
  name: nginx-deployment
  labels:
    run: nginx-deployment
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    app: nginx
EOF
service/nginx-deployment created
</code></pre>
<p>Verify that the service exists:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl --namespace ktest get svc nginx-deployment
NAME       TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
nginx-deployment   ClusterIP   10.43.32.89   &lt;none&gt;        80/TCP    58s
</code></pre>
<p>A Service is backed by a group of Pods. Pods are exposed through endpoints. A Service uses POST actions to populate Endpoints objects named <code>nginx-deployment</code>. Should a Pod die, it's removed from the endpoints, but new Pods matching the same selector are added to the endpoints. This is how Kubernetes ensures your application's uptime.</p>
<p>To see more information:</p>
<pre class="bash"><code>[k100c]$ k3s kubectl \
--namespace ktest \
describe svc nginx-deployment
Name:              nginx-deployment
Namespace:         ktest
Labels:            run=nginx-deployment
Annotations:       &lt;none&gt;
Selector:          app=nginx
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.43.251.104
IPs:               10.43.251.104
Port:              &lt;unset&gt;  80/TCP
TargetPort:        80/TCP
Endpoints:         10.42.2.8:80,10.42.2.9:80,10.42.3.7:80
Session Affinity:  None
Events:            &lt;none&gt;
</code></pre>
<p>Notice that the <code>Endpoints</code> value is set to a series of IP addresses. This confirms that instances of Nginx are accessible. The IP of the service is set to 10.43.251.104, and it's running on port 80/TCP. That means you can log onto any of your nodes (referred to as "inside the cluster") to interact with your Nginx app. This does not work from your control plane, only from a node.</p>
<pre class="bash"><code>[k101c]$ curl https://10.43.251.104
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>Nginx is accessible.</p>
<p>The only thing left to do now is to route traffic from the outside world.</p>
<h2 id="exposing-a-deployment">Exposing a deployment</h2>
<p>For a deployed application to be visible outside your cluster, you need to route network traffic to it. There are many tools that provide that functionality.</p>
<p>Install metallb:</p>
<pre class="bash"><code>$ k3s kubectl apply \
-f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/namespace.yaml
$ k3s kubectl apply \
-f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/metallb.yaml
$ k3s kubectl create secret generic \
-n metallb-system memberlist \
--from-literal=secretkey=&quot;$(openssl rand -base64 128)&quot;
</code></pre>
<p>Determine what network range you want your cluster to use. This must not overlap with what your DHCP server is managing.</p>
<pre class="bash"><code>---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: address-pool-0
      protocol: layer2
      addresses:
      - 10.0.1.1/26
</code></pre>
<p>Save this as <code>metallb.yaml</code> and apply the configuration:</p>
<pre class="bash"><code>$ k3s kubectl apply -f metallb.yaml
</code></pre>
<p>You now have a configmap for metallb, and metallb is running.</p>
<p>Create a load balance service mapping your deployment's ports (port 80 in this case, which you can verify with <code>k3s kubectl -n ktest get all</code>). Save this as <code>loadbalance.yaml</code>:</p>
<pre class="bash"><code>---
apiVersion: v1
kind: Service
metadata:
  name: ktest-ext
  namespace: ktest
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer
</code></pre>
<p>This service selects any deployment in the <code>ktest</code> namespace with an <code>app</code> name of <code>nginx</code>, and maps the container's port 80 to a port 80 for an IP address within your address range (in my example, that's 10.0.1.1/26, or 10.0.1.1-10.0.1.62).</p>
<pre class="bash"><code>$k3s kubectl apply -f loadbalance.yaml
</code></pre>
<p>Find out what external IP address it got:</p>
<pre><code>$ k3s kubectl get service ktest-ext -n ktest
NAME        TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)
ktest   LoadBalancer   10.43.138.91   10.0.1.3                  80:31790/TCP
</code></pre>
<p>Open a web browser and navigate to the external IP address listed (in this example, 10.0.1.3).</p>
<p><img src="https://gnuworldorder.info/tank/nginx-welcome.webp" alt="Nginx" /></p>
