K8s primer - Part I

November 22, 2022

This primer is targeted at developers who mainly work on product features but may also need to understand the basics of Kubernetes at a practical level, so they know how their applications are deployed and managed.

Requirements: Docker and minikube.

Second part in K8s Primer - Part II

Let’s start

We want to deploy a Go application using Kubernetes (K8s). The application is a simple HTTP server that listens on port 7070 and responds to any request with “Hello world” followed by a random integer.

File ./main.go

package main

import (
	"fmt"
	"log"
	"math/rand"
	"net/http"
	"time"
)

func main() {
	rand.Seed(time.Now().UnixNano())
	id := rand.Int()

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "Hello world (ID: %d)\n", id)
	})

	fmt.Printf("Serving on http://localhost:7070 (ID: %d)\n", id)
	log.Fatal(http.ListenAndServe(":7070", nil))
}

We can run this HTTP server locally:

$ go build main.go && ./main # Or just: go run main.go
Serving on http://localhost:7070 (ID: 3628456987918824506)

And we can query the server using curl:

$ curl localhost:7070
Hello world (ID: 3628456987918824506)

So far, so good.

K8s runs containerized applications

We cannot just deploy our Go app by “handing over” the compiled binary main to K8s. K8s runs applications within containers. We need to containerize our Go app; I’ll use Docker for that:

File ./Dockerfile

FROM golang:1.19-alpine3.15 as build
WORKDIR /app
COPY main.go ./
RUN go build -o /app/HelloWorld main.go
EXPOSE 7070

FROM alpine:3.15
USER nobody:nobody
COPY --from=build /app/HelloWorld /HelloWorld

CMD ["/HelloWorld"]

The ./Dockerfile contains all the commands needed to build an image that will include our application binary. Let’s build the image:

$ docker build -t YOUR-HANDLE/hello-world-go:1.0.0 .

The image is now stored on our machine:

$ docker image ls
REPOSITORY                     TAG       IMAGE ID       CREATED       SIZE
YOUR-HANDLE/hello-world-go   1.0.0     a2fe58fdd36f   2 seconds ago    11.6MB

We can run our containerized Go app locally by doing:

$ docker run --rm -p 7070:7070 YOUR-HANDLE/hello-world-go:1.0.0

Serving on http://localhost:7070 (ID: 3888774625775762015)

And we can query our containerized Go app just like before: curl localhost:7070.

Let’s publish our Go application image

The image we just built is stored on our personal machine, but K8s doesn’t know how to retrieve the image from our file system. The standard approach is to make our container image available via a container registry. I’ll use Docker Hub to publish (for free) the Go app image:

$ docker login -u YOUR-HANDLE
$ docker push YOUR-HANDLE/hello-world-go:1.0.0

(docker login will ask for your personal access token to login in to Docker Hub.)

Now our Go app image is publicly available via Docker Hub.

We need some kind of machine/server/node where K8s can run

If we want to run applications on K8s, we first need to figure out where K8s can run. We need hardware. We can use a fully managed K8s service like GKE or EKS, or we can use minikube, which is open source, to run K8s locally on our machine. I’ll use minikube.

K8s has the notion of clusters. A cluster is a set of nodes. A node can be a physical or a virtual machine. We can run K8s itself and our application on nodes.

Let’s create a local K8s cluster of three (virtual) nodes:

$ minikube start --nodes 3 -p minikube-lab

Nice. We got three nodes. Let’s use kubectl, a CLI that lets us interact with K8s, to ask K8s about the status of our cluster:

$ kubectl get nodes

NAME               STATUS   ROLES           AGE    VERSION
minikube-lab       Ready    control-plane   1h    v1.25.2
minikube-lab-m02   Ready    <none>          58s   v1.25.2
minikube-lab-m03   Ready    <none>          58s   v1.25.2

K8s is telling us that we have three nodes, each with a unique name and ready for whatever we may need them for. One of the nodes works as the control plane of our cluster (where most of the K8s components run to manage the cluster), while the other two nodes are what K8s calls worker nodes (nodes on which our application can run).

Let’s deploy our Go app in our local K8s cluster

We have a cluster and a containerized application, but we can’t yet deploy our containerized application on the cluster. First, we need a Pod. A Pod can be thought of as a wrapper that allows a container to run on K8s. It’s true that we can run containerized apps on K8s, but K8s requires that every container runs inside a Pod.

Fig. 1 Go app within container within Pod.

Let’s create our Pod in a declarative way:

File ./k8s/pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: helloworld-app
  labels:
    app: helloworld-backend
spec:
  containers:
    - image: mypublicrepos/hello-world-go:1.0.0
      name: helloworld-container
      ports:
        - containerPort: 7070
          name: http
          protocol: TCP

This YAML file, also known as a Pod manifest, declares the desired state of our cluster. We are telling K8s:

Hey K8s, create one Pod named helloworld-app which contains a single container named helloworld-container whose image can be pulled from YOUR-HANDLE/hello-world-go:1.0.0 (from DockerHub) and that runs on port 7070

kind: identifies the type of object being declared. We are declaring a Pod
apiVersion: defines the schema version we’ll use when creating the Pod object. We are using the Pod schema v1
metadata.*: this section describes the Pod (its name, its labels, annotations, etc.). We identify Pods by their name and use labels to group Pods together
spec.*: this part describes the list of containers that will run within the Pod. We are declaring a single container, its image and its port

Now we are finally ready to deploy our app.

Let’s deploy a Pod

We can use kubectl (alias k) to submit our ./k8s/pod.yaml manifest to K8s. Behind the curtains, many things will happen, but what matters here is that K8s will find a healthy node in our cluster on which our Pod can run:

$ k apply -f k8s/pod.yaml
pod/helloworld-app created

We can check if the Pod is in our cluster, but you may want to wait a little before running the following command, as it may take some time to pull the container image:

$ k get pods -o wide
NAME             READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
helloworld-app   1/1     Running   0          2m11s   10.244.2.2   minikube-lab-m03   <none>           <none>

Our Pod is there. It’s running on the node minikube-lab-m03 (a worker node), and it even has its own IP address (10.244.2.2). Let’s get more details about our Pod:

$ k describe pod helloworld-app
Name:         helloworld-app
Namespace:    default
Priority:     0
Node:         minikube-lab-m03/192.168.49.4
Start Time:   Fri, 11 Nov 2022 20:41:23 +0100
Labels:       app=helloworld-backend
Annotations:  <none>
Status:       Running
IP:           10.244.2.2
IPs:
  IP:  10.244.2.2
Containers:
  helloworld-container:
    Container ID:   docker://284cf78c1abd23ba1229b493db35c605004f122e0f7e32222fc8340ea6855efe
    Image:          YOUR-HANDLE/hello-world-go:1.0.0
    Image ID:       docker-pullable://YOUR-HANDLE/hello-world-go@sha256:56345565aae8bd519084145e2990ebdeabba12
    Port:           7070/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 11 Nov 2022 20:41:23 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vh2vb (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-vh2vb:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  4m13s  default-scheduler  Successfully assigned default/helloworld-app to minikube-lab-m03
  Normal  Pulled     4m13s  kubelet            Container image "YOUR-HANDLE/hello-world-go:1.0.0" already present on machine
  Normal  Created    4m13s  kubelet            Created container helloworld-container
  Normal  Started    4m13s  kubelet            Started container helloworld-container

That’s a lot of information. Some notable points:

The namespace of the Pod is default. Namespaces provide a mechanism for isolating groups of resources within a single cluster. We didn’t specify any namespace in our Pod manifest, so our Pod was assigned the default one
Our Pod has one label (app=helloworld-backend), which is useful for filtering and grouping Pods, as we’ll see later
Our Pod is running one container named helloworld-container, which hasn’t suffered any restarts yet
There are some events attached to our Pod (container creation, image pulling, scheduling of the Pod to a node)

What can (or can’t) we do with our Pod?

can query its state (as seen before)
can delete it (k delete pod helloworld-app)
can recreate it (first delete, then create again)
can reach it by its IP address
cannot modify it (Pods are immutable)
cannot reach it from outside the K8s cluster

From our local machine, we cannot query the Go app running in the Pod. While the Pod has an IP address, the IP is only reachable from within the K8s cluster. Check this out:

$ curl 10.244.2.2:7070
curl: (28) Failed to connect to 10.244.2.2 port 7070: Operation timed out

Can’t reach the Pod from my local machine. Let’s SSH into one of the nodes of the K8s cluster and try querying our Go app from there:

$ minikube ssh -p minikube-lab -n minikube-lab-m02
Last login: Fri Nov 11 20:00:05 2022 from 192.168.49.1
docker@minikube-lab-m02:~$ curl 10.244.2.2:7070
Hello world (ID: 8356456912919624511)

Our Go app is reachable from within the cluster! But to reach our Pod we need to know its IP address, and as we’ll see later, that’s not ideal. Pods come and go, they may die unexpectedly. If Pods are ephemeral, so are their IP addresses. If we don’t want to keep track of which IP corresponds to which Pod, we’ll need to reach Pods in a different way (e.g., by name). But first, let’s deploy more stuff.

Let’s deploy two identical Pods

A simple way to (horizontally) scale our Go app would be to deploy many instances of it. What if we try to deploy the same Pod again?

$ k apply -f k8s/pod.yaml
pod/helloworld-app unchanged

We cannot deploy two identical Pods. We could write another manifest (e.g., ./k8s/pod2.yaml) where we give the Pod a different name (e.g., helloworld-app2) and try to deploy that:

$ git diff --no-index k8s/pod.yaml k8s/pod2.yaml

diff --git a/pod.yaml b/pod2.yaml
index d04ad55..9e15e0a 100644
--- a/pod.yaml
+++ b/pod2.yaml
@@ -1,7 +1,7 @@
 apiVersion: v1
 kind: Pod
 metadata:
-  name: helloworld-app
+  name: helloworld-app2
   labels:
     app: helloworld-backend
 spec:

$ k apply -f k8s/pod2.yaml
pod/helloworld-app2 created

$ k get pods -o wide
NAME              READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
helloworld-app    1/1     Running   0          75m     10.244.2.2   minikube-lab-m03   <none>           <none>
helloworld-app2   1/1     Running   0          1m25s   10.244.1.2   minikube-lab-m02   <none>           <none>

There it is. Now we have two instances of our Go app running in two different Pods. The second Pod has been scheduled to a different node (minikube-lab-m02) than the first Pod, but that won’t be always the case. Notice as well that our second Pod has a different IP address (10.244.1.2).

Let’s deploy two identical Pods (a better way)

You almost never deploy Pods using only a Pod manifest. K8s offers a better way to deploy one or more instances of your application (without having to manually create N different Pod manifest files): via the K8s Deployment object. A Deployment is a higher-level abstraction than a Pod, and it allows us to manage our application’s lifecycle with features such as scaling, zero-downtime deployments and rollbacks, self-healing, and more.

An application might consist of N identical Pods running on multiple Nodes, each with its own IP address and name. We don’t want to deal with each of them individually; we should focus on the bigger picture (we have one application running), not on internal details (the application is composed of N Pods).

Let’s create a Deployment manifest to deploy our Go application:

File ./k8s/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-deployment
  labels:
    app: helloworld-backend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: helloworld-backend
  template:
    metadata:
      labels:
        app: helloworld-backend
    spec:
      containers:
      - image: mypublicrepos/hello-world-go:1.0.0
        name: helloworld-container
        ports:
          - containerPort: 7070
            name: http
            protocol: TCP

With this manifest file, we’re telling K8s:

Hey K8s, create a Deployment named helloworld-deployment that creates two replicated Pods. The Pods should be identical (i.e., same container image and port).

Notice how the section under the spec.template.spec key in the manifest is exactly the same as what we had in our Pod manifest (./k8s/pod.yaml) under the spec key. We essentially embedding the Pod spec inside the Deployment spec. Let’s submit the Deployment manifest to K8s and see if our Pods are created:

$ k apply -f k8s/deployment.yaml
deployment.apps/helloworld-deployment created

$ k get pods -o wide
NAME                                     READY   STATUS    RESTARTS   AGE     IP           NODE               NOMINATED NODE   READINESS GATES
helloworld-deployment-79fcbf57bc-4zdvg   1/1     Running   0          13s     10.244.2.3   minikube-lab-m03   <none>           <none>
helloworld-deployment-79fcbf57bc-g8ksc   1/1     Running   0          13s     10.244.1.3   minikube-lab-m02   <none>           <none>

We have two pods. Each was scheduled to a different node and assigned a unique, cluster-wide IP address. Nice. At this point, we no longer need our ./k8s/pod.yaml and ./k8s/pod2.yaml manifest files, since ./k8s/deployment.yaml is sufficient. Let’s see what K8s has to say about our Deployment:

$ k get deployments -o wide
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS             IMAGES                               SELECTOR
helloworld-deployment   2/2     2            2           1m    helloworld-container   mypublicrepos/hello-world-go:1.0.0   app=helloworld-backend

Our Deployment is managing two Pods as expected. How does the Deployment know which Pods “belong” to it? Through the selector labels. In our Deployment manifest, the spec.selector defines the labels that Pods must have to be managed by the Deployment. The label spec.selector.matchLabels.app must match the label defined in the Pod template (spec.template.metadata.labels.app).

Fig. 2 Go app within container within Pods within Deployment.

Deployment’s niceties

Pods deployed via Pod manifests are also known as static Pods. If you delete a static Pod (e.g., k delete pod helloworld-app), the Pod is gone forever. The only way to scale static Pods is manually (as we saw, by creating a duplicated Pod manifest file). That’s why in real-world applications we typically don’t deploy Pods using Pod manifests, but rather through Deployments. Deployments offer all the benefits of being monitored and managed by K8s. Let’s examine this.

Run the following command in one terminal:

$ k get pods --watch
NAME                                     READY   STATUS    RESTARTS   AGE
helloworld-deployment-79fcbf57bc-8nmnh   1/1     Running   0          11s
helloworld-deployment-79fcbf57bc-ppcb8   1/1     Running   0          19m

The --watch flag allows us to monitor updates to the Pods managed by our Deployment. In a second terminal, let’s delete one of those Pods:

$ k delete pod helloworld-deployment-79fcbf57bc-ppcb8
pod "helloworld-deployment-79fcbf57bc-ppcb8" deleted

On the first terminal we see:

$ k get pods --watch
NAME                                     READY   STATUS              RESTARTS   AGE
helloworld-deployment-79fcbf57bc-8nmnh   1/1     Running             0          11s
helloworld-deployment-79fcbf57bc-ppcb8   1/1     Running             0          19m
helloworld-deployment-79fcbf57bc-ppcb8   1/1     Terminating         0          19m
helloworld-deployment-79fcbf57bc-mw8gg   0/1     Pending             0          0s
helloworld-deployment-79fcbf57bc-mw8gg   0/1     Pending             0          0s
helloworld-deployment-79fcbf57bc-mw8gg   0/1     ContainerCreating   0          0s
helloworld-deployment-79fcbf57bc-mw8gg   1/1     Running             0          1s
helloworld-deployment-79fcbf57bc-ppcb8   0/1     Terminating         0          19m
helloworld-deployment-79fcbf57bc-ppcb8   0/1     Terminating         0          19m
helloworld-deployment-79fcbf57bc-ppcb8   0/1     Terminating         0          19m

The Pod whose name ends in -ppcb8 is being terminated as requested but not before a new Pod (the one ending in -mw8gg) is created and becomes ready to serve. Why does this happen? Because in our Deployment manifest, we have specified spec.replicas: 2, so K8s continuously monitors the state of the cluster to ensure it matches the desired state described in the mmanifest. In the end, we’ll always have two Pods running (the remaining original one -8nmh and the newly created one -mw8gg):

$ k get pods
NAME                                     READY   STATUS    RESTARTS   AGE
helloworld-deployment-79fcbf57bc-8nmnh   1/1     Running   0          25s
helloworld-deployment-79fcbf57bc-mw8gg   1/1     Running   0          5s

Accessing our Go application

K8s Deployments allow us to manage multiple replicas of the same Pod. They also offer other niceties like self-healing (e.g., if a Pod, managed by a Deployment, dies, K8s automatically creates a new one to replace it), and auto-scaling (e.g., the number of Pods can increase or decrease automatically based on load). But we still can’t do the following:

refer to our application by name (instead of by IP address)
access our application from outside the K8s cluster

For the first point we need another K8s object, the Service. But, that’s enough for today. Let’s continue tomorrow in Part II. Let’s shutdown our local K8s cluster:

$ minikube stop -p minikube-lab
✋  Stopping node "minikube-lab"  ...
🛑  Powering off "minikube-lab" via SSH ...
✋  Stopping node "minikube-lab-m02"  ...
🛑  Powering off "minikube-lab-m02" via SSH ...
✋  Stopping node "minikube-lab-m03"  ...
🛑  Powering off "minikube-lab-m03" via SSH ...
🛑  3 nodes stopped.

Continues in K8s Primer - Part II