Kubernetes logs

This is a short entry and pretty similar to the certmanager post I wrote last month. By running simple commands as kubectl describe or kubectl logs sometimes we can get enough information to figure out the issue.

A common issue with new kubernetes cluster using kubeadm is that you need to add a network addon such as Flannel, Weave, Calico, Romana or Cillum in this case. For instance, you may need to define a specific POD_CIDR like Calico suggests.

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

There are two scenarios you may encounter:

  1. you forget to defined the specific POD_CIDR

  2. you define the right network without taking into account that your instances are already using the same range, overlapping issues acoming up.

All these examples and the one below can be address just reviewing logs. After installing Cillium in Ubuntu 16.04 I noticed the network wasn’t coming up, this was due to and old linux kernel version as the logs pointed me out.

$ k logs pod/cilium-57s6d -n kube-system

el=info msg="  --tofqdns-proxy-port='0'" subsys=daemon
level=info msg="  --tofqdns-proxy-response-max-delay='100ms'" subsys=daemon
level=info msg="  --trace-payloadlen='128'" subsys=daemon
level=info msg="  --tunnel='vxlan'" subsys=daemon
level=info msg="  --version='false'" subsys=daemon
level=info msg="  --write-cni-conf-when-ready=''" subsys=daemon
level=info msg="     _ _ _" subsys=daemon
level=info msg=" ___|_| |_|_ _ _____" subsys=daemon
level=info msg="|  _| | | | | |     |" subsys=daemon
level=info msg="|___|_|_|_|___|_|_|_|" subsys=daemon
level=info msg="Cilium 1.8.2 aa42034f0 2020-07-23T15:02:39-07:00 go version go1.14.6 linux/amd64" subsys=daemon
level=info msg="cilium-envoy  version: 0a9743dda269a0b0039c9db3cf7e0a637caad7a9/1.13.3/Modified/RELEASE/BoringSSL" subsys=daemon
level=fatal msg="kernel version: NOT OK: minimal supported kernel version is 4.8.0; kernel version that is running is: 4.4.0" subsys=linux-datapath

This was pretty simple to detect and solve, after upgrading the kernel to version 4.15 and rebooted, everything came up. Sometimes, in the worst case scneario, you just have to reinstall.