Kubernetes Upgrade Process: Step-by-Step Guide

  1. Check the current version of node
kubectl get nodes -o wide

2. Check the Kubernetes API Server Version

kubectl version

3. Verify pods are running and distributed across nodes

kubectl get pods -o wide

4. Check pod distribution

kubectl get pdb

5. SSH to contact the control panel

ssh <user>@<control-plane-ip>
or
ssh -i key.pem <user>@<control-plane-ip>

6. Verify etcd Pod

kubectl get pods -n kube-system | grep etcd

7. Backup etcd database before upgrade

export ETCDCTL_API=3
etcdctl snapshot save /opt/backup/etcd-snapshot.db \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key

8. Verify the status backup

etcdctl snapshot status /opt/backup/etcd-snapshot.db

9. Check current kubeadm version

kubeadm version

10. update package list

sudo apt-get update

11. Check available kubeadm versions

apt-cache madison kubeadm

12. Add kubernetes 1.34 versions

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /" \
| sudo tee /etc/apt/sources.list.d/kubernetes.list

13. update package list

sudo apt-get update

14. Check available kubeadm version again

apt-cache madison kubeadm

15. Allow kubeadm to be upgraded

sudo apt-mark unhold kubeadm

16. Install kubeadm 1.34.2

sudo apt-get install -y kubeadm=1.34.2-1.1

17. verify the kubeadm version

kubeadm version

18. Prevent kubeadm from autoupgrade

sudo apt-mark hold kubeadm

19. Preview upgrade plan

sudo kubeadm upgrade plan

20. Verify cluster still on old version

kubectl get nodes

21. Upgrade the control panel

sudo kubeadm upgrade apply v1.34.2

22. Verify pod still running

kubectl get pods

23. Check nodes still on 1.33

kubectl get nodes

24. Check api server version

kubectl version

25. Allow the kubelet and kubectl to be upgraded

sudo apt-mark unhold kubelet kubectl

26. Install the new version

sudo apt-get install -y kubelet=1.34.2-1.1 kubectl=1.34.2-1.1

27. Again stop the autoupgrade configuration

sudo apt-mark hold kubelet kubectl

28. reload system manager configuration

sudo systemctl daemon-reload

29. restart kubelet with new version

sudo systemctl restart kubelet

30. Get the nodes again: now you see control panel is upgraded

kubectl get nodes

31. Verify the pods

kubectl get pods 0n kube-system

32. Check pod distribution with wide command

kubectl get pods -o wide
kubectl get pdb

Upgrade the Node1 present in cluster

33. Mark node 1 as unschedulable

kubectl cordon node1

34. verify scheduling disable

kubectl get nodes

35. Drain node1

kubectl drain node1 --ignore-daemonsets --delete-emptydir-data --timeout=120s

36. Verify pod move to another nodes

kubectl get pods -o wide

37. Test app still working during maintenance by port forwarding

kubectl port-forward nigniz 8080:90

38. SSH to the node1

ssh -i key.pem username@ipaddressofnode1

39. updrade the node1

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /" \
| sudo tee /etc/apt/sources.list.d/kubernetes.list

40. update package list

sudo apt-get update

41. Allow kubeadm to update

sudo apt-mark unhold kubeadm

42. Install new version of kubeadm

sudo apt-get install -y kubeadm=1.34.2-1.1

43. Disable auto upgrade for kubeadm

sudo apt-mark hold kubeadm

44. Upgrade worker node configuration

sudo kubeadm upgrade node

45. Allow kubectl and kubelet to upgrade

sudo apt-mark unhold kubelet kubectl

46. Install kubectl and kubelet latest version

sudo apt-get install -y kubectl=1.34.2-1.1 kubelet=1.34.2-1.1

47. disable autoupgrade for both

sudo apt-mark hold kubelet kubectl

48. Reload the systemd manager configuration

sudo systemctl daemon-reload

49. Restart kubelet with new version

sudo systemctl restart kubelet

50. Get nodes and verify its upgrade to new version

kubectl get nodes

51. Mark node1 as schedulable again

kubeclt uncordon node1

Do the node upgarde process for all nodes present in your cluster

Verify all nodes are upgrade

kubeclt get nodes -o wide