Kubernetes Cluster Upgrade Process (kubeadm)
This document explains the end-to-end Kubernetes cluster upgrade process using kubeadm.
- Why Kubernetes Cluster Upgrade is Needed
Kubernetes clusters must be upgraded periodically to:
- Receive security patches
- Avoid deprecated APIs
- Maintain supportability
- Improve performance and stability
Kubernetes follows a strict version skew policy, so upgrades must be done one minor version at a time.
2. Important Concepts Before Upgrade
- Pods cannot be moved; they are recreated
- Control plane components run as static pods
- Worker nodes host application workloads
- cordon & drain are for workload movement, not version upgrade
3. Pre-Upgrade Checklist
- At least 2 worker nodes available
- Application replicas >= 2
- PodDisruptionBudgets configured
- Backup taken using Velero or etcd snapshot
Command:
kubectl get nodes
kubectl version — short
4. Control Plane Upgrade (NO cordon/drain)
The control plane does not run application pods, so cordoning or draining is not required.
Step 1: Check upgrade plan
Command:
sudo kubeadm upgrade plan
Purpose:
- Shows available Kubernetes versions
- Validates compatibility
Step 2: Upgrade control plane
Command:
sudo kubeadm upgrade apply v1.27.x
Purpose:
- Upgrades API server, scheduler, controller manager, etcd
- Workloads continue running
Step 3: Upgrade kubelet and kubectl on control plane
Command:
sudo apt-get update
sudo apt-get install -y kubelet=1.27.x-00 kubectl=1.27.x-00
sudo systemctl daemon-reload
sudo systemctl restart kubelet
5. Worker Node Upgrade (Rolling Upgrade)
Worker nodes run application workloads, so we must move pods safely.
Step 1: Cordon worker node
Command:
kubectl cordon node01
Purpose:
- Prevents new pods from scheduling on node
Step 2: Drain worker node
Command:
kubectl drain node01 — ignore-daemonsets — delete-emptydir-data
Purpose:
- Evicts existing pods gracefully
- Recreates pods on other nodes
- Maintains zero downtime
Explanation of flags:
— ignore-daemonsets : Skips DaemonSet pods (kube-proxy, CNI, logging)
— delete-emptydir-data : Allows deletion of temporary local storage
Step 3: Upgrade worker node
Command (on worker):
sudo kubeadm upgrade node
sudo apt-get install -y kubelet=1.27.x-00
sudo systemctl restart kubelet
Purpose:
- Updates node components to new version
Step 4: Uncordon worker node
Command:
kubectl uncordon node01
Purpose:
- Makes node schedulable again
Repeat steps for each worker node one by one.
6. Post-Upgrade Validation
Command:
kubectl get nodes
kubectl get pods -A
kubectl version — short
Purpose:
- Verify cluster health
- Ensure all nodes are upgraded
- Confirm workloads are running
7. Important Notes
- Pods do not move back automatically after uncordon
- Scheduler only acts on newly created pods
- DaemonSet pods recreate automatically
- Never skip Kubernetes minor versions
8. Interview Summary
“In kubeadm-based clusters, we upgrade the control plane first without draining it.
Then we perform rolling upgrades on worker nodes using cordon and drain to ensure zero downtime.”
