Error: kubernetes pod stuck in terminating

Error: kubernetes pod stuck in terminating

Delete Pods

You can perform a graceful pod deletion with the following command:

kubectl delete pods <pod>

Graceful deletion is safe and will ensure that the Pod shuts down gracefully before the kubelet deletes the name from the apiserver. But there are situation when we know that all the other related components are deleted but its entry has not been deleted from the apiserver, in that case we need to delete them forcefully.

Kubernetes (versions 1.5 or newer) will not delete Pods just because a Node is unreachable. The Pods running on an unreachable Node enter the ‘Terminating’ or ‘Unknown’ state after a timeout. Pods may also enter these states when the user attempts graceful deletion of a Pod on an unreachable Node. The only ways in which a Pod in such a state can be removed from the apiserver are as follows:

1. The Node object is deleted (either by you, or by the Node Controller).
2. The kubelet on the unresponsive Node starts responding, kills the Pod and removes the entry from the apiserver.
3. Force deletion of the Pod by the user.

Delete Forcefully:

Force deletions do not wait for confirmation from the kubelet that the Pod has been terminated. Irrespective of whether a force deletion is successful in killing a Pod, it will immediately free up the name from the apiserver.

 kubectl version >= 1.5

 kubectl delete pods <pod> --grace-period=0 --force

 kubectl <= 1.4, you should omit the --force option and use:

kubectl delete pods <pod> --grace-period=0

If even after these commands the pod is stuck on Unknown state, use the following command to remove the pod from the cluster:

kubectl patch pod <pod> -p '{"metadata":{"finalizers":null}}'

Example:

[root@monitoring kubeadm-ha]# kubectl delete pods grafana-5c5c46f797-8dzv5 -n monitoring --grace-period=0 --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "grafana-5c5c46f797-8dzv5" force deleted