Merge pull request #59767 from sjenning/check-phase-transition

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubelet: check for illegal phase transition

I have been unable to root cause the transition from `Failed` phase to `Succeeded`.  I have been unable to recreate as well.  However, our CI in Origin, where we have controllers that look for these transitions and rely on the phase transition rules being respected, obviously indicate that the illegal transition does occur.

This PR will prevent the illegal phase transition from propagating into the kubelet caches or the API server.

Fixes https://github.com/kubernetes/kubernetes/issues/58711

xref https://github.com/openshift/origin/issues/17595

@dashpole @yujuhong @derekwaynecarr @smarterclayton @tnozicka
This commit is contained in:
Kubernetes Submit Queue 2018-02-12 15:13:21 -08:00 committed by GitHub
commit 4efbb71bf6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -1359,6 +1359,15 @@ func (kl *Kubelet) generateAPIPodStatus(pod *v1.Pod, podStatus *kubecontainer.Po
spec := &pod.Spec
allStatus := append(append([]v1.ContainerStatus{}, s.ContainerStatuses...), s.InitContainerStatuses...)
s.Phase = getPhase(spec, allStatus)
// Check for illegal phase transition
if pod.Status.Phase == v1.PodFailed || pod.Status.Phase == v1.PodSucceeded {
// API server shows terminal phase; transitions are not allowed
if s.Phase != pod.Status.Phase {
glog.Errorf("Pod attempted illegal phase transition from %s to %s: %v", pod.Status.Phase, s.Phase, s)
// Force back to phase from the API server
s.Phase = pod.Status.Phase
}
}
kl.probeManager.UpdatePodStatus(pod.UID, s)
s.Conditions = append(s.Conditions, status.GeneratePodInitializedCondition(spec, s.InitContainerStatuses, s.Phase))
s.Conditions = append(s.Conditions, status.GeneratePodReadyCondition(spec, s.ContainerStatuses, s.Phase))