The controller would recover eventually because it double-checks when seeing
the pod status update, but when reacting to events it recovers faster.
Kubernetes-commit: 03c34ede467ea16580beee3937f9997070421529
When updating a ResourceSlice, a more recent copy than in the underlying store
gets cached by the MutationCache. Then when that ResourceSlice gets deleted on
the apiserver and then the store, the MutationCache kept the stale copy and
returned it when the controller synced again, causing the controller to
erroneously not recreate the deleted ResourceSlice.
Depending on timing, this could have happened while updating the DRA driver:
- Driver restarts, updating the existing ResourceSlices.
- kubelet deletes the slices before realizing that the driver
is running again.
The fix is to address a long-standing TODO in the MutationCache: it needs to
react to changes in the store. This allows it to drop cached objects sooner and
let's it remove cached objects that are known to be deleted. Users of a
MutationCache must inform the cache in their own informer event
handlers *before* triggering sync operations.
The integration test demonstrates the problem when run against master:
core.go:560: I0629 09:43:22.656382] Deleting slice...
resourceslicecontroller.go:610: I0629 09:43:22.659446] ResourceSlice delete slice="00000-ga-publishresourceslices-5s47j-slice"
resourceslicecontroller.go:612: I0629 09:43:22.659496] Scheduled sync poolName="global" at="2026-06-29 09:43:25.659481942 +0200 CEST m=+5.886096424"
resourceslicecontroller.go:755: I0629 09:43:24.639116] Matched existing slice by index poolName="global" slice="00000-ga-publishresourceslices-5s47j-slice" matchIndex=0
resourceslicecontroller.go:788: I0629 09:43:24.639317] Completed comparison poolName="global" numObsolete=0 numMatchedSlices=1 numChangedMatchedSlices=0 numNewSlices=0
resourceslicecontroller.go:804: I0629 09:43:24.639391] Kept generation because at most one update API call is necessary poolName="global" generation=1
core.go:566: FATAL ERROR: I0629 09:43:30.661588]
Timed out after 8.002s.
Expected
<resourceslice.Stats>: {NumCreates: 0, NumUpdates: 1, NumDeletes: 0}
to equal
<resourceslice.Stats>: {NumCreates: 1, NumUpdates: 1, NumDeletes: 0}
The "matched existing slice" doesn't actually exist anymore, so the controller
does nothing and the test fails.
Kubernetes-commit: 495f81169d8b2fe9e6e5dafd6b31fca1ae0483fa
The context is used for cancellation and to support contextual logging.
In most cases, alternative *WithContext APIs get added, except for
NewIntegerResourceVersionMutationCache where code searches indicate that the
API is not used downstream.
An API break around SharedInformer couldn't be avoided because the
alternative (keeping the interface unchanged and adding a second one with
the new method) would have been worse. controller-runtime needs to be updated
because it implements that interface in a test package. Downstream consumers of
controller-runtime will work unless they use those test package.
Converting Kubernetes to use the other new alternatives will follow. In the
meantime, usage of the new alternatives cannot be enforced via logcheck
yet (see https://github.com/kubernetes/kubernetes/issues/126379 for the
process).
Passing context through and checking it for cancellation is tricky for event
handlers. A better approach is to map the context cancellation to the normal
removal of an event handler via a helper goroutine. Thanks to the new
HandleErrorWithLogr and HandleCrashWithLogr, remembering the logger is
sufficient for handling problems at runtime.
Kubernetes-commit: 4638ba971661497b147906b8977ae206c9dd6e44
2. Remove staging/src/k8s.io/client-go/tools/cache from .golint_failures;
3. Fix some typo from comments.
Kubernetes-commit: 0e0e1f7daba0a6ae6dd59df0a1bb643c323ad8cb
- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
* github.com/kubernetes/repo-infra
* k8s.io/gengo/
* k8s.io/kube-openapi/
* github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods
Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135
Kubernetes-commit: 954996e231074dc7429f7be1256a579bedd8344c
