When a container restarts before kubelet restarts, containerMap has
multiple entries (old exited + new running). GetContainerID() may
return the exited container, causing the running check to fail. Fixed
by checking if ANY container for the pod/name is running.
Also filter terminal pods from podresources since they no longer
consume resources, and fix test error handling to avoid exiting
Eventually immediately on transient errors.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
When a pod has a sidecar (initContainer with restartPolicy: Always) with
a startupProbe, and one or more regular containers crash after a kubelet
restart, the kubelet fails to restart the regular containers. RestartCount
stays at 0 indefinitely.
When ChangeContainerStatusOnKubeletRestart is disabled (default in v1.35),
the prober worker skips seeding probe results for containers that predate
the kubelet restart. For a sidecar with a startupProbe this means
startupManager.Get() returns found=false permanently. In
computeInitContainerActions, the sidecar Running case breaks out early at
the !found check, leaving podHasInitialized=false. computePodActions then
returns early at the !hasInitialized guard without restarting the crashed
regular containers.
Fix: when the gate is off and a restartable init container's startup probe
is being seeded for the first time after a kubelet restart, check the
container's Started field in the pod status. If Started=true, the sidecar
had already passed startup before the restart, so seed the startup manager
with Success. This allows computeInitContainerActions to detect pod
initialization via the sidecar Running path without altering readiness or
liveness probe seeding behaviour.
Add and update tests to cover the fix:
- worker unit tests for sidecar startup/readiness/liveness restart behaviour
- e2e node regression test for sidecar with startupProbe across kubelet restart
Fixes: https://github.com/kubernetes/kubernetes/issues/136910
Update the procMount test expectations to match the intentional PSA
policy relaxation introduced in commit e8bd3f629d.
As of Kubernetes 1.35+, Pod Security Admission Baseline policy
allows UnmaskedProcMount for pods with user namespaces (hostUsers:
false). This was an intentional change to support nested container
use cases while maintaining security through user namespace isolation.
The test "will fail to unmask proc mounts if not privileged" was
written before this relaxation and expected Baseline level to reject
UnmaskedProcMount. Since Baseline now allows it (for user namespace
pods), the test needs to use Restricted level instead, which
unconditionally blocks UnmaskedProcMount regardless of user namespace
settings.
Changes:
- Change PSA level from Baseline to Restricted
- Update test name to clarify it's testing Restricted level behavior
- Update framework name from "proc-mount-baseline-test" to
"proc-mount-restricted-test"
Fixes the ci-crio-userns-e2e-serial test failure that started occurring
when runtimes began reporting user namespace support.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
This considerably impacts the ability to merge PRs right now because it fails
in the merge-blocking pull-kubernetes-node-e2e-containerd. Attempts to fix
it seem to be incomplete and need further
discussion (https://github.com/kubernetes/kubernetes/pull/135142),
so let's instead disable the test in the job.
The Kubelet's DRA manager was failing to report device health status in a pod's status for certain types of resource claims. The logic incorrectly assumed that the claim name used within the container's spec (`container.resources.claims[*].name`) was the same as the metadata name of the actual ResourceClaim object.
This assumption is false in two key scenarios:
1. When a claim is generated from a `ResourceClaimTemplate`, Kubernetes creates a `ResourceClaim` object with a randomized suffix in its name.
2. When a user defines a pre-existing claim in `pod.spec.resourceClaims`, they can provide a local name that differs from the actual `ResourceClaim` object's name.
In both cases, the code would fail to find the claim's information in its internal cache, resulting in the health status not being populated, as reported in issue #134482.
This fix corrects the logic by using `pod.Status.ResourceClaimStatuses` as the authoritative map to look up the actual, generated name of the `ResourceClaim` object. This ensures that both templated and renamed claims are resolved correctly before their health status is retrieved from the cache.
Additionally, this change introduces a new node e2e test that specifically covers templated and renamed claims to prevent future regressions.
This commit migrates the container manager package to use
contextual logging.
This follows the contextual logging migration pattern where:
- Logger is passed from the boundary down to implementations
- klog.TODO() is used at call sites where context is not yet available
- Functions with context.Context extract logger via klog.FromContext(ctx)
- klog.InfoS/ErrorS/V().InfoS calls are replaced with logger.Info/Error/V().Info
- Sub-managers receive proper logger context from their callers
Some call sites still use klog.TODO() as placeholders, with TODO
comments indicating these will be replaced with proper contextual loggers
as the migration continues upward through the call stack.
Test/fake implementations use klog.Background() which is the appropriate
choice for test code where no real context is available.
Mock file updates (pkg/kubelet/cm/testing/mocks.go) were done manually
due to mockery tool incompatibility with Go 1.25 and will be committed
separately with an explanation.
Key changes:
1. PodContainerManager and ContainerManager interface updates:
- PodContainerManager.EnsureExists: add logger parameter
- PodContainerManager.Destroy: add logger parameter
- PodContainerManager.ReduceCPULimits: add logger parameter
- PodContainerManager.SetPodCgroupConfig: add logger parameter
- ContainerManager.UpdateQOSCgroups: add logger parameter
- Update all implementations (Linux, Windows, stub, noop, fake)
2. Helper function updates:
- GetKubeletContainer: add logger parameter (Linux and unsupported platforms)
- Update cmd/kubelet/app/server.go to pass logger to GetKubeletContainer
- Comment out type assertion in helpers.go due to signature change
3. Cgroup manager contextual logging:
- CgroupManager interface methods updated to accept logger:
* Destroy: add logger parameter
* ReduceCPULimits: add logger parameter
* SetCgroupConfig: add logger parameter
- Update cgroupCommon and unsupportedCgroupManager implementations
- Migrate klog.InfoS/V().InfoS calls to logger.Info/V().Info
4. Container manager implementation updates:
- Extract logger from context in NewContainerManager() and Start()
- Pass logger to sub-managers (deviceManager, topologyManager)
- Update DRA manager initialization to use logger from context
- Migrate klog.InfoS/ErrorS to logger.Info/Error throughout
5. Call sites updated with TODO comments:
- pkg/kubelet/kubelet.go: UpdateQOSCgroups, EnsureExists
- pkg/kubelet/kubelet_pods.go: UpdateQOSCgroups, ReduceCPULimits, Destroy
- pkg/kubelet/kuberuntime/kuberuntime_manager.go: SetPodCgroupConfig
- pkg/kubelet/kuberuntime/kuberuntime_manager_test.go: test expectations
Most call sites currently use klog.TODO() as placeholders, with TODO
comments indicating these will be replaced with proper contextual loggers
as the migration continues upward through the call stack.
Test/fake implementations use klog.Background() which is the appropriate
choice for test code where no real context is available.
Mock file updates (pkg/kubelet/cm/testing/mocks.go) were done manually
due to mockery tool incompatibility with Go 1.25 and will be committed
separately with an explanation.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
With kubernetes/kubernetes#132028 merged, pods in terminal states are no longer
reported by the podresources API. The previous test logic accounted for the old
behavior where even failed pods appeared in the API response (tracked under k/k
issue #119423). As a result, we used to expect the failed test pod to be present in the
response but with an empty device set.
This change updates the test to reflect the new, correct behavior:
1. The failed test pod should no longer appear in the podresources API response.
2. The test now asserts absence of the failed pod rather than checking for an empty device assignment.
This simplifies the test logic and aligns expectations with the current upstream
behavior of the podresources API.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
