This change completes the contextual logging migration for the memory
manager by updating the Manager interface and all implementations to
accept logr.Logger parameters instead of context.Context.
Key changes:
- Update Manager interface methods to accept logr.Logger:
* AddContainer, RemoveContainer, GetMemoryNUMANodes
* GetAllocatableMemory, GetMemory (removed context entirely)
- Update Policy interface methods to accept logr.Logger instead of context.Context
- Pass logger to NewManager() and policy constructors (NewPolicyStatic, NewPolicyNone, NewPolicyBestEffort)
- Update internal_container_lifecycle to use klog.TODO() when calling memory manager methods
- Update fake manager to accept and use logger parameter
- Update all test code to pass logger instead of context
This aligns with the contextual logging migration pattern where:
- Functions that need a logger accept logr.Logger parameter directly
- Logger is passed from the boundary (e.g., Start()) down to implementation
- klog.TODO() is used temporarily in call sites where logger is not yet available
- Context is only used where truly needed (e.g., Start() method)
This follows the same pattern as recent migrations in:
- pkg/kubelet/cm/topologymanager (#134174)
- pkg/kubelet/cm/devicemanager (#134293)
- pkg/kubelet/cm/cpumanager (#125912)
Related to the initial memory manager contextual logging work in #130727.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
Updated allocateContainerResources to skip resources referring
extended resource claims as they are managed by DRA drivers.
These resources shouldn't be processed by device plugins.
migrate the cpumanager code to contextual logging
design musings:
- golang contexts "[...] carries deadlines, cancellation signals, and
other request-scoped values across API boundaries and between
processes." (see: https://pkg.go.dev/context#pkg-overview).
Thus, it generally makes sense to add contexts to most of the APIs
and functions as the resource management is done in the admission flow, when
ultimately we process a request and we have time bounds;
- however, receiving a context from the outside and depending on it
kind also conflicts with the goal of having uniform logging and "
- attach key/value pairs that get included in all log messages
- add names that describe which component or operation triggered a log messages"
(see:
https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/3077-contextual-logging#summary)
Some of this conflict is due to the transitional nature of the PRs
which we do when updating the codelet codebase. We can envision a
final state on which the top level caller adds identifying information
like pod UID and pod namespace/name pair; but up until all the calling
chain is updated and wired, the results are gonna be inconsistent.
- continuing from the point above, the most realistic transition plan
for the kubelet is outlined in
https://github.com/kubernetes/kubernetes/issues/130069 .
Since we do want to move in parallel and migrate on a
subsystem-by-subsystem basis, each subsystem will be in charge to add
the data they want to their contexts. I think this is the only
reasonnable way forward, and the cost will be overhead and redundancy.
We should do sweeping changes to uniform the flows once we completed
the full transition to ensure uniformity.
- there are some well known packages which want to operate by design
only in in-memory data structure, which arguably can't block.
Shoehorning a context feels not great. Instead, we intentionally pass
only the logger, which still is the core idea of contextual logging
(see:
https://github.com/kubernetes/kubernetes/pull/125912#discussion_r1935919144)
examples are the topology subpackage and the cpu_assignment logic
(takeByTopology and friends)
Signed-off-by: Francesco Romani <fromani@redhat.com>
unit tests for the policy options add test options to the
global state without proper cleanup; this is done only in tests
so it has limited room for doing damage, but still weakens
the test signal. Let's avoid that adding proper cleanup.
Signed-off-by: Francesco Romani <fromani@redhat.com>
Fixes TestConnectionHandling/no-wipe-on-reconnect which was failing with
"Only one usage of each socket address is normally permitted" because the
teardown function wasn't waiting for the gRPC server to fully shut down
before returning, causing socket cleanup issues in subsequent test runs.
The fix ensures proper synchronization during server shutdown so that the
Unix socket is fully released before the teardown completes. This prevents
socket binding conflicts when tests run in sequence.
When gRPC notifies the kubelet that a connection ended, the kubelet tries to
reconnect because it needs to know when a DRA driver comes back. The same code
gets called when a connection goes idle, by default after 30 minutes. In that
and only that case the conn.Connect call deadlocks while calling into the gRPC
idle manager.
This can be reproduced with a new unit test which artificially shortens the
idle timeout. This fix is to move the Connect call into a goroutine because
then both HandleConn and Connect can proceed. It's sufficient that Connect
finishes at some point, it doesn't need to be immediately.
DRA also calls Register at pkg/kubelet/cm/container_manager_linux.go NewContainerManager(), causing volume stats collector being ignored.
Fix this by moving it out of `sync.Once()`, allowing multiple calls to `Register()` func.
mockery has introduced breaking changes and switched to a v3 branch,
this migrates to that, mostly using the built-in migration tool. Mocks
are now generated in single files per package, except in packages
containing mocks for multiple interface packages (in
pkg/kubelet/container/testing).
Signed-off-by: Stephen Kitt <skitt@redhat.com>
When pod-level resources are detected, the cpu and memory manages
cannot engage because the feature is not yet compatible,
one of the main reasons being the managers only work at container level.
So, the managers has to detect if pod level resources are in use,
and turn themselves to no-operation skipping resource allocation
should that be the case.
We add an intentional loud log to inform the user, because
pods with pod-level resources landing on a node which cannot
actuate the desired spec is likely to be undesirable.
Signed-off-by: Francesco Romani <fromani@redhat.com>
This change introduces the ability for the Kubelet to monitor and report
the health of devices allocated via Dynamic Resource Allocation (DRA).
This addresses a key part of KEP-4680 by providing visibility into
device failures, which helps users and controllers diagnose pod failures.
The implementation includes:
- A new `v1alpha1.NodeHealth` gRPC service with a `WatchResources`
stream that DRA plugins can optionally implement.
- A health information cache within the Kubelet's DRA manager to track
the last known health of each device and handle plugin disconnections.
- An asynchronous update mechanism that triggers a pod sync when a
device's health changes.
- A new `allocatedResourcesStatus` field in `v1.ContainerStatus` to
expose the device health information to users via the Pod API.
Update vendor
KEP-4680: Fix lint, boilerplate, and codegen issues
Add another e2e test, add TODO for KEP4680 & update test infra helpers
Add Feature Gate e2e test
Fixing presubmits
Fix var names, feature gating, and nits
Fix DRA Health gRPC API according to review feedback
