The current behavior is to select only based on pod priority, where any
pod with higher priority can preempt any pod with lower priority.
In our case, we have multiple priority classes but where only a subset
of those are considered preemptible. Here's roughly how this looks:
- High priority: higher in the scheduling queue, not preemptible
- Low priority: lower in the scheduling queue, not preemptible
- Preemptible priority: lowest in the scheduling queue, preemptible
This PR allows the preemption selection to be configured against the
`DefaultPreemption` plugin, rather than needing to reimplement the
plugin itself. The structure used here mimics [what's currently being
done for the `Evaluator`](https://github.com/kubernetes/kubernetes/blob/release-1.32/pkg/scheduler/framework/preemption/preemption.go#L161-L165),
where a `PreemptPod` callback may be overridden to customize what
happens when performing a preemption.
This PR also updates the plugin's tests to call the updated `New()`
function rather than constructing `DefaultPreemption` directly, so
that `New()` is now being exercised in tests.
Currently, the NodeResourcesFit plugin always returns Unschedulable when a pod's
resource requests exceed a node's available resources. However, when a pod's
requests exceed the node's total allocatable, preemption cannot help since even
an empty node would not have enough resources.
This change modifies the NodeResourcesFit plugin to return UnschedulableAndUnresolvable
when a pod's resource requests exceed the node's total allocatable. This helps
optimize the scheduling process in large clusters by:
1. Reducing the number of candidate nodes that need to be considered for preemption
2. Providing clearer feedback about unresolvable resource constraints
3. Improving scheduling performance by avoiding unnecessary preemption calculations
The change is particularly beneficial in heterogeneous clusters where node sizes
vary significantly, as it helps quickly identify nodes that are fundamentally
too small for certain pods.
Fixes https://github.com/kubernetes/kubernetes/issues/131310
Co-authored-by: Kensei Nakada <handbomusic@gmail.com>
Thanks to the tracker, the plugin sees all taints directly in the device
definition and can compare it against the tolerations of a request while
trying to find a device for the request.
When the feature is turnedd off, taints are ignored during scheduling.
The controller is derived from the node taint eviction controller.
In contrast to that controller it tracks the UID of pods to prevent
deleting the wrong pod when it got replaced.
If there was an unexpected status, the code extracting the expected error
message crashed with a panic. Happened once so far, for unknown reasons
because the unexpected status then didn't get logged.
The tests and comments have also been updated because while
VolumeCapacityPriority preferred a node with the least allocatable,
StorageCapacityScoring preferred a node with the maximum allocatable.
This was previously caught during Filter by the allocator check. Doing it
sooner avoids wasting resources on a pod which ultimately cannot get scheduled.
While at it, be a bit more clear about which feature is disabled. The user
might not know that.
- Refactored `PreScore` method in `balanced_allocation.go` to skip
best-effort pods.
- Updated unit tests in `balanced_allocation_test.go` to check for
the new status codes.
