* First version of batching w/out signatures.
* First version of pod signatures.
* Integrate batching with signatures.
* Fix merge conflicts.
* Fixes from self-review.
* Test fixes.
* Fix a bug that limited batches to size 2
Also add some new high-level logging and
simplify the pod affinity signature.
* Re-enable batching on perf tests for now.
* fwk.NewStatus(fwk.Success)
* Review feedback.
* Review feedback.
* Comment fix.
* Two plugin specific unit tests.:
* Add cycle state to the sign call, apply to topo spread.
Also add unit tests for several plugi signature
calls.
* Review feedback.
* Switch to distinct stats for hint and store calls.
* Switch signature from string to []byte
* Revert cyclestate in signs. Update node affinity.
Node affinity now sorts all of the various
nested arrays in the structure. CycleState no
longer in signature; revert to signing fewer
cases for pod spread.
* hack/update-vendor.sh
* Disable signatures when extenders are configured.
* Update pkg/scheduler/framework/runtime/batch.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Update staging/src/k8s.io/kube-scheduler/framework/interface.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Review feedback.
* Disable node resource signatures when extended DRA enabled.
* Review feedback.
* Update pkg/scheduler/framework/plugins/imagelocality/image_locality.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Update pkg/scheduler/framework/interface.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Update pkg/scheduler/framework/plugins/nodedeclaredfeatures/nodedeclaredfeatures.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Update pkg/scheduler/framework/runtime/batch.go
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
* Review feedback.
* Fixes for review suggestions.
* Add integration tests.
* Linter fixes, test fix.
* Whitespace fix.
* Remove broken test.
* Unschedulable test.
* Remove go.mod changes.
---------
Co-authored-by: Maciej Skoczeń <87243939+macsko@users.noreply.github.com>
Currently volume and dynamic-resource plugin shares an AssumeCache
implementation. However, they features significantly different use case. DRA
call Assume() on objects returned by APIServer, but volume call Assume() on
objects yet to be sent to the APIServer.
VolumeBinding plugin only makes one update request, while DynamicResource makes
2 requests (add finalizer then update allocation status). Taking advantage of
this, currently the volume cache is simpler:
1. Reserve: assume PV/PVC will be updated
2. PreBind: really send the update request
3. AssumeCache receives an update from informer and overwrite the assumed state.
a. if Prebind succeeded, this will surely include the update from step 2.
b. if PreBind is not finished yet, and this is an irrelevant update, it is safe to
overwrite the assumed state, because our update in PreBind will surely fail with Conflict.
While for DynamicResource:
1. Reserve: add devices to inFlightAllocations
2. PreBind:
a. send the 2 update requests
b. add the returned object into AssumeCache
c. AssumeCache dispatch events synchronously to update allocatedDevices
d. remove devices from inFlightAllocations
DynamicResource needs some features from AssumeCache that is not necessary for VolumeBinding:
1. DynamicResource needs strictly ordered update events to update allocatedDevices,
including those from Assume() and Restore()
2. DynamicResource needs to compare ResourceVersion to prevent the assumed state from being
overwritten by older version from informer. While this works, the doc[1] says:
"you must not compare resource versions for greater-than or less-than relationships".
Given so much difference, it can be beneficial to fork another simpler
AssumeCache for VolumeBinding plugin. Because of no need to send events, the
lite AssumeCache is a passive component. It only record the assumed version
without copying all objects from informer into its local cache. When reading,
we read from both informer and local cache. So it will always be up-to-date
with informer, no need to wait for event handler.
This resolves a race condition where AssumeCache and scheduler queue both
receive events from informer. When a pod is scheduling due to PV update event,
the PVCache may be not updated yet because it has not processed the relevant
event.
The passive version still listens events from informer, but only for cleaning up
its local cache to save memory.
[1]: https://kubernetes.io/docs/reference/using-api/api-concepts/#resource-versions
* Move ClusterEvent type to staging repo, leaving some functions (that contain logic internal to scheduler) in kubernetes/kubernetes
apply review comment and fix linter warning
* update-vendor.sh
* update doc comments
* run update-vendor.sh
The tests and comments have also been updated because while
VolumeCapacityPriority preferred a node with the least allocatable,
StorageCapacityScoring preferred a node with the maximum allocatable.
fix if condition
add test
add log
eliminate unnecessary args from log
fix Queue condition
check original pod status
fix return value when can scheduleable
fix tweak
fix testcase
There's no reason for having the interface because there is only one
implementation. Makes the implementation of the test functions a bit
simpler (no casting). They are still stand-alone functions instead of methods
because they should not be considered part of the "normal" API.
This is now used by both the volumebinding and dynamicresources plugin, so
promoting it to a common helper package is better.
In terms of functionality, nothing was changed. Documentation got
updated (warns about storing locally modified objects, clarifies what the Get
parameters are). Code coverage should be a bit better than before (tested with
and without indexer, exercises event handlers, more error paths).
Checking for specific errors can now be done via errors.Is.
