mirror of
https://github.com/k3s-io/kubernetes.git
synced 2025-07-19 18:02:01 +00:00
DRA API: bump maximum size of ReservedFor to 256
The original limit of 32 seemed sufficient for a single GPU on a node. But for shared non-local resources it is too low. For example, a ResourceClaim might be used to allocate an interconnect channel that connects all pods of a workload running on several different nodes, in which case the number of pods can be considerably larger. 256 is high enough for currently planned systems. If we need something even higher in the future, an alternative approach might be needed to avoid scalability problems. Normally, increasing such a limit would have to be done incrementally over two releases. In this case we decided on Slack (https://kubernetes.slack.com/archives/CJUQN3E4T/p1734593174791519) to make an exception and apply this change to current master for 1.33 and backport it to the next 1.32.x patch release for production usage. This breaks downgrades to a 1.32 release without this change if there are ResourceClaims with a number of consumers > 32 in ReservedFor. In practice, this breakage is very unlikely because there are no workloads yet which need so many consumers and such downgrades to a previous patch release are also unlikely. Downgrades to 1.31 already weren't supported when using DRA v1beta1.
This commit is contained in:
parent
117a48fcdf
commit
a5de75458e
4
api/openapi-spec/swagger.json
generated
4
api/openapi-spec/swagger.json
generated
@ -15123,7 +15123,7 @@
|
||||
"x-kubernetes-list-type": "map"
|
||||
},
|
||||
"reservedFor": {
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"items": {
|
||||
"$ref": "#/definitions/io.k8s.api.resource.v1alpha3.ResourceClaimConsumerReference"
|
||||
},
|
||||
@ -15956,7 +15956,7 @@
|
||||
"x-kubernetes-list-type": "map"
|
||||
},
|
||||
"reservedFor": {
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"items": {
|
||||
"$ref": "#/definitions/io.k8s.api.resource.v1beta1.ResourceClaimConsumerReference"
|
||||
},
|
||||
|
@ -847,7 +847,7 @@
|
||||
"x-kubernetes-list-type": "map"
|
||||
},
|
||||
"reservedFor": {
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"items": {
|
||||
"allOf": [
|
||||
{
|
||||
|
@ -869,7 +869,7 @@
|
||||
"x-kubernetes-list-type": "map"
|
||||
},
|
||||
"reservedFor": {
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"description": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"items": {
|
||||
"allOf": [
|
||||
{
|
||||
|
@ -689,7 +689,7 @@ type ResourceClaimStatus struct {
|
||||
// which issued it knows that it must put the pod back into the queue,
|
||||
// waiting for the ResourceClaim to become usable again.
|
||||
//
|
||||
// There can be at most 32 such reservations. This may get increased in
|
||||
// There can be at most 256 such reservations. This may get increased in
|
||||
// the future, but not reduced.
|
||||
//
|
||||
// +optional
|
||||
@ -717,9 +717,9 @@ type ResourceClaimStatus struct {
|
||||
Devices []AllocatedDeviceStatus
|
||||
}
|
||||
|
||||
// ReservedForMaxSize is the maximum number of entries in
|
||||
// ResourceClaimReservedForMaxSize is the maximum number of entries in
|
||||
// claim.status.reservedFor.
|
||||
const ResourceClaimReservedForMaxSize = 32
|
||||
const ResourceClaimReservedForMaxSize = 256
|
||||
|
||||
// ResourceClaimConsumerReference contains enough information to let you
|
||||
// locate the consumer of a ResourceClaim. The user must be a resource in the same
|
||||
|
4
pkg/generated/openapi/zz_generated.openapi.go
generated
4
pkg/generated/openapi/zz_generated.openapi.go
generated
@ -47405,7 +47405,7 @@ func schema_k8sio_api_resource_v1alpha3_ResourceClaimStatus(ref common.Reference
|
||||
},
|
||||
},
|
||||
SchemaProps: spec.SchemaProps{
|
||||
Description: "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
Description: "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
Type: []string{"array"},
|
||||
Items: &spec.SchemaOrArray{
|
||||
Schema: &spec.Schema{
|
||||
@ -48902,7 +48902,7 @@ func schema_k8sio_api_resource_v1beta1_ResourceClaimStatus(ref common.ReferenceC
|
||||
},
|
||||
},
|
||||
SchemaProps: spec.SchemaProps{
|
||||
Description: "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
Description: "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
Type: []string{"array"},
|
||||
Items: &spec.SchemaOrArray{
|
||||
Schema: &spec.Schema{
|
||||
|
@ -675,7 +675,7 @@ message ResourceClaimStatus {
|
||||
// which issued it knows that it must put the pod back into the queue,
|
||||
// waiting for the ResourceClaim to become usable again.
|
||||
//
|
||||
// There can be at most 32 such reservations. This may get increased in
|
||||
// There can be at most 256 such reservations. This may get increased in
|
||||
// the future, but not reduced.
|
||||
//
|
||||
// +optional
|
||||
|
@ -687,7 +687,7 @@ type ResourceClaimStatus struct {
|
||||
// which issued it knows that it must put the pod back into the queue,
|
||||
// waiting for the ResourceClaim to become usable again.
|
||||
//
|
||||
// There can be at most 32 such reservations. This may get increased in
|
||||
// There can be at most 256 such reservations. This may get increased in
|
||||
// the future, but not reduced.
|
||||
//
|
||||
// +optional
|
||||
@ -715,9 +715,9 @@ type ResourceClaimStatus struct {
|
||||
Devices []AllocatedDeviceStatus `json:"devices,omitempty" protobuf:"bytes,4,opt,name=devices"`
|
||||
}
|
||||
|
||||
// ReservedForMaxSize is the maximum number of entries in
|
||||
// ResourceClaimReservedForMaxSize is the maximum number of entries in
|
||||
// claim.status.reservedFor.
|
||||
const ResourceClaimReservedForMaxSize = 32
|
||||
const ResourceClaimReservedForMaxSize = 256
|
||||
|
||||
// ResourceClaimConsumerReference contains enough information to let you
|
||||
// locate the consumer of a ResourceClaim. The user must be a resource in the same
|
||||
|
@ -291,7 +291,7 @@ func (ResourceClaimSpec) SwaggerDoc() map[string]string {
|
||||
var map_ResourceClaimStatus = map[string]string{
|
||||
"": "ResourceClaimStatus tracks whether the resource has been allocated and what the result of that was.",
|
||||
"allocation": "Allocation is set once the claim has been allocated successfully.",
|
||||
"reservedFor": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"reservedFor": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"devices": "Devices contains the status of each device allocated for this claim, as reported by the driver. This can include driver-specific information. Entries are owned by their respective drivers.",
|
||||
}
|
||||
|
||||
|
@ -683,7 +683,7 @@ message ResourceClaimStatus {
|
||||
// which issued it knows that it must put the pod back into the queue,
|
||||
// waiting for the ResourceClaim to become usable again.
|
||||
//
|
||||
// There can be at most 32 such reservations. This may get increased in
|
||||
// There can be at most 256 such reservations. This may get increased in
|
||||
// the future, but not reduced.
|
||||
//
|
||||
// +optional
|
||||
|
@ -695,7 +695,7 @@ type ResourceClaimStatus struct {
|
||||
// which issued it knows that it must put the pod back into the queue,
|
||||
// waiting for the ResourceClaim to become usable again.
|
||||
//
|
||||
// There can be at most 32 such reservations. This may get increased in
|
||||
// There can be at most 256 such reservations. This may get increased in
|
||||
// the future, but not reduced.
|
||||
//
|
||||
// +optional
|
||||
@ -723,9 +723,9 @@ type ResourceClaimStatus struct {
|
||||
Devices []AllocatedDeviceStatus `json:"devices,omitempty" protobuf:"bytes,4,opt,name=devices"`
|
||||
}
|
||||
|
||||
// ReservedForMaxSize is the maximum number of entries in
|
||||
// ResourceClaimReservedForMaxSize is the maximum number of entries in
|
||||
// claim.status.reservedFor.
|
||||
const ResourceClaimReservedForMaxSize = 32
|
||||
const ResourceClaimReservedForMaxSize = 256
|
||||
|
||||
// ResourceClaimConsumerReference contains enough information to let you
|
||||
// locate the consumer of a ResourceClaim. The user must be a resource in the same
|
||||
|
@ -300,7 +300,7 @@ func (ResourceClaimSpec) SwaggerDoc() map[string]string {
|
||||
var map_ResourceClaimStatus = map[string]string{
|
||||
"": "ResourceClaimStatus tracks whether the resource has been allocated and what the result of that was.",
|
||||
"allocation": "Allocation is set once the claim has been allocated successfully.",
|
||||
"reservedFor": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 32 such reservations. This may get increased in the future, but not reduced.",
|
||||
"reservedFor": "ReservedFor indicates which entities are currently allowed to use the claim. A Pod which references a ResourceClaim which is not reserved for that Pod will not be started. A claim that is in use or might be in use because it has been reserved must not get deallocated.\n\nIn a cluster with multiple scheduler instances, two pods might get scheduled concurrently by different schedulers. When they reference the same ResourceClaim which already has reached its maximum number of consumers, only one pod can be scheduled.\n\nBoth schedulers try to add their pod to the claim.status.reservedFor field, but only the update that reaches the API server first gets stored. The other one fails with an error and the scheduler which issued it knows that it must put the pod back into the queue, waiting for the ResourceClaim to become usable again.\n\nThere can be at most 256 such reservations. This may get increased in the future, but not reduced.",
|
||||
"devices": "Devices contains the status of each device allocated for this claim, as reported by the driver. This can include driver-specific information. Entries are owned by their respective drivers.",
|
||||
}
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user