This reports and fixes for test/e2e:
ERROR: E2E suite initialization was faulty, these errors must be fixed:
ERROR: apimachinery/mutatingadmissionpolicy.go:184: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicy API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:184, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:606)
ERROR: apimachinery/mutatingadmissionpolicy.go:412: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicyBinding API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:412, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:834)
ERROR: common/node/pod_level_resources.go:250: full test name is not unique: "[sig-node] Pod Level Resources [Serial] [Feature:PodLevelResources] [FeatureGate:PodLevelResources] [Beta] Guaranteed QoS pod with container resources" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/common/node/pod_level_resources.go:250 (2x))
ERROR: dra/dra.go:1899: full test name is not unique: "[sig-node] [DRA] kubelet [Feature:DynamicResourceAllocation] [FeatureGate:DRAConsumableCapacity] [Alpha] [Feature:OffByDefault] [FeatureGate:DynamicResourceAllocation] must allow multiple allocations and consume capacity [KubeletMinVersion:1.34]" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/dra/dra.go:1899 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io] [Serial] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
And for test/e2e_node:
ERROR: cpu_manager_test.go:1622: full test name is not unique: "[sig-node] CPU Manager [Serial] [Feature:CPUManager] when checking the CFS quota management should disable for guaranteed pod with exclusive CPUs assigned" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1622, /nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1642)
ERROR: eviction_test.go:800: full test name is not unique: "[sig-node] LocalStorageCapacityIsolationFSQuotaMonitoring [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolationQuota] [Feature:LSCIQuotaMonitoring] [Feature:UserNamespacesSupport] when we run containers that should cause use quotas for LSCI monitoring (quotas enabled: true) should eventually evict all of the correct pods" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/eviction_test.go:800 (2x))
As before when adding v1beta2, DRA drivers built using the
k8s.io/dynamic-resource-allocation helper packages remain compatible with all
Kubernetes release >= 1.32. The helper code picks whatever API version is
enabled from v1beta1/v1beta2/v1.
However, the control plane now depends on v1, so a cluster configuration where
only v1beta1 or v1beta2 are enabled without the v1 won't work.
With OrderedNamespaceDeletion enabled by default, the namespace
controller now deletes resources before deleting the namespace itself.
At the same time, some existing tests create many resources. When
those tests finish around the same time, the namespace controller can
become overloaded. As a result, it may take longer to start deleting
resources created by this test.
Initially, this patch increased the test timeout to account for that
delay. However, the test still flaked in 1 out of 10 jobs, which
suggests increasing the timeouts isn't enough.
Once the test was made serial, no flakes were observed.
When expectation is that something does *not* happen, the test has to check for
a certain time. Checking just once is not sufficient because it might not have
happened immediately.
As can be seen in the output of a test failure in the CI, the test waits for
"quota-not-terminating" to be updated, but then fails because creating the pod
exceeds the "quota-terminating" quota which has not been updated yet. As both
ResourceQuotas apply, it needs to wait for both of them to be updated.
STEP: Deleting the pod - k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:1597 @ 05/18/25 07:50:46.531
STEP: Ensuring resource quota status released the pod usage - k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:1601 @ 05/18/25 07:50:46.547
I0518 07:50:46.551520 75078 resource_quota.go:2508] Got expected ResourceQuota:
...
name: quota-not-terminating
namespace: scope-selectors-1836
resourceVersion: "15914"
uid: db72668e-2fe0-4543-ada2-a3ff0ac77dd5
spec:
hard:
limits.cpu: "2"
limits.memory: 1Gi
pods: "1"
requests.cpu: "1"
requests.memory: 500Mi
scopeSelector:
matchExpressions:
- operator: Exists
scopeName: NotTerminating
status:
hard:
limits.cpu: "2"
limits.memory: 1Gi
pods: "1"
requests.cpu: "1"
requests.memory: 500Mi
used:
limits.cpu: "0"
limits.memory: "0"
pods: "0"
requests.cpu: "0"
requests.memory: "0"
STEP: Creating a terminating pod - k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:1610 @ 05/18/25 07:50:46.551
I0518 07:50:46.560944 75078 resource_quota.go:1612] Unexpected error:
<*errors.StatusError | 0xc002dba500>:
pods "terminating-pod" is forbidden: exceeded quota: quota-terminating, requested: pods=1, used: pods=1, limited: pods=1
{
ErrStatus:
code: 403
details:
kind: pods
name: terminating-pod
message: 'pods "terminating-pod" is forbidden: exceeded quota: quota-terminating,
requested: pods=1, used: pods=1, limited: pods=1'
metadata: {}
reason: Forbidden
status: Failure,
}
[FAILED] pods "terminating-pod" is forbidden: exceeded quota: quota-terminating, requested: pods=1, used: pods=1, limited: pods=1
This is meant to help debug this flake:
[It] [sig-api-machinery] ResourceQuota should verify ResourceQuota with terminating scopes through scope selectors.
[FAILED] pods "test-pod" is forbidden: exceeded quota: quota-terminating, requested: pods=1, used: pods=1, limited: pods=1
In [It] at: k8s.io/kubernetes/test/e2e/apimachinery/resource_quota.go:1572 @ 05/14/25 08:49:07.456
Dumping the ResourceQuota object each time the test progresses will
help with understanding the test and the actual behavior.
We only need one special "DynamicResourceAllocation" feature for the optional
node support of DRA (plugin registration, CDI support in the container
runtime). For individual features, the automatic labeling through
WithFeatureGate is sufficient.
To find DRA-related tests in a label filter, instead of plain-text "DRA" a
"DRA" label now gets added.
This change depends on an update of the DRA jobs.
We plan to add more formats in upcoming releases.
This tracks which formats are introduced at a version.
Unrecognized formats remain ignored. That is,
if a format is not supported at the emulated version,
the format is not enforced. This differs from typical
field handling, where unsupported field values are
forbidden. This is pre-existing behavior and is
in compliance with JSON Schema's format handling.
Ratcheting of custom resources helps with the introduction
of new formats. When a cluster is upgraded to a version
of Kubernetes that supports a format already set (but not enforced)
in a custom resource definition, the format will start
being enforced against custom resources. Ratcheting will
tolerate unchanged values of custom resources, even
if the value is not valid according to the format.