As discovered in ticket 134737 `hostname` is buggy on busybox
due to the musl backend that it uses. something in the /etc/hosts
that k8s generates trips its parser and it doesn't work properly
in the ipv6 tests.
To workaround that use an image that has a glibc backend,
so that the hostname command works.
In the device plugin node reboot e2e test, the test previously waited a
short period for the resources exported by the sample device plugin to
appear on the local node. On slower test nodes, the plugin may take
longer to register, causing flakes where the expected devices are not
yet available.
This change increases the polling duration to 2 minutes, ensuring the test
waits long enough for the expected device capacity and allocatable resources
to appear, improving test stability.
This commit also updates the assertion message to be more explicit making
failures clearer and improving test reliability.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
In the device plugin node reboot e2e test, the registration trigger
(control file deletion) was being executed immediately after pod creation.
This could create a race condition: the device plugin container might not
be fully running, causing the test to flake when devices were not reported
as available on the node.
This change explicitly waits for the sample device plugin pod to reach the
Running/Ready state before deleting the registration control file. This
ensures that the device plugin is ready to register its devices with the
kubelet, eliminating a possible source of test flakiness.
Signed-off-by: Swati Sehgal <swsehgal@redhat.com>
This reports and fixes for test/e2e:
ERROR: E2E suite initialization was faulty, these errors must be fixed:
ERROR: apimachinery/mutatingadmissionpolicy.go:184: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicy API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:184, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:606)
ERROR: apimachinery/mutatingadmissionpolicy.go:412: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicyBinding API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:412, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:834)
ERROR: common/node/pod_level_resources.go:250: full test name is not unique: "[sig-node] Pod Level Resources [Serial] [Feature:PodLevelResources] [FeatureGate:PodLevelResources] [Beta] Guaranteed QoS pod with container resources" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/common/node/pod_level_resources.go:250 (2x))
ERROR: dra/dra.go:1899: full test name is not unique: "[sig-node] [DRA] kubelet [Feature:DynamicResourceAllocation] [FeatureGate:DRAConsumableCapacity] [Alpha] [Feature:OffByDefault] [FeatureGate:DynamicResourceAllocation] must allow multiple allocations and consume capacity [KubeletMinVersion:1.34]" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/dra/dra.go:1899 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io] [Serial] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
And for test/e2e_node:
ERROR: cpu_manager_test.go:1622: full test name is not unique: "[sig-node] CPU Manager [Serial] [Feature:CPUManager] when checking the CFS quota management should disable for guaranteed pod with exclusive CPUs assigned" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1622, /nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1642)
ERROR: eviction_test.go:800: full test name is not unique: "[sig-node] LocalStorageCapacityIsolationFSQuotaMonitoring [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolationQuota] [Feature:LSCIQuotaMonitoring] [Feature:UserNamespacesSupport] when we run containers that should cause use quotas for LSCI monitoring (quotas enabled: true) should eventually evict all of the correct pods" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/eviction_test.go:800 (2x))
8fed05c5b7 fixed the cleanup of tests which start
registrar and service in a single call. But tests which first started the
registrar and then the service separately still had the problem:
- registrar is started with test context
- pods remain running at end of test
- registrar stops because of test context cancellation
- pods remain pending despite deletion because the driver gets
unregistered (timing dependent, so this may have flaked)
The fix is to also clean up the registrar after the test, in reverse startup
order.
Added e2e_node test to verify that the Kubelet establishes only
a single gRPC connection with the DRA plugin for all service calls
during the plugin lifecycle.
The test uses a custom listener to count accepted connections and
asserts that only one connection is used for NodePrepareResources,
NodeUnprepareResources, and NodeWatchResources calls.
dd9917ddce fixed one test which did not wait for
pods to be deleted and then, depending on the timing, left ResourceClaims
prepared because the driver stopped before kubelet could call
NodeUnprepareResources.
But this is a more systematic issue also with other tests, so now the any test
which starts a DRA plugin automatically uses the same common cleanup code:
- delete pods in the test names
- wait for the driver to not have any active ResourceClaims
- stop the driver
The E2E node test "[DRA] Two resource Kubelet Plugins [Serial] must provide metrics" was failing flakily due to a race condition.
The preceding test, "should not add health status to Pod when feature gate is disabled," was leaking an in-use ResourceClaim. It deleted its pod but did not wait for the Kubelet to finish unprepared the resources, leaving the `dra_resource_claims_in_use` metric at a non-zero value.
This commit makes the cleanup process synchronous so now it deletes the pod and explicitly waits for the `NodeUnprepareResources` gRPC call to complete making sure resources are released before the test finishes.
Additionally, I fixed the cleanup logic in the `createHealthTestPodAndClaim` helper function to prevent a `DeviceClass` leak.
getLocalNode tried to get a ready node and fails if there's none.
The e2e test sends termination signal to kubelet and it's expected to have no ready nodes. Because of this, the e2e was permafailing.
Signed-off-by: Ayato Tokubi <atokubi@redhat.com>
`framework.WithLabel("[FeatureGate:ResourceHealthStatus:Disabled]")` has no
effect unless a job explicitly uses it in a --label-filter, which is not what
"generic" alpha/beta jobs are meant to do. The test therefore ran in the new
dra-alpha-beta job and failed because it expected the feature to be off.
In addition, the square brackets got added twice (once via the string
parameter, once by `framework.WithLabel`).
There is no generic way to filter out tests in advance which depend on feature
gates to be turned off. In e2e_node tests the active feature gates can be
checked at runtime, so this is what the test now does.
