This avoids the risk of having a slow test started towards the end of a run,
which then would cause the run to take longer. When started early they can run
in parallel to other tests. In serial runs it doesn't matter.
The implementation maps the Slow label to the new ginkgo.SpecPriority. The
default is 0. Tests with priority 1 run first.
This reverts commit cff07e7551.
The commit caused several kubeadm jobs to fail while executing all conformance
tests (including slow ones) in parallel. Sometimes execution took longer and
ran into the overall timeout, sometimes there was:
[FAILED] Expected
<int>: 440
to be ==
<int>: 400
In [It] at: k8s.io/kubernetes/test/e2e/apimachinery/chunking.go:202
It looks like the tests are flaky and/or reveal a real bug when slow tests run
all in parallel at the same time.
This should work, but doesn't right now, so let's revert until that problem is fixed.
This avoids the risk of having a slow test started towards the end of a run,
which then would cause the run to take longer. When started early they can run
in parallel to other tests. In serial runs it doesn't matter.
The implementation maps the Slow label to the new ginkgo.SpecPriority. The
default is 0. Tests with priority 1 run first.
This reports and fixes for test/e2e:
ERROR: E2E suite initialization was faulty, these errors must be fixed:
ERROR: apimachinery/mutatingadmissionpolicy.go:184: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicy API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:184, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:606)
ERROR: apimachinery/mutatingadmissionpolicy.go:412: full test name is not unique: "[sig-api-machinery] MutatingAdmissionPolicy [Privileged:ClusterAdmin] [Feature:MutatingAdmissionPolicy] [FeatureGate:MutatingAdmissionPolicy] [Beta] [Feature:OffByDefault] should support MutatingAdmissionPolicyBinding API operations" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:412, /nvme/gopath/src/k8s.io/kubernetes/test/e2e/apimachinery/mutatingadmissionpolicy.go:834)
ERROR: common/node/pod_level_resources.go:250: full test name is not unique: "[sig-node] Pod Level Resources [Serial] [Feature:PodLevelResources] [FeatureGate:PodLevelResources] [Beta] Guaranteed QoS pod with container resources" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/common/node/pod_level_resources.go:250 (2x))
ERROR: dra/dra.go:1899: full test name is not unique: "[sig-node] [DRA] kubelet [Feature:DynamicResourceAllocation] [FeatureGate:DRAConsumableCapacity] [Alpha] [Feature:OffByDefault] [FeatureGate:DynamicResourceAllocation] must allow multiple allocations and consume capacity [KubeletMinVersion:1.34]" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/dra/dra.go:1899 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
ERROR: storage/testsuites/volume_group_snapshottable.go:173: full test name is not unique: "[sig-storage] CSI Volumes [Driver: pd.csi.storage.gke.io] [Serial] [Testpattern: (delete policy)] volumegroupsnapshottable [Feature:volumegroupsnapshot] VolumeGroupSnapshottable should create snapshots for multiple volumes in a pod" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_group_snapshottable.go:173 (2x))
And for test/e2e_node:
ERROR: cpu_manager_test.go:1622: full test name is not unique: "[sig-node] CPU Manager [Serial] [Feature:CPUManager] when checking the CFS quota management should disable for guaranteed pod with exclusive CPUs assigned" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1622, /nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/cpu_manager_test.go:1642)
ERROR: eviction_test.go:800: full test name is not unique: "[sig-node] LocalStorageCapacityIsolationFSQuotaMonitoring [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolationQuota] [Feature:LSCIQuotaMonitoring] [Feature:UserNamespacesSupport] when we run containers that should cause use quotas for LSCI monitoring (quotas enabled: true) should eventually evict all of the correct pods" (/nvme/gopath/src/k8s.io/kubernetes/test/e2e_node/eviction_test.go:800 (2x))
When building the test binary without race detection, we don't
need the post-processing of the JUnit file because it cannot
contain data race reports. This can be done via build tags.
Promoting real tests turned out to be harder than expected (should be rewritten
to be self-contained, additional reviews, etc.).
They would not achieve 100% endpoint+operation coverage because real tests only
use some of the operations. Therefore each API type has to be covered with
CRUD-style tests which only exercise the apiserver, then maybe additional
functional tests can be added later (depending on time and motivation).
The machinery for testing different API types is meant to be reusable, so it
gets added in the new e2e/framework/conformance helper package.
Ginkgo itself doesn't do this, in which case prun-junit-xml drops the output
and Spyglass wouldn't show the test as failed. If the data race warning is
captured, we now treat that as the failure of a test if it hasn't already
failed for other reasons.
While at it, the entire report cleanup gets moved to our junit package.
This doubles the termination timeout for the eviction test from 5min to
10min. Reason for that is that the eviction manager relies on pod stats
metrics, which may not be acceessible during a period of time because of
the kubelet API unreachable. This could be reasoned in hardware or
network pressure when multiple tests run in parallel.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Slightly-more-generic replacement for validateEndpointsPortsOrFail()
(but only validates EndpointSlices, not Endpoints).
Also, add two new unit tests to the Endpoints controller, to assert
the correct Endpoints-generating behavior in the cases formerly
covered by the "should serve endpoints on same port and different
protocols" and "should be updated after adding or deleting ports" e2e
tests (since they are now EndpointSlice-only). (There's not much point
in testing the Endpoints controller in "end to end" tests, since
nothing in a normal cluster ever looks at its output, so there's
really only one "end" anyway.)
These tests were using validateEndpointsPortsOrFail() not because they
cared about ports, but just because it was there, or in some cases
because they needed to wait for one pod to exit and a different pod to
start, which can't be done with framework.WaitForServiceEndpointsNum()
(or e2eendpointslice.WaitForEndpointCount) without racing. Update
these tests using the new e2eendpointslice.WaitForEndpointPods, which
can wait for specific expected pods.
(This also means these tests now only watch EndpointSlices, rather
than watching both Endpoints and EndpointSlices, which is fine,
because none of them are doing tricky things that actually require
making assertions about the exact contents of the
Endpoints/EndpointSlices. They just want to know when the controller
has updated things to point to the expected pods.)
