Previously it was necessary to use the Ginkgo wrappers when
using any of the custom arguments like WithSlow(). Now the
hook within Ginkgo for modifying arguments is used such that
e.g. the original ginkgo.It also works.
The spaces are unnecessary because Ginkgo adds spaces automatically.
This was detected before only for tests using the wrapper functions,
now it also gets detected for ginkgo methods.
* tested how many errors
* added exceptions
* added scoped exceptions per API group
* added struct.field specification
* improved regex match and included core and resources with the new struct.field format
* condensed exceptions using regex as requested
* fixed the scope kal nomaps exceptions to match existing fields
Quite a lot of unit tests set up informers with a fake client, do
informerFactory.WaitForCacheSync, then create or modify objects. Such tests
suffered from a race: because the fake client only delivered objects to the
watch after the watch has been created, creating an object too early caused
that object to not get delivered to the informer.
Usually the timing worked out okay because WaitForCacheSync typically slept a
bit while polling, giving the Watch call time to complete, but this race has
also gone wrong occasionally. Now with WaitForCacheSync returning more promptly
without polling (work in progress), the race goes wrong more often.
Instead of working around this in unit tests it's better to improve the fake
client such that List+Watch works reliably, regardless of the timing. The fake
client has traditionally not touched ResourceVersion in stored objects and
doing so now might break unit tests, so the added support for ResourceVersion
is intentionally limited to List+Watch.
The test simulates "real" usage of informers. It runs in a synctest bubble and
completes quickly:
go test -v .
=== RUN TestListAndWatch
listandwatch_test.go:67: I0101 01:00:00.000000] Listed configMaps="&ConfigMapList{ListMeta:{ 1 <nil>},Items:[]ConfigMap{ConfigMap{ObjectMeta:{cm1 default 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,},},}" err=null
listandwatch_test.go:79: I0101 01:00:00.000000] Delaying Watch...
listandwatch_test.go:90: I0101 01:00:00.100000] Caches synced
listandwatch_test.go:107: I0101 01:00:00.100000] Created second ConfigMap
listandwatch_test.go:81: I0101 01:00:00.100000] Continuing Watch...
--- PASS: TestListAndWatch (0.00s)
PASS
ok k8s.io/client-go/testing/internal 0.009s
Some users of the fake client need to be updated to avoid test failures:
- ListMeta comparisons have to be updated.
- Optional: pass ListOptions into tracker.Watch. It's optional because
the implementation behaves as before when options are missing,
but the List+Watch race fix only works when options are passed.
Test_isSchedulableAfterClaimChange was sensitive to system load because of the
arbitrary delay when waiting for the assume cache to catch up. Running inside
a synctest bubble avoids this. While at it, the unit tests get converted
to ktesting (nicer failure output, no extra indention needed for
tCtx.SyncTest).
TestPlugin/prebind-fail-with-binding-timeout relied on setting up a claim with
certain time stamps and then getting that test case tested within a certain
real-world time window. It's surprising that this didn't flake more often
because test execution order is random. Now the time stamp gets set right
before the test case is about to be tested. Conversion to a synctest would
be nicer, but synctests cannot have sub-tests, which are used here to track
where log output and failures come from within the larger test case.
Inside the plugin itself some log output gets added to explain why a claim is
unavailable on a node in case of a binding timeout or error during Filter.
The script is broken because it relies on grepping the source code.
This has always been fragile (currently it finds the search term
in comments) and stopped working years ago when changing how tests
are labeled.
Instead of fixing the script let's remove it because it's clearly unused.
The tests assumed that instantiating a DRAManager followed by
informerFactory.WaitForCacheSync would be enough to have the manager
up-to-date, but that's not correct: the test only waits for informer *caches*
to be synced, but syncing *event handlers* like the one in the manager may
still be going on. The flake rate is low, though:
$ GOPATH/bin/stress -p 256 ./noderesources.test
5s: 0 runs so far, 0 failures, 256 active
10s: 256 runs so far, 0 failures, 256 active
15s: 256 runs so far, 0 failures, 256 active
20s: 512 runs so far, 0 failures, 256 active
25s: 567 runs so far, 0 failures, 256 active
30s: 771 runs so far, 0 failures, 256 active
/tmp/go-stress-20251226T181044-974980161
--- FAIL: TestCalculateResourceAllocatableRequest (0.81s)
--- FAIL: TestCalculateResourceAllocatableRequest/DRA-backed-resource-with-shared-device-allocation (0.00s)
extendedresourcecache.go:197: I1226 18:11:14.431337] Updated extended resource cache for explicit mapping extendedResource="extended.resource.dra.io/something" deviceClass="device-class-name"
extendedresourcecache.go:204: I1226 18:11:14.431380] Updated extended resource cache for default mapping extendedResource="deviceclass.resource.kubernetes.io/device-class-name" deviceClass="device-class-name"
extendedresourcecache.go:220: I1226 18:11:14.431394] Updated device class mapping deviceClass="device-class-name" extendedResource="extended.resource.dra.io/something"
resource_allocation_test.go:595: Expected requested=2, but got requested=1
FAIL
It becomes higher when changing WaitForCacheSync such that it doesn't poll and
therefore returns more promptly, which is where this flake was first observed.
The fix is to run the test in a syntest bubble where Wait can be used to wait
for all background activity, including event handling, to be finished before
proceeding with the test.
synctest is less forgiving about lingering goroutines. A synctest bubble must
wait for gouroutines to stop, which in this case means that there has to be
a way to wait for the metric recorder shutdown. Event handlers have to be
removed.
This could be done with plain Go, but here test/utils/ktesting is used instead
because it offers some advantages:
- less boilerplate code
- automatic cancellation of the context (i.e. less manual context.WithCancel)
- tCtx.SyncTest is a direct substitute for t.Run, which avoids re-indenting
sub-tests. synctest itself needs another anonymous function, which makes
the line too long and forced re-indention:
t.Run(... func(...) {
synctest.Test(... func() {
})
})
For the sake of consistency all tests get updated.
While at it, some code gets improved:
- t.Fatal(err) is not a good way to report an error because
there is no additional markup in the test output that indicates
that there was an unexpected error. It just logs err.Error(),
which might not be very informative and/or obvious.
- newTestDRAManager aborts in case of a failure instead of
returning an error.
Verify scripts are run such that stderr is captured and included in the JUnit
files. Stdout is not. Therefore the instructions in case of a failure where
only visible by searching the entire job log file, but not in the Prow summary.
