Quite a lot of unit tests set up informers with a fake client, do
informerFactory.WaitForCacheSync, then create or modify objects. Such tests
suffered from a race: because the fake client only delivered objects to the
watch after the watch has been created, creating an object too early caused
that object to not get delivered to the informer.
Usually the timing worked out okay because WaitForCacheSync typically slept a
bit while polling, giving the Watch call time to complete, but this race has
also gone wrong occasionally. Now with WaitForCacheSync returning more promptly
without polling (work in progress), the race goes wrong more often.
Instead of working around this in unit tests it's better to improve the fake
client such that List+Watch works reliably, regardless of the timing. The fake
client has traditionally not touched ResourceVersion in stored objects and
doing so now might break unit tests, so the added support for ResourceVersion
is intentionally limited to List+Watch.
The test simulates "real" usage of informers. It runs in a synctest bubble and
completes quickly:
go test -v .
=== RUN TestListAndWatch
listandwatch_test.go:67: I0101 01:00:00.000000] Listed configMaps="&ConfigMapList{ListMeta:{ 1 <nil>},Items:[]ConfigMap{ConfigMap{ObjectMeta:{cm1 default 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,},},}" err=null
listandwatch_test.go:79: I0101 01:00:00.000000] Delaying Watch...
listandwatch_test.go:90: I0101 01:00:00.100000] Caches synced
listandwatch_test.go:107: I0101 01:00:00.100000] Created second ConfigMap
listandwatch_test.go:81: I0101 01:00:00.100000] Continuing Watch...
--- PASS: TestListAndWatch (0.00s)
PASS
ok k8s.io/client-go/testing/internal 0.009s
Some users of the fake client need to be updated to avoid test failures:
- ListMeta comparisons have to be updated.
- Optional: pass ListOptions into tracker.Watch. It's optional because
the implementation behaves as before when options are missing,
but the List+Watch race fix only works when options are passed.
Kubernetes-commit: 56448506075c3db1d16b5bbf0c581b833a4646f1
This mirrors most (maybe all?) other fake clients. Example of a real
world use case this would have been useful:
https://github.com/istio/istio/pull/35434
Kubernetes-commit: 86c9fef160b37ce834b680e49cf237917d2e5c6c
The fake clientset used a slice to store each kind of objects, it's
quite slow to init the clientset with massive objects because it checked
existence of an object by traversing all objects before adding it, which
leads to O(n^2) time complexity. Also, the Create, Update, Get, Delete
methods needs to traverse all objects, which affects the time statistic
of code that calls them.
This patch changed to use a map to store each kind of objects, reduced
the time complexity of initializing clientset to O(n) and the Create,
Update, Get, Delete to O(1).
For example:
Before this patch, it took ~29s to init a clientset with 30000 Pods,
and 2~4ms to create and get an Pod.
After this patch, it took ~50ms to init a clientset with 30000 Pods,
and tens of µs to create and get an Pod.
Kubernetes-commit: 7e15e31e11e48a6db855e30ca9b07dbce3047577
This commit performs two refactors and fixes a bug.
Refactor 1 changes the signature of Request to take a RESTClient, which
removes the extra copy of everything on RESTClient from Request. A pair
of optional constructors are added for testing. The major functional
change is that Request no longer has the shim HTTPClient interface and
so some test cases change slightly because we are now going through
http.Client code paths instead of direct to our test stubs.
Refactor 2 changes the signature of RESTClient to take a
ClientContentConfig instead of ContentConfig - the primary difference
being that ClientContentConfig uses ClientNegotiator instead of
NegotiatedSerializer and the old Serializers type. We also collapse
some redundancies (like the rate limiter can be created outside the
constructor).
The bug fix is to negotiate the streaming content type on a Watch()
like we do for requests. We stop caching the decoder and simply
resolve it on the request. We also clean up the dynamic client
and remove the extra WatchSpecificVersions() method by providing
a properly wrapped dynamic client.
Kubernetes-commit: 3b780c64b89606f4e6b21f48fb9c305d5998b9e5
These will be used by the garbage collector controller and others that
use higher level primitives.
Kubernetes-commit: bc89c37f32aa6cfd0f9ca975d9221d0a89320623
