Commit Graph

276 Commits

Author SHA1 Message Date
Odin Ugedal
a01b5a2a96 client-go/cache: rewrite Replace to check queue first
This is useful to both reduce the code complexity, and to ensure clients
get the "newest" version of an object known when its deleted. This is
all best-effort, but for clients it makes more sense giving them the
newest object they observed rather than an old one.

This is especially useful when an object is recreated. eg.

Object A with key K is in the KnownObjects store;
- DELETE delta for A is queued with key K
- CREATE delta for B is queued with key K
- Replace without any object with key K in it.

In this situation its better to create a DELETE delta with
DeletedFinalStateUnknown with B (with this patch), than it is to give
the client an DeletedFinalStateUnknown with A (without this patch).

Signed-off-by: Odin Ugedal <ougedal@palantir.com>
Signed-off-by: Odin Ugedal <odin@uged.al>

Kubernetes-commit: 7bcc3e00fc28b2548886d04639a2e352ab37fb55
2023-02-13 11:12:37 +00:00
Odin Ugedal
b250bf51ae client-go/cache: merge ReplaceMakesDeletionsForObjectsInQueue tests
Signed-off-by: Odin Ugedal <ougedal@palantir.com>
Signed-off-by: Odin Ugedal <odin@uged.al>

Kubernetes-commit: cd3e98b65c1339a8adc157175630de099a057d3f
2023-02-10 14:30:10 +00:00
Odin Ugedal
8279635aa4 client-go/cache: fix missing delete event on replace without knownObjects
This fixes an issue where a relist could result in a DELETED delta
with an object wrapped in a DeletedFinalStateUnknown object; and then on
the next relist, it would wrap that object inside another
DeletedFinalStateUnknown, leaving the user with a "double" layer
of DeletedFinalStateUnknown's.

Signed-off-by: Odin Ugedal <ougedal@palantir.com>
Signed-off-by: Odin Ugedal <odin@uged.al>

Kubernetes-commit: 0bf0546d9f75d92c801e81c9f7adf040bba64102
2023-02-10 14:16:26 +00:00
Odin Ugedal
2ded6b6eb8 client-go/cache: fix missing delete event on replace
This fixes a race condition when a "short lived" object
is created and the create event is still present on the queue
when a relist replaces the state. Previously that would lead in the
object being leaked.

The way this could happen is roughly;

1. new Object is added O, agent gets CREATED event for it
2. watch is terminated, and the agent runs a new list, L
3. CREATE event for O is still on the queue to be processed.
4. informer replaces the old data in store with L, and O is not in L
  - Since O is not in the store, and not in the list L, no DELETED event
    is queued
5. CREATE event for O is still on the queue to be processed.
6. CREATE event for O is processed
7. O is <leaked>; its present in the cache but not in k8s.

With this patch, on step 4. above it would create a DELETED event
ensuring that the object will be removed.

Signed-off-by: Odin Ugedal <ougedal@palantir.com>
Signed-off-by: Odin Ugedal <odin@uged.al>

Kubernetes-commit: 25d77218acdac2f793071add9ea878b08c7d328b
2023-02-08 14:57:23 +00:00
xuzhenglun
7685b51912 Fix bug in reflector not detecting "Too large resource version" error before 1.17.0
Kubernetes-commit: 11e5e92dc65c1caaeb248a60d90ccc8e29eb9ff0
2023-01-16 11:36:28 +08:00
Clayton Coleman
08e22c4b64 cache: Reflector should have the same injected clock as its informer
While refactoring the backoff manager to simplify and unify the code
in wait a race condition was encountered in
TestSharedInformerWatchDisruption. The new implementation failed
because the fake clock was not propagated to the backoff managers
when the reflector was used in a controller. After ensuring the
mangaers, reflector, controller, and informer shared the same
clock the test needed was updated to avoid the race condition by
advancing the fake clock and adding real sleeps to wait for
asynchronous propagation of the various goroutines in the controller.

Due to the deep structure of informers it is difficult to inject
hooks to avoid having to perform sleeps. At a minimum the FakeClock
interface should allow a caller to determine the number of waiting
timers (to avoid the first sleep).

Kubernetes-commit: 91b3a81fbd916713afe215f7d701950e13a02869
2023-01-14 14:17:33 -05:00
Daniel Smith
cb28a0e57e Fix N^2 startup for webhook configurations
Add a "lazy" type to track when an update is needed. It uses a nested
locking technique to avoid extra evaluation calls.

Kubernetes-commit: 5a1091d88d95bd1dd5c27f2c72cee4ecb4219dda
2023-01-09 23:29:25 +00:00
Daniel Smith
5d70a118df Enable propagration of HasSynced
* Add tracker types and tests
* Modify ResourceEventHandler interface's OnAdd member
* Add additional ResourceEventHandlerDetailedFuncs struct
* Fix SharedInformer to let users track HasSynced for their handlers
* Fix in-tree controllers which weren't computing HasSynced correctly
* Deprecate the cache.Pop function

Kubernetes-commit: 8100efc7b3122ad119ee8fa4bbbedef3b90f2e0d
2022-11-18 00:12:50 +00:00
Andy Goldstein
fbb7f087d1 reflector: refactor setting typeDescription & expectedGVK
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 784ec157e67c86bc3383b326bbfe8ee70737aa4d
2022-12-02 12:39:58 -05:00
Andy Goldstein
37897aff8d Reflector: support logging Unstructured type
Add an annotation that can be added to the exampleType passed to
NewReflector to indicate the expected type for the Reflector. This is
useful for types such as unstuctured.Unstructured, which, when used with
a dynamic informer, do not have their TypeMeta filled in.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 474fc8c5234000bce666a6b02f7ffbb295ef135f
2022-08-17 15:49:26 -04:00
Wojciech Tyczyński
5e7ba1f8d7 Minor cleanup of thread safe store
Kubernetes-commit: 72194de96f0bb5c1c1e3247a129cf9e3f11972ef
2022-11-02 13:02:29 +01:00
Wojciech Tyczyński
b69a16cf38 Refactor store index into its structure
Kubernetes-commit: 7c94ce3076a96acab4c7e88489cd596f1aad40e0
2022-05-05 17:30:24 +02:00
astraw99
f549acf3ce Fix duplicate code block of ListAll function
Kubernetes-commit: d8db1e9ba3c110012def6faa5579ad3abb71a6a6
2022-03-24 14:47:20 +08:00
Nick Turner
5870c622c7 Remove log line from expiration cache
Kubernetes-commit: e7bfd5909f5540f884a8bb914a9230c9bf7c2803
2022-10-04 11:01:37 -07:00
Lurong
59765b8784 fix typo error
Kubernetes-commit: 0b2ad4cc42e727ec93011411033cac1abfb637ea
2022-09-22 14:42:41 +08:00
Alexander Zielenski
a300ae0dfe return when test is done
Kubernetes-commit: cc0b9ffbd5a4edfe7ce01293c2bf963d4335d8d6
2022-08-30 14:38:21 -07:00
Alexander Zielenski
93e5e0e8a0 hold listener lock while waiting for goroutines to finish
Kubernetes-commit: 7ce19b75a8dbca12837ed9f4c5c2828c38b82a03
2022-08-29 12:34:35 -07:00
Alexander Zielenski
e11a988e1c simplify control flow
Kubernetes-commit: ee24648300f8575f503156f45b44034054c2e49d
2022-08-29 11:52:35 -07:00
Alexander Zielenski
ac7f6579ff fix spelling
Kubernetes-commit: 7685b393c2ee8f8548d62f9dd8485df764a0c8c3
2022-08-22 17:34:02 -07:00
Alexander Zielenski
0f4a6cf319 reset listenersStarted
for correctness. technically shouldnt be an issue since restarting a stopped processor is not supported

Kubernetes-commit: 3a81341cfa6f7e2ca1b9bfc195c567dcdfaa4dea
2022-08-08 14:19:37 -07:00
Alexander Zielenski
449817f7b5 add multithreaded test to shared informer
Kubernetes-commit: 8af0a31a15079725e5910d538a6ace03df2e382a
2022-08-08 13:39:18 -07:00
Alexander Zielenski
de0b7671e3 remove duplicate test
became a duplicate while refactoring original PR

Kubernetes-commit: df9a7afa63164035f1e94c3278d470b347b697b9
2022-08-08 11:46:05 -07:00
Alexander Zielenski
0565962dd0 address review comments
asd

Kubernetes-commit: 5f372e1867305679d8f9d8a013c5500763e5c875
2022-08-08 11:44:37 -07:00
Alexander Zielenski
5a25eb0de8 switch listeners to use a map, adapt tests
Kubernetes-commit: 063ef090e7fb5823ca18a10a83e8847eedac9599
2022-06-16 10:54:03 -07:00
Uwe Krueger
90c6a46b32 active remove/add tests for event handlers
Kubernetes-commit: e9cd17170b1646672796e713dee6c4fb0e6693bb
2021-10-06 12:40:46 +02:00
Uwe Krueger
de4dd3aaf3 tests for invalid registration removals
Kubernetes-commit: 152b6e11af4c8c50b768630bf7724e0bb3414e9c
2021-08-22 13:27:36 +02:00
Uwe Krueger
33eff64a05 apply desired changes for handler registration
Kubernetes-commit: 92f04baac98186673c684bba419087d6285807b1
2021-08-22 13:03:18 +02:00
Uwe Krueger
d73e40f207 rename handle to registration
Kubernetes-commit: 7054ac16d43a4a55d6e7b69943eb209f6495c4ce
2021-08-04 21:54:32 +02:00
Alexander Zielenski
b3a61c6731 remove informational informer methods again
Kubernetes-commit: f52f4a8e3045fd3eeba0315a347ed653012cd5c5
2022-06-16 00:09:20 -07:00
Alexander Zielenski
ecdc8bf729 support removal of event handlers from SharedIndexInformers
To be able to implement controllers that are dynamically deciding
on which resources to watch, it is required to get rid of
dedicated watches and event handlers again. This requires the
possibility to remove event handlers from SharedIndexInformers again.
Stopping an informer is not sufficient, because there might
be multiple controllers in a controller manager that independently
decide which resources to watch.

Unfortunately the ResourceEventHandler interface encourages to use
value objects for handlers (like the ResourceEventHandlerFuncs
struct, that uses value receivers to implement the interface).
Go does not support comparison of function pointers and therefore
the comparison of such structs is not possible, also. To be able
to remove all kinds of handlers and to solve the problem of
multi-registrations of handlers a registration handle is introduced.
It is returned when adding a handler and can later be used to remove
the registration again. This handle directly stores the created
listener to simplify the deletion.

Kubernetes-commit: 7436af3302088c979b431856c432b95dd230f847
2022-06-16 00:07:54 -07:00
Marcel Zięba
ff6bf679aa Add option to retry internal api error in reflector.
Kubernetes-commit: 0b2b6489de8f75d5299f54180617601126bb8878
2022-07-25 08:02:54 +00:00
Davanum Srinivas
2a6c116e40 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Lukasz Szaszkiewicz
441e2c8c97 reflector: simplify reading the resourceVersion
Kubernetes-commit: 63b125d4061d267ad8998ea30e054bb1445b1a5a
2022-07-15 10:14:26 +02:00
Lukasz Szaszkiewicz
04f67d5d4f reflector: move LIST to its own method
Kubernetes-commit: 6fc09008def0b63fdd31bd26a14f7fcd3ed63c7e
2022-06-24 16:25:08 +02:00
Lukasz Szaszkiewicz
4e28921c86 reflector: refactor watchHandler
Kubernetes-commit: e9e26068b746315b616a26b91ff2613f98e934bc
2022-06-24 13:44:32 +02:00
weilaaa
64585cf823 correct input params and add godoc
Kubernetes-commit: 9847b2eeb43e3fd13581dfdb4eeb288139ea7f3b
2022-06-22 11:26:07 +08:00
Karl Isenberg
876406445a fix: reflector to return wrapped list errors
This fix allows Reflector/Informer callers to detect API errors using the standard Go errors.As unwrapping methods used by the apimachinery helper methods. Combined with a custom WatchErrorHandler, this can be used to stop an informer that encounters specific errors, like resource not found or forbidden.

Kubernetes-commit: 9ace604b63045ebbb066cab5e8508b51d0900a05
2022-05-16 16:33:30 -07:00
Andrew Stoycos
f19a514ff5 Fix Panic Condition
Currenlty an event recorder can send an event to a
broadcaster that is already stopped, resulting
in a panic.  This ensures the broadcaster holds
a lock while it is shutting down and then forces
any senders to drop queued events following
broadcaster shutdown.

It also updates the Action, ActionOrDrop,  Watch,
and WatchWithPrefix functions to return an error
in the case where data is sent on the closed bradcaster
channel rather than panicing.

Lastly it updates unit tests to ensure the fix works correctly

fixes: https://github.com/kubernetes/kubernetes/issues/108518

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

Kubernetes-commit: 6aa779f4ed3d3acdad2f2bf17fb27e11e23aabe4
2022-02-11 14:50:19 -05:00
Michael Bolot
e540ebe994 Addresses the issue which caused #109115
Kubernetes-commit: cbbb5f70a47644f9830073d9d0329bf247a328a1
2022-03-29 12:35:13 -05:00
Wojciech Tyczyński
1cab68940f Add test for indexer with multiple values
Kubernetes-commit: 56159f258ca380600b0bc08b2e99cbc745db3560
2022-03-30 08:52:10 +02:00
Jian Li
6b59aa0b4a make comments of updateIndices optimization code more accurate
Kubernetes-commit: 0eae1f3addbd044793dac89f9e097e5f4c6fb2aa
2022-03-14 13:56:17 +08:00
Alexander Zielenski
54928eef9f modify SharedIndexInformer to use newInformer constructor which supports transformers
avoids code duplication, allows transformer to be used with SharedIndexInformer

Kubernetes-commit: 754bf3b3d02c66d1dd030460b03dddd3d6c7196d
2022-01-19 11:40:42 -08:00
Ismayil Mirzali
664b1a6c8c client-go: refactor: Fix styling issues (#107248)
* client-go: Remove unreachable return

Due to the way the switch statement is done,
the return at the end of the function will neverbe reached.

Signed-off-by: Ismayil Mirzali <ismayilmirzeli@gmail.com>

* client-go: Refactor for clarity

Fixed one instance where the error message should be lowercase.
Made the fields in the struct literal more explicit

Signed-off-by: Ismayil Mirzali <ismayilmirzeli@gmail.com>

Kubernetes-commit: 75c0987de3cb9a0380873745f68dea2f0835a7a2
2022-01-18 12:03:08 +02:00
Davanum Srinivas
3bf0eac274 OWNERS cleanup - Jan 2021 Week 1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9682b7248fb69733c2a0ee53618856e87b067f16
2022-01-03 10:59:47 -05:00
wq
3480737c9e fix typos in comment
Kubernetes-commit: 3e2932179084fa322ba1bc53997d44e1d383db00
2022-01-08 23:39:23 +09:00
astraw99
a8ff96d887 fix log typo
Kubernetes-commit: abce7ab534d0c667dd1438ff24bb3130d17d3f6e
2021-11-19 11:06:14 +08:00
Davanum Srinivas
73f2731e23 Cleanup OWNERS files (No Activity in the last year)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 497e9c1971c9e7d0193bc6d11503ec4ad527f1d5
2021-12-10 15:18:50 -05:00
Davanum Srinivas
70f09c4943 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
hyschumi
547d6c809a refactor: remove dup code
Signed-off-by: hyschumi<msliudongsheng@gmail.com>

Kubernetes-commit: da5d0a72d9d2a0948e21b746a8c6951dc40c0ce9
2021-11-04 21:11:53 +08:00
Antoni Zawodny
9af7db3c25 Add more info to the ListAndWatch trace
Kubernetes-commit: 093aa210be67f45efb1005f273d8d68444156473
2021-10-21 14:33:57 +02:00