If, for whatever reason, the context was context.Background(), the additional
goroutine was started and then got stuck forever because
context.Background().Done() is a nil channel. Found when indirectly
instantiating a broadcaster with such a context:
found unexpected goroutines:
[Goroutine 9106 in state chan receive (nil chan), with k8s.io/kubernetes/vendor/k8s.io/client-go/tools/record.NewBroadcaster.func1 on top of the stack:
goroutine 9106 [chan receive (nil chan)]:
k8s.io/kubernetes/vendor/k8s.io/client-go/tools/record.NewBroadcaster.func1()
/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/tools/record/event.go:206 +0x2c
created by k8s.io/kubernetes/vendor/k8s.io/client-go/tools/record.NewBroadcaster in goroutine 8957
/home/prow/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/client-go/tools/record/event.go:205 +0x1a5
This can be fixed by checking for a nil channel.
Another problem also gets addressed: if Shutdown was called without canceling
the context, the goroutine also didn't stop. Now it waits for the cancelation
context and thus terminates in both cases.
Kubernetes-commit: eed6e29a5b8cfaa20fbc426541d9c74105d430ee
until #115478(use streaming against the etcd storage)
is resolved the cacher need a way to disable the streaming.
Kubernetes-commit: 41e706600aea7468f486150d951d3b8948ce89d5
27a68aee3a4834 introduced context support. In order to use that
also with NewEventBroadcasterAdapter, a variant of the call is needed to allow
the caller to specify the context.
The `logcheck:context` comment ensures that code which is meant to be
contextual uses the new call.
Kubernetes-commit: f8e25eff926c640c86daa46222bfaf8d625e75d7
checkWatchListConsistencyIfRequested performs a data consistency check only when
the KUBE_WATCHLIST_INCONSISTENCY_DETECTOR environment variable was set during a binary startup.
The consistency check is meant to be enforced only in the CI, not in production.
The check ensures that data retrieved by the watch-list api call
is exactly the same as data received by the standard list api call.
Note that this function will panic when data inconsistency is detected.
This is intentional because we want to catch it in the CI.
Kubernetes-commit: b31e7793d0d873a71c90caf8455556aa905cf88d
Because the EventBroadcaster code now has a a context, changing the EventSink
interface so that the methods accepts a context instead of using context.TODO
becomes possible.
Kubernetes-commit: 5dc540ff57eed5d0e6bc8476f95fe03b7a983cf5
Using StartRecordingToSinkWithContext instead of StartRecordingToSink and
StartLogging instead of StartStructuredLogging has several advantages:
- Spawned goroutines no longer get stuck for extended periods of
time during shutdown when passing in a context that gets canceled.
- Log output can be directed towards a specific logger instead of the global
default, for example one which writes to a testing.T instance.
- The new methods return an error when something went wrong instead of
merely recording the error.
That last point is the reason for deprecating the old methods instead of merely
adding new alternatives.
Setting a context when constructing an EventBroadcaster makes calling Shutdown
optional. It can also be used to specify the logger.
Both EventRecorder interfaces in tools/events and tools/record now have a
WithLogger helper. Using that method is optional, but recommended to support
contextual logging properly. Without it, errors that occur while emitting an
event are not associated with the caller.
Kubernetes-commit: 27a68aee3a48340f7c14235f7fc24aa69aaeb8f6
originally we honored only apierrors.IsInvalid
but decided to fallback on every error
because it is better to make progress than deadlocking
Kubernetes-commit: 4b3915017950a114124a88c5d308bd8bfb9ec48e
* client-go: add DNS resolver latency metrics
* client-go: add locking to DNS latency metrics
* client-go: add locking for whole DNSStart and DNSDone
Signed-off-by: Vu Dinh <vudinh@outlook.com>
* Fix a mismatched ctx on the request
Signed-off-by: Vu Dinh <vudinh@outlook.com>
* Clean up request code and fix comments
Signed-off-by: Vu Dinh <vudinh@outlook.com>
---------
Signed-off-by: Vu Dinh <vudinh@outlook.com>
Co-authored-by: Vu Dinh <vudinh@outlook.com>
Kubernetes-commit: 1c7e87cff27aa009488a9d55342220e223d5c146
Create events are forbidden in terminating namespaces, use info
instead of error to log the failed event.
Signed-off-by: Sunil Shivanand <sunil.shivanand@statnett.no>
Kubernetes-commit: 7a6d58001b7d824f92601fd246b3aad9fbb9c583
T.Setenv ensures that the environment is returned to its prior state
when the test ends. It also panics when called from a parallel test to
prevent racy test interdependencies.
Kubernetes-commit: ac5ce5cbce7ddb6ffeff755d1cf670afadb8e1fb
`SetWatchErrorHandler` claims it will fail if Run() has already started.
But if they are called concurrently, it will actually trigger a data
race.
With this PR:
```
62702 runs so far, 0 failures (100.00% pass rate). 59.152682ms avg, 189.068387ms max, 26.623785ms min
```
Without this PR:
```
5012 runs so far, 38 failures (99.25% pass rate). 58.675502ms avg, 186.018084ms max, 29.468104ms min
```
Kubernetes-commit: 35d2431b3a89c5bd693846952e9d27ce4e3a0754
Before, we've used two separate backoff managers for List and Watch
calls, now they share single backoff manager.
Kubernetes-commit: 337728b02559dec8a613fdef174f732da9cae310
Ensure that the lock identity is not empty. This can cause unexpected issues during leader election.
Kubernetes-commit: 5519b89a28eeea5a5b134092242aff770fcc07eb
* Add warning log callback in client-go loading rules
This provides a way to consumers use their own custom warning
mechanisms instead default klog warning.
* Use typed error instead plain string
* Fix interface change in unit test
Kubernetes-commit: 2ea6896f90c8b757d8a247d393b9a13fff2dab58
Add two new metrics to monitor the client-go logic that
generate http.Transports for the clients.
- rest_client_transport_cache_entries is a gauge metrics
with the number of existin entries in the internal cache
- rest_client_transport_create_calls_total is a counter
that increments each time a new transport is created, storing
the result of the operation needed to generate it: hit, miss
or uncacheable
Change-Id: I2d8bde25281153d8f8e8faa249385edde3c1cb39
Kubernetes-commit: 3f3e1d507d081528ce8081f0daf4446d8a3b9bc0
