Commit Graph

65 Commits

Author SHA1 Message Date
Antonio Ojea
374e3638f2 workqueue unit tests to assert collapsing behaviors
Change-Id: If4bf54c3af603c17de49055960f9a76e7d38aab3

Kubernetes-commit: 8a31bb6786c2d3600bade49de7a64a026802bd6d
2023-09-21 10:48:01 +00:00
Dave McCormick
e9b0103a41 Fix a race where a call to Shutdown was happening after ShutDownWithDrain() had performed the check q.isProcessing() && q.shouldDrain() and before waitForProcessing() has reached its Wait(). This is because waitForProcessing() is only checking the length part of the condition instead of both the length and whether we still need to drain.
It turned out that there were lots of unnecessary accessor functions being called locking and unlocking the cond which are not needed because Wait() automatically unlocks and locks the cond for us and best practice says we should run it in a for checking for the condition (so this is what I have done).

Kubernetes-commit: 2b12df56b0be93bc2cac7c5a66342c0ffaa72311
2023-07-24 22:40:54 +01:00
austin ce
21b4553e44 Allow custom metric provider for retry metrics
Kubernetes-commit: a60624ffbc265d61b35cab16ea9ded485909f3ed
2023-03-10 17:48:10 -05:00
austin ce
66fa9d308f Unify queue constructors, deprecate current constructors
Kubernetes-commit: fea83b78243c77e81b91f3f0a812979809aeacaf
2023-03-10 17:46:26 -05:00
austin ce
9668243c04 Integrate new metrics provider option in test case
Kubernetes-commit: 9e83e7e975dd5971e60e9768e06520aa5ebf9bf8
2022-12-02 14:08:11 -05:00
austin ce
e0785d6713 Remove options from queue ctors w/o metrics
Kubernetes-commit: 31f16db1959bb1db02fcc578f4aa7d47414a26f4
2022-12-02 14:06:37 -05:00
austin ce
89a4c1271f Add license header to new file
Kubernetes-commit: 95ad089ff8164d455f766694b1292ca8f1d200df
2022-12-01 18:28:02 -05:00
austin ce
98d0d1a9ba Allow setting per-queue metrics providers
Kubernetes-commit: da71811514ac6e8fedd265831fed71f897b97781
2022-11-29 20:47:33 -05:00
Aldo Culquicondor
3dfaef50d7 Add clock interface to disruption controller
To be able to write more precise unit tests in the future

Change-Id: I8f45947dfacca501acd856849bd978fad0f735cd

Kubernetes-commit: dad8454ebb87ba7af897537db6d34033127bbfef
2022-07-26 15:11:53 -04:00
Davanum Srinivas
2a6c116e40 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
cncal
152e68ff13 Remove the duplicate code snippet in client-go delaying_queue tests
Kubernetes-commit: af438abbf332e26e2f3f729627bf9764c9b1feef
2022-04-17 15:55:33 +08:00
Spencer Peterson
5a537879c2 Document when workqueue metrics are dropped
Two simple choices for workqueues do not document that they do not emit
metrics. Using their named variants fixes this, but was undocumented.

Change-Id: I100ad08a4859513987941ed35d12abb4cbb39873

Kubernetes-commit: f468bee672b0ccf9b97a85f17ec1f5645aced926
2021-12-09 16:59:54 -08:00
Karthik K N
881e30ae78 Changes to fix the flake test case in ratelimiter
Kubernetes-commit: 7134657304a6d5b3a62c51cad82af6ca703d40c4
2021-09-28 12:23:27 +05:30
luyou86
a6277bbabf client-go bucket rate limiter add maxDelay
Kubernetes-commit: 57ec8336195702dd0d5f6f19fd3907e2a6a615a8
2021-08-29 17:45:58 +08:00
John Howard
eb364c77d8 workqueue: fix leak in queue preventing objects from being GCed
See https://github.com/grpc/grpc-go/issues/4758 for a real world example
of this leaking 2gb+ of data.

Basically, when we do `q.queue[1:]` we are just repositioning the slice.
The underlying array is still active, which contains the object formerly
known as `q.queue[0]`. Because its referencing this object, it will not
be GCed. The only thing that will trigger it to free is eventually when
we add enough to the queue that we allocate a whole new array.

Instead, we should explicitly clear out the old space when we remove it
from the queue. This ensures the object can be GCed, assuming the users'
application doesn't reference it anymore.

Kubernetes-commit: 2a34801168dc1c70ba25b1d4200b534bf515cbc2
2021-09-14 15:51:09 -07:00
xuzhonghu
2f21912570 Fix workqueue memory leak
Kubernetes-commit: 25060beef0c536e80b53507ec445695a7305db00
2021-09-14 09:53:53 +08:00
wojtekt
14a51589c3 Migratet to k8s.io/utils/clock in workqueue
Kubernetes-commit: 392292ba81964e6ee7badec360a800b1f8645f2f
2021-09-09 16:38:59 +02:00
Alexander Constantinescu
22aa998def [queue] Implement ShutDownWithDrain allowing the queue to drain when shutting down
Signed-off-by: Alexander Constantinescu <aconstan@redhat.com>

Kubernetes-commit: 5b740f430e0a4892e9db3a1fea9f349a06267755
2021-05-12 02:26:22 +02:00
astraw99
9d15ff74ca fix typo of rate limiter
Kubernetes-commit: 7953cf0ce9f890b6cba7de3325a7ce68609832a0
2021-08-15 17:48:37 +08:00
Dan Ramich
f39ca994bd Don't start goroutine for noMetrics
Problem:
When calling newQueue metrics can be of type noMetrics when just calling
New. When doing this a new goroutine is created to update the metrics
but in this case there are no metrics so it's just creating goroutines
that don't do anything but consume resources.

Solution:
If the incoming metrics is of type noMetrics, don't start the goroutine

Kubernetes-commit: de021396f81ff438899297a6f464c70113b58475
2020-09-15 16:21:48 -06:00
knight42
f2963a02f6 test(workqueue): deflake TestMetrics
Signed-off-by: knight42 <anonymousknight96@gmail.com>

Kubernetes-commit: b70e1d836c2d438af6b01fd297677b900b364a33
2020-09-05 18:50:45 +08:00
fatedier
1ded602dc0 Export new constructor for DelayingQueue
Kubernetes-commit: b5d989a6fc78fd8d1743eb2b80b9be4cb6e3c6ac
2020-03-23 12:15:29 +08:00
Aldo Culquicondor
7ee24064f0 Use sqrt(n) chunk size in pod affinity and core scheduler
Kubernetes-commit: e902e70d0d9bca1a1c823ce9d04d8fd68e8f3396
2020-03-16 14:12:11 -04:00
Aldo Culquicondor
326ffd9341 Add chunk size option to ParallelizeUntil
Signed-off-by: Aldo Culquicondor <acondor@google.com>

Kubernetes-commit: 36efa035e292480d4b0ccc7a4ffbf9aa579fbf30
2020-03-11 17:13:55 -04:00
Maciej Borsz
8fbf5159e5 Revert "Implement ItemBucketRateLimiter"
Kubernetes-commit: 01c62f672611024445a846e3cdf60e5e99fb8a1e
2020-03-05 14:35:19 +01:00
Maciej Borsz
94f6bd1785 Implement ItemBucketRateLimiter
Kubernetes-commit: 6846a0a6b62725a5888d66e2789f2af4fb172045
2020-02-18 11:24:20 +01:00
danielqsj
51f3d77844 clean SinceInMicroseconds, convert to SinceInSeconds
Kubernetes-commit: ab182552b485ceed96c3b05d284ae9480377daf8
2019-04-15 16:18:01 +08:00
Jacek Kaniuk
d81654af9d pkg/util/workqueue/delaying_queue: export contructor with custom clock
Kubernetes-commit: 638c02f6cdb4e269f46b60fe1f91e553cfc59749
2019-10-23 16:07:38 +02:00
Michael Morello
8a1fbdc8bb Fix memory and timer leak in work queue
Kubernetes-commit: f9c3d246d32cd440a8ea5ea40f02b6a9bfe4b485
2019-10-15 10:06:06 +02:00
Sergiusz Urbaniak
ee078c72e2 pkg/util/workqueue: delete deprecated metrics
This deletes deprecated metrics and simplifies registration.

Kubernetes-commit: 4532cfd85c00da6f64e03fcf05f5636adc1151c7
2019-05-13 13:22:08 +02:00
Mikhail Mazurskiy
c90a87409a Cleanup math/rand package usage
Kubernetes-commit: 4ce1fb7d40beb9010e56d60792c4da25e8d86ed0
2018-11-17 16:45:36 +11:00
Sukeesh
c8a491caea Update doc.go in staging/src/k8s.io/
Kubernetes-commit: c8bd20d1b289eadb6abc20f0868f4a5a6a668035
2019-06-25 22:43:44 +09:00
Clayton Coleman
bdead75b4b DelayingQueue.ShutDown() should be reentrant
All queue ShutDown() calls should be able to be invoked multiple times.

```
Observed a panic: "close of closed channel" (close of closed channel)
/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:76
/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/asm_amd64.s:573
/usr/local/go/src/runtime/panic.go:502
/usr/local/go/src/runtime/chan.go:333
/go/src/github.com/openshift/cluster-version-operator/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:137
```

Use sync.Once to guarantee a single close.

Kubernetes-commit: d2f7eb5235a93556261c8947e7a87342aeeaee2b
2019-04-27 16:16:55 -04:00
danielqsj
f12b22a8d9 clean the deprecated func Parallelize
Kubernetes-commit: a10021787b2d037e3c1d44a205980fd48ac23eff
2019-04-15 18:15:28 +08:00
danielqsj
6658e1f4a3 Using histogram metrics instead of summary
Kubernetes-commit: 42214c5ac423289a6c87e0f3c1014f652d58bf62
2018-12-12 16:50:32 +08:00
danielqsj
c270b352df Use prometheus conventions for workqueue metrics
Kubernetes-commit: b828bc1a50dfa29ab85248e5af6da5acb27189c1
2018-11-21 11:43:47 +08:00
Lv Jiawei
2059e953fc Comment error in ItemExponentialFailureRateLimiter
The backoff value is baseDelay*2^<num-failures> in ItemExponentialFailureRateLimiter.When . But the comment is baseDelay*10^<num-failures>.

Kubernetes-commit: c1fa760b75970fbd0c142971f1142754cb4ea3fc
2018-11-08 12:13:23 +08:00
danielqsj
d25d9a3c52 Fix typos like limitting
Kubernetes-commit: 3c055aa4b47232bf7d6b5d5a0901dae239e33c59
2018-12-04 11:01:40 +08:00
Daniel Smith
f0e9de6741 fixup! add longest_running_processor_microseconds metric
fix data race

Kubernetes-commit: 680ddd49d8d23a88744f9de720f266022effd409
2018-11-12 12:57:42 -08:00
Daniel Smith
952b05be23 add longest_running_processor_microseconds metric
Kubernetes-commit: fd77aa5a41bbce7490dd4538c0d5743cb59b2be4
2018-11-12 10:52:18 -08:00
Daniel Smith
cc2731ad7e fixup! Test workqueue metrics
change units to seconds

Kubernetes-commit: 578962d934df19cb2cb7ec0536dcb76f53951e68
2018-11-10 18:46:43 -08:00
Daniel Smith
e403f1715c fixup! Test workqueue metrics
Kubernetes-commit: 44a87baf09aec4f2e8e159de4c409d27aebbb1da
2018-11-10 18:24:59 -08:00
Daniel Smith
26f9385b8e Test workqueue metrics
Kubernetes-commit: 5a8444ceec9e28e8a7dbf36bfd7cb55554c5b865
2018-11-09 16:12:11 -08:00
Daniel Smith
75d4dad922 add a metric that can be used to notice stuck worker threads
Kubernetes-commit: 6195d1005d81eaa5dd49da744f5beab178340f5a
2018-11-09 10:43:44 -08:00
Guoliang Wang
b5eba4c261 Replace Parallelize with function ParallelizeUntil and formally deprecate the Parallelize
Kubernetes-commit: c2622dd9d800280bcaeea9473340a4efb94946d2
2018-09-07 17:17:27 +08:00
tanshanshan
0fa5515631 fix syntax error:'its'
Kubernetes-commit: 7dfe0dae64aa2f6cd579f9c88bdf84cccad94504
2018-09-20 09:45:46 +08:00
Guoliang Wang
35223a0681 Not split nodes when searching for nodes but doing it all at once
Kubernetes-commit: 6c63dcfffebb9a8bcc5e1cee748ad16d7ed7e293
2018-08-18 09:05:58 +08:00
Mikhail Mazurskiy
9446f4ef9d Stop() for Ticker to enable leak-free code
Kubernetes-commit: 1f393cdef96fe6e4ddcbf93825d65a9980463406
2018-03-31 19:41:43 +11:00
Kubernetes Publisher
fff8c3d73e sync: initially remove files BUILD */BUILD BUILD.bazel */BUILD.bazel 2018-03-15 09:19:38 +00:00
Jeff Grafton
fca8bb2928 Autogenerated: hack/update-bazel.sh
Kubernetes-commit: ef56a8d6bb3800ab7803713eafc4191e8202ad6e
2018-02-16 13:43:01 -08:00