Commit Graph

79 Commits

Author SHA1 Message Date
Jordan Liggitt
25d4fa6ba8 Make Request#RequestURI honor configured context root
Kubernetes-commit: 5677f7ab0b1b1ab22a71d01e4a710bf0f5e633aa
2025-04-03 12:51:46 -04:00
Patrick Ohly
5d128adc87 client-go/rest: finish conversion to contextual logging
The remaining calls can be converted without API changes.

Kubernetes-commit: 7821abf2ae289673bbfa3b9a6b8b34f5196c7c7e
2024-09-02 17:55:12 +02:00
Patrick Ohly
7aa9904196 client-go/rest: backoff with context support
The BackoffManager interface sleeps without considering the caller's context,
i.e. cancellation is not supported. This alone is reason enough to deprecate it
and to replace it with an interface that supports a context parameter.

The other reason is that contextual logging needs that parameter.

Kubernetes-commit: b15a1943d51adfb8c5e0185d58d25e038c3d6ade
2024-09-02 20:18:47 +02:00
Patrick Ohly
2b2015d460 client-go/test: warning handler with contextual logging
The default handler now uses contextual logging. Instead of

     warnings.go:106] warning 1

it now logs the caller of client-go and uses structured, contextual
logging

     main.go:100] "Warning" message="warning 1"

Users of client-go have the choice whether the handler that they provide uses
the traditional API (no API break!) or contextual logging.

Kubernetes-commit: 48fb886325fce4b16e4067caadb7bcd3044d460f
2024-09-02 17:51:01 +02:00
Monis Khan
1647efd5c4 client-go/rest: move content type wiring from client to request
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: fe1eda0649fdb6a15200d279be214c67246d3b12
2024-08-18 15:01:57 -04:00
Patrick Ohly
3d02d42465 client-go/rest: contextual logging of request/response
Logging in rest.Request.Body cannot be made context-aware without an API
change. Such a change is complicated if done in a backwards-compatible
fashion (must change lots of callers in Kubernetes) and prohibitive if not (all
callers of Body would have to pass a context).

Instead, logging of the request body gets moved into the functions which send
the request. This is a change of behavior, but it is limited to log levels >= 8
and thus should have no impact in production.

A request which gets sent multiple times will also log the body multiple
times. This might even be a good thing because it serves as reminder what is
being sent when it is being sent.

While at it, stack backtracing gets enhanced so that the caller of the REST API
is logged and tests for the new behavior get added.

Kubernetes-commit: 57f9b7c7a2412865e7817dbf7638881b00ac9721
2024-08-29 19:44:55 +02:00
Quan Tian
009514ecc1 Enrich the error returned from Request.Watch method
The Error method of the error returned from Request.Watch was "unknown"
even the server returned clear message in the Status struct. It was
because Request.Watch used the Result's err member directly, which is an
unstructured error from the response which the Result object may use if
the caller did not return a structured error.

The patch fixes it by calling the Result's Error method instead, which
returns the structured error when it's present.

It also removes the wrong expectation about events.

Kubernetes-commit: 596c5696c64023808af164284263647d795b0ac2
2024-07-22 17:13:43 +08:00
aimuz
39f724a06e cleanup: duplicate request_test TestRequestAbsPathPreservesTrailingSlash
Signed-off-by: aimuz <mr.imuz@gmail.com>

Kubernetes-commit: 7da7468a3fc3f914c7b032c25c3e35097fd85449
2023-05-15 23:14:30 +08:00
HirazawaUi
84dc0417b2 fix fd leaks and failed file removing for pkg client-go
Kubernetes-commit: 73aeed8766c2c42a6cb4fc8632b1b974f4508dde
2023-05-03 01:35:16 +08:00
Stephen Kitt
20433f9965 api-machinery: replace intstr.FromInt with intstr.FromInt32
This touches cases where FromInt() is used on numeric constants, or
values which are already int32s, or int variables which are defined
close by and can be changed to int32s with little impact.

Signed-off-by: Stephen Kitt <skitt@redhat.com>

Kubernetes-commit: 94410ee8078971b8894e5c400ce3fc79f02bc510
2023-03-14 16:17:48 +01:00
Tim Hockin
12553015e2 Replace uses of ObjectReflectDiff with cmp.Diff
ObjectReflectDiff is already a shim over cmp.Diff, so no actual output
or behavior changes

Kubernetes-commit: bc302fa4144d21a338683cd83701661f97be4aba
2023-03-23 11:34:03 -07:00
Stephen Kitt
22e2a9b6b1 client-go: add an Error() function on Request
Requests can accumulate errors with no obvious indication, e.g. if
their primary purpose is to construct a URL: URL() itself doesn't
return an error if r.err is non-nil.

Instead of changing URL() to return an error, which has quite a large
impact, add an Error() function and indicate on URL() that it should
be checked.

Signed-off-by: Stephen Kitt <skitt@redhat.com>

Kubernetes-commit: f69c1c47463ff70ad61adf6f38c4d5b7373e9d0a
2023-02-10 14:44:07 +01:00
Abu Kashem
d2388d199c client-go: add metric to count retries
Kubernetes-commit: b6c369f5c90bbef8058b3d44949f4e815dd6607f
2022-02-28 12:22:09 -05:00
Abu Kashem
67f1485c1d client-go: add unit test for Request thread safety
Kubernetes-commit: ca492a9b62a586caf3e3cd22bc020108a63fb395
2022-08-26 13:24:36 -04:00
Abu Kashem
79c4b4060a client-go: add test to document retry conditions
Kubernetes-commit: 68c542d52246b0b750e332616884350164018695
2022-05-27 09:13:30 -04:00
Jordan Liggitt
e2f402cda3 Limit request retrying to []byte request bodies
Kubernetes-commit: 69fad419a7666ac416307de4c3ee88c7e61b94db
2022-11-15 17:47:35 -05:00
André Martins
c501ee0eba Revert "client-go: remove no longer used finalURLTemplate"
The functionality provided by the finalURLTemplate is still used by
certain external projects to track the request latency for requests
performed to kube-apiserver.

Using a template of the URL, instead of the URL itself, prevents the
explosion of label cardinality in exposed metrics since it aggregates
the URLs in a way that common URLs requests are reported as being the
same.

This reverts commit bebf5a608f68523fc430a44f6db26b16022dc862.

Signed-off-by: André Martins <aanm90@gmail.com>

Kubernetes-commit: f8f190cdd2fa76296f8b1b019ac77128b5d40b79
2022-08-08 23:27:45 +02:00
inosato
27de641f75 Remove ioutil from client-go
Signed-off-by: inosato <si17_21@yahoo.co.jp>

Kubernetes-commit: 88dfa51b6003c90e8f0a0508939a1d79950a40df
2022-07-30 20:54:41 +09:00
zhoumingcheng
84e1219642 Correct some wrong syntax
Signed-off-by: zhoumingcheng <zhoumingcheng@beyondcent.com>

Kubernetes-commit: e590ab77dcb68486a154b755aae658509b9290ea
2022-07-12 10:46:56 +08:00
saltbo
7c3fa18b11 fix: update the typo code comment
Kubernetes-commit: d2bab218ddef3fc3f444038420dd500c0e8bc068
2022-07-13 10:16:51 +08:00
Abu Kashem
71b8c174e4 client-go: fix backoff delay
Kubernetes-commit: 60e74a9ff73b206e5dd439b5df51ac9826635579
2022-05-17 17:19:26 -04:00
Abu Kashem
d8531f5ff0 client-go: make retry in Request thread safe
Kubernetes-commit: 6618b8ef7c0b552839555d4578b64427d20524ef
2022-03-29 13:09:26 -04:00
Abu Kashem
01ab7fb211 client-go: reset request body after response is read and closed
This commit refactors the retry logic to include resetting the
request body. The reset logic will be called iff it is not the
first attempt. This refactor is nescessary mainly because now
as per the retry logic, we always ensure that the request body
is reset *after* the response body is *fully* read and closed
in order to reuse the same TCP connection.

Previously, the reset of the request body and the call to read
and close the response body were not in the right order, which
leads to race conditions.

This commit also adds a test that verifies the order in which
the function calls are made to ensure that we seek only after
the response body is closed.

Co-authored-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 68c8c458ee8f6629eef806c48c1a776dedad3ec4
2022-03-28 20:40:49 -04:00
Abu Kashem
7c9347d386 client-go: wrap error from previous attempt to provide more context
Kubernetes-commit: 868b5a31d382b325f49e1b831e6b094282d50cc7
2022-03-14 18:57:56 -04:00
Jordan Liggitt
83029168cc Temporarily disable TestCheckRetryClosesBody
Kubernetes-commit: 9c13a27d182c36296b7f2e381970b82c69d5c405
2022-03-24 10:11:06 -04:00
Abu Kashem
34f3aff43e client-go: refactor retry logic for backoff, rate limiter and metric
Kubernetes-commit: cecc563d3b9a9438cd3e6ae1576baa0a36f2d843
2022-02-17 16:57:45 -05:00
Abu Kashem
e4ecde2cf0 client-go: add unit test to verify order of calls
Kubernetes-commit: f6a66bbe051d2c7d537cf9033cc800babbb1a74b
2022-02-21 13:53:23 -05:00
Antonio Ojea
83bb1e3ff2 client-go: remove no longer used finalURLTemplate
The restclient metrics were updated to track only the host field of the
url, the finalURLTemplate is not longer needed, its only goal was to
replace name and namespace in the path to avoid cardinality.

Kubernetes-commit: bebf5a608f68523fc430a44f6db26b16022dc862
2022-02-17 16:51:55 +01:00
Antonio Ojea
115ee0e475 bump TestHTTP1DoNotReuseRequestAfterTimeout timeout
the test TestHTTP1DoNotReuseRequestAfterTimeout has to wait for
request to time out to assert that subsequent requests does not
reuse the TCP connection.

It seems that current value of 100ms causes issues on some CI
environments and bumping the timeout seems to solve this flakiness,

We can bump the timeout value because is really low compared to real
scenarios and the bump still keeps it in the millisecond order.

Kubernetes-commit: 85797eba7075d83b116b5c91ff3b17c2d5118d01
2021-11-29 15:37:31 +01:00
Antonio Ojea
34c99305b5 remove unused fakeUpgradeConnection
Kubernetes-commit: bf9ce7fd76068903e909358d2b25b05da7e4a431
2021-11-16 22:54:18 +01:00
Antonio Ojea
d62dc4e20e no lint unused variables
Kubernetes-commit: 0019f986130fdd8ca17c6e5511b15168d9181b1e
2021-11-16 19:00:31 +01:00
Antonio Ojea
f34af1a39f CloseIdleConnections for wrapped Transport
It iterates over the wrapped transports until it finds one
that implements the CloseIdleConnections method and executes it.

add test for closeidle http1 connections

add test for http1.1 reconnect with inflight request

add test to reuse connection request

add test for request connect after timeout

add test for client-go request concurrency

Kubernetes-commit: b9d865a8185b62d83e9ff81b0e3499a26ac6960d
2021-09-10 15:27:23 +02:00
wojtekt
608b5216df Migrate to k8s.io/utils/clock in client-go - part 1
Kubernetes-commit: 950f655bdd19c5af275846cf1940d767a5ec0115
2021-09-15 10:58:56 +02:00
wojtekt
8d49a0b940 Migrate to k8s.io/utils/clock in flowcontrol backoff
Kubernetes-commit: 4ce452989bef21ab6d15bc659f463d8c706ad33e
2021-09-10 09:37:30 +02:00
Abu Kashem
fc210d907d client-go: fix flake in test TestRequestWatchWithRetry
Kubernetes-commit: e797a5a1989373275311b745faf0ff97f5d61000
2021-06-04 15:12:48 -04:00
Abu Kashem
bbd71da939 Revert "Merge pull request #102581 from liggitt/revert-watch-retry"
This reverts commit 5a59a43957c6743995dac67fdda42bf8e0a9ca77, reversing
changes made to 81b9789eaa7bc067f417b5e74d5695dd6dd88a46.

Kubernetes-commit: 892d4fabb845e2461e3655aa414beb6ac322fc99
2021-06-04 13:45:26 -04:00
Jordan Liggitt
5ba99a7f80 Revert "client-go: add retry logic for Watch and Stream"
This reverts commit 607d3819498e64d969407c3d7cbbb8f53d98f0d4.

Kubernetes-commit: e35af41a1236943b6510a25cfb8cb47855aaa16e
2021-06-04 01:11:25 -04:00
Abu Kashem
55854fadb1 client-go: add retry logic for Watch and Stream
Kubernetes-commit: 607d3819498e64d969407c3d7cbbb8f53d98f0d4
2021-05-18 15:15:28 -04:00
Abu Kashem
239ec44d74 client-go: refactor retry logic so it can be reused
Kubernetes-commit: 5fdf196b4e9bbba036a43f4c3b5d9ed8af1672cc
2021-05-20 16:54:19 -04:00
Jordan Liggitt
61471be615 Deflake ThrottledLogger test
Kubernetes-commit: c84df5d66ea8d949374b67c5205b9869b984ea84
2020-11-19 13:59:27 -05:00
André Martins
2396a52017 client-go/rest: fix finalURLTemplate for url base == "/"
In some environments, where url base is "/", it can cause all paths to
be presented in metrics with "{prefix}" as `groupIndex` is with the wrong
index. To fix the behavior in such environments, it was added a
conditional branch to check if the URL base is "/" and, thus, print the
metrics with the correct path, for example "api/v1/nodes/{name}" instead
of "{prefix}".

Fixes: 99248b8fe1fe ("Rewrite finalURLTemplate used only for metrics because of dynamic client change")
Signed-off-by: André Martins <aanm90@gmail.com>

Kubernetes-commit: c039b02fa7281fc061455e23b6530ed8b4d19645
2019-02-21 20:25:36 +01:00
Davanum Srinivas
75fea27a27 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
yue9944882
3e9d5317f0 make client-side max-retry override'ble
Kubernetes-commit: f373697c64e9dc5a6618e1d360ad9a51bc8b79c7
2020-03-27 16:15:46 +08:00
jennybuckley
2d3138825e Improve rate limiter latency logging and metrics
Kubernetes-commit: 2bcf99f05fdc47fb4bc3601b9134408483f59773
2020-02-13 13:32:12 -08:00
Mike Danese
09009e85ca cleanup req.Context() and ResponseWrapper
Kubernetes-commit: 968adfa99362f733ef82f4aabb34a59dbbd6e56a
2020-01-27 18:52:27 -08:00
Clayton Coleman
9bbcc2938d Always negotiate a decoder using ClientNegotiator
This commit performs two refactors and fixes a bug.

Refactor 1 changes the signature of Request to take a RESTClient, which
removes the extra copy of everything on RESTClient from Request. A pair
of optional constructors are added for testing. The major functional
change is that Request no longer has the shim HTTPClient interface and
so some test cases change slightly because we are now going through
http.Client code paths instead of direct to our test stubs.

Refactor 2 changes the signature of RESTClient to take a
ClientContentConfig instead of ContentConfig - the primary difference
being that ClientContentConfig uses ClientNegotiator instead of
NegotiatedSerializer and the old Serializers type. We also collapse
some redundancies (like the rate limiter can be created outside the
constructor).

The bug fix is to negotiate the streaming content type on a Watch()
like we do for requests. We stop caching the decoder and simply
resolve it on the request. We also clean up the dynamic client
and remove the extra WatchSpecificVersions() method by providing
a properly wrapped dynamic client.

Kubernetes-commit: 3b780c64b89606f4e6b21f48fb9c305d5998b9e5
2019-11-10 16:52:08 -05:00
clarklee92
d3a5e5f798 Modify the status code number to HTTP status semantics
Signed-off-by: clarklee92 <clarklee1992@hotmail.com>

Kubernetes-commit: f86f5ee14ef3c8adf9855ce16dcc57beca949719
2019-11-06 00:45:35 +08:00
Ricardo Maraschini
e6a1dc4b13 Creating function for preflight check.
Migrated code that checks for common programmer errors to a separated
function and added test coverage for it. Wrong comment stating that a
typed error is returned was also removed.

Kubernetes-commit: ad5fafd6ade2838098890a4e7727c8e347686867
2019-09-12 19:22:46 +02:00
Odin Ugedal
2ee65257e5 Fix file permissions for non executable files
Normal files should have permissions 644 by default,
and does not require the last bit to be
executable

Signed-off-by: Odin Ugedal <odin@ugedal.com>

Kubernetes-commit: 35cb87f9cf71776e99a970dfff751cd29ba7ebfb
2019-06-14 21:04:39 +02:00
wojtekt
deab738de6 Fix client-go request benchmark
Kubernetes-commit: 85763127c0b9399e6c48ec144c4716543a6cbe2d
2019-06-27 17:18:56 +02:00