Commit Graph

7906 Commits

Author SHA1 Message Date
yue9944882
563ab26819 addressing reviews from lavalamp/deads2k 2019-09-26 01:06:01 -04:00
yue9944882
9f5903d351 rename post-start-hooks 2019-09-26 01:06:01 -04:00
yue9944882
1a9c427757 minor fixes 2019-09-26 01:06:01 -04:00
yue9944882
21439b64d1 refactor/inline event-clock to fairqueuing package 2019-09-26 01:06:01 -04:00
yue9944882
ea845aa59a review: addressing yliaog pt2 2019-09-26 01:06:01 -04:00
yue9944882
f34166c922 [generated] run hack scripts 2019-09-26 01:06:01 -04:00
yue9944882
bfa9b4e50c review: addressing comments from yliaog 2019-09-26 01:06:01 -04:00
yue9944882
d2da381bbf [PATCH] Fixed the debug log format in flowcontrol 2019-09-26 01:06:01 -04:00
yue9944882
cf399d2e7a [PATCH] Fix bugs and names in API priority and fairness 2019-09-26 01:06:01 -04:00
yue9944882
c1259dee97 prometheus metrics 2019-09-26 01:06:01 -04:00
yue9944882
79b587fe8d refactor and reviews
fixes insecure handler chaining

reviews

revert clock

review clock
2019-09-26 01:06:01 -04:00
MikeSpreitzer
04388e22c5 Fix protobuf tag for Subjects in PolicyRuleWithSubjects 2019-09-26 01:06:00 -04:00
Mike Spreitzer
9b093f5639 Introduce NameAll in prioritization and fairness config models 2019-09-26 01:06:00 -04:00
Mike Spreitzer
05abd0c07d More work on shuffle sharding utils
Changes following up on PR #807810 .

Made the validation checking function return a slice of error messages
rather than just a bit.

Replaced all the `int32` with `int` because this is intended for more
than just the priority-and-faireness feature and so should not be a
slave to its configuration datatypes.

Introduced ShuffleAndDealIntoHand, to make memory allocation the
caller's problem/privilege.

Made the hand uniformity tester avoid reflection, evaluate the
histogram against the expected range of counts, and run multiple test
cases, including one in which the number of hash values is a power of
two with four extra bits (as the validation check requires) and one in
which the deck size is not a power of two.

Updated the fairqueuing implementation to use the shuffle sharding
utility package.
2019-09-26 01:06:00 -04:00
yue9944882
639be9133f ensure flow-control api enabled 2019-09-26 01:06:00 -04:00
Mike Spreitzer
313597b96b More uniform and informative debugging of request handling 2019-09-26 01:06:00 -04:00
Mike Spreitzer
fda5d9277c Better goroutine tracking in the API priority and fairness filter
Introduced `OptionalWaitGroup` as a convenient abstraction of
`*WaitGroup`.

Made fairqueuing and emptyRelay::HandleEmpty update the
OptionalWaitGroup and plumbed requestManagementSystem to be able to do
that. Making HandleEmpty update the OptionalWaitGroup illustrates how
a test function that is testing the quiesce functionality of a
QueueSet would do that. This is also a step toward making
requestManagementSystem update the OptionalWaitGroup properly when
there are configuration changes; the remaining steps will include
plumbing the OptionalWaitGroup through the work queue and its worker.

Even without this change it is already possible to test the rest of
QueueSet functionality without any special entry methods. It is also
easy to test the functionality of a requestManagementSystem excepting
its config controller.
2019-09-26 01:06:00 -04:00
yue9944882
77edfe378c fixes some problems in status syncing 2019-09-26 01:06:00 -04:00
Aaron Prindle
fb9dc1b2bb add fair queuing implementation and passing test suite 2019-09-26 01:06:00 -04:00
yue9944882
7a156d2914 flow-control system
compute flow-distinguisher

config controller

plumbs default-confifuration into post-start-hook

addressing reviews

apiserver flow-control
2019-09-26 01:06:00 -04:00
yue9944882
99a4e51fee fixes local-up failures
disable server-side printing

fix test
2019-09-26 01:06:00 -04:00
Bruce Ma
6e0f50c974 make benchmark tests ever-changing and add uniform distribution test
Signed-off-by: Bruce Ma <brucema19901024@gmail.com>
2019-09-26 01:06:00 -04:00
Mike Spreitzer
a8e9921880 Start drafting a unit test of QueueSet
The test runs a scenario over time and evaluates fairness of the
result.

There is no real QueueSet implementation now, so only a dummy is
tested.

In other news, this change introduces a dummy QueueSet implementation
that exerts no control.
2019-09-26 01:06:00 -04:00
Bruce Ma
e7f224651c some refactors on shuffle sharding
Signed-off-by: Bruce Ma <brucema19901024@gmail.com>
2019-09-26 01:06:00 -04:00
Bruce Ma
3328046a34 shuffle sharding: add some benchmarks and tests
Signed-off-by: Bruce Ma <brucema19901024@gmail.com>
2019-09-26 01:06:00 -04:00
Bruce Ma
1d30ce12bc add shuffle sharding utils
Signed-off-by: Bruce Ma <brucema19901024@gmail.com>
2019-09-26 01:06:00 -04:00
Mike Spreitzer
3926050733 Continue refactoring started in PR 80628
Moved the top-level logic associated with the data in
k8s.io/apiserver/pkg/util/flowcontrol into the same package, greatly
simplifying its interface.

Also plumbed the call to NewRequestManagementSystem (1) into
`RecommendedOptions::ApplyTo(RecommendedConfig)` for running in
aggregated servers and (2) into buildGenericConfig in the
kube-apiserver code.

Also moved the definitions of FairQueuingSystem and FairQueuingFactory
into the directory where Aaron is developing his implementation of
those.

Also did some renaming to satisfy golint: FairQueuingSytem became
QueueSet, and FairQueuingFactory became QueueSetFactory.

Also realized that the clock should be passed in every Create... call
to the Factory, the Factory should hold the clock to use (as well as
the WaitGroup that will be helpful for testing).
2019-09-26 01:06:00 -04:00
yue9944882
7b6ef2555c refactor request management filter 2019-09-26 01:05:59 -04:00
Aaron Prindle
dfadb6c9ff update generated code 2019-09-26 01:05:59 -04:00
Aaron Prindle
e17f0671dd update api/flowcontrol/v1alpha1 to have correct Group and Version 2019-09-26 01:05:59 -04:00
Mike Spreitzer
c42702f87a Simplified FairQueuingSystem interface
.. and introduced a tiny bit of starting to work with config objects.
2019-09-26 01:05:59 -04:00
Mike Spreitzer
faf3723ea6 A little more thought about concurrency in the reqmgmt filter
(1) removed the idea of multiple config queue workers; it will be
simpler and adequate if there is only 1.

(2) attempted to clarify the comment on FairQueuingSystem::DoOnEmpty
to (a) make the division of responsibilities clearer, (b) make a
precise statement of the semantics, and (c) be less prescriptive of
how the filter uses this method.

(this is the third edition of this change)
2019-09-26 01:05:59 -04:00
Mike Spreitzer
c79ec2fcf3 Expanded comment on FairQueuingSystem::DoOnEmpty 2019-09-26 01:05:59 -04:00
Mike Spreitzer
1c8f89b554 update bazel by hack 2019-09-26 01:05:59 -04:00
Mike Spreitzer
89caade3cf Started drafting skeleton of the request management filter 2019-09-26 01:05:59 -04:00
yue9944882
24f7323248 [DO NOT REVIEW]: trival codes 2019-09-26 01:05:59 -04:00
yue9944882
601da29814 versioned flow-control api model 2019-09-26 01:05:59 -04:00
Kubernetes Prow Robot
fb1db2eba7 Merge pull request #82957 from RainbowMango/pr_handle_stability_level_default_better
Handle stability level default better
2019-09-25 19:43:07 -07:00
Kubernetes Prow Robot
45f7f70479 Merge pull request #80465 from answer1991/remove-apiserver-loopback-client-qps-limit
remove apiserver loopback client QPS limit
2019-09-25 17:15:07 -07:00
Kubernetes Prow Robot
28bcf55acc Merge pull request #83113 from yastij/bump-utils
bump k8s.io/utils to pickup ipallocator changes
2019-09-25 13:52:36 -07:00
Kubernetes Prow Robot
ea9a559093 Merge pull request #83075 from yutedz/pod-ready-time
Remove unnecessary traversal of pod.Status.Conditions
2019-09-25 13:52:03 -07:00
Kubernetes Prow Robot
6555c34743 Merge pull request #83022 from mm4tt/controller_ref
Optimize GetControllerOf method and add a benchmark for it
2019-09-25 13:51:14 -07:00
Kubernetes Prow Robot
08f46e425b Merge pull request #82932 from ashish-billore/mybranch2
Corrected the pod reporting and messages
2019-09-25 11:07:13 -07:00
Mahendra Kariya
3698100224 Fix golint errors in pkg/apis/core (#82919)
* Fix lint errors related to receiver name

Ref #68026

* Fix lint errors related to comments

Ref #68026

* Fix package name in comments

Ref #68026

* Rename Cpu to CPU

Ref #68026

* Fix lint errors related to naming convention

Ref #68026

* Remove deprecated field

DoNotUse_ExternalID has been deprecated and is not in use anymore.
It has been removed to fix lint errors related to underscores in field
names.

Ref #68026, #61966

* Include pkg/apis/core in golint check

Ref #68026

* Rename var to fix lint errors

Ref #68026

* Revert "Remove deprecated field"

This reverts commit 75e9bfc168077fcb9346e334b59d60a2c997735b.

Ref #82919

* Remove math from godoc

Ref #82919, #68026

* Remove underscore from var name

Ref #68026

* Rename var in staging core api type

Ref #68026
2019-09-25 11:06:51 -07:00
Kubernetes Prow Robot
dd8a23ca2e Merge pull request #82643 from ZP-AlwaysWin/dev02
Fix bug The statefulset have duplicate revision after resource was up…
2019-09-25 11:06:16 -07:00
Kubernetes Prow Robot
973629f742 Merge pull request #82410 from nikhita/informer-gen-package-name-dots
[v1.16.1] informer-gen: allow package names containing dots
2019-09-25 11:05:50 -07:00
Kubernetes Prow Robot
07025a5d9e Merge pull request #82409 from RainbowMango/pr_add_authentication_overall_latency_metrics
Add authentication overall latency metrics
2019-09-25 11:05:32 -07:00
Kubernetes Prow Robot
0676af36c5 Merge pull request #82388 from marun/unstructured-reflector
Add support for type checking Unstructured event objects via GVK to client-go's reflector
2019-09-25 11:05:14 -07:00
Kubernetes Prow Robot
327f53ba57 Merge pull request #83064 from liggitt/propagate-context
Propagate context to remote authorize/authenticate webhook calls
2019-09-25 09:32:01 -07:00
Yassine TIJANI
16fe4d7627 bump k8s.io/utils to pickup ipallocator changes
Signed-off-by: Yassine TIJANI <ytijani@vmware.com>
2019-09-25 16:15:30 +02:00