Automatic merge from submit-queue (batch tested with PRs 64181, 65737). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
API linter framework and namesMatch API rule
**What this PR does / why we need it**:
Bump kube-openapi dependency to use the [API linter framework](https://github.com/kubernetes/kube-openapi/pull/83) in k/k OpenAPI spec generation procedure.
Currently one API rule is enforced:
"Go field names must be CamelCase. JSON field names must be camelCase. Other than capitalization of the initial letter, the two should almost always match. No underscores nor dashes in either."
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65244
**Special notes for your reviewer**:
Most code change in this PR was generated (~1700 lines). Please see commits for detail.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/cc @pwittrock @mbohlool
Kubernetes-commit: 614e3adda05c5d7189c8485ba7abe1ba7ee501a7
Automatic merge from submit-queue (batch tested with PRs 65899, 65900). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiserver: use self-signed cert fixtures in integration test servers
Kubernetes-commit: 932766231342f1f7d2b8d3063b139c46a8bf9f2b
Automatic merge from submit-queue (batch tested with PRs 65052, 65594). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Derive kubelet serving certificate CSR template from node status addresses
xref https://github.com/kubernetes/features/issues/267fixes#55633
Builds on https://github.com/kubernetes/kubernetes/pull/65587
* Makes the cloud provider authoritative when recording node status addresses
* Makes the node status addresses authoritative for the kube-apiserver determining how to speak to a kubelet (stops paying attention to the hostname label when determining how to reach a kubelet, which was only done to support kubelets < 1.5)
* Updates kubelet certificate rotation to be driven from node status
* Avoids needing to compute node addresses a second time, and differently, in order to request serving certificates.
* Allows the kubelet to react to changes in its status addresses by updating its serving certificate
* Allows the kubelet to be driven by external cloud providers recording node addresses on the node status
test procedure:
```sh
# setup
export FEATURE_GATES=RotateKubeletServerCertificate=true
export KUBELET_FLAGS="--rotate-server-certificates=true --cloud-provider=external"
# cleanup from previous runs
sudo rm -fr /var/lib/kubelet/pki/
# startup
hack/local-up-cluster.sh
# wait for a node to register, verify it didn't set addresses
kubectl get nodes
kubectl get node/127.0.0.1 -o jsonpath={.status.addresses}
# verify the kubelet server isn't available, and that it didn't populate a serving certificate
curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
ls -la /var/lib/kubelet/pki
# set an address on the node
curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
-H "Content-Type: application/merge-patch+json" \
--data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"}]}}'
# verify a csr was submitted with the right SAN, and approve it
kubectl describe csr
kubectl certificate approve csr-...
# verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname, but NOT the IP
curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
ls -la /var/lib/kubelet/pki
# set an hostname and IP address on the node
curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
-H "Content-Type: application/merge-patch+json" \
--data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"},{"type":"InternalIP","address":"127.0.0.1"}]}}'
# verify a csr was submitted with the right SAN, and approve it
kubectl describe csr
kubectl certificate approve csr-...
# verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname AND IP
curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
ls -la /var/lib/kubelet/pki
```
```release-note
* kubelets that specify `--cloud-provider` now only report addresses in Node status as determined by the cloud provider
* kubelet serving certificate rotation now reacts to changes in reported node addresses, and will request certificates for addresses set by an external cloud provider
```
Kubernetes-commit: 337dfe0a9cde3894eb6a26f9184df659d54007c6
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Handle errors in generated client scheme
**What this PR does / why we need it**:
Adds missing error handling for schema construction code in generated clientsets.
**Which issue(s) this PR fixes**
Together with #64654fixes#51457.
**Release note**:
```release-note
NONE
```
/kind bug
/sig api-machinery
Kubernetes-commit: 0e7376d9cb1ec74bcb3c10a664a330bc9d60df64
Automatic merge from submit-queue (batch tested with PRs 65946, 65904, 65913, 65906, 65920). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
track schemes by name for error reporting
Getting an error message about a type not being in the scheme is hard to fix if you don't know which scheme is failing. This adds a name to the scheme which can be set during creation or can be set based on the calling stack. If you use the old constructor a name is generated for you based on the stack. Something like "k8s.io/client-go/dynamic/scheme.go:28" for instance.
Also moves a typer to its point of use. This was debt from previous refactors which I noticed going through.
@kubernetes/sig-api-machinery-misc
@sttts
```release-note
NONE
```
Kubernetes-commit: 8e2fdb32bc84103b15310a221a375470bf567bdc
Automatic merge from submit-queue (batch tested with PRs 65805, 65811, 65833, 65488, 65857). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix print format string
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 7597e952d17882dca2b464d56cf744f126a037fa
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typos for TestBackoffHighWaterMark
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Kubernetes-commit: 692f9bb7b1fa6ca72ddd5a305607d79f9684e907
Automatic merge from submit-queue (batch tested with PRs 65648, 65700, 64976, 65692, 65667). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
leader election: fix dead loop bug
**What this PR does / why we need it**:
bug fix: leader election calling apiserver without time interval. It is a dead loop.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65666
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Kubernetes-commit: fc081b5941b90aa6dedbc796f97a19da373c5df9
Automatic merge from submit-queue (batch tested with PRs 65094, 65533, 63522, 65694, 65702). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set leader election client and renew timeout
**What this PR does / why we need it**:
set leader-election client timeout
set timeout for tryAcquireOrRenew
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#65090#65257
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 47020f331877ca77f0541c7622c33894282e9a20
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add missing error handling in schema-related code
**What this PR does / why we need it**:
Adds missing error handling to a few places.
**Which issue(s) this PR fixes**
Updates #51457. Still more work to do to fix the issue - client generation code needs to be updated (addressed in https://github.com/kubernetes/kubernetes/pull/64664).
**Release note**:
```release-note
NONE
```
/kind bug
/sig api-machinery
Kubernetes-commit: 7786bd8c9a99974e2cda31940dd4a1ef0a31c2e5
Automatic merge from submit-queue (batch tested with PRs 65518, 65624, 65380, 65390, 65586). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cleanup: remove deadcode
/kind cleanup
remove deadcode
remove unnecessary conversion
```release-note
NONE
```
Kubernetes-commit: 16dbb95c51532e8a937a3ccb90223e8568b130c3
Automatic merge from submit-queue (batch tested with PRs 63773, 65411). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
client-go/examples/fake-client: add doc.go to fix go build warnings
The publishing bot is currently broken (https://github.com/kubernetes/kubernetes/issues/56876#issuecomment-399763681). This is due to the following error:
```
[24 Jun 18 15:08 UTC]: Running smoke tests for branch master
[24 Jun 18 15:08 UTC]: /bin/bash -xec "godep restore\ngo build ./...\ngo test $(go list ./... | grep -v /vendor/)\n"
+ godep restore
+ go build ./...
go build k8s.io/client-go/examples/fake-client: no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client
[24 Jun 18 15:09 UTC]: exit status 1
+ godep restore
+ go build ./...
go build k8s.io/client-go/examples/fake-client: no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client
[24 Jun 18 15:09 UTC]: exit status 1```
```
The fakeclient package does not have any non-test go files. The test file was added in https://github.com/kubernetes/kubernetes/pull/65291 2 days ago.
This causes `go build` to give a warning: `no non-test Go files in /go-workspace/src/k8s.io/client-go/examples/fake-client`, which breaks the publishing bot. This PR adds a dummy doc.go file in the package to avoid this warning and fix the publishing bot.
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 7a9a30a4f7eae900bffd4b477a431a999de350dd
Automatic merge from submit-queue (batch tested with PRs 64122, 64936, 65288, 65383). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update to rules_go 0.12.1 and gazelle 0.12.0 and perform related cleanups
**What this PR does / why we need it**: my initial intent was to simply update to rules_go 0.12.1 and gazelle 0.12.0.
A few internal changes / deprecations meant that I finally needed to clean up some technical debt. This also fixes#64122.
I've attempted to keep the steps as separate commits to make it easier to review:
1. Disable gazelle proto rule generation; legacy proto rules are deprecated, and we don't (currently) build protos at build time anyway, instead generating them with `hack/update-generated-protobuf.sh` and then checking them in. We can revisit this in the future if we'd like.
2. Remove the legacy `go_default_library_protos` filegroups using [buildozer](https://github.com/bazelbuild/buildtools/tree/master/buildozer). We don't use these, anyway.
3. Update the rules_go bazel workspace dependency to 0.12.1.
4. Vendor gazelle 0.12.0 and update BUILD files with `hack/update-bazel.sh`. This causes a lot of diffs, because `select()`s are no longer used in `srcs` attributes, external tests are folded into non-external tests, and vendored targets get an `importmap` attribute.
5. Set `gazelle:prefix` on `staging/src/BUILD` to make gazelle treat these correctly(ish). This allows us to remove the sed rewrite hack in `hack/update-bazel.sh`.
6. Explicitly set `# gazelle:importmap_prefix k8s.io/kubernetes/vendor` on `vendor/`, so that all vendored dependencies get the right importmap. gazelle 0.12.0 uses the bazel workspace name + `vendor/` as a prefix, which doesn't work with native go. Newer gazelle will use the go prefix (https://github.com/bazelbuild/bazel-gazelle/pull/207), but it's not released yet. Setting this correctly now also fixes later `BUILD` churn.
7. Re-run `hack/update-bazel.sh`. This causes a bunch of diffs, since anything under `staging/src` now uses the `staging/src/` path instead of `vendor/`. (Both would work for bazel, but gazelle uses the former, since `vendor/` uses symlinks.) Also `importmap`s under `vendor/` are fixed.
8. Reformat a few files (using [buildifier](https://github.com/bazelbuild/buildtools/tree/master/buildifier)) to make later diffs easier to read.
9. Rework the `go_genrule` rules to use the new `go_genrule` from https://github.com/kubernetes/repo-infra/pull/72, which is more bazely, since it uses the rules_go `go_path` rule instead of lots of shell.
10. Remove the deprecated `go_prefix` rule from the root BUILD.bazel file.
11. Set `# gazelle:importmap_prefix k8s.io/kubernetes/vendor` on `staging/src` as well, which ensures that these repos are treated as vendored dependencies. (It's basically the bazel-y way of doing the `vendor/k8s.io` symlinks.)
12. Run `hack/update-bazel.sh` one last time to fix all of the `importmap`s under `staging/src`.
Note re: point 6 above - we're pretty much ignoring the `vendor/k8s.io` symlinks entirely now under bazel. Using the `gazelle:prefix` directive ensures these get mapped into the right go importpath, and the `go_path` rule installs these correctly now too.
**Special notes for your reviewer**: this should not be submitted before https://github.com/kubernetes/repo-infra/pull/72, obviously.
**Release note**:
```release-note
NONE
```
/assign @BenTheElder @fejta @thockin
cc @cblecker @jayconrod
Kubernetes-commit: 1ad1c8c7f80d99b9625924b2102a04a555162bfb
Automatic merge from submit-queue (batch tested with PRs 65301, 65291, 65307, 63845, 65313). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add client-go example using fake client in test.
**What this PR does / why we need it**:
This adds an example showing the steps needed to get a working
SharedInformerFactory with a fake client for testing.
**Special notes for your reviewer**:
I had a really hard time figuring out how to use this utility correctly. I don't think this example is sufficient documentation, but it's a good start.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind documentation
Kubernetes-commit: 8008c7dad945f7ac0748774a99148c864932288e
The fakeclient package does not have any non-test go files. This
causes `go build` to give a warning: `no non-test Go files in ...`.
This also causes the publishing bot to fail.
This PR adds a dummy doc.go file in the package to avoid this warning.
Kubernetes-commit: 764ae69671428987d77898b2d3c48c6c003e2fd4
Automatic merge from submit-queue (batch tested with PRs 64895, 64938, 63700, 65050, 64957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
enhance leaderelection code
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 8850871891f345628f01dbef4ede727d74dd2113
Automatic merge from submit-queue (batch tested with PRs 65256, 64236, 64919, 64879, 57932). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cancellable leader election
**What this PR does / why we need it**:
Adds ability to cancel leader election. Useful in integration tests where the whole app is started and stopped in each test.
**Special notes for your reviewer**:
I used the `context` package - it is impossible/hard to achieve the same behaviour with just channels without spawning additional goroutines but it is trivial with `context`. See `acquire()` and `renew()` methods.
**Release note**:
```release-note
NONE
```
/kind enhancement
/sig api-machinery
Kubernetes-commit: 571b9beac5bdaa65fb581bead7464926fa81cdbf
Automatic merge from submit-queue (batch tested with PRs 65256, 64236, 64919, 64879, 57932). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix CRD OpenAPI schema
fixes#65243
depends on https://github.com/kubernetes/kube-openapi/pull/84
without this PR, kubectl complains about creating this CRD with a validation schema (which worked in 1.10):
```yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: resources.mygroup.example.com
spec:
group: mygroup.example.com
version: v1alpha1
scope: Namespaced
names:
plural: resources
singular: resource
kind: Kind
listKind: KindList
validation:
openAPIV3Schema:
properties:
spec:
type: array
items:
type: number
```
> error: error validating "/Users/jliggitt/projects/snippets/crd/crd.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec.validation.openAPIV3Schema.properties.spec.items): unknown field "type" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaPropsOrArray, ValidationError(CustomResourceDefinition.spec.validation.openAPIV3Schema.properties.spec.items): missing required field "Schema" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaPropsOrArray, ValidationError(CustomResourceDefinition.spec.validation.openAPIV3Schema.properties.spec.items): missing required field "JSONSchemas" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaPropsOrArray]; if you choose to ignore these errors, turn validation off with --validate=false
that is because the types used to serialize JSONSchema require custom marshaling/unmarshaling, and the OpenAPI generator was not informed of that, so it produced this:
```json
{
"io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaPropsOrArray": {
"description": "JSONSchemaPropsOrArray represents a value that can either be a JSONSchemaProps or an array of JSONSchemaProps. Mainly here for serialization purposes.",
"required": [
"Schema",
"JSONSchemas"
],
"properties": {
"JSONSchemas": {
"type": "array",
"items": {
"$ref": "#/definitions/io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaProps"
}
},
"Schema": {
"$ref": "#/definitions/io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.JSONSchemaProps"
}
}
}
}
```
OpenAPI isn't able to represent oneOf/anyOf types correctly currently. Until it can, we definitely shouldn't publish a schema containing required fields which aren't even part of the JSON serialization. This PR implements custom openapi type functions, which omit the properties/required/schema attributes for four specific JSONSchema types. This allows kubectl to continue creating these objects without complaining.
/sig api-machinery
/assign @sttts
```release-note
fixed incorrect OpenAPI schema for CustomResourceDefinition objects
```
Kubernetes-commit: ed6c8b7326bd1a1b845719f4bfb302073a18f03f
This adds an example showing the steps needed to get a working
SharedInformerFactory with a fake client for testing.
Kubernetes-commit: aed487b42d1b49df9ac01f7234896f6d1b4da166
Automatic merge from submit-queue (batch tested with PRs 64416, 63625, 60967, 64767, 64588). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Collapse memcached discovery client onto parallelized discovery method
```release-note
NONE
```
Kubernetes-commit: f637a259ea2852a15a4321fd059972c55aaefc9b
Automatic merge from submit-queue (batch tested with PRs 64688, 64451, 64504, 64506, 56358). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
replace fmt.Sprintf(%s, i.Type()) with i.Type().String()
**What this PR does / why we need it**:
/kind cleanup
replace fmt.Sprintf(%s, i.Type()) with i.Type().String()
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Kubernetes-commit: b309d18a48d8054904fe88fdc49cae33a276f722
Automatic merge from submit-queue (batch tested with PRs 64142, 64426, 62910, 63942, 64548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove inaccurate comment about watch timeout
when watch timeout, the tcp connection does not close, it is reused for another new watch.
But this makes users misunderstanding.
**Release note**:
```release-note
NONE
```
Kubernetes-commit: d0a2bfd22e8c129e0a156a4304a62e3563dc6a59
