Automatic merge from submit-queue
Add a heap to client-go. Heap orders items with heap invariant ordering.
**What this PR does / why we need it**:
Heap is useful in implementing priority queues. Some components may need such ordering to process their highest priority objects first. Scheduler is going to be the first user of the heap. It will store pending pods ordered by their priority, so that the highest priority pods are popped first to be scheduled.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
ref/ #47604
ref/ #48646
@kubernetes/api-reviewers @kubernetes/sig-scheduling-pr-reviews @davidopp
/assign @caesarxuchao
Automatic merge from submit-queue
Simplify a command for unmounting mounted directories
**What this PR does / why we need it**:
This PR simplifies a command for unmounting mounted directories. It replaces `cat | awk | grep` by a single `awk` expression.
**Special notes for your reviewer**:
Link about useless cat/grep usage: http://porkmail.org/era/unix/award.html#grep
**Credit**:
I saw this code fragment in the Julia Evans (@jvns) [write up](https://jvns.ca/blog/2017/07/30/a-couple-useful-ideas-from-google/).
**Release note**:
```release-note
NONE
```
CC @simo5
Automatic merge from submit-queue
Task 2: Added toleration to DaemonSet pods for node condition taints
**What this PR does / why we need it**:
If TaintByCondition was enabled, added toleration to DaemonSet pods for node condition taints.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
remove the duplicate address of glusterfs
**What this PR does / why we need it**:
remove the duplicate endpoint address of glusterfs.
If there is no duplicate address of user config or the duplicate address is ok, we can remove the unused addr map.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
Typedef visitor to document parameters
adds a typedef to clarify the parameters of the visitor
updates the unit test to verify each namespace/name pair
Automatic merge from submit-queue
add fuzzer dir for each apigroup
Fixes https://github.com/kubernetes/kubernetes/issues/50081.
We should put fuzzer functions in each apigroups separately to make the code more modular instead of monolithic.
Automatic merge from submit-queue
code format in master_utils.go
**What this PR does / why we need it**:
code format
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
add some checks for fedration-apiserver options
**What this PR does / why we need it**:
I find there is a TODO, see https://github.com/kubernetes/kubernetes/blob/master/federation/cmd/federation-apiserver/app/options/validation.go#L30
This PR add some checks for fedration-apiserver options
@sttts
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Rewrite staging import verifier in Go
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
**What this PR does / why we need it**:
Maintaining Bash is nobody's idea of fun. The declarative config is infinitely easier to read, as well.
**Special notes for your reviewer**:
@sttts still trying to figure out the rules you have encoded in the shell script... very confusing. Please review the `kube-gen` specifically.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
move logs to kubectl/util
Move `pkg/util/logs` to `pkg/kubectl/util/logs` per https://github.com/kubernetes/kubernetes/issues/48209#issuecomment-311730681
This will make kubeadm, kubefed, gke-certificates-controller and e2e have dependency on kubectl, which should be fine.
partially addresses: kubernetes/community#598
```release-note
NONE
```
/assign @apelisse @monopole
Automatic merge from submit-queue
Remove deprecated ESIPP beta annotations
**What this PR does / why we need it**:
Remove deprecated ESIPP beta annotations.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50187
**Special notes for your reviewer**:
/assign @MrHohn
/sig network
**Release note**:
```release-note
Beta annotations `service.beta.kubernetes.io/external-traffic` and `service.beta.kubernetes.io/healthcheck-nodeport` have been removed. Please use fields `service.spec.externalTrafficPolicy` and `service.spec.healthCheckNodePort` instead.
```
Automatic merge from submit-queue
Added field CollisionCount to StatefulSetStatus
**What this PR does / why we need it**:
This PR added a new field `CollisionCount` into `StatefulSetStatus`, similarly in terms of both name and semantics to the existing `CollisionCount` field in `DaemonSetStatus`. The field will be used for collision avoidance when the `StatefulSet` controller creates name for the newest ControllerRevision, which will be done in another PR.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: #49909.
**Special notes for your reviewer**:
A second PR will include logic that actually uses the field for collision avoidance.
**Release note**:
```release-note
Added field CollisionCount to StatefulSetStatus in both apps/v1beta1 and apps/v1beta2
```
Automatic merge from submit-queue
add newline for unimplemented RequiredDuringSchedulingRequiredDuringExecution
**What this PR does / why we need it**:
Unimplemented `RequiredDuringSchedulingRequiredDuringExecution` should not be treated as comments of `RequiredDuringSchedulingIgnoredDuringExecution`
* [pkg/api/types.go#L1937](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go#L1937)
* [staging/src/k8s.io/api/core/v1/types.go#L2155](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/core/v1/types.go#L2155)
> // NOT YET IMPLEMENTED. TODO: Uncomment field once it is implemented.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:fixes #50483
**Special notes for your reviewer**:
* Add an empty line to separate them.
/cc @aveshagarwal @bsalamat @gyliu513 @k82cn @timothysc
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
FC volume plugin: Support WWID for volume identifier
**What this PR does / why we need it**:
This PR adds World Wide Identifier (WWID) parameter to FCVolumeSource as an unique volume identifier.
**Which issue this PR fixes**: fixes#48639
**Special notes for your reviewer**:
/cc @rootfs @jsafrane @msau42
**Release note**:
```
FC volume plugin: Support WWID for volume identifier
```
Automatic merge from submit-queue
Migrate to controller references helpers in meta/v1
**What this PR does / why we need it**:
This is a follow up for #48319 that migrates all method usages to new methods in meta/v1.
**Special notes for your reviewer**:
Looking at each commit individually might be easier.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind cleanup
Automatic merge from submit-queue
Fix my incorrect username
My mistake - used goog username rather than github.
Again, this is for kubectl extraction, currently blocked by need for many approvers in, e.g. #48580#48581#47011, etc.
Automatic merge from submit-queue
kubeadm: enhanced TLS validation for token-based discovery in `kubeadm join`
**What this PR does / why we need it**:
This PR implements enhanced TLS validation for `kubeadm join` when using token-based TLS discovery. Without this enhancement, `kubeadm join` has some less-than-ideal security properties. Specifically, in the case where a bootstrap token is compromised, the attacker can impersonate the API server to newly bootstrapping clients ([more discussion in the design proposal](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a)).
The gist of this enhancement is to support public key pinning in the style of [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4). When bootstrapping, `kubeadm` can now be configured with a whitelist of root CA public keys. It can then validate that the cluster it connects to is operated by the owner of one of those public keys.
These public key hashes are short enough that the entire `kubeadm join` command can still be copy-pasted relatively easily (not as easily as before, but ~160 characters). Using a public key hash rather than a hash over the entire certificate allows certificates to be reissued with updated expirations without invalidating existing key pins.
This change adds two new command line flags (and associated config parameters):
- **`--discovery-token-ca-cert-hash sha256:<hash>`:**
Validates that the cluster root CA has a public key fingerprint that matches one of the specified values. If this flag is not passed when token-based discovery is being used, a warning is printed. This warning will become an error in 1.9.
- **`--discovery-token-unsafe-skip-ca-verification`:**
Disables the warning message when no keys are pinned. In 1.9, this flag will be required _unless_ `--discovery-token-unsafe-skip-ca-verification` is used.
This is fully backwards compatible and client side (kubeadm) only. It will be a breaking change when the flag becomes required in v1.9.
This validation is done after and in addition to the existing bootstrap token signing/MAC mechanism.
#### Example from `kubeadm init`:
```
$ kubeadm init
[...]
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
```
#### Example from `kubeadm join`:
```
$ kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "192.168.42.10:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.42.10:6443"
[discovery] Requesting info from "https://192.168.42.10:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.42.10:6443"
[discovery] Successfully established connection with API Server "192.168.42.10:6443"
Node join complete:
* Certificate signing request sent to master and response
received.
* Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
```
**Which issue this PR fixes**:
ref https://github.com/kubernetes/features/issues/130
fixes: https://github.com/kubernetes/kubeadm/issues/365
**Special notes for your reviewer**:
This was proposed and discussed briefly by SIG-cluster-lifecycle and SIG-auth. The design proposal is [in Google Docs](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a).
There is a documentation change needed to explain the security properties of `kubeadm join` with and without `--discovery-token-ca-cert-hash`. This page should be linked by to by the warning message when you don't pass either of the new flags (I have it pointing [here](https://kubernetes.io/docs/admin/kubeadm/#kubeadm-join) for now, which I think will be the right place). I will follow up with this documentation shortly.
**Release note**:
```release-note
kubeadm: added enhanced TLS validation for token-based discovery in `kubeadm join` using a new `--discovery-token-ca-cert-hash` flag.
```
/cc @luxas @jbeda @ericchiang
This change adds the `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join` and corresponding fields on the kubeadm NodeConfiguration struct. These flags configure enhanced TLS validation for token-based discovery.
The enhanced TLS validation works by pinning the public key hashes of the cluster CA. This is done by connecting to the `cluster-info` endpoint initially using an unvalidated/unsafe TLS connection. After the cluster info has been loaded, parsed, and validated with the existing symmetric signature/MAC scheme, the root CA is validated against the pinned public key set. A second request is made using validated/safe TLS using the newly-known CA and the result is validated to make sure the same `cluster-info` was returned from both requests.
This validation prevents a class of attacks where a leaked bootstrap token (such as from a compromised worker node) allows an attacker to impersonate the API server.
This change also update `kubeadm init` to print the correct `--discovery-token-ca-cert-hash` flag in the example `kubeadm join` command it prints at the end of initialization.
This change adds a `k8s.io/kubernetes/cmd/kubeadm/app/util/pubkeypin` package which implements x509 public key pinning in the style of RFC7469. This is the public key hash format used by the new `kubeadm join --discovery-token-ca-cert-hash` flag.
Hashes are namespaced with a short type, with "sha256" being the only currently-supported format. Type "sha256" is a hex-encoded SHA-256 hash over the Subject Public Key Info (SPKI) object in DER-encoded ASN.1.
Automatic merge from submit-queue
Add Cluster Autoscaler scalability test suite
This suite is intended for manually testing Cluster Autoscaler on large clusters. It isn't supposed to be run automatically (at least for now).
It can be run on Kubemark (with #50440) with the following setup:
- start Kubemark with NUM_NODES=1 (as we require there to be exactly 1 replica per hollow-node replication controller in this setup)
- set kubemark-master machine type manually to appropriate type for the Kubemark cluster size. Maximum Kubemark cluster size reached in test run is defined by maxNodes constant, so for maxNodes=1000, please upgrade to n1-standard-32. Adjust if modifying maxNodes.
- start Cluster Autoscaler pod in the external cluster using image built from version with Kubemark cloud provider (release pending)
- for grabbing metrics from ClusterAutoscaler (with #50382), add "--include-cluster-autoscaler=true" parameter in addition to regular flags for gathering components' metrics/resource usage during e2e tests
cc @bskiba
Automatic merge from submit-queue
Make ClusterID required for AWS.
**What this PR does / why we need it**:
Makes ClusterID required for AWS and provides a flag to run in un-tagged mode
fixes#48954
**Release note**:
```release-note
A cluster using the AWS cloud provider will need to label existing nodes and resources with a ClusterID or the kube-controller-manager will not start. To run without a ClusterID pass --allow-untagged-cloud=true to the kube-controller-manager on startup.
```
