Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make easy validation admission plugins into validators
This switches "easy" admission plugins that need to be validators to be validators. I also did one simple plugin to do both mutation and validation for practice.
@kubernetes/sig-api-machinery-pr-reviews @caesarxuchao
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Node autoprovisioning e2e test.
This PR adds test scenario for cluster-autoscaler in GKE for node autoprovisioning.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dependency on drv_cfg binary for querying ScaleIO devices
**What this PR does / why we need it**:
This PR fixes the issue where a ScleIO kubernetes plugin required additional binary `drv_cfg` to be present on the local node to work properly, making it harder for the kubelet binary to be containerized without providing access to local paths inside the container.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54954
**Special notes for your reviewer**:
**Release note**:
```release-note
ScaleIO driver completely removes dependency on drv_cfg binary so a Kubernetes cluster can easily run a containerized kubelet.
```
Automatic merge from submit-queue (batch tested with PRs 54906, 54120, 54934, 54915, 54848). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve unit test coverage for kube-proxy configuration.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/cc @ncdc
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54906, 54120, 54934, 54915, 54848). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve unit test coverage for network policy validation.
**What this PR does / why we need it**:
`ok k8s.io/kubernetes/pkg/apis/networking/validation 0.074s coverage: 87.8% of statements`
to
`ok k8s.io/kubernetes/pkg/apis/networking/validation 0.082s coverage: 100.0% of statements`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54906, 54120, 54934, 54915, 54848). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Only parse ClusterCIDR, ServiceCIDR if AllocateNodeCIDRs
**What this PR does / why we need it**:
Avoid unnecessary spam in kube-controller-manager log if --cluster-cidr is not specified and --allocate-node-cidrs is false. Add clarification in kube-controller-manager help about that.
**Release note**
```release-note
Avoid unnecessary spam in kube-controller-manager log if --cluster-cidr is not specified and --allocate-node-cidrs is false.
```
Automatic merge from submit-queue (batch tested with PRs 54906, 54120, 54934, 54915, 54848). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix kube-proxy fall back to iptables firstly when unable to load kernel modules
**What this PR does / why we need it**:
Fix kube-proxy fall back to iptables firstly when unable to load kernel modules
**Which issue this PR fixes**:
fixes#54121
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/kind bug
/sig network
/area kube-proxy
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update storageclass version to v1 in examples
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
/cc @jsafrane
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up legacy ipv4/32 in ipvs proxy
**What this PR does / why we need it**:
clean up legacy ipv4/32 in ipvs proxy
**Which issue(s) this PR fixes**: closes#51866
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig network
/area ipv6
/assign @thockin
cc @danehans
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kubeadm way in ipvs proxy README
**What this PR does / why we need it**:
As #53962 which support specify proxy mode for kubeadm is already in, we should add ipvs proxy kubeadm way in README.md.
**Which issue(s) this PR fixes**:
Fixes#54978
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
StopPodSandbox should not log when container is already removed
**What this PR does / why we need it**:
StopPodSandbox should not log when a container is already gone. It should only log if it could not stop and the container was still present.
Fixes https://github.com/kubernetes/kubernetes/issues/55021
**Special notes for your reviewer**:
This was seen in our production logs, need to eliminate spam.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reduce metadata-proxy cpu requests to 30m
After the recent change enabling metadata-proxy in tests (https://github.com/kubernetes/kubernetes/pull/54150) we started seeing problems with scheduling cluster autoscaler on master. Metadata-proxy eats all of the available space leaving nothing for CA to run on.
This PR reduces the cpu requests for metadata-proxy allowing other components to fit in.
cc: @kubernetes/sig-autoscaling-bugs
Automatic merge from submit-queue (batch tested with PRs 54488, 54838, 54964). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support to for alternative container runtime in `kube-up.sh`
For https://github.com/kubernetes/features/issues/286.
This PR added 4 new environment variables in `kube-up.sh` to support alternative container runtime:
1) `KUBE_MASTER_EXTRA_METADATA` and `KUBE_NODE_EXTRA_METADATA`. Add extra metadata on master and node instance. With this we could specify different cloud-init for a different container runtime, and also add extra metadata for the new cloud-init, e.g. [master.yaml](7d73966214/test/e2e/master.yaml)
2) `KUBE_CONTAINER_RUNTIME_ENDPOINT`. Specify different sock for different container runtime. It's only used when it's not empty.
3) `KUBE_LOAD_IMAGE_COMMAND`. Specify different load image command for different container runtime.
An example for cri-containerd:
```
export KUBE_MASTER_EXTRA_METADATA="user-data=${GOPATH}/src/github.com/kubernetes-incubator/cri-containerd/test/e2e/master.yaml,cri-containerd-configure-sh=${GOPATH}/src/github.com/kubernetes-incubator/cri-containerd/test/configure.sh"
export KUBE_NODE_EXTRA_METADATA="user-data=${GOPATH}/src/github.com/kubernetes-incubator/cri-containerd/test/e2e/node.yaml,cri-containerd-configure-sh=${GOPATH}/src/github.com/kubernetes-incubator/cri-containerd/test/configure.sh"
export KUBE_CONTAINER_RUNTIME="remote"
export KUBE_CONTAINER_RUNTIME_ENDPOINT="/var/run/cri-containerd.sock"
export KUBE_LOAD_IMAGE_COMMAND="/home/cri-containerd/usr/local/bin/cri-containerd load"
export NETWORK_POLICY_PROVIDER="calico"
```
Signed-off-by: Lantao Liu <lantaol@google.com>
```release-note
none
```
/cc @yujuhong @dchen1107 @feiskyer @mikebrow @abhi @mrunalp @runcom
/cc @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add admission handler for device resources allocation
**What this PR does / why we need it**:
Add admission handler for device resources allocation to fail fast during pod creation
**Which issue this PR fixes**
fixes#51592
**Special notes for your reviewer**:
@jiayingz Sorry, there is something wrong with my branch in #51895. And I think the existing comments in the PR might be too long for others to view. So I closed it and opened the new one, as we have basically reach an agreement on the implement :)
I have covered the functionality and unit test part here, and would set about the e2e part ASAP
/cc @jiayingz @vishh @RenaudWasTaken
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
discovery client not depend on pkg/api/legacyscheme
Switch kubectl's cached discovery client to use kubect's scheme (pkg/kubectl/scheme) which only registered with external version types.
The encoding and decoding here uses only `k8s.io/apimachinery/pkg/apis/meta/v1.APIGroupList` and `k8s.io/apimachinery/pkg/apis/meta/v1.APIResourceList` which are not internal version. So it should be safe.
```release-note
NONE
```
/assign @monopole
Automatic merge from submit-queue (batch tested with PRs 55016, 50887). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
If command.Execute() return err, print to stdErr
The current kube-proxy not print error if (execute() failed) . not so good and not so friendly to user.
If print err to stdError, will show us why it failed.
Automatic merge from submit-queue (batch tested with PRs 55016, 50887). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup resouce test
- Make client-go testing support decoding to external version
- Cleanup bad dependency of `pkg/kubectl/resource` pkg
```release-note
NONE
```
/assign @monopole @caesarxuchao
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a file store utility package in kubelet
More and more components checkpoints (i.e., persist their states) in
kubelet. Refurbish and move the implementation in dockershim to a
utility package to improve code reusability.
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Lift embedded structure out of ManifestURLHeader field
Related: #53833
```release-note
It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files.
```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support retainkeys strategy for new merge code
- Some prefactoring for retainkeys
- Add retainkeys strategy
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should check and return err when visit failure
**What this PR does / why we need it**:
should check and return err when visit failure
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cmd: genutils: remove entry from .golint_failures
**What this PR does / why we need it**:
`golint` emits warnings
```
exported function OutDir should have comment or be unexported
strings should not be capitalized or end with punctuation or a newline
```
- Add documentation comment to exported function OutDir.
- Remove newline from error string.
- Remove `hack\.golint_failures` entry for `cmd/genutils`
**Special notes for your reviewer**:
Don't know which sig to use?
**Release note**:
```release-note
NONE
```
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Basic GCE PodSecurityPolicy Config
**What this PR does / why we need it**:
This PR lays the foundation for enabling PodSecurityPolicy in GCE and other default deployments. The 3 commits are:
1. Add policies, roles & bindings for the default addons on GCE.
2. Enable the PSP admission controller & load the addon policies when the`ENABLE_POD_SECURITY_POLICY=true` environment variable is set.
3. Support the PodSecurityPolicy in the E2E environment & add PSP tests.
NOTES:
- ~~Depends on https://github.com/kubernetes/kubernetes/pull/52301 for privileged capabilities~~
- ~~Depends on https://github.com/kubernetes/kubernetes/pull/52849 for sane mutations~~
- ~~Depends on https://github.com/kubernetes/kubernetes/pull/53479 for aggregator tests to pass~~
- ~~Depends on https://github.com/kubernetes/kubernetes/pull/54175 for dedicated fluentd service~~ account
- This PR is a fork of https://github.com/kubernetes/kubernetes/pull/46064, credit to @Q-Lee
**Which issue this PR fixes**: #43538
**Release note**:
```release-note
Add support for PodSecurityPolicy on GCE: `ENABLE_POD_SECURITY_POLICY=true` enables the admission controller, and installs policies for default addons.
```
Automatic merge from submit-queue (batch tested with PRs 55004, 54957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update kube-openapi to use validation
**What this PR does / why we need it**: Moves openapi validation code to kube-openapi, so that we can move the rest of the code to apimachinery repository, so that later we can use it from both the client and the server.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #Nothing
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55004, 54957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Explicitly set route_localnet on nodes & masters.
Allow for loopback addresses to be used for routing, specifically to
enable metadata proxy on master nodes.
**What this PR does / why we need it**: Enables metadata proxy (#8867) to work on the master. This is already being done on the nodes by kube-proxy, but this makes it explicit, and sets it on the master where kube-proxy doesn't run. Thanks to @dnardo for figuring this out!
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: fixes#54736
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
ClusterCIDR and ServiceCIDR are settings that are only used if at least
AllocateNodeCIDRs is set. The route controller requires in addition to
it for ConfigureCloudRoutes to be true as well. Since
AllocateNodeCIDRs is by default false, if guard the parsing of these
settings in order to not unnecessarily spam logs. Amend the
documentation of kube-controller-manager for the 2 settings to point
out the requirement of AllocateNodeCIDRs to be true as well
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
admission: wire through validating-only admission
Based on https://github.com/kubernetes/kubernetes/pull/54232.
This is important and required for beta because it affects the shape of the webhook admission plugins that are going to be produced and is needed to make sure that our existing chain continues to properly verify and protect the API objects based on their final state after webhook admission mutators run.
We discussed this in the October 11 API machinery call with @erictune and @caesarxuchao and we agreed to do this as a requirement for beta. See this part of the recording: https://www.youtube.com/watch?v=mrgDPHyr4VY#t=325 .
Automatic merge from submit-queue (batch tested with PRs 54800, 53898, 54812, 54921, 53558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix error for strategic merge patch of custom resources
Fixes#50037.
We need the go struct tags `patchMergeKey` and `patchStrategy` for fields that support a strategic merge patch. For native resources, we can easily figure out these tags since we know the fields.
Because custom resources are decoded as Unstructured and because we're missing the metadata about how to handle each field in a strategic merge patch, we can't find the go struct tags. Hence, we can't easily do a strategic merge for custom resources.
So we should fail fast and return an error.
**Release note**:
```release-note
NONE
```
/cc @sttts @deads2k @ncdc
Automatic merge from submit-queue (batch tested with PRs 54800, 53898, 54812, 54921, 53558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
rename metric reflector_xx_last_resource_version
**What this PR does / why we need it**:
mv reflector name from metric name to metric label
before:
```
reflector_k8s_io_kubernetes_pkg_client_informers_informers_generated_internalversion_factory_go:73_8664_last_resource_version{instance="104.154.20.21:443",job="kubernetes-apiservers"}
```
after
```
reflector_last_resource_version{instance="10.0.2.15:6443",job="kubernetes-apiservers",name="k8s_io_kubernetes_pkg_client_informers_informers_generated_internalversion_factory_go_73_8664"}
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#52121
**Special notes for your reviewer**:
None
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54800, 53898, 54812, 54921, 53558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow override of cluster level (default, whitelist) tolerations by namespace level empty (default, whitelist) tolerations.
Currently In PodTolerationRestriction admission plugin, if namespace level default and whitelist of tolerations are nil or empty, they do not override cluster level default and whitelist tolerations.
This PR fixes the plugin to not override cluster level tolerations only when namespace level toleration are nil. IOW, if namespace level toleration are empty, they override cluster level tolerations. To be more clear, if following annotations are set to empty, they override cluster level tolerations.
```
scheduler.alpha.kubernetes.io/defaultTolerations : ""
scheduler.alpha.kubernetes.io/tolerationsWhitelist: ""
```
This behavior is inline with PodNodeSelector admission plugin too.
@sjenning @derekwaynecarr
**Release Note**:
```release-note
In PodTolerationRestriction admisson plugin, if namespace level tolerations are empty, now they override cluster level tolerations.
```
Automatic merge from submit-queue (batch tested with PRs 54800, 53898, 54812, 54921, 53558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
NewProxierWithNoProxyCIDR: fix handling IPv6 URLs
**What this PR does / why we need it**:
Current logic of splitting hostname from URL does not work if URL
is for IPv6 address and does not explicitly specify port number.
Example: "https://[2001:db8::1]/".
Use standard library function to get hostname out of URL string.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
- Fix handling of IPv6 URLs in NO_PROXY.
```
/area ipv6
/sig api-machinery
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubenet: yield lock while executing CNI plugin.
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
This caused problems with the pod latency tests - previously, CNI plugins executed
in under 20ms. Now they must wait for DAD to finish and addresses to leave
tentative state.
Fixes: #54651
**What this PR does / why we need it**:
After upgrading CNI plugins to v0.6 in #51250, the pod latency tests began failing. This is because the plugins, in order to support IPv6, need to wait for DAD to finish. Because this
delay is while the kubenet lock is held, it significantly slows down the pod creation rate.
**Special notes for your reviewer**:
The CNI plugins also do locking for their critical paths, so it is safe to run them concurrently.
**Release note**:
```release-note
NONE
```
