Automatic merge from submit-queue (batch tested with PRs 61203, 61071). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix issue with race condition during pod deletion
This PR fixes two issues
1. When desired_state_populator removes podvolume state, it should check
whether the actual state already has the volume before deleting it to
make sure actual state has a chance to add the volume into the state
2. When checking podVolume still exists, it not only checks the actual
state, but also the volume disk directory because actual state might not
reflect the real world when kubelet starts.
fixes issue #60645
Automatic merge from submit-queue (batch tested with PRs 61203, 61071). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix deprecated gcloud compute networks --mode switches.
"create --mode" becomes "create --subnet-mode", and switch-mode has been
folded into "update".
Create --mode was deprecated in October and will be removed in the next
gcloud release. It is already failing in staging tests.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** Fixes#54238
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
This PR fixes two issues
1. When desired_state_populator removes podvolume state, it should check
whether the actual state already has the volume before deleting it to
make sure actual state has a chance to add the volume into the state
2. When checking podVolume still exists, it not only checks the actual
state, but also the volume disk directory because actual state might not
reflect the real world when kubelet starts.
Automatic merge from submit-queue (batch tested with PRs 60888, 61225). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump fluentd-gcp-scaler version
**What this PR does / why we need it**:
This version verifies on its own whether resources should be updated or not, instead of relying on `kubectl set resources`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61190
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @shyamjvs
Automatic merge from submit-queue (batch tested with PRs 60888, 61225). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Mark reconstructed volumes as reported InUse
When a newly started kubelet finds a directory where a volume should be,
it can be fairly confident that the volume was mounted by previous kubelet
and therefore the volume must have been in node.status.volumesInUse.
Therefore we can mark reconstructed volumes as already reported so
subsequent reconcile() can fix the directory and put the mounted volume
into actual state of world.
Fixes: #60645
**Release note**:
```release-note
NONE
```
/sig storage
/sig node
cc: @gnufied @jingxu97
"create --mode" becomes "create --subnet-mode", and switch-mode has been
folded into "update".
Create --mode was deprecated in October and will be removed in the next
gcloud release. It is already failing in staging tests.
Automatic merge from submit-queue (batch tested with PRs 61118, 60579). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increase loging verbosity for deleting stateful set pods
We should always log reasons for deleting StatefulSet Pods.
@jdumars - what's the current process for putting such changes into the release? It's literally 0-risk change that helps with debugging.
cc @ttz21
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61118, 60579). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increase apiserver mem-threshold in density test
Ref: https://github.com/kubernetes/kubernetes/issues/60500#issuecomment-372682659 (fixes part of that issue)
/sig scalability
/kind bug
/priority important-soon
/cc @wojtek-t
/cc @crassirostris (for the release-note)
```release-note
Audit logging with buffering enabled can increase apiserver memory usage (e.g. up to 200MB in 100-node cluster). The increase is bounded by the buffer size (configurable). Ref: issue #60500
```
Automatic merge from submit-queue (batch tested with PRs 61129, 60359). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cleanup old upgrading code that is v1.8->v1.9-specific
**What this PR does / why we need it**:
Cleanup old upgrading code that is v1.8->v1.9-specific
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/622
This will finish the task in the issue.
**Special notes for your reviewer**:
/cc @luxas @vbmade2000
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61111, 61069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use pod UID as cache key instead of namespace/name
UID uniquely identifies pods across lifecycles, while namespace/name
could be 2 different pods across lifecycles. This could result in
tricky scheduler bugs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60966
**Special notes for your reviewer**: @bsalamat
**Release note**:
```release-note
Fix a bug in scheduler cache by using Pod UID as the cache key instead of namespace/name
```
Automatic merge from submit-queue (batch tested with PRs 61111, 61069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix subpath e2e tests on multizone cluster.
Use dynamically provisioned PV to run GCE PD tests. This will make sure that the pod is scheduled to the right zone and GCE PD can be attached to a node.
**Which issue(s) this PR fixes**:
Fixes#61101
**Release note**:
```release-note
NONE
```
/sig storage
@msau42 @verult
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Find most recent modified date for fluentd buffers recursively.
Fixes#60762
**What this PR does / why we need it**:
Due to updates in Fluent v0.14, the buffers directory modified date is no
longer updated when files inside the directory are changed. Therefore we
must find the most recent modified date recursively to fix liveness probe.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix broken gke regional logging test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60882
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Detect backsteps correctly in base path detection
Avoids false positives with atomic writer `..<timestamp>` directories
Fixes#61076
/assign @msau42 @jsafrane
```release-note
Fix a regression that prevented using `subPath` volume mounts with secret, configMap, projected, and downwardAPI volumes
```
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix default auditing options.
- Log backend defaults to blocking mode (backwards compatability)
- Webhook backend defaults to throttled
- Fix webhook validation
- Add options test
**Which issue(s) this PR fixes**:
Fixes#60719
**Special notes for your reviewer**:
This PR is an alternative fix to https://github.com/kubernetes/kubernetes/pull/60727. If the rollback goes in first, I'll rebase this on a roll-forward.
**Release note**:
-->
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update Kubelet command option description for IPv6
**What this PR does / why we need it**:
The restriction for a /66 cidr was removed in PR #60089.
Removing this reference from the command options description.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60734
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
UID uniquely identifies pods across lifecycles, while namespace/name
could be 2 different pods across lifecycles. This could result in
tricky scheduler bugs.
Fixes#60966
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump to etcd 3.1.12 to pick up critical fix
etcd [3.1.12](https://github.com/coreos/etcd/releases/tag/v3.1.12) (as well as 3.2.17 and 3.3.2) was released yesterday to fix a bug critical to kubernetes:
Fix [mvcc "unsynced" watcher restore operation](https://github.com/coreos/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/coreos/etcd/issues/9086).
This will be backported to 1.9 as well.
Release note:
```release-note
Upgrade the default etcd server version to 3.1.12 to pick up critical etcd "mvcc "unsynced" watcher restore operation" fix.
```
cc @gyuho @wojtek-t @shyamjvs @timothysc @jdumars
Automatic merge from submit-queue (batch tested with PRs 61004, 60981). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use grpc to improve the CPU utilization of the logging agent.
Fixes#60762
**What this PR does / why we need it**:
Using gRPC improves the CPU utilization of the logging agent be reducing
serialization overhead and reusing TCP connections.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix option --audit-webhook-initial-backoff
Before this change, --audit-webhook-initial-backoff has no effect
@crassirostris @sttts
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Use dynamically provisioned PV to run GCE PD tests. This will make sure
that the pod is scheduled to the right zone and GCE PD can be attached
to a node.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Exclude commas when pulling the tag out of the git export-subst format string
**What this PR does / why we need it**: the version tag is not guaranteed to be the last item in the ref names substituted into the format string, so we need to be sure not to match on the trailing comma.
For example, v1.9.3 was exported with `HEAD -> release-1.9, tag: v1.9.3, origin/release-1.9`, where we only want to match `v1.9.3`, not `v1.9.3,`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61029, though we need to backport this to all active branches.
**Release note**:
```release-note
NONE
```
/assign @david-mcmahon
/priority important-soon
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added missing error check that could cause kubelet to crash
**What this PR does / why we need it**:
Adds missing error check. An error can happen due to a race condition when watched files change, or become inaccessible. This can happen if a file was added to the driver directory then quickly removed, in which case the callback will be called with non-nil `err` and nil `info`, which is not checked, causing kubelet to crash.
**Which issue(s) this PR fixes**:
Fixes#60861
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add missing container-runtime "remote" option
**What this PR does / why we need it**:
Added the "remote" option to the auto-generated documentation for the
`--container-runtime` flag.
The kubelet flag `--container-runtime` lists the possible values as part of the auto-generated documentation but is missing the "remote" possibility.
**Which issue(s) this PR fixes** :
Fixes#60992
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60772, 61011, 61053). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet initial flag parse should normalize flags instead of exiting
**What this PR does / why we need it**:
Sets normalize func for kubelet cleanFlagSet, otherwise kubelet will exit if non-normalized flag is used. For example, this happened on a kubelet running v1.10.0-beta.2:
```
F0312 17:59:03.138986 25317 server.go:144] unknown flag: --cluster_dns
```
This is where the kubelet exits: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L142-L145. This would be a breaking change for anyone upgrading Kubernetes and using `_` in their flag names.
```release-note
kubelet initial flag parse should normalize flags instead of exiting.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Set readOnly for CSI mounter
**What this PR does / why we need it**:
Currently the `csiMountMgr .readOnly` field is never set, we should set it to `Spec.ReadOnly`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61008
**Special notes for your reviewer**:
Currently, most of the volume plugins use a `getVolumeSourceFromSpec` method to fetch `VolumeSource` and `ReadOnly` from `volume.Spec`. If the volume is an inline volume, `ReadOnly` is fetched from `Spec.Volume.<SpecificVolumeSource>.ReadOnly`, and if the volume is a `PersistentVolume`, `ReadOnly` is set to `Spec.Readonly`, which comes from `PersistentVolumeClaimVolumeSource.ReadOnly`.
However, as CSI volume plugin is only supported in `PersistentVolume`, so we can just set `ReadOnly` to `Spec.ReadOnly`.
**Release note**:
```release-note
NONE
```
/sig storage
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Code cleanup: group consts togather
**What this PR does / why we need it**:
This is a code cleanup, which groups all consts togather.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Due to updates in Fluent v0.14, the buffers directory modified date is no
longer updated when files inside the directory are changed. Therefore we
must find the most recent modified date recursively to fix liveness probe.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Setting REMOUNT_VOLUME_PLUGIN_DIR for COS images in kube-env
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60725
**Special notes for your reviewer**: Not sure if it's the best place to set `REMOUNT_VOLUME_PLUGIN_DIR`.
/sig storage
/sig cluster-lifecycle
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update documentation for azure-shared-securityrule
**What this PR does / why we need it**:
Azure augmented rules for NSGs has been GA https://azure.microsoft.com/en-us/updates/agumented-rules-ga-nsg/. This PR updates documentation for "service.beta.kubernetes.io/azure-shared-securityrule" to reflect this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
