Automatic merge from submit-queue (batch tested with PRs 51590, 48217, 51209, 51575, 48627)
Skip system container cgroup stats if undefined
**What this PR does / why we need it**:
the kubelet /stats/summary endpoint tried to look up cgroup stats for containers that are not required. this polluted logs with messages about not finding stats for "" container. this pr skips cgroup stats if the cgroup name is not specified (they are optional anyway)
**Special notes for your reviewer**:
i think this was a regression from recent refactor.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51590, 48217, 51209, 51575, 48627)
Deviceplugin jiayingz
**What this PR does / why we need it**:
This PR implements the kubelet Device Plugin Manager.
It includes four commits implemented by @RenaudWasTaken and a commit that supports allocation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Design document: kubernetes/community#695
PR tracking: kubernetes/features#368
**Special notes for your reviewer**:
**Release note**:
Extending Kubelet to support device plugin
```release-note
```
Automatic merge from submit-queue
Fix InstanceTypeByProviderID for Azure
**What this PR does / why we need it**:
Fix change in #46940, should return InstanceType in function InstanceTypeByProviderID
Otherwise:
```
I0830 05:01:08.497989 15347 node_controller.go:328] Adding node label from cloud provider: beta.kubernetes.io/instance-type=/subscriptions/{id}/resourceGroups/{id}/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
@brendandburns @realfake
Automatic merge from submit-queue
Adding Flexvolume plugin dir piping for controller manager on COS
**What this PR does / why we need it**: Sets the default Flexvolume plugin directory correctly for controller manager running on COS images.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51563
```release-note
NONE
```
/release-note-none
/sig storage
/assign @msau42
/cc @wongma7
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Update defaults.go
**What this PR does / why we need it**:
This PR is a TODO
Add explain about TaintTolerationPriority function.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Address TestEtcdStoragePath flakes
- Wait for the master to be healthy
- Wait longer for the master to start
- Fail gracefully if starting the master panics
Signed-off-by: Monis Khan <mkhan@redhat.com>
```release-note
NONE
```
Fixes#49423
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
add apps/v1beta2 conversion tests
**What this PR does / why we need it**:
add apps/v1beta2 conversion test
Depend on ~~#49751~~(Merged), ~~#49719~~(Merged)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
/cc @janetkuo
**Release note**:
```release-note
add apps/v1beta2 conversion test
```
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Remove deprecated and experimental fields from KubeletConfiguration
As we work towards providing a stable (v1) kubeletconfig API,
we cannot afford to have deprecated or "experimental" (alpha) fields
living in the KubeletConfiguration struct. This removes all existing
experimental or deprecated fields, and places them in KubeletFlags
instead.
I'm going to send another PR after this one that organizes the remaining
fields into substructures for readability. Then, we should try to move
to v1 ASAP (maybe not v1 in 1.8, given how close we are, but definitely in 1.9).
It makes far more sense to focus on a clean API in kubeletconfig v2,
than to try and further clean up the existing "API" that everyone
already depends on.
fixes: #51657
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50381, 51307, 49645, 50995, 51523)
Bugfix: Use local JSON log buffer in parseDockerJSONLog.
**What this PR does / why we need it**:
The issue described in #47800 is due to a race condition in `ReadLogs`: Because the JSON log buffer (`dockerJSONLog`) is package-scoped, any two goroutines modifying the buffer could race and overwrite the other's changes. In particular, one goroutine could unmarshal a JSON log line into the buffer, then another goroutine could `Reset()` the buffer, and the resulting `Stream` would be empty (`""`). This empty `Stream` is caught in a `case` block and raises an `unexpected stream type` error.
This PR creates a new buffer for each execution of `parseDockerJSONLog`, so each goroutine is guaranteed to have a local instance of the buffer.
**Which issue this PR fixes**: fixes#47800
**Release note**:
```release-note
Fixed an issue (#47800) where `kubectl logs -f` failed with `unexpected stream type ""`.
```
Automatic merge from submit-queue
fix bad url
**What this PR does / why we need it**:
fix bad url in the readme file
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
Split APIVersion into APIGroup and APIVersion in audit events
audit.Event.ObjectRef.APIVersion currently holds both the the API group and
version, separated by a /. This change break these out into separate fields.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
test/e2e/auth: fix audit log test format parsing
Fixes https://github.com/kubernetes/kubernetes/issues/51556
```release-note
NONE
```
cc @CaoShuFeng
Still need to figure out how to run this test locally.
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
fixing a typo in staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go
**What this PR does / why we need it**:
Fix a typo in `staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
`NONE`
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
GCE: Reserve address for ILBs during sync
**What this PR does / why we need it**:
This PR adds the ability for the service controller to hold the ILB's IP during sync which may delete/recreate the forwarding rule.
Fixes: #47531
**Release note**:
```release-note
GCE: Internal load balancer IPs are now reserved during service sync to prevent losing the address to another service.
```
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
[Scheduler] Fix typo in info message
Signed-off-by: sakeven <jc5930@sina.cn>
**What this PR does / why we need it**:
Missed a `'` in info message.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Use json-iterator instead of ugorji for JSON.
@smarterclayton @wojtek-t
Fixes#36120
xref #18762
```release-note
Switch JSON marshal/unmarshal to json-iterator library. Performance should be close to previous with no generated code.
```
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
Separate feature gates for dynamic kubelet config vs loading from a file
This makes it so these two features can be turned on independently, rather than bundling both under dynamic kubelet config.
fixes: #51664
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
Add some initial shell parsing tests.
These just test to see if there is a bash syntax error in these shell
libraries.
For #51642
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
kubectl: Remove ending punctuation from error strings
**What this PR does / why we need it**:
`golint` emits 2 warnings of type:
`error strings should not end with punctuation`
Remove punctuation from end of error strings.
**Release note**:
```release-note
NONE
```
/sig cli
/kind cleanup
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
CPU manager wiring and `none` policy
Blocker for CPU manager #49186 (4 of 6)
* Previous PR in this series: #51140
* Next PR in this series: #51180
cc @balajismaniam @derekwaynecarr @sjenning
**Release note**:
```release-note
NONE
```
TODO:
- [X] In-memory CPU manager state
- [x] Kubelet config value
- [x] Feature gate
- [X] None policy
- [X] Unit tests
- [X] CPU manager instantiation
- [x] Calls into CPU manager from Kubelet container runtime
Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)
add information for subresource kind determination
xref https://github.com/kubernetes/kubernetes/issues/38810https://github.com/kubernetes/kubernetes/issues/38756
Polymorphic subresources usually have different groupVersions for their discovery kinds than their "native" groupVersions. Even though the APIResourceList shows the kind properly, it does not reflect the group or version of that kind, which makes it impossible to unambiguously determine if the subresource matches you and it is impossible to determine how to serialize your data. See HPA controller.
This adds an optional Group and Version to the discovery doc, which can be used to communicate the "native" groupversion of an endpoint. Doing this does not preclude fancier contenttype negotiation in the future and doesn't prevent future expansion from indicating equivalent types, but it does make it possible to solve the problem we have today or polymorphic categorization.
@kubernetes/sig-api-machinery-misc @smarterclayton
@cheftako since @lavalamp is out.
```release-note
Adds optional group and version information to the discovery interface, so that if an endpoint uses non-default values, the proper value of "kind" can be determined. Scale is a common example.
```
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Ensure that DaemonSet respects termination
**What this PR does / why we need it**:
#43077 correctly prevents the DaemonSet controller from adopting deleted Pods, but, as pointed out in #50477, the controller now has no sensitivity to the termination lifecycle (i.e TerminationGracePeriodSeconds) of the Pods it creates. This PR attempts to balance the two. DaemonSet controller will now consider deleted Pods owned by a DaemonSet during creation, but it will not consider deleted Pods as targets for adoption.
fixes#50477
```release-note
#43077 introduced a condition where DaemonSet controller did not respect the TerminationGracePeriodSeconds of the Pods it created. This is now corrected.
```
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix pod local ephemeral storage usage calculation
We use podDiskUsage to calculate pod local ephemeral storage which is not correct, because podDiskUsage also contains HostPath volume which is considered as persistent storage
This pr fixes it
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51489
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @jingxu97 @vishh
cc @ddysher
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix printISCSIVolumeSource to show kubectl describe properly
**What this PR does / why we need it**:
After merging #51189, 'kubectl describe' doesn't show persistent volume properly if the volume is iSCSI.
This PR fixes the problem.
**Which issue this PR fixes** : Fixes#51635
**Special notes for your reviewer**:
Result without InitiatorName.
```
% k describe pv pv0001
Name: pv0001
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-class=slow
StorageClass: slow
Status: Available
Claim:
Reclaim Policy: Recycle
Access Modes: RWO
Capacity: 1Gi
Message:
Source:
Type: ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
TargetPortal: 192.168.122.85:3260
IQN: iqn.2017-05.com.example:rhel7
Lun: 0
ISCSIInterface default
FSType: ext4
ReadOnly: true
Portals: [192.168.122.7:3260]
DiscoveryCHAPAuth: false
SessionCHAPAuth: false
SecretRef: <nil>
InitiatorName: <none>
Events: <none>
```
Result with InitiatorName.
```
% k describe pv pv0001
Name: pv0001
Labels: <none>
Annotations: volume.beta.kubernetes.io/storage-class=slow
StorageClass: slow
Status: Available
Claim:
Reclaim Policy: Recycle
Access Modes: RWO
Capacity: 1Gi
Message:
Source:
Type: ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
TargetPortal: 192.168.122.85:3260
IQN: iqn.2017-05.com.example:rhel7
Lun: 0
ISCSIInterface default
FSType: ext4
ReadOnly: true
Portals: [192.168.122.7:3260]
DiscoveryCHAPAuth: false
SessionCHAPAuth: false
SecretRef: <nil>
InitiatorName: iqn.1994-05.com.redhat:185ce16b55ad
Events: <none>
```
@rootfs @humblec @jsafrane
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)
Fix iSCSI WaitForAttach not mounting a volume
WaitForAttach failed consistently with this error:
```
Heuristic determination of mount point failed:stat /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/10.128.0.3:3260-iqn.2003-01.org.linux-iscsi.f21.x8664:sn.4b0aae584f7c-lun-0: no such file or directory
```
We should ignore "no such file or directory" eror, the directory is created just few lines below.
Fixes: #51625
**Release note**:
```release-note
NONE
```
/sig storage
/assign @mtanino
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
audit policy: support subresources and resource names
Updates #48561
policy:
- level: Metadata
resources:
- group: ""
resources ["pods/logs"]
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
The top level resource no longer matches the subresource. For example "pods"
no longer matches requests to the logs subresource on pods.
```release-note
Audit policy supports matching subresources and resource names, but the top level resource no longer matches the subresouce. For example "pods" no longer matches requests to the logs subresource of pods. Use "pods/logs" to match subresources.
```
/cc @sttts @soltysh @crassirostris @ihmccreery @timstclair
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
Add a persistent volume label controller to the cloud-controller-manager
Part of https://github.com/kubernetes/features/issues/88
Outstanding concerns needing input:
- [x] Why 5 threads for controller processing?
- [x] Remove direct linkage to aws/gce cloud providers [#51629]
- [x] Modify shared informers to allow added event handlers ability to include uninitialized objects/using unshared informer #48893
- [x] Use cache.MetaNamespaceKeyFunc in event handler?
I'm willing to work on addressing the removal of the direct linkage to aws/gce after this PR gets in.
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
Task 1: Tainted node by condition.
**What this PR does / why we need it**:
Tainted node by condition for MemoryPressure, OutOfDisk and so on.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001
**Release note**:
```release-note
Tainted nodes by conditions as following:
* 'node.kubernetes.io/network-unavailable=:NoSchedule' if NetworkUnavailable is true
* 'node.kubernetes.io/disk-pressure=:NoSchedule' if DiskPressure is true
* 'node.kubernetes.io/memory-pressure=:NoSchedule' if MemoryPressure is true
* 'node.kubernetes.io/out-of-disk=:NoSchedule' if OutOfDisk is true
```
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
update scheduler to return structured errors instead of process exit
The scheduler Run method returns an error that is properly handled at higher levels. Instead of existing the process, we should return the error and handle it at higher level logic to allow testing of error conditions and composition of commands. The changes are relatively minor.
@sjenning @aveshagarwal
