Automatic merge from submit-queue
Make generic registry easier to understand
**What this PR does / why we need it**:
This PR makes the generic registry and some areas of the api REST abstractions easier to understand by adding and clarifying comments. These comments are based on digging that was done to implement a new API server and REST storage for resources in a wholly-new API group.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
make client-go tools/auth and tools/clientcmd authoritative
These moved easily. Pure mechanical. I'll have a couple snip pulls for some of the more complicated packages. `pkg/util/homedir` was a transitive that moved with. It was only used in `kubectl` and you'd need it to be consistent with `clientcmd` loading order.
@sttts
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
kubeadm: Secure apiserver -> kubelet communication and set storage backend to etcd3
**What this PR does / why we need it**:
Switch storage backend to etcd3
Writes ca.crt down to disk so we can set `--client-ca-file=/etc/kubernetes/ca.crt` for the kubelet.
Adds --kubelet-client-{certificate,key} to the apiserver args and make it allowed to access the kubelets
In some cases the `path` dependency is switched to `filepath`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes https://github.com/kubernetes/kubeadm/issues/118
fixes https://github.com/kubernetes/kubeadm/issues/129
**Special notes for your reviewer**:
This PR is should make it possible to secure the apiserver -> kubelet communication.
**Release note**:
```release-note
NONE
```
@pires @mikedanese @andrewrynhard @liggitt @deads2k
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue
separate announce factories
We caught this part way through Daniel's refactor from 1.5. Have to separate announce factories to be able to double register types for client-go.
@kubernetes/sig-apimachinery-misc
Automatic merge from submit-queue (batch tested with PRs 40196, 40143, 40277)
avoid incorrect short-circuit of client-ca setup
PasswordFile options should only affect computation of the SupportsBasicAuth field, not short-circuit setting up the client ca certpool
Automatic merge from submit-queue (batch tested with PRs 40196, 40143, 40277)
Emit warning event when CronJob cannot determine starting time
**What this PR does / why we need it**:
In #39608, we've modified the error message for when a CronJob has too many unmet starting times to enumerate to figure out the next starting time. This makes it more "actionable", and the user can now set a deadline to avoid running into this. However, the error message is still only controller level AFAIK and thus not exposed to the user. From his perspective, there is no way to tell why the CronJob is not scheduling the next instance.
The PR adds a warning event in addition to the error in the controller manager's log.
**Which issue this PR fixes**: This is an addition to PR #39608 regarding #36311.
**Special notes for your reviewer**: cc @soltysh
**Release note**:
```release-note
```
Automatic merge from submit-queue
Remove TODOs to refactor kubelet labels
To address #39650 completely.
Remove label refactoring TODOs, we don't need them since CRI rollout is on the way.
Automatic merge from submit-queue
[Federation] Expose autoscaling apis through federation api server
This PR implements first part of federated pod autoscaler.
The issue to handle the whole feature is https://github.com/kubernetes/kubernetes/issues/38974
cc @kubernetes/sig-cluster-federation
@shashidharatd @kshafiee @deepak-vij
**Release note**:
```
federation users can now use federated autoscaling resources and create federated horizontalpodautoscalers
```
Automatic merge from submit-queue (batch tested with PRs 40066, 36250, 40255, 40225, 40204)
Fix validation errors in cli are cryptic
fix issue https://github.com/kubernetes/kubernetes/issues/39811
now the error message is like:
`error validating "/home/shiywang/template/test.yaml": error validating data: field ports for v1.ServiceSpec: is required; if you choose to ignore these errors, turn validation off with --validate=false`
still need add some unit test, plz don't test now
Automatic merge from submit-queue
Set custom PollingDelay of 5 seconds for Azure VirtualMachinesClient
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Release Nodes:
```release-note
Reduce time needed to attach Azure disks
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
declare it, but no use it
**What this PR does / why we need it**:
declare it, but no use it
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Refactor/improve the kubeadm generation of certificates
**What this PR does / why we need it**:
Continues to refactor/improve kubeadm towards beta.
**Special notes for your reviewer**:
Modify the certs that are generated; generate on demand (if not exist) and only four files instead of eight previously. Basically implements what has been discussed so far in https://github.com/kubernetes/kubeadm/pull/100
**Release note**:
```release-note
NONE
```
cc @mikedanese @pires @lukemarsden @errordeveloper @dgoodwin @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Invalidate updates to non-convertible selectors
Follow-up to https://github.com/kubernetes/kubernetes/pull/24733
@deads2k @lavalamp @smarterclayton @bgrant0607 @liggitt @mfojtik
First commit contains the necessary validation for replication controllers with non-convertible selectors.
Second commit updates the name for the annotation added during conversion since it is invalid currently:
```
+++ [0719 11:19:54] Running tests without code coverage
--- FAIL: TestValidateUpdate (0.00s)
strategy_test.go:191: unexpected error: [metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName', metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName']
FAIL
FAIL k8s.io/kubernetes/pkg/registry/controller 0.015s
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Add a log at the start of a watch request.
Fixes https://github.com/kubernetes/kubernetes/issues/16899 (> than a year old!)
@lavalamp
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
fixing Cassandra shutdown example to avoid data corruption
Hi,
I was playing with Cassandra example stored in the Kubernetes project and I encountered issues on shutdown (not anytime). After checking it looks like the shutdown of a node is brutal and data corruption may occur during a flush on disk. To avoid that, I'm suggesting a hook to gracefully shutdown Cassandra before stopping the container.
Here are logs of corruption after a pod delete:
```
/10.2.76.4:[-8699848499000118463, -8567123670484406873, -8496767951391579058, -8426990834929543369, -7697118318683556771, -6942779781591907873, -6795880495022459877, -6496399078175245235, -5450122121479522544, -5002551029990001224, -4914532712178218138, -4884518674849288097, -3667338763252443465, -3316742521554936832, -2844544359955291760, -1291351295404368159, -794348397160283083, -705240847455001090, -652995206518489298, -284127251294286231, 173240967232234690, 616476682204879844, 826670457841382100, 1815369334084765465, 4431706613761077084, 4743606016174161647, 5637469692783959686, 5802957011124852712, 6759688243703331970, 7679657413128857702, 7713766696628426028, 9098158217036036188]
ERROR 16:23:06 Exception in thread Thread[CompactionExecutor:2,1,main]
org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /cassandra_data/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/mc-2-big-Data.db
at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:232) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:184) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264) ~[apache-cassandra-3.9.jar:3.9]
```
It works well for me now and do not have data corruption anymore.
Automatic merge from submit-queue
Fix Elasticsearch image for logging and bump version
Follow-up from https://github.com/kubernetes/kubernetes/pull/39861
`elasticsearch_logging_discovery.go` had to be updated to respect API changes.
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Typo fix: Change logging function to formatting version
**What this PR does / why we need it**:
Slightly broken logging message:
```
I0120 10:56:08.555712 7575 kubelet_node_status.go:135] Deleted old node object %qkubernetes-cit-kubernetes-cr0-0
```
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Remove conversion package
This package is empty, it would be nice if we could remove it.
