Automatic merge from submit-queue
[Federation] Expose autoscaling apis through federation api server
This PR implements first part of federated pod autoscaler.
The issue to handle the whole feature is https://github.com/kubernetes/kubernetes/issues/38974
cc @kubernetes/sig-cluster-federation
@shashidharatd @kshafiee @deepak-vij
**Release note**:
```
federation users can now use federated autoscaling resources and create federated horizontalpodautoscalers
```
Automatic merge from submit-queue (batch tested with PRs 40066, 36250, 40255, 40225, 40204)
Fix validation errors in cli are cryptic
fix issue https://github.com/kubernetes/kubernetes/issues/39811
now the error message is like:
`error validating "/home/shiywang/template/test.yaml": error validating data: field ports for v1.ServiceSpec: is required; if you choose to ignore these errors, turn validation off with --validate=false`
still need add some unit test, plz don't test now
Automatic merge from submit-queue
Set custom PollingDelay of 5 seconds for Azure VirtualMachinesClient
The default polling delay of 1 minute results in very long delays when
an Azure Disk is attached to a node. It gets worse as go-autorest
doubles the default delay to 2 minutes.
Please see: https://github.com/kubernetes/kubernetes/issues/35180#issuecomment-273085063
Only the PollingDelay for VirtualMachinesClient is modified here to
avoid too much pressure on Azure quotas.
Release Nodes:
```release-note
Reduce time needed to attach Azure disks
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
declare it, but no use it
**What this PR does / why we need it**:
declare it, but no use it
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Refactor/improve the kubeadm generation of certificates
**What this PR does / why we need it**:
Continues to refactor/improve kubeadm towards beta.
**Special notes for your reviewer**:
Modify the certs that are generated; generate on demand (if not exist) and only four files instead of eight previously. Basically implements what has been discussed so far in https://github.com/kubernetes/kubeadm/pull/100
**Release note**:
```release-note
NONE
```
cc @mikedanese @pires @lukemarsden @errordeveloper @dgoodwin @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Invalidate updates to non-convertible selectors
Follow-up to https://github.com/kubernetes/kubernetes/pull/24733
@deads2k @lavalamp @smarterclayton @bgrant0607 @liggitt @mfojtik
First commit contains the necessary validation for replication controllers with non-convertible selectors.
Second commit updates the name for the annotation added during conversion since it is invalid currently:
```
+++ [0719 11:19:54] Running tests without code coverage
--- FAIL: TestValidateUpdate (0.00s)
strategy_test.go:191: unexpected error: [metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName', metadata.annotations: Invalid value: "kubernetes.io/non-convertible/spec.selector": must match the regex ([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9] (e.g. 'MyName' or 'my.name' or '123-abc') with an optional DNS subdomain prefix and '/' (e.g. 'example.com/MyName']
FAIL
FAIL k8s.io/kubernetes/pkg/registry/controller 0.015s
```
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Add a log at the start of a watch request.
Fixes https://github.com/kubernetes/kubernetes/issues/16899 (> than a year old!)
@lavalamp
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
fixing Cassandra shutdown example to avoid data corruption
Hi,
I was playing with Cassandra example stored in the Kubernetes project and I encountered issues on shutdown (not anytime). After checking it looks like the shutdown of a node is brutal and data corruption may occur during a flush on disk. To avoid that, I'm suggesting a hook to gracefully shutdown Cassandra before stopping the container.
Here are logs of corruption after a pod delete:
```
/10.2.76.4:[-8699848499000118463, -8567123670484406873, -8496767951391579058, -8426990834929543369, -7697118318683556771, -6942779781591907873, -6795880495022459877, -6496399078175245235, -5450122121479522544, -5002551029990001224, -4914532712178218138, -4884518674849288097, -3667338763252443465, -3316742521554936832, -2844544359955291760, -1291351295404368159, -794348397160283083, -705240847455001090, -652995206518489298, -284127251294286231, 173240967232234690, 616476682204879844, 826670457841382100, 1815369334084765465, 4431706613761077084, 4743606016174161647, 5637469692783959686, 5802957011124852712, 6759688243703331970, 7679657413128857702, 7713766696628426028, 9098158217036036188]
ERROR 16:23:06 Exception in thread Thread[CompactionExecutor:2,1,main]
org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /cassandra_data/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/mc-2-big-Data.db
at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:351) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:232) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:184) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) ~[apache-cassandra-3.9.jar:3.9]
at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264) ~[apache-cassandra-3.9.jar:3.9]
```
It works well for me now and do not have data corruption anymore.
Automatic merge from submit-queue
Fix Elasticsearch image for logging and bump version
Follow-up from https://github.com/kubernetes/kubernetes/pull/39861
`elasticsearch_logging_discovery.go` had to be updated to respect API changes.
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Typo fix: Change logging function to formatting version
**What this PR does / why we need it**:
Slightly broken logging message:
```
I0120 10:56:08.555712 7575 kubelet_node_status.go:135] Deleted old node object %qkubernetes-cit-kubernetes-cr0-0
```
Automatic merge from submit-queue (batch tested with PRs 40250, 40134, 40210)
Remove conversion package
This package is empty, it would be nice if we could remove it.
Automatic merge from submit-queue
bazel: add a config setting to control embedding kubernetes-src.tar.gz
**What this PR does / why we need it**: currently a change anywhere in the tree will cause `kubernetes-src.tar.gz` to need to be regenerated, and thus also the server and node tarballs. All of these operations are slow, so for the sake of developer productivity, only include `kubernetes-src.tar.gz` when we need it (e.g. if we were doing a real release).
I don't have metrics on how much of an effect this has, but I expect it should help incremental builds, especially those that don't affect any node/server targets.
To embed the srcs tarball with this change, you'd run
```console
bazel build //build/release-tars --define EMBED_LICENSE_TARGETS=true
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40232, 40235, 40237, 40240)
Fixup pet terminology in log and user-facing events
**What this PR does / why we need it**:
Removes some user-facing strings for pet terminology.
Automatic merge from submit-queue (batch tested with PRs 40232, 40235, 40237, 40240)
move listers out of cache to reduce import tree
Moving the listers from `pkg/client/cache` snips links to all the different API groups from `pkg/storage`, but the dreaded `ListOptions` remains.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
Update client/server cert generation utilities
Limit generated cert usage to client or server use.
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
Add .bazelrc with flaky_test_attempts.
**What this PR does / why we need it**: This PR adds `.bazelrc`, which will set defaults for all bazel commands. The only controversial default I set is `--flaky_test_attempts=3`.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
remove the delete-pod-uid spec
**What this PR does / why we need it**:
remove the specifications about "kubectl delete pod uid" cause it's no longer supported.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40121
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
update short help message in set resource
To keep the help message well-formed.
```
#kubectl set -h
Configure application resources
These commands help you make changes to existing application resources.
Available Commands:
image Update image of a pod template
resources update resource requests/limits on objects with pod templates
selector Set the selector on a resource
Usage:
kubectl set SUBCOMMAND [options]
```
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
Cleanup temp dirs
So funny story my /tmp ran out of space running the unit tests so I am cleaning up all the temp dirs we create.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
move apimachinery/announced to apimachinery
This also rewires the announce package to allow installation into multiple groups via multiple calls to `install.Install`. At some point, we'll want to unwire the hardcoded, "install to this scheme" call.
@lavalamp something we've wanted for a while and need for genericapiserver
@seh you've asked related questions
@sttts ptal. First commit is interesting, second commit is a straight move.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
