Automatic merge from submit-queue (batch tested with PRs 44626, 45641)
Update Google Cloud DNS provider Rrset.Get(name) method to return a list and change the `Rrset.List()` implementation to perform a paged walk
Some federated service e2e tests and a few ingress tests would become flaky after a few hundred runs. @csbell spent quite a lot of time debugging this and found out that this flakiness was due to a bug in the federated service controller deletion logic. Deletion of a federated service object triggers a logic in the controller to update the DNS records corresponding to that object. This DNS record update logic would return an error in failed runs which would in-turn cause the controller to reschedule the operation. This led to an infinite retry-failure cycle that never gave the API server a chance to garbage collect the deleted service object.
A couple of days ago we started seeing a correlation between the number of resource records in a DNS managed zone and these test failures. If you look at the test runs before and after run 2900 in the test grid - https://k8s-testgrid.appspot.com/cluster-federation#gce, you will notice that the grid became super green at 2900. That's when I deleted all the dangling DNS records from the past runs.
After some investigation yesterday, we found that `ResourceRecordSet.Get()` interface and its implementation, and `ResourceRecordSet.List()` implementation at least for Google Cloud DNS were incorrect.
This PR makes minimal set of changes (read: least invasive) in Google Cloud DNS provider implementation to fix these problems:
1. Modifies DNS provider Rrset.Get(name) interface to return multiple records and updates federated service controller.
There can be multiple DNS resource records for a given name. They can vary by type, ttl, rrdata and a number of various other parameters. It is incorrect to return a single resource record for a given name.
This change updates the Get interface to return multiple records for a given name and uses this list in the federated service controller to perform DNS operations.
2. Update Google Cloud DNS List implementation to perform a paged walk of lists to aggregate all the DNS records.
The current `List()` implementation just lists the DNS resorce records in a given managed zone once and retruns the list. It neither performs a paged walk nor does it consider the `page_token` in the returned response.
This change walks all the pages and aggregates the records in the pages and returns the aggregated list. This is potentially dangerous as it can blow up memory if there are a huge number of records in the given managed zone. But this is the best we can do without changing the provider interface too much.
Next step is to define a new paged list interface and implement it.
**Release note**:
```release-note
NONE
```
/assign @csbell
cc @justinsb @shashidharatd @quinton-hoole @kubernetes/sig-federation-pr-reviews
When we fetch the dns records by name, we get a list of records that match
the given name. As an optimization we look up to see if the new record we
want to create is already in the returned list to avoid performing any updates.
However, when the new record we want to create isn't in the returned list, it
is hard to say if the returned list contains the list of records that we want
to retain. For example, we might get a list of A records and we want to create
a CNAME record. Creating a new CNAME record without removing the A records is
a DNS misconfiguration. So to play safe we just remove all the existing records
in the list and create the new desired record.
**Note**: This is the opposite of what I said here - https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/44626#-Ki9xQOzybryHvsxNrra.
Automatic merge from submit-queue (batch tested with PRs 45556, 45561, 45256)
[Federation] Replace the indexing lister with a regular store in the replicaset controller
This is part of the refactoring work to allow the replicaset controller to use the generic sync controller.
None of the other controllers use a lister, including the deployment controller
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45382, 45384, 44781, 45333, 45543)
[Federation] Provide updater timeout to instance rather than to Update()
This PR changes the federated updater to receive its timeout at construction rather than on every call to Update(). This provides a slight decrease in coupling by removing the need for the deletion handler to be provided the timeout along with the updater.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue
[Federation] Improve the logging and user feedback in 'kubefed init'
This is a follow-up to #41849, which added some status information. This PR is based off of that one, and includes its changes as well.
See #41725.
```release-note
None
```
The current `List()` implementation just lists the DNS resorce records in
a given managed zone once and retruns the list. It neither performs a paged
walk nor does it consider the `page_token` in the returned response.
This change walks all the pages and aggregates the records in the pages
and returns the aggregated list. This is potentially dangerous as it can
blow up memory if there are a huge number of records in the given
managed zone. But this is the best we can do without changing the
provider interface too much. Next step is to define a new paged list
interface and implement it.
There can be multiple DNS resource records for a given name. They can
vary by type, ttl, rrdata and a number of various other parameters. It
is incorrect to return a single resource record for a given name.
This change updates the Get interface to return multiple records for a given
name and uses this list in the federated service controller to perform
DNS operations.
Automatic merge from submit-queue
Keep UserAgentName style consistent
Keep using UserAgentName for controllers and add some logs for debugging
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
[Federation] Record events in federated updater
Controllers for federated types were previously recording events when generating the list of operations. This change delegates responsibility for recording events to the federated updater so that events are recorded when the operations are actually executed, and ensures consistency across recording of both operation initiation and failure.
The deletion helper was similarly updated to rely on the federated updater for event recording. To support this change to the deletion helper, controllers have been updated to provide a namespace qualified name via the objNameFunc function to ensure that the updater can record events for deletions with the same detail as for add and update operations.
cc: @kubernetes/sig-federation-pr-reviews @perotinus
Automatic merge from submit-queue (batch tested with PRs 45362, 45159, 45321, 45238)
Remove redundent GetObjectKind() defined on types
Embedding TypeMeta is enough.
Automatic merge from submit-queue (batch tested with PRs 45283, 45289, 45248, 44295)
Use docker_bundle rule from new rules_docker repo
**What this PR does / why we need it**: switched to using the new `docker_bundle` rule from `rules_docker` instead of my patched `docker_build` rule. This also brings in some fixes for the docker rules that were missing from my fork.
Additionally, I switched out the `git_repository` rules for `http_archive` rules, since that seems to be recommended by the bazel docs (and might be faster).
Lastly, I updated the `pkg_tar` rules to use my patch, which doesn't prepend `./` to files inside the tarballs.
This one should likely be merged upstream in the near future.
I think this is the last of the changes necessary to have `bazel run //:ci-artifacts` working properly to support using bazel for e2e in CI.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40544, 44338, 45225)
[Federation]Update event type
Use EventTypeWarning instead of EventTypeNormal when get failure
Left behind after pr #40296
Automatic merge from submit-queue
Fixes broken link in PodSpec
The PodSpec reference to NodeSelector misses
the `.md` extension in the file.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)
[Federation] Generate the secret name in kubefed join.
Addresses part of #42324. A follow-up PR will address annotating Federation resources.
```release-note
Remove the `--secret-name` flag from `kubefed join`, instead generating the secret name arbitrarily.
```
Automatic merge from submit-queue
Remove GetClientsetForCluster()
The newClusterClientset() has insteaded of GetClientsetForCluster(),
and GetClientsetForCluster() run wrong. Let's remove it.
Automatic merge from submit-queue
README.md: Update outdated links
**What this PR does / why we need it**:
the PR aims to update some links.
Some links with "#" would not redirect to right point of pages.
Other links without "#" can work, but they are outdated. I change them by the way.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
none
Automatic merge from submit-queue
Ignore IsNotFound error
IsNotFound error is fine since that means the object is
deleted already, so we should check err and ignore err
before returning.
Automatic merge from submit-queue
Delete "hard-coded" default value in flags usage.
**What this PR does / why we need it**:
Some flags of kubernetes components have "hard-coded" default values in their usage info. In fact, [pflag pkg](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/spf13/pflag/flag.go#L602-L608) has already added a string `(default value)` automatically in the usage info if the flag is initialized. Then we don't need to hard-code the default value in usage info. After this PR, if we want to update the default value of a flag, we only need to update the flag where it is initialized. `pflag` will update the usage info for us. This will avoid inconsistency.
For example:
Before
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s (default 10s)
...
```
After
```
kubelet -h
...
--node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. (default 10s)
...
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
This PR doesn't delete some "hard-coded" default values because they are not explicitly initialized. We still need to hard-code them to give users friendly info.
```
--allow-privileged If true, allow containers to request privileged mode. [default=false]
```
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
Prepare for move zz_generated_deepcopy.go to k8s.io/api
This is in preparation to move deep copies to with the types to the types repo (see https://github.com/kubernetes/gengo/pull/47#issuecomment-296855818). The init() function is referring the `SchemeBuilder` defined in the register.go in the same packge, so we need to revert the dependency.
This PR depends on https://github.com/kubernetes/gengo/pull/49, otherwise verification will fail.
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
[Federation] Convert Daemonset to use the generic sync controller
To be rebased on master when @perotinus's configmaps PR merges.
Tested integration and e2e.
Automatic merge from submit-queue (batch tested with PRs 44942, 41258)
[Federation] Use federated informer for service controller and annotations to store lb ingress
**What this PR does / why we need it**:
This is breaking up of the PR #40296 into smaller one. please refer to #41253
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Handles 2 tasks in #41253
Fixes issues in #27623, #35827
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
cc @quinton-hoole @nikhiljindal @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41287, 41636, 44881, 44826)
[Federation] Only attempt to log information about pods in federation/cluster/log-dump.sh if those pods exist.
Fixes an issue where the script fails, because bash considers an empty array an unbound variable.
**Release note**:
```release-note
NONE
```
