Automatic merge from submit-queue
Allow mounts to run in parallel for non-attachable volumes
This PR:
* Fixes https://github.com/kubernetes/kubernetes/issues/28616
* Enables mount volume operations to run in parallel for non-attachable volume plugins.
* Enables unmount volume operations to run in parallel for all volume plugins.
* Renames `GoRoutineMap` to `GoroutineMap`, resolving a long outstanding request from @thockin: `"Goroutine" is a noun`
Automatic merge from submit-queue
ImagePuller refactoring
A plain refactoring
- Moving image pullers to a new pkg/kubelet/images directory
- Hiding image pullers inside the new ImageManager
The next step is to consolidate the logic of the serialized and the parallel image pullers inside ImageManager
xref: #25577
Automatic merge from submit-queue
Fix node e2e (kubelet metrics) panic
On systems (rhel7, and fedora 24), the kubelet metrics node e2e panics:
```
[k8s.io] Kubelet metrics api when querying /stats/summary
it should report resource usage through the stats api
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:146
[BeforeEach] [k8s.io] Kubelet
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/framework.go:132
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Jul 18 10:45:45.004: INFO: Skipping waiting for service account
[It] it should report resource usage through the stats api
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:146
W0718 10:45:45.036201 17112 request.go:347] Field selector: v1 - pods - metadata.name - stats-busybox-4edb73ec-4cf6-11e6-9ecc-52540041b7801: need to check if this is versioned correctly.
W0718 10:45:45.036213 17112 request.go:347] Field selector: v1 - pods - metadata.name - stats-busybox-4edb73ec-4cf6-11e6-9ecc-52540041b7800: need to check if this is versioned correctly.
E0718 10:50:45.036630 17112 streamwatcher.go:109] Unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)
Jul 18 10:50:45.036: INFO: Unexpected error occurred: timed out waiting for the condition
panic:
Your test failed.
Ginkgo panics to prevent subsequent assertions from running.
Normally Ginkgo rescues this panic so you shouldn't see it.
But, if you make an assertion in a goroutine, Ginkgo can't capture the panic.
To circumvent this, you should call
defer GinkgoRecover()
at the top of the goroutine that caused this panic.
goroutine 68 [running]:
panic(0x1958fc0, 0xc8204ba6b0)
/usr/lib/golang/src/runtime/panic.go:481 +0x3e6
k8s.io/kubernetes/vendor/github.com/onsi/ginkgo.Fail(0xc820302160, 0xb0, 0xc8204ba2c8, 0x1, 0x1)
/root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:244 +0x116
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc820820080, 0x2b2337d20268, 0x30a2da8, 0x0, 0x0, 0x0, 0x0, 0x30a2da8)
/root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:69 +0x32d
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc820820080, 0x2b2337d20268, 0x30a2da8, 0x0, 0x0, 0x0, 0x2b2337d20220)
/root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:43 +0x92
k8s.io/kubernetes/test/e2e/framework.ExpectNoError(0x2b2336968028, 0xc8200b30c0, 0x0, 0x0, 0x0)
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:1811 +0x203
k8s.io/kubernetes/test/e2e/framework.(*Framework).CreatePods.func1(0xc8200a6540, 0xc820204f40, 0x33, 0xc8201576c0)
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/pods.go:68 +0x68
created by k8s.io/kubernetes/test/e2e/framework.(*Framework).CreatePods
/root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/pods.go:70 +0x213
Ginkgo ran 1 suite in 5m20.524270699s
Test Suite Failed
!!! Error in hack/make-rules/test-e2e-node.sh:132
'"${ginkgo}" --focus=$focus --skip=$skip "${KUBE_ROOT}/test/e2e_node/" --report-dir=${report} -- --alsologtostderr --v 2 --node-name $(hostname) --disable-kubenet=true --build-services=true --start-services=true --stop-services=true "$test_args"' exited with status 1
Call stack:
1: hack/make-rules/test-e2e-node.sh:132 main(...)
Exiting with status 1
make: *** [test-e2e-node] Error 1
```
Automatic merge from submit-queue
Kubelet: Set PruneChildren when removing image.
This is a bug introduced during switching to engine-api. https://github.com/kubernetes/kubernetes/issues/23563.
When removing image, there is an option `noprune`:
```
If prune is true, ancestor images will each attempt to be deleted quietly.
```
In go-dockerclient, the default value of the option is ["noprune=false"](https://github.com/fsouza/go-dockerclient/blob/master/image.go#L171), which means that ancestor images should be also removed. This is the expected behaviour.
However in engine-api, the option is changed to `PruneChildren`, and the default value is `PruneChildren=false`, which means that ancestor images won't be removed.
This makes `ImageRemove` only remove the first layer of the image, which causes the image garbage collection not working as expected.
This should be fixed in 1.3.
And thanks to @ronnielai for finding the bug! :)
/cc @kubernetes/sig-node
Automatic merge from submit-queue
docker_manager: Correct determineContainerIP args
This could result in the network plugin not retrieving the pod ip in a
call to SyncPod when using the `exec` network plugin.
The CNI and kubenet network plugins ignore the name/namespace arguments,
so they are not impacted by this bug.
I verified the second included test failed prior to correcting the
argument order.
Fixes#29161
cc @yujuhong
Automatic merge from submit-queue
pkg/probe/http: don't compare error strings in tests
TestHTTPProbeChecker fails on the Go1.7 release candidates. The
package's history show that this was the case for Go1.5 and Go1.6
as well.
The test depend on errors holding specific string values, behavior
not guarenteed in the standard library API, and causing new test
failures every minor Go release. Just look for an error rather than
trying to inspect it using string comparison. If we feel this
impacts coverage we can add more test cases.
Fixes#15952
Automatic merge from submit-queue
Fix RBAC authorizer of ServiceAccount
RBAC authorizer assigns a role to a wrong service account.
How to reproduce
1.Create role and rolebinding to allow default user in kube-system namespace to read secrets in kube-system namespace.
```
# kubectl create -f role.yaml
# kubectl create -f binding.yaml
```
```yaml
# role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: secret-reader
namespace: kube-system
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
nonResourceURLs: []
```
```yaml
# binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: read-secrets
namespace: kube-system
subjects:
- kind: ServiceAccount
name: default
namespace: kube-system
roleRef:
kind: Role
namespace: kube-system
name: secret-reader
apiVersion: rbac.authorization.k8s.io/v1alpha1
```
2.Set a credential of default user
```
$ kubectl config set-credentials default_user --token=<token_of_system:serviceaccount:kube-system:default>
$ kubectl config set-context default_user-context --cluster=test-cluster --user=default_user
$ kubectl config use-context default_user-context
```
3.Try to get secrets as default user in kube-system namespace
```
$ kubectl --namespace=kube-system get secrets
the server does not allow access to the requested resource (get secrets)
```
As shown above, default user could not access to secrets.
But if I have kube-system user in default namespace, it is allowed access to secrets.
4.Create a service account and try to get secrets as kube-system user in default namespace
```
# kubectl --namespace=default create serviceaccount kube-system
serviceaccount "kube-system" created
$ kubectl config set-credentials kube-system_user --token=<token_of_system:serviceaccount:default:kube-system>
$ kubectl config set-context kube-system_user-context --cluster=test-cluster --user=kube-system_user
$ kubectl config use-context kube-system_user-context
$ kubectl --namespace=kube-system get secrets
NAME TYPE DATA AGE
default-token-8pyb3 kubernetes.io/service-account-token 3 4d
```
Automatic merge from submit-queue
Information is opposite to real meaning to express
master is not equal to expectedMaster, the meaning should be the master is unexpected:
master, err := mesosCloud.Master(clusterName)
if master != expectedMaster {
t.Fatalf("Master returns the expected value: (expected: %#v, actual: %#v", expectedMaster, master)
Automatic merge from submit-queue
format number not consistent with real variable number
glog.Infof format number not consistent with real variable number， should add %s for second var because loadBalancerName is string:
func (c *Cloud) ensureLoadBalancer(namespacedName types.NamespacedName, loadBalancerName string, ...
Automatic merge from submit-queue
"server.go" directory error
In file "docs\devel\profiling.md", line 55:
"In 'pkg/master/server/server.go' more servers are created“
Here server.go directory is wrong, should be :pkg/kubelet/server/server.go
Automatic merge from submit-queue
Proposal for ControllerReference
Proposal for including the reference pointing to the owning "collection" (controller) for objects that can be grouped. The goal is to prevent a situation when two controllers are fighting over some resources.
cc @bgrant0607 @lavalamp @caesarxuchao @davidopp @fgrzadkowski @wojtek-t @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Allow handlers earlier in a request flow to inject a UID for an object
This lets admission controllers specify a stable UID for an object prior to its creation. That lets the admission controller then record a reference to the object on another resource using that stable UID prior to the object being created. This would be a prerequisite for supporting quota reservations.
/cc @smarterclayton @lavalamp @deads2k
Automatic merge from submit-queue
Start namespace controller in node e2e
Fix https://github.com/kubernetes/kubernetes/issues/28320.
Based on https://github.com/kubernetes/kubernetes/pull/28807, only the last 2 commits are new.
Before this PR, there was no namespace controller running in node e2e test infrastructure. We can not enable the [`delete-namespace`](f2ddd60eb9/test/e2e/framework/test_context.go (L109)) flag in the test framework.
So after the test running, there will be running pod left on the test node. This seems to be acceptable in our test infrastructure because we create an new instance each time.
However, in 1.4 we may want to provide part of the test as node conformance test to the user, they definitely don't want the test to leave tons of pods on their node after test running.
Currently, there is no easy way to only start namespace controller in kube-controller-manager (confirmed with @mikedanese), so in this PR I started a "uncontainerized" one in the test infrastructure.
This PR:
* Started the namespace controller in the node e2e test infrastructure and enable the automatic namespace deletion.
* Change the privileged test to use framework (@yujuhong), so that all node e2e tests are using the framework and test pods will be cleaned up by namespace controller.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Info about enabling the hostpath provisioner
The README for persistent volume provisioning had information about the
hostpath provisioner for testing purposes, but lacked instructions on
how to enable it.
Automatic merge from submit-queue
Revert "Follow on for 1.4 to default HTTP2 on by default"
This reverts commit efe25553cd
in order to address: #29001#28537
Automatic merge from submit-queue
Bump the default etcd version in the Makefile to 3.0.3
Fixes: #29132
I haven't had time to manually validate the arm and arm64 version yet, but I think it should be fine.
cc @xiang90 @hongchaodeng @timothysc @lavalamp @wojtek-t @thockin @kubernetes/sig-scalability @Pensu @laboger
Automatic merge from submit-queue
Add proposal for service externalName
This is a proposal to address: #13748.
@smarterclayton @ncdc @thockin. Please check this out when you have time, hopefully this is okay :-D
I created the proposal because was unsure if the feature would be able to go in if there isn't a proposal already merged, because of this mail to kubernetes-pm: https://groups.google.com/forum/#!topic/kubernetes-pm/Ki63EztfZMo.
So, IIUC it would be nice to have the proposal merged ASAP (I think the interface looks ok for all, so hopefully this will be easy) so we can have this feature in 1.4 as you guys ( @smarterclayton @ncdc ) need.
Automatic merge from submit-queue
Change the name of the secret that delivers federation kubeconfig.
```release-note
Federation API server kubeconfig secret consumed by federation-controller-manager has a new name.
If you are upgrading your Cluster Federation components from v1.3.x, please run this command to migrate the federation-apiserver-secret to federation-apiserver-kubeconfig serect;
$ kubectl --namespace=federation get secret federation-apiserver-secret -o json | sed 's/federation-apiserver-secret/federation-apiserver-kubeconfig/g' | kubectl create -f -
You might also want to delete the old secret using this command:
$ kubectl delete secret --namespace=federation federation-apiserver-secret
```
The current name, federation-apiserver-secret, is very similar to the
other secret we have, federation-apiserver-secrets, that delivers
somewhat similar data but in a different format. This is extremely
confusing, particularly while debugging.
This change should soothe the pain.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Make "attach" and "exec" rejection in proxy more explicit
```release-note
kubectl proxy changed to now allow urls to pods with "attach" or "exec" in the pod name
```
The more explicit regular expression for rejection makes a possibility of accessing pods (or any other resources) which contain "attach" or "exec" in their names via proxy API. It was not possible before.
Also, the reference for "run" resource was removed, because it doesn't exist in any of k8s APIs currently.
Fixes: #21464
Automatic merge from submit-queue
Move ExtractPodBandwidthResources test into appropriate package
Found during #28511, this test is in the wrong package currently.
cc @kubernetes/sig-network
Allow mount volume operations to run in parallel for non-attachable
volume plugins.
Allow unmount volume operations to run in parallel for all volume
plugins.
Automatic merge from submit-queue
AWS: Added experimental option to skip zone check
This pull request resolves#28380. In the vast majority of cases, it is appropriate to validate the AWS region against a known set of regions. However, there is the edge case where this is undesirable as Kubernetes may be deployed in an AWS-like environment where the region is not one of the known regions.
By adding the optional **DisableStrictZoneCheck true** to the **[Global]** section in the aws.conf file (e.g. /etc/aws/aws.conf) one can bypass the ragion validation.
Automatic merge from submit-queue
dnsprovider: Use route53 page functions to avoid truncated results
The List<Type>Pages functions make it pretty easy to avoid result truncation;
switch to using them
Automatic merge from submit-queue
Format apiserver options
Trivial change, code formatting only: it was hard to read long lines, and my editor was really slow when scrolling through them.
Automatic merge from submit-queue
Ubuntu: Enable ssh compression when downloading binaries during cluster creation
<!--
Checklist for submitting a Pull Request
Please remove this comment block before submitting.
1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
resolves#20971 by using the options provided by ssh.
Native ssh compression has existed for years, and the server is free to disregard the setting, so this should be safe.
With things like the kube binaries I see about a 2x speed increase.
```
λ time scp kubes-bin.tar 9.30.182.251:/mnt/build/kubin
kubes-bin.tar 100% 344MB 10.7MB/s 00:32
real 0m32.284s
user 0m1.679s
sys 0m1.263s
λ time scp -C kubes-bin.tar 9.30.182.251:/mnt/build/kubin
kubes-bin.tar 100% 344MB 22.9MB/s 00:15
real 0m14.810s
user 0m12.858s
sys 0m0.994s
λ ls -lah kubes-bin.tar
-rw-r--r-- 1 mhb staff 344M Jun 2 15:29 kubes-bin.tar
λ tar -tf kubes-bin.tar
kubectl
master/
master/etcd
master/etcdctl
master/flanneld
master/kube-apiserver
master/kube-controller-manager
master/kube-scheduler
node/
node/flanneld
node/kube-proxy
node/kubelet
```
