Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
ExecProbes should be able to do simple env var substitution
For containers that don't have bash, we should support env substitution
like we do on command and args. However, without major refactoring
valueFrom is not supportable from inside the prober. For now, implement
substitution based on hardcoded env and leave TODOs for future work.
Improves the state of #40846, will spawn a follow up issue for future refactoring after CRI settles down
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
Don't sync IPtables before underlying store/reflector is fully synced
Ref #42108
Build on top of #42108 - only the second commit is unique.
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
Keep entries of apimachinery in client-go's Godeps.json
1. copy.sh now copies staging/ to a temp dir, creates snapshots for them, and then adds the temp dir to GOPATH before running `godep save`. This way, we don't need the `go list` check added in #41987 so that's removed @sttts.
2. copy.sh fills the entries for k8s.io/apimachinery with dummy SHA1 in client-go's Godeps.json. The publish robot will later fill the actually published one.
3. copy.sh removes the entire `_vendor` from client-go. We cannot keep `_vendor` because it makes hard to use the `git filter-branch` magic for client-go, because the filtered commits might contain changes to `_vendor` while client-go only has `vendor/`.
Some notes on point 3: the robot will restore the vendor/ later, but exclude k8s.io/* and glog, so that users retrieving client-go with `go get` won't end up with multiple copies of them. I think this should be a ***general rule*** for published k8s.io/* repos: every published repos should have complete Godeps.json with latest commit hashes of other k8s.io/* repos, but vendor/k8s.io/* and vendor/glog will be removed from all repos.
@sttts @lavalamp @deads2k
Automatic merge from submit-queue (batch tested with PRs 41672, 42084, 42233, 42165, 42273)
remove azure getting kube-ups.
Haven't been touched in > 7 months.
@colemickens , i"m going to send out an email about this.
```release-note
Remove Azure kube-up as the Azure community has focused efforts elsewhere.
```
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
apiserver/pkg/server: include scheme in insecure self client config
Noticed this during bootkube development: https://github.com/kubernetes-incubator/bootkube/issues/325
In Go 1.8's `url.Parse` became more strict, and `url.Parse("127.0.0.1:8080")` now fails.
https://beta.golang.org/doc/go1.8#net_urlhttps://play.golang.org/p/dw_cPeotG4
Accidentally compiled bootkube with 1.8 and tracked a panic down to the loopback client config. Though we're still using the old "genericapiserver", this seems to be translated from the old one.
The actual panic we observed was
```
E0222 19:40:11.364949 5 server.go:254] Failed to create clientset: parse 127.0.0.1:8080: first path segment in URL cannot contain colon
panic: parse 127.0.0.1:8080: first path segment in URL cannot contain colon
goroutine 35 [running]:
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/client/clientset_generated/internalclientset/typed/core/internalversion.NewForConfigOrDie(0xc420728ea0, 0xc420738e30)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/client/clientset_generated/internalclientset/typed/core/internalversion/core_client.go:132 +0x62
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/master.(*Config).Complete(0xc420739db8, 0x0)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/pkg/master/master.go:180 +0x40a
github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc42007a3c0, 0x0, 0x0)
/home/eric/src/github.com/kubernetes-incubator/bootkube/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:347 +0x1e8d
github.com/kubernetes-incubator/bootkube/pkg/bootkube.(*bootkube).Run.func1(0xc4206b01e0, 0xc420164300)
/home/eric/src/github.com/kubernetes-incubator/bootkube/pkg/bootkube/bootkube.go:124 +0x2f
created by github.com/kubernetes-incubator/bootkube/pkg/bootkube.(*bootkube).Run
/home/eric/src/github.com/kubernetes-incubator/bootkube/pkg/bootkube/bootkube.go:124 +0xb0
```
I don't actually know if this is the correct fix or if there should be changes to `NewForConfigOrDie`. Am looking for comments more than anything.
edit: @abourget pointed out over on bootkube that the actual panic was fixed in the internalclient by https://github.com/kubernetes/kubernetes/pull/38519.
cc @deads2k @sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
allow subject access review to non-existent namespace
A localsubjectaccessreview is a special kind of resource which can be created even when the namespace doesn't exist. Since permissions can be granted at different scopes, you can reasonably check if someone *could* do something at a lower scope that isn't there yet. In addition, the permission to do an access check is separate from the permission to list all namespaces, so we're leaking information.
@liggitt @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
fix LeaderTransitions always zero
on leader transition, LeaderTransitions is increased to 1, but
then cleared to zero by next renew.
External monitoring system may watch LeaderTransitions and rely
on it's facticity.
Automatic merge from submit-queue (batch tested with PRs 42126, 42130, 42232, 42245, 41932)
Update fluentd-gcp configuration for hosted masters
This PR makes use of the new fluentd-gcp image, which is not configured per se, for the hosted masters, which cannot use configmaps.
Mirroring https://github.com/kubernetes/kubernetes/pull/42126
Automatic merge from submit-queue
Move fluentd DS config to configmap
This is the logical continuation of https://github.com/kubernetes/kubernetes/pull/41998. This PR makes fluentd-gcp DaemonSet use the new image configured using ConfigMap.
This PR doesn't change the way fluentd-gcp works in case master is not registered, that'll be fixed in a separate PR
CC @ixdy @timstclair @igorpeshansky @qingling128 @dhrupadb
**Release note:**
```release-note
Fluentd-gcp containers spawned by DaemonSet are now configured using ConfigMap
```
Automatic merge from submit-queue (batch tested with PRs 41980, 42192, 42223, 41822, 42048)
Adjust parameters of GCL cluster logging load tests
This PR increases the amount of logs produced in load tests to match the number of nodes and provide the predictable load of 100 KB/sec on each node.
Also this PR reduces in half amount of time, given for ingesting logs.
Automatic merge from submit-queue (batch tested with PRs 41980, 42192, 42223, 41822, 42048)
Take into account number of restarts in cluster logging tests
Before, in cluster logging tests, we only measured e2e number of lines delivered to the backend.
Also, befure https://github.com/kubernetes/kubernetes/pull/41795 was merged, from the k8s perspective, fluentd was always working properly, even if it's crashlooping inside.
Now we can detect whether fluentd is truly working properly, experiencing no, or almost no OOMs duing its operation.
Automatic merge from submit-queue (batch tested with PRs 41980, 42192, 42223, 41822, 42048)
CRI: Make dockershim better implements CRI.
When thinking about CRI Validation test, I found that `PodSandboxStatus.Linux.Namespaces.Options.HostPid` and `PodSandboxStatus.Linux.Namespaces.Options.HostIpc` are not populated. Although they are not used by kuberuntime now, we should populate them to conform to CRI.
/cc @yujuhong @feiskyer
Automatic merge from submit-queue (batch tested with PRs 41980, 42192, 42223, 41822, 42048)
Modified kubemark startup scripts to restore master on reboot
Fixes#41735
As discussed in the issue, modified the scripts to satisfy the conditions of restoring master env, running non-idempotent operations only for the first time and persist important data like pki/auth files on a PD.
Also attached `start-kubemark-master.sh` as startup-script metadata to master instance (on GCE) so that it is called automatically on each boot.
cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Use `docker logs` directly if the docker logging driver is not `json-file`
Fixes https://github.com/kubernetes/kubernetes/issues/41996.
Post the PR first, I still need to manually test this, because we don't have test coverage for journald logging pluggin.
@yujuhong @dchen1107
/cc @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Adding legacy ABAC for 1.6
This is a fork of a previous [pull request](https://github.com/kubernetes/kubernetes/pull/42014) to include feedback as the original author is unavailable.
Adds a mechanism to optionally enable legacy abac for 1.6 to provide a migration path for existing users.
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Revert conditional updates to configmap that breaks ingress
Don't prevent configmap updates to happen on existing ingress-uid configmap despite the lack of ingress objects. Otherwise, ingress objects get created with the wrong name.
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Apiserver: wait for Etcd to become available on startup
fixes#37704
Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)
Admission Controller: Add Pod Preset
Based off the proposal in https://github.com/kubernetes/community/pull/254
cc @pmorie @pwittrock
TODO:
- [ ] tests
**What this PR does / why we need it**: Implements the Pod Injection Policy admission controller
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Added new Api `PodPreset` to enable defining cross-cutting injection of Volumes and Environment into Pods.
```
Automatic merge from submit-queue
Add RBAC roles for bootstrap controllers
Supercedes https://github.com/kubernetes/kubernetes/pull/42221
When locking down controllers to individual RBAC roles we need to make sure that the bootstrap controllers have the right permissions.
This adds the roles and bindings at the correct namespace scopes for the bootstrap-signer and token-cleaner controllers.
@liggitt ptal
@jbeda @luxas you got a good way to test this? It must not be covered in normal e2e or we'd've seen the issue before.
Automatic merge from submit-queue
AWS: run k8s master in different account or different provider
Currently the master and the nodes must run in the same account. With this change the master can run in a different AWS account, on a different cloud provider or on premise.
## Release Notes
```release-note
AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes.
```
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Update defaultbackend image to 1.3
Update `gcr.io/google-containers/defaultbackend` to the latest version.
See https://github.com/kubernetes/contrib/pull/2386
/cc @ixdy
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Burstable QoS cgroup has cpu shares assigned
**What this PR does / why we need it**:
This PR sets the Burstable QoS cgroup cpu shares value to the sum of the pods cpu requests in that tier. We need it for proper evaluation of CPU shares in the new QoS hierarchy.
**Special notes for your reviewer**:
It builds against the framework proposed for https://github.com/kubernetes/kubernetes/pull/41833
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Ingress-glbc upgrade tests
Basically #41676 but with some fixes and added comments. @bprashanth has been away this week and it's desirable to have this in before code freeze.
Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)
Ensure pod cgroup is deleted prior to deletion of pod
**What this PR does / why we need it**:
This PR ensures that the kubelet removes the pod cgroup sandbox prior to deletion of a pod from the apiserver. We need this to ensure that the default behavior in the kubelet is to not leak resources.
Automatic merge from submit-queue
Re-writing of the resolv.conf file generated by docker
Fixes#17406
Docker 1.12 will contain feature "The option --dns and --net=host should not be mutually exclusive" (docker/docker#22408)
This patch adds optional support for this ability in kubelet (for now in case of "hostNetwork: true" set all dns settings are ignored if any).
To enable feature use newly added kubelet flag: --allow-dns-for-hostnet=true
export functions from pkg/api/validation
add settings API
add settings to pkg/registry
add settings api to pkg/master/master.go
add admission control plugin for pod preset
add new admission control plugin to kube-apiserver
add settings to import_known_versions.go
add settings to codegen
add validation tests
add settings to client generation
add protobufs generation for settings api
update linted packages
add settings to testapi
add settings install to clientset
add start of e2e
add pod preset plugin to config-test.sh
Signed-off-by: Jess Frazelle <acidburn@google.com>
Automatic merge from submit-queue
HPA Controller: Use Custom Metrics API
This commit switches over the HPA controller to use the custom metrics
API. It also converts the HPA controller to use the generated client
in k8s.io/metrics for the resource metrics API.
In order to enable support, you must enable
`--horizontal-pod-autoscaler-use-rest-clients` on the
controller-manager, which will switch the HPA controller's MetricsClient
implementation over to use the standard rest clients for both custom
metrics and resource metrics. This requires that at the least resource
metrics API is registered with kube-aggregator, and that the controller
manager is pointed at kube-aggregator. For this to work, Heapster
must be serving the new-style API server (`--api-server=true`).
Before this merges, this will need kubernetes/metrics#2 to merge, and a godeps update to pull that in.
It's also semi-dependent on kubernetes/heapster#1537, but that is not required in order for this to merge.
**Release note**:
```release-note
Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs.
```
Automatic merge from submit-queue
statefulset: wait for pvc cache sync
#42056 switched the statefulset controller to use the pvc shared informer/lister, but accidentally left out waiting for its cache to sync.
cc @kubernetes/sig-apps-pr-reviews @kargakis @foxish @kow3ns @smarterclayton @deads2k
Automatic merge from submit-queue
fix rsListerSynced and podListerSynced for DeploymentController
**What this PR does / why we need it**:
There is a mistake when initializing `DeploymentController`'s `rsListerSynced` and `podListerSynced` in `NewDeploymentController`, they are all initialized to `Deployment`'s `Informer`, so the `DeploymentController` maybe running before the `ReplicaSet` cache and `Pod` cache has been synced.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Indeed according unit test is neccessary, but this bug fix is simple, and if the tests is neccessary I will submit another PR later.
**Release note**:
```release-note
```
Automatic merge from submit-queue
Implement support for mount options in PVs
**What this PR does / why we need it**:
This PR implements support for mount options in PersistentVolume via `volume.beta.kubernetes.io/mount-options` annotation.
**Which issue this PR fixes**
Fixes https://github.com/kubernetes/features/issues/168
**Release note**:
```
Enable additional, custom mount options to be passed to PersistentVolume objects via volume.beta.kubernetes.io/mount-options annotation.
```
Automatic merge from submit-queue
Extensible Userspace Proxy
This PR refactors the userspace proxy to allow for custom proxy socket implementations.
It changes the the ProxySocket interface to ensure that other packages can properly implement it (making sure all arguments are publicly exposed types, etc), and adds in a mechanism for an implementation to create an instance of the userspace proxy with a non-standard ProxySocket.
Custom ProxySockets are useful to inject additional logic into the actual proxying. For example, our idling proxier uses a custom proxy socket to hold connections and notify the cluster that idled scalable resources need to be woken up.
Also-Authored-By: Ben Bennett bbennett@redhat.com
Introduced chages:
1. Re-writing of the resolv.conf file generated by docker.
Cluster dns settings aren't passed anymore to docker api in all cases, not only for pods with host network:
the resolver conf will be overwritten after infra-container creation to override docker's behaviour.
2. Added new one dnsPolicy - 'ClusterFirstWithHostNet', so now there are:
- ClusterFirstWithHostNet - use dns settings in all cases, i.e. with hostNet=true as well
- ClusterFirst - use dns settings unless hostNetwork is true
- Default
Fixes#17406
