Commit Graph

5917 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
d19773d855 Merge pull request #47835 from juju-solutions/feature/security
Automatic merge from submit-queue (batch tested with PRs 47850, 47835, 46197, 47250, 48284)

Securing the cluster created by Juju

**What this PR does / why we need it**: This PR secures the deployments done with Juju master. Works around certain security issues inherent to kubernetes (see for example dashboard access)

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
Securing Juju kubernetes dashboard
```
2017-06-29 15:16:39 -07:00
Kubernetes Submit Queue
d3aa0d5a8a Merge pull request #46850 from x13n/nanny-version
Automatic merge from submit-queue (batch tested with PRs 46850, 47984)

Update addon-resizer version

Update addon-resizer version and remove the flags that have been deprecated in the new version.

**What this PR does / why we need it**:
ref kubernetes/contrib#2623

**Special notes for your reviewer**:
Need to wait for merging kubernetes/contrib#2623 first.

**Release note**:
```release-note
addon-resizer flapping behavior was removed.
```
2017-06-29 07:18:32 -07:00
Kubernetes Submit Queue
7018479968 Merge pull request #48204 from shyamjvs/logdump-only-n-nodes
Automatic merge from submit-queue

Allow log-dumping only N randomly-chosen nodes in the cluster

This should let us save "lots" (~3-4 hours) of time in our 5000-node cluster scale tests as we copy logs from all the nodes to jenkins worker and then upload all of them to gcs (while we don't need too many).
This will also prevent the jenkins container facing "No space left on device" error while dumping logs, that we saw in runs 12-13 of gce-enormous-cluster.

The longterm fix will be to enable [logexporter](https://github.com/kubernetes/test-infra/tree/master/logexporter) for our tests.

cc @kubernetes/sig-scalability-misc @kubernetes/test-infra-maintainers @gmarek @fejta
2017-06-29 04:23:58 -07:00
Daniel Kłobuszewski
63ccedcfa7 Update addon-resizer version
Also, remove the flags that have been deprecated in the new version.
2017-06-29 11:03:43 +02:00
Shyam Jeedigunta
b960a0da12 Allow log-dumping only N randomly-chosen nodes in the cluster 2017-06-28 23:01:08 +02:00
Shyam Jeedigunta
cc8bb857f9 Allow creating special node for heapster in GCE 2017-06-28 21:27:36 +02:00
Kubernetes Submit Queue
a17f15a8a9 Merge pull request #48205 from piosz/heapster-1.4
Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Bumped Heapster to v1.4.0

``` release-note
Bumped Heapster to v1.4.0.
More details about the release https://github.com/kubernetes/heapster/releases/tag/v1.4.0
```

follow up #47961
The release candidate `v1.4.0-beta.0` turned out to be stable.
2017-06-28 10:35:12 -07:00
Kubernetes Submit Queue
63d4af44ac Merge pull request #48004 from dnardo/gke
Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Do not set CNI in cases where there is a private master and network policy provider is set.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
In GCE and in a "private master" setup, do not set the network-plugin provider to CNI by default if a network policy provider is given.
```
2017-06-28 10:35:10 -07:00
Kubernetes Submit Queue
ec729ad66d Merge pull request #48182 from gmarek/fluentd
Automatic merge from submit-queue (batch tested with PRs 48192, 48182)

Add generic NoSchedule toleration to fluentd in gcp config as a quick…

…-fix for #44445
2017-06-28 09:33:08 -07:00
Piotr Szczesniak
43280e274d Bumped Heapster to v1.4.0 2017-06-28 16:40:35 +02:00
gmarek
10ce8e2c0d Fix bug cluster-subnet logic 2017-06-28 14:27:52 +02:00
gmarek
3f57d8dba3 Add generic NoSchedule toleration to fluentd in gcp config as a quick-fix for #44445 2017-06-28 10:35:58 +02:00
Konstantinos Tsakalozos
0525b84a45 Disable anonymous-auth 2017-06-28 10:47:45 +03:00
Zach Loafman
903bc643b1 Bump GCE ContainerVM to container-vm-v20170627
Remove the built-in kubelet (finally), pick up security fixes.
2017-06-27 16:14:55 -07:00
Kubernetes Submit Queue
89579c45a4 Merge pull request #48054 from juju-solutions/bug/terminate-etcd
Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Add a failsafe for etcd not returning a connection string

**What this PR does / why we need it**: Removing a kubernetes-master will fail as described on this issue: https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311

**Special notes for your reviewer**: This is a two liner defensive code. I am not totally sold on this patch. I might not be the right place to address the above issue. However, solving the problem on the etcd side and updating the interface scope to be unit (as suggested) seems much more involving.

**Release note**:

```
Fix error when removing juju kubernetes-master unit
```
2017-06-27 14:08:19 -07:00
Kubernetes Submit Queue
f1b58f4e5f Merge pull request #48139 from crassirostris/fix-fluentd-config
Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Fix fluentd-gcp configuration to facilitate JSON parsing

There's a bug in https://github.com/kubernetes/kubernetes/pull/45734, because of which each records gets additional field and google-cloud plugin thinks it's not JSON (https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/blob/master/lib/fluent/plugin/out_google_cloud.rb#L569)

Fixes https://github.com/kubernetes/kubernetes/issues/48108

/cc @piosz @fgrzadkowski
2017-06-27 14:08:07 -07:00
Kubernetes Submit Queue
ede78d9ee7 Merge pull request #47513 from gmarek/subnet
Automatic merge from submit-queue

Make big clusters work again after introduction of subnets

This PR does two things: 
  - make IP aliases automatically pick Node IP Range based on number of Nodes,
  - fix logic for starting clusters >4095 Nodes that was broken by introduction of subnets,

cc @wojtek-t @shyamjvs 

```release-note
Setting env var ENABLE_BIG_CLUSTER_SUBNETS=true will allow kube-up.sh to start clusters bigger that 4095 Nodes on GCE.
```

Ref https://github.com/kubernetes/kubernetes/issues/47344
2017-06-27 08:52:50 -07:00
Kubernetes Submit Queue
d65b87a00d Merge pull request #47847 from chuckbutler/cluster-juju-approvers
Automatic merge from submit-queue

Insert Cynerva and Kjackal to approvers list

**What this PR does / why we need it**:
Per the membership reviews, we're looking to promote Konstantinos and
George to approvers to help distribute the review/bug load for the `cluster/juju` code
tree.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:
cc @marcoceppi and @tvansteenburgh 

**Release note**:

```release-note
NONE
```
2017-06-27 08:36:30 -07:00
Mik Vyatskov
b6a0e442ce Fix fluentd-gcp configuration to facilitate JSON parsing 2017-06-27 16:16:00 +02:00
Maciej Pytel
04f7a96340 Fix typo in cluster-autoscaler config 2017-06-27 13:49:19 +02:00
Maciej Pytel
b11175f73f Set cluster-autoscaler node balancing flag 2017-06-27 12:10:33 +02:00
Konstantinos Tsakalozos
0b01cd743b Improve security of Juju deployed clusters 2017-06-27 12:19:21 +03:00
Kubernetes Submit Queue
0dad2d0803 Merge pull request #47983 from yguo0905/memcg
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)

Enables memcg notification in cluster/node e2e tests

Ref: https://github.com/kubernetes/kubernetes/issues/42676

This PR sets Kubelet flag `--experimental-kernel-memcg-notification=true` when running cluster/node e2e tests on COS and Ubuntu images.

Tested:
```
e2e-node-cos:
I0623 00:09:06.641776    1080 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-777178888.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170739/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170739/pod-manifest571288056 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170739/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170739/cni/net.d --hostname-override tmp-node-e2e-bfe5799d-cos-stable-59-9460-64-0 --experimental-mounter-path=/tmp/node-e2e-20170622T170739/cluster/gce/gci/mounter/mounter --experimental-kernel-memcg-notification=true"

e2e-node-ubuntu:
I0623 00:03:28.526984    2279 server.go:147] Starting server "kubelet" with command "/usr/bin/systemd-run --unit=kubelet-1407651753.service --slice=runtime.slice --remain-after-exit /tmp/node-e2e-20170622T170203/kubelet --kubelet-cgroups=/kubelet.slice --cgroup-root=/ --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T170203/pod-manifest083943734 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%%,nodefs.inodesFree<5%% --eviction-minimum-reclaim nodefs.available=5%%,nodefs.inodesFree=5%% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T170203/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T170203/cni/net.d --hostname-override tmp-node-e2e-e48cdd73-ubuntu-gke-1604-xenial-v20170420-1 --experimental-kernel-memcg-notification=true"

e2e-node-containervm:
I0623 00:14:35.392383    2774 server.go:147] Starting server "kubelet" with command "/tmp/node-e2e-20170622T171318/kubelet --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --cgroup-root=/ --system-cgroups=/system --api-servers http://localhost:8080 --address 0.0.0.0 --port 10250 --read-only-port 10255 --volume-stats-agg-period 10s --allow-privileged true --serialize-image-pulls false --pod-manifest-path /tmp/node-e2e-20170622T171318/pod-manifest507536807 --file-check-frequency 10s --pod-cidr 10.100.0.0/24 --eviction-pressure-transition-period 30s --feature-gates  --eviction-hard memory.available<250Mi,nodefs.available<10%,nodefs.inodesFree<5% --eviction-minimum-reclaim nodefs.available=5%,nodefs.inodesFree=5% --v 4 --logtostderr --network-plugin=kubenet --cni-bin-dir /tmp/node-e2e-20170622T171318/cni/bin --cni-conf-dir /tmp/node-e2e-20170622T171318/cni/net.d --hostname-override tmp-node-e2e-9e3fdd7c-e2e-node-containervm-v20161208-image"

e2e-cos:
Jun 23 17:54:38 e2e-test-ygg-minion-group-t5r0 kubelet[2005]: I0623 17:54:38.646374    2005 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-ubuntu:
Jun 23 18:25:27 e2e-test-ygg-minion-group-19qp kubelet[1547]: I0623 18:25:27.722253    1547 flags.go:52] FLAG: --experimental-kernel-memcg-notification="true"

e2e-containervm:
I0623 18:55:51.886632    3385 flags.go:52] FLAG: --experimental-kernel-memcg-notification="false"
```

**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @dchen1107 @dashpole
2017-06-26 21:08:10 -07:00
Kubernetes Submit Queue
e8701e6746 Merge pull request #47894 from xuxinkun/fixclustercentos20170622
Automatic merge from submit-queue (batch tested with PRs 48092, 47894, 47983)

fix systemd service file for custom args.

`KUBE_SCHEDULER_ARGS` and `KUBELET_ARGS` are used to custom args for scheduler or kubelet by users. 
But if there are more than one params in `KUBELET_ARGS`, for example, if I set  KUBELET_ARGS="--cgroups-per-qos=false --enforce-node-allocatable=", the kubelet will judge the `false --enforce-node-allocatable=` as the value of `cgroups-per-qos`.  Because `${KUBELET_ARGS}` in kubelet.service will expands the variable into one word. And if I take `$KUBELET_ARGS` instead, kubelet will worker perfectly.
For more info, please click [EnvironmentFiles and support for /etc/sysconfig files](http://fedoraproject.org/wiki/Packaging:Systemd#EnvironmentFiles_and_support_for_.2Fetc.2Fsysconfig_files). This bug is reported by @huanxingyouyoutoo. And I make this PR for her to fix it.

**Release note**:

```
NONE
```
2017-06-26 21:08:08 -07:00
Kubernetes Submit Queue
9a4a0cb31d Merge pull request #47178 from ktsakalozos/bug/worker-labels
Automatic merge from submit-queue (batch tested with PRs 48012, 47443, 47702, 47178)

Fix setting juju worker labels during deployment

**What this PR does / why we need it**: Allows for setting the labels of juju workers during deployment (eg inside a bundle)

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #47176

**Special notes for your reviewer**:

**Release note**:

```
Fix bug in setting Juju kubernetes-worker labels in bundle.yaml files. 
```
2017-06-26 17:48:06 -07:00
Kubernetes Submit Queue
ec1fcbed74 Merge pull request #47170 from ktsakalozos/bug/restart
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)

Fix restart action on juju kubernetes-master

**What this PR does / why we need it**: Restart action of kubernetes-master of Juju is not functioning. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/299

**Special notes for your reviewer**:

**Release note**:

```
Fix: Restart action of juju's kubernetes-master restarts the respective snap based services
```
2017-06-26 14:03:06 -07:00
Kubernetes Submit Queue
93ef2f2f59 Merge pull request #47860 from crassirostris/fluentd-log-to-stdio
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)

Make fluentd log to stdio instead of a dedicated file

Lower verbosity also, to reduce volume of system logs exported to the backend.

Fix https://github.com/kubernetes/kubernetes/issues/43772

/cc @piosz
2017-06-26 14:03:04 -07:00
gmarek
536f48ef15 Fix test commands in cluster/gce/util.sh 2017-06-26 21:27:04 +02:00
gmarek
64f6606833 Make big clusters work again after introduction of subnets 2017-06-26 21:27:04 +02:00
Kubernetes Submit Queue
8cbe10497e Merge pull request #48044 from bowei/kube-dns-1.14.4
Automatic merge from submit-queue (batch tested with PRs 48074, 47971, 48044, 47514, 47647)

Update kube-dns to 1.14.4

- Fixes broken arm dnsmasq image
- Fixes kube-dns log spam issue

https://github.com/kubernetes/dns/issues/111

```release-note
none
```
2017-06-26 11:40:56 -07:00
Yang Guo
50d49d9c51 Enables memcg notification in cluster/node e2e tests 2017-06-26 11:40:22 -07:00
Kubernetes Submit Queue
cb1f5605a6 Merge pull request #48074 from mwielgus/ca-0.6.0
Automatic merge from submit-queue

Bump Cluster Autoscaler to 0.6.0

```release-note
Cluster Autoscaler 0.6
```
cc: @MaciekPytel @aleksandra-malinowska @wasylkowski
2017-06-26 11:08:38 -07:00
Marcin Wielgus
083f46a7f7 Bump Cluster Autoscaler to 0.6.0 2017-06-26 17:32:48 +02:00
Piotr Szczesniak
77fd8a1898 Revert "Decrese fluentd cpu request" 2017-06-26 16:18:24 +02:00
Konstantinos Tsakalozos
b3a1115796 Add a failsafe for etcd not returning a connection string 2017-06-26 13:00:17 +03:00
Kubernetes Submit Queue
e725c8fca7 Merge pull request #47961 from piosz/heapster-1.4
Automatic merge from submit-queue (batch tested with PRs 47961, 46276)

Bumped Heapster to v1.4.0-beta.0

Heapster release candidate for Kubernetes 1.7

cc @dchen1107 @caesarxuchao
2017-06-26 02:11:52 -07:00
Bowei Du
85025ef7dc Update kube-dns to 1.14.4
- Fixes broken arm dnsmasq image
- Fixes kube-dns log spam issue
2017-06-25 23:53:37 -07:00
Kubernetes Submit Queue
d0ee6bb12a Merge pull request #47776 from jianzhangbjz/build-hyperkube-ppc64le
Automatic merge from submit-queue

Parameterize the binary path and host arch for the hyperkube image

As the [cluster/images/hyperkube/README.md](https://github.com/kubernetes/kubernetes/tree/master/cluster/images/hyperkube) shows, I run the command: `make build VERSION=test ARCH=ppc64le`, but got the below errors, so this PR will fix it.
```
ARCH=ppc64le
cp -r ./* /tmp/hyperkubeTFbYrI
mkdir -p /tmp/hyperkubeTFbYrI/cni-bin
cp ../../../_output/dockerized/bin/linux/ppc64le/hyperkube /tmp/hyperkubeTFbYrI
cp: cannot stat '../../../_output/dockerized/bin/linux/ppc64le/hyperkube': No such file or directory
Makefile:62: recipe for target 'build' failed
make: *** [build] Error 1
```
2017-06-24 04:12:42 -07:00
Kubernetes Submit Queue
14edc46c2e Merge pull request #47892 from ajitak/npd-config
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)

Bump up npd version to v0.4.1

```
Bump up npd version to v0.4.1
```

Fixes #47219
2017-06-23 18:05:46 -07:00
Kubernetes Submit Queue
de86a83535 Merge pull request #47993 from dnardo/ip-masq-agent
Automatic merge from submit-queue (batch tested with PRs 47993, 47892, 47591, 47469, 47845)

Use a different env var to enable the ip-masq-agent addon.

We shouldn't mix setting the non-masq-cidr with enabling the addon.



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

https://github.com/kubernetes/kubernetes/issues/47865
2017-06-23 18:05:44 -07:00
Daniel Nardo
00b62df554 Do not set CNI on a private master when enabling network policy. 2017-06-23 17:07:56 -07:00
Kubernetes Submit Queue
a78898964c Merge pull request #47982 from timstclair/audit-policy
Automatic merge from submit-queue (batch tested with PRs 47883, 47179, 46966, 47982, 47945)

Strip versions from known api groups in audit policy

Props to @CaoShuFeng for catching this.

Issue: kubernetes/features#22

/cc @ericchiang
2017-06-23 15:48:43 -07:00
Daniel Nardo
c75de4e39f Use a different env var to enable the ip-masq-agent addon.
We shouldn't mix setting the non-masq-cidr with enabling the addon.
2017-06-23 14:47:49 -07:00
Tim St. Clair
ed8ea24f65 Strip versions from known api groups in audit policy 2017-06-23 11:55:45 -07:00
Marcin Wielgus
f90d894ede Bump Cluster Autoscaler to 0.6.0-beta2 2017-06-23 19:41:32 +02:00
Piotr Szczesniak
98e599b40d Bumped Heapster to v1.4.0-beta.0 2017-06-23 14:35:14 +02:00
Kubernetes Submit Queue
5f82f129df Merge pull request #46203 from simt2/fluentd-elasticsearch-rbac
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)

Add RBAC support to fluentd-elasticsearch cluster addon

**What this PR does / why we need it**:
Adds rbac support to the fluentd-elasticsearch addon

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #46023 

**Special notes for your reviewer**:

**Release note**:

```release-note
Add RBAC support to fluentd-elasticsearch cluster addon
```
2017-06-23 05:08:28 -07:00
Kubernetes Submit Queue
509c4351df Merge pull request #47507 from yiqinguo/yiqinguo_es_addargs
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)

es discovery support args apiserver-host and kubeconfig

Now discovery elasticsearch through kubernetes client,but now does not support specifying the apiserver-host or kubeconfig create client.
2017-06-23 05:08:26 -07:00
Kubernetes Submit Queue
0cbd0ca189 Merge pull request #47915 from crassirostris/fix-event-exporter-noise
Automatic merge from submit-queue

Bump event-exporter version to reduce warnings noise

Fixes https://github.com/kubernetes/kubernetes/issues/47914
2017-06-23 03:45:20 -07:00
Kubernetes Submit Queue
cdc9770346 Merge pull request #46792 from ianchakeres/avoid-redundant-copy-to-staging
Automatic merge from submit-queue (batch tested with PRs 47403, 46646, 46906, 46527, 46792)

Avoid redundant copying of tars during kube-up for gce if the same file already exists

**What this PR does / why we need it**: 

Whenever I execute cluster/kube-up.sh it copies my tar files to google cloud, even if the files haven't changed. This PR checks to see whether the files already exist, and avoids uploading them again. These files are large and can take a long time to upload.

**Which issue this PR fixes**: fixes #46791

**Special notes for your reviewer**:

Here is the new output:

cluster/kube-up.sh 
... Starting cluster in us-central1-b using provider gce
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
Project: PROJECT
Zone: us-central1-b
+++ Staging server tars to Google Storage: gs://kubernetes-staging-PROJECT/kubernetes-devel
+++ kubernetes-server-linux-amd64.tar.gz uploaded earlier, cloud and local file md5 match (md5 = 3a095kcf27267a71fe58f91f89fab1bc)


**Release note**:
```cluster/kube-up.sh on gce now avoids redundant copying of kubernetes tars if the local and cloud files' md5 hash match```
2017-06-23 02:59:31 -07:00