Commit Graph

841 Commits

Author SHA1 Message Date
Koonwah Chen
46ff8e6b57 Add env var(CNI_STORAGE_PATH) for cni storage path. 2019-06-24 11:47:14 -07:00
Gustavo Paiva
ca3519c7ad Clean up selft-set node labels 2019-06-20 00:07:31 -03:00
Vallery Lancey
dc0f14312e Removed deprecated --resource-container flag from kube-proxy. 2019-06-16 08:36:42 -07:00
Maciej Borsz
31f18c0a6d Modify firewall rules names to make them shorter. 2019-06-05 13:18:53 +02:00
Michael Taufen
a66cb353d4 Disable KubeletPodResources on Windows
The feature caused tests to fail when it was enabled.

- https://github.com/kubernetes/kubernetes/issues/78628

Work is in progress to fix the feature, but until that work is complete,
we will disable it in the GCE scripts.
2019-06-03 15:39:55 -07:00
Kubernetes Prow Robot
6d70e7ff01
Merge pull request #77930 from mm4tt/upload_kubeconfig
Upload kubeconfig to master metadata in tests.
2019-05-30 06:30:24 -07:00
Kubernetes Prow Robot
5c314535d1
Merge pull request #78183 from cheftako/masterTaint
Ensure kubernetes master is properly tainted in GCE.
2019-05-29 17:24:46 -07:00
Matt Matejczyk
7e7570d6ff Upload kubeconfig to master metadata in tests.
Till a few days ago, it was possible to ssh into master and access cluster via insecure master port.
Now, the master insecure port has been disabled, we're not able to do that anymore.

This PR aims to fix that by uploading the kubeconfig to the master metadata during cluster setup in tests.
2019-05-24 13:25:19 +02:00
Matt Matejczyk
6ced6491c6 Change etcd's --listen-client-urls to 0.0.0.0 in tests
This is to allow scraping etcd metrics in scalability tests.

Ref. https://github.com/kubernetes/perf-tests/issues/522
2019-05-23 15:11:22 +02:00
Kubernetes Prow Robot
4f33b5f421
Merge pull request #77820 from dims/charrywanganthony-pr-71835-delete-kubelet-security-controls
Remove deprecated Kubelet security controls
2019-05-22 13:52:25 -07:00
walter
d574ab00e4 Ensure kubernetes master is properly tainted in GCE.
Factored in mtaufens suggestion.
2019-05-21 23:41:56 -07:00
Kubernetes Prow Robot
72f6954614
Merge pull request #77889 from Random-Liu/support-using-containerd-in-cos
Support using docker containerd in COS and Ubuntu on GCE.
2019-05-17 20:26:59 -07:00
Lantao Liu
bc1a78d31e Support using docker containerd in COS and Ubuntu on GCE.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-05-16 13:53:57 -07:00
chaowang
9ba0d47465
Remove deprecated Kubelet security controls
Change-Id: I9f770e6780b60e881092690f6f177d726f474fe3
2019-05-16 13:33:52 -04:00
Maciej Borsz
59af63c687 Make coredns memory limit configurable 2019-05-15 13:35:28 +02:00
immutablet
1ae9cb2f88 Remove encryption via locally stored key. 2019-05-02 15:09:02 -07:00
Krzysztof Siedlecki
8dc0be6b9e creating instance groups in parallel 2019-04-30 18:42:09 +02:00
Peter Hornyack
81cc384a9a Pin GCE Windows node image to 1809 v20190312.
This is to work around
https://github.com/kubernetes/kubernetes/issues/76666.
2019-04-17 09:45:43 -07:00
Matt Matejczyk
2efea76263 Ability to set up Private GCE Clusters with NAT.
Ref. https://github.com/kubernetes/kubernetes/issues/76374
2019-04-12 13:21:56 +02:00
Lantao Liu
ad5b64ca10 Add envs for CNI version.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-09 16:38:53 -07:00
Ke Wu
925dc7f2e0 Fix empty array expansion error in cluster/gce/util.sh
Empty array expansion causes "unbound variable" error in
bash 4.2 and bash 4.3.
2019-04-04 16:46:58 -07:00
Rohit Ramkumar
8ac15cd54a Plumb CUSTOM_INGRESS_YAML into other setup scripts so that it can be set before a call to kubetest 2019-03-15 13:36:30 -07:00
Michael Taufen
0d59d2a43e Windows kubeproxy flags cleanup
- Pass most of the flags via kube-env, similar to Kubelet, to make it
  easier to reconfigure.
- Use kube-proxy's kubeconfig file to pass the API server address,
  rather than the --master flag.
- Make KUBEPROXY_ARGS a required variable in Windows kube-env, similar
  to KUBELET_ARGS.
2019-03-07 14:45:34 -08:00
Kubernetes Prow Robot
ab7a48d796
Merge pull request #70036 from pbarker/audit-etoe
dynamic audit e2e test
2019-03-06 17:58:58 -08:00
Kubernetes Prow Robot
45e5f6053b
Merge pull request #74424 from liggitt/drop-k8s-io-node-labels
Clean up self-set node labels
2019-03-06 08:24:26 -08:00
Patrick Barker
45d715cdc6 adds dynamic audit e2e test 2019-03-05 14:38:58 -07:00
Zhen Wang
efa96f7eb8 allows configuring NPD release and flags on GCI and add cluster e2e test 2019-02-26 21:21:54 -08:00
Jordan Liggitt
943b32a289 Prepare switch from beta.kubernetes.io/kube-proxy-ds-ready to node.kubernetes.io/kube-proxy-ds-ready 2019-02-26 11:42:23 -05:00
Peter Hornyack
3efd4ca1dc Enhance/repair detect-node-names() and related env vars for Windows nodes. 2019-02-22 14:56:55 -08:00
Peter Hornyack
6d78f2b666 Default to Windows Server version 1809 for Windows nodes.
Removes all references to 1803, including moving "win1803" directory to
just "windows". A single Windows directory suffices for now, if
necessary in the future we can shard it into directories for each
Windows version.

We've been running tests with Windows 1809 nodes for a couple days in
our fork without major problems:
https://testgrid.k8s.io/google-windows#windows-prototype&width=20.
Testing on Azure is already using 1809:
https://testgrid.k8s.io/sig-windows#Conformance%20acs-engine%20on%20Azure&width=20.
2019-02-21 09:44:44 -08:00
Peter Hornyack
063725230d Delete RDP firewall rule during kube-down.
This rule is only created if the cluster has one or more Windows nodes,
but delete-firewall-rules() ignores firewall rule arguments that do not
exist so it's safe to always attempt to delete this rule.
2019-02-12 15:19:16 -08:00
Matt Matejczyk
4dcc017d7a Propagate API_SERVER_TEST_LOG_LEVEL in cluster/gce/util.sh 2019-02-05 10:30:17 +01:00
Maciej Borsz
afeabe99b7 Fix typo in create-heapster-node 2019-02-01 11:04:23 +01:00
Peter Hornyack
f0f7829934 Update cluster/gce scripts to support Windows nodes. 2019-01-30 19:49:07 -08:00
Wenjia Zhang
c17233c41c add mTLS encription between etcd and kube-apiserver in GCE 2019-01-07 13:52:20 -08:00
Yu-Ju Hong
f87fcf6ce8 GCE: enable downloading and staging of the node binaries
Currently, only kubernetes-server-linux-<arch>.tar.gz will be downloaded
and staged for tests. It is not possible to test a cluster where the
platform/arch of the nodes differs from that of the master
control-plane. This changes allows downloading and staging node
binaries, in addition to the existing server binaries.
2018-12-10 16:58:44 -08:00
Matt Brannock
d99422884b Create managed instance groups for nodes in parallel 2018-12-04 13:56:52 -08:00
k8s-ci-robot
79dab474c0
Merge pull request #70555 from prameshj/nodelocaldnscache
Support running a nodelocal dns cache
2018-11-14 00:18:56 -08:00
Pavithra Ramesh
73b548db06 Support running a nodelocal dns cache
This change includes the yaml files and gce startup script changes
to run this addon. It is disabled by default, can be enabled by setting
KUBE_ENABLE_NODELOCAL_DNS=true
An ip address is required for the cache instance to listen for
requests on, default is a link local ip address of value 169.254.25.10

addressed review comments, updated image location
Picked a different prometheus port so stats port is not same as the
coredns deployment

Removed the nodelocaldns-ready label.
Set memory limit to 30Mi
2018-11-12 14:54:20 -08:00
Mike Danese
f3611a6264 always enable TokenRequest in GCE kube-up.sh 2018-11-12 11:24:52 -08:00
Chris O'Haver
0cfb4bbe2c make coredns kubeup default, update manifest 2018-10-18 14:52:38 -04:00
Marian Lobur
3928818b64 Enable audit logging truncating backend. 2018-10-02 07:58:01 +02:00
Yu Liao
f4482e3abb "gcloud beta compute networks subnets" stopped working, fails with the
erros:  - The request did not match the specified API.
"gcloud compute networks subnets" works with the latest gcloud release.
2018-09-28 15:42:20 -07:00
Shyam JVS
73b014b51a
Revert "Enable CoreDNS as default for kube-up deployments" 2018-09-13 18:57:01 +02:00
Davanum Srinivas
ae0dde5c7d
Remove e2e-image-puller
A long time ago, We added the image prepulling as a workaround due to
the overwhelming amount of flake caused by pulling during the tests.
This functionality has been broken for a while now when we switched to a
COS image where mounting `docker` binary into `busybox` stopped working.
So we just have dead code we should clean up.

Change-Id: I538171a5c1d9361eee7f9e0a99655b88b1721e3e
2018-09-03 20:43:54 -04:00
Kubernetes Submit Queue
1cfeede6d5
Merge pull request #68143 from shyamjvs/garbage-collect-kubemark-image
Automatic merge from submit-queue (batch tested with PRs 68127, 68143). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

GC kubemark image after cluster starts up

Last piece to fix https://github.com/kubernetes/kubernetes/issues/59567

/cc @wojtek-t 

```release-note
NONE
```
2018-09-03 04:45:32 -07:00
Kubernetes Submit Queue
e4b0a935fa
Merge pull request #68152 from mikedanese/nopush
Automatic merge from submit-queue (batch tested with PRs 67571, 67284, 66835, 68096, 68152). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

cluster: delete some push remanents

```release-note
NONE
```
2018-08-31 20:41:46 -07:00
Kubernetes Submit Queue
2d7b92ee74
Merge pull request #68100 from vishh/add-node-termination-handle
Automatic merge from submit-queue (batch tested with PRs 64283, 67910, 67803, 68100). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Adding GCE node termination handler as an optional addon. 

This step is a pre-requisite for auto-deploying that addon in GKE

cc @mikedanese
2018-08-31 16:46:45 -07:00
Mike Danese
1c41211456 cluster: delete some kube-push remanents 2018-08-31 14:48:31 -07:00
Vishnu kannan
ee65e6ac04 Adding GCE node termination handler as an optional addon.
This step is a pre-requisite for auto-deploying that addon in GKE.

Signed-off-by: Vishnu kannan <vishnuk@google.com>
2018-08-31 12:47:37 -07:00
Shyam Jeedigunta
63ce112d9e GC kubemark image after cluster starts up 2018-08-31 18:56:36 +02:00
Kubernetes Submit Queue
eff30a31a7
Merge pull request #68120 from wojtek-t/configurable_etcd_servers
Automatic merge from submit-queue (batch tested with PRs 67756, 64149, 68076, 68131, 68120). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Allow for configuring etcd servers addresses in gce
2018-08-31 09:06:42 -07:00
wojtekt
122b7a5f82 Allow for configuring etcd servers 2018-08-31 12:37:13 +02:00
Kubernetes Submit Queue
58ead1c1e8
Merge pull request #65862 from loburm/remove_basic_audit
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Promote AdvancedAuditing to GA

**What this PR does / why we need it**:
Removes deprecated legacy code used for basic audit logging in favor of advanced audit logging.

```release-note
Promote AdvancedAuditing to GA, replacing the previous (legacy) audit logging mechanisms.
```
2018-08-30 07:31:23 -07:00
Kubernetes Submit Queue
7c2a9a0150
Merge pull request #67943 from Szetty/master
Automatic merge from submit-queue (batch tested with PRs 67745, 67432, 67569, 67825, 67943). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add flag for disabling prometheus-to-sd only for daemon sets

```release-note
NONE
```
2018-08-29 15:19:45 -07:00
Kubernetes Submit Queue
ae6840fa3a
Merge pull request #67569 from fturib/coredns-default
Automatic merge from submit-queue (batch tested with PRs 67745, 67432, 67569, 67825, 67943). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Enable CoreDNS as default for kube-up deployments

**What this PR does / why we need it**:
Enable CoreDNS as default (for kube-up installations)
It will allow to run CI tests to prepare graduation criteria for CoreDNS as Default

see : KEP - https://github.com/kubernetes/community/blob/master/keps/sig-network/0012-20180518-coredns-default-proposal.md
see also : https://github.com/kubernetes/features/issues/566

NOTE for release : I guess that CoreDNS as default server for k8s needs a longer description. This specific PR is to ensure we validate all e2e.

```release-note
Make CoreDNS be the default DNS server in kube-up (instead of kube-dns formerly). 
It is still possible to deploy kube-dns by setting CLUSTER_DNS_CORE_DNS=false.
```
2018-08-29 15:19:37 -07:00
Arnold Szederjesi
134f7f2a9d Add flag for disabling prometheus-to-sd only for daemon sets 2018-08-28 14:09:07 +02:00
administrator
96922ceef0 set coredns as default for kube-up eployments 2018-08-27 20:30:41 +00:00
Kubernetes Submit Queue
64de381a34
Merge pull request #67377 from shyamjvs/add-mig-startup-timeout
Automatic merge from submit-queue (batch tested with PRs 67377, 67801). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add timeout to MIG wait-until-stable operation

Fixes https://github.com/kubernetes/kubernetes/issues/67374

/cc @wojtek-t @mborsz 
/sig cluster-lifecycle

```release-note
NONE
```
2018-08-24 04:50:00 -07:00
Marian Lobur
3f730d4c25 Remove deprecated legacy audit logging code. 2018-08-23 12:08:54 +02:00
liangwei
5ea138f4e9 remove rescheduler 2018-08-22 11:49:14 +08:00
Shyam Jeedigunta
65850950b4 Add timeout to MIG wait-until-stable operation 2018-08-14 11:13:52 +02:00
Jordan Liggitt
cc1a7735e8
move easyrsa check to gce prereq check 2018-08-06 19:56:10 -04:00
Kubernetes Submit Queue
1b6a505451
Merge pull request #66690 from spiffxp/verify-no-libressl
Automatic merge from submit-queue (batch tested with PRs 66284, 66690). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Exit gce kube-up.sh early if openssl is LibreSSL

**What this PR does / why we need it**:

macOS has an openssl binary, but it's actually LibreSSL, which doesn't play well with the easyrsa script that cluster/gce/util.sh uses to generate certs

Instead of waiting until we generate certs to discover easyrsa doesn't work, consider openssl a prereq for gce, and include a check for the version string starting with OpenSSL

Also, mirror kube-up.sh's "... calling" output in kube-down.sh

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/community#1954

```release-note
NONE
```
2018-07-30 20:01:58 -07:00
Aaron Crickenberger
a9aa892177 Exit gce kube-up.sh early if openssl is LibreSSL
macOS has an openssl binary, but it's actually LibreSSL, which
doesn't play well with the easyrsa script that cluster/gce/util.sh
uses to generate certs

Instead of waiting until we generate certs to discover easyrsa doesn't
work, consider openssl a prereq for gce, and include a check for the
version string starting with OpenSSL

Also, mirror kube-up.sh's "... calling" output in kube-down.sh
2018-07-26 14:48:51 -07:00
Koonwah Chen
7933edd290 Add KUBE_CUSTOM_CALICO_NODE_DAEMONSET_YAML and KUBE_CUSTOM_TYPHA_DEPLOYMENT_YAML 2018-07-26 11:59:00 -07:00
Kubernetes Submit Queue
e49e3baa83
Merge pull request #64939 from hzxuzhonghu/rm-etcd-quoram-read-flag
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

stop using deprecated --etcd-quorum-read

etcd-quorum-read was deprecated, but it is still used. 
This pr stops using it.

**Release note**:

```release-note
NONE
```
2018-06-30 19:32:34 -07:00
wojtekt
226aa7306c Create heapster node first 2018-06-21 11:00:18 +02:00
xuzhonghu
cfcd0e4d45 stop using deprecated --etcd-quorum-read 2018-06-09 12:38:46 +08:00
Cao Shufeng
d767ce24a9 remove deprecated option '--enable-custom-metrics' 2018-06-05 11:19:23 +08:00
Kubernetes Submit Queue
898831ad9d
Merge pull request #64592 from ravisantoshgudimetla/revert-64364-remove-rescheduler
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "Remove rescheduler and corresponding tests from master"

Reverts kubernetes/kubernetes#64364

After discussing with @bsalamat on how DS controllers(ref: https://github.com/kubernetes/kubernetes/pull/63223#discussion_r192277527) cannot create pods if the cluster is at capacity and they have to rely on rescheduler for making some space, we thought it is better to 

- Bring rescheduler back.
- Make rescheduler priority aware.
- If cluster is full and if **only** DS controller is not able to create pods, let rescheduler be run and let it evict some pods which have less priority.
- The DS controller pods will be scheduled now.

So, I am reverting this PR now. Step 2, 3 above are going to be in rescheduler.

/cc @bsalamat @aveshagarwal @k82cn 

Please let me know your thoughts on this. 

```release-note
Revert #64364 to resurrect rescheduler. More info https://github.com/kubernetes/kubernetes/issues/64725 :)
```
2018-06-04 16:56:11 -07:00
Koonwah Chen
37059e7efa Code clean up 2018-06-03 19:41:47 -07:00
Koonwah Chen
bb8272ead4 support netd on k8s 2018-06-03 01:35:27 -07:00
Koonwah Chen
d903d32856 Add netd as an addon for GKE. 2018-05-31 19:25:15 -07:00
RaviSantosh Gudimetla
872addf9e3
Revert "Remove rescheduler and corresponding tests from master" 2018-05-31 22:18:49 -04:00
ravisantoshgudimetla
aeccffc339 Phase out rescheduler in favor of priority and preemption 2018-05-29 19:52:06 -04:00
Michael Taufen
d4e48fd789 graduate DynamicKubeletConfig feature to beta 2018-05-24 09:59:29 -07:00
Marian Lobur
c1d0004013 Add environment variable to control truncating backend. 2018-05-18 15:52:47 +02:00
Maciej Borsz
128d6d3498 Add a way to pass extra arguments to etcd. 2018-05-17 10:48:13 +02:00
Kubernetes Submit Queue
7b8bb6e7d3
Merge pull request #63357 from Random-Liu/install-and-use-crictl
Automatic merge from submit-queue (batch tested with PRs 63167, 63357). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Install and use crictl in gce kube-up.sh

Download and use crictl in gce kube-up.sh.

This PR:
1. Downloads crictl `v1.0.0-beta.0` onto the node, which supports CRI v1alpha2. We'll upgrade it to `v1.0.0-beta.1` soon after the release is cut.
2. Change `kube-docker-monitor` to `kube-container-runtime-monitor`, and let it use `crictl` to do health monitoring.
3. Change `e2e-image-puller` to use `crictl`. Because of https://github.com/kubernetes/kubernetes/issues/63355, it doesn't work now. But in `crictl v1.0.0-beta.1`, we are going to statically link it, and the `e2e-image-puller` should work again.
4. Use `systemctl kill --kill-who=main` instead of `pkill`, the reason is that:
  a. `pkill docker` will send `SIGTERM` to all processes including `dockerd`, `docker-containerd`, `docker-containerd-shim`. This is not a problem for Docker 17.03 CE, because `containerd-shim` in containerd 0.2.x doesn't exit with SIGERM (see [code](https://github.com/containerd/containerd/blob/v0.2.x/containerd-shim/main.go#L123)). However, `containerd-shim` in containerd 1.0+ does exit with SIGTERM (see [code](https://github.com/containerd/containerd/blob/master/cmd/containerd-shim/main_unix.go#L200)). This means that `pkill docker` and `pkill containerd` will kill all shim processes for Docker 17.11+ and containerd 1.0+.
  b. We can use `pkill -x` instead. However, docker systemd service name is `docker`, but daemon process name is `dockerd`. We have to introduce another environment variable to specify "daemon process name". Given so, it seems easier to just use `systemctl kill` which only requires systemd service name. `systemctl kill --kill-who=main` will make sure only main process receives SIGTERM.

Signed-off-by: Lantao Liu <lantaol@google.com>

/cc @filbranden @yujuhong @feiskyer @mrunalp @kubernetes/sig-node-pr-reviews @kubernetes/sig-cluster-lifecycle-pr-reviews 

**Release note**:

```release-note
Kubernetes cluster on GCE have crictl installed now. Users can use it to help debug their node. The documentation of crictl can be found https://github.com/kubernetes-incubator/cri-tools/blob/master/docs/crictl.md.
```
2018-05-15 21:18:12 -07:00
Kubernetes Submit Queue
b617748f7b
Merge pull request #62905 from serathius/event-exporter-region
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[fluentd-gcp addon] Pass region in seperate field

This PR makes location passed to event-exporter based on `MULTIZONE` env.

Fixes https://github.com/kubernetes/kubernetes/issues/62399
```release-note
NONE
```
/cc @loburm
2018-05-11 06:00:44 -07:00
Marek Siarkowicz
f351b00a99 [fluentd-gcp addon] Pass region in seperate field 2018-05-11 09:50:07 +02:00
yankaiz
3989ec66eb Add MAX_PODS_PER_NODE env allowing kubelet to be max-pods aware. 2018-05-04 11:09:55 -07:00
Lantao Liu
d94a2b39d9 Install and use crictl in gce kube-up.sh
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-05-03 17:17:55 -07:00
Kubernetes Submit Queue
b5f61ac129
Merge pull request #62657 from matthyx/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update all script shebangs to use /usr/bin/env interpreter instead of /bin/interpreter

This is required to support systems where bash doesn't reside in /bin (such as NixOS, or the *BSD family) and allow users to specify a different interpreter version through $PATH manipulation.
https://www.cyberciti.biz/tips/finding-bash-perl-python-portably-using-env.html
```release-note
Use /usr/bin/env in all script shebangs to increase portability.
```
2018-05-02 19:44:32 -07:00
Karol Wychowaniec
6fb42aea4a Remove METADATA_AGENT_VERSION config option 2018-04-23 12:15:48 +02:00
Matthias Bertschy
9b15af19b2 Update all script to use /usr/bin/env bash in shebang 2018-04-19 13:20:13 +02:00
Kubernetes Submit Queue
bb8f58b6e6
Merge pull request #62195 from serathius/prometheus
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add prometheus cluster monitoring addon.

This PR adds new cluster monitoring addon based on prometheus.
It adds prometheus deployment with e2e tests.
Additional components will be added iterativly in future.
Manifests based on current Helm chart.
At current state it's not intended for production use.

cc @piosz @kawych @miekg
```release-note
Add prometheus cluster monitoring addon to kube-up
```
/sig instrumentation
/kind feature
/priority important-soon
2018-04-18 02:17:48 -07:00
Michael Taufen
420edc7b50 provision Kubelet config file for GCE
This PR extends the client-side startup scripts to provision a Kubelet
config file instead of legacy flags. This PR also extends the
master/node init scripts to install this config file from the GCE
metadata server, and provide the --config argument to the Kubelet.
2018-04-13 13:08:38 -07:00
Marek Siarkowicz
9544222e91 Test e2e prometheus addon 2018-04-13 11:12:10 +02:00
Kubernetes Submit Queue
72b7dacf07
Merge pull request #58178 from mikedanese/token-auth
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

enable token authentication for kubelets in GCE

```release-note
NONE
```
2018-04-12 15:06:07 -07:00
Mike Danese
23d02c8f07 enable token auth for kubelets in GCE 2018-04-12 09:31:00 -07:00
Shyam Jeedigunta
be2e5e65d3 Fix subnet cleanup logic when using IP-aliases with custom subnets 2018-04-11 15:44:28 +02:00
Shyam Jeedigunta
da01243af1 Fix IP-alias subnet creation logic 2018-04-06 13:23:38 +02:00
Kubernetes Submit Queue
4cfa2e4dfd
Merge pull request #60102 from satyasm/gcloud_net_flag
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fixes #54017, remove deprecated --mode flag

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #54017

**Special notes for your reviewer**:

**Release note**:

```release-note
remove deprecated --mode flag in check-network-mode
```
2018-04-05 17:53:00 -07:00
Supriya Garg
e350c46116 Update the stackdriver agents yaml to include a deployment for cluster level resources 2018-04-05 10:09:11 -04:00
Kubernetes Submit Queue
4685df26dd
Merge pull request #60590 from immutableT/enc_config_automation
Automatic merge from submit-queue (batch tested with PRs 60420, 60590). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable AESGCM encryption of secrets in etcd by default.

**What this PR does / why we need it**:
Enable encryption of secrets in etcd via AESGCM transform (as described here https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) during kube-up.sh build of a cluster.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-03-28 23:53:06 -07:00
Kubernetes Submit Queue
e3af2374a6
Merge pull request #60801 from jingax10/gce_util_branch
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Suppress error message from grep when checking whether a subnet has a secondary range or not.

**What this PR does / why we need it**:

Get rid of stdrr caused by grep command when running cluster/kube-up.sh for GCE.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

N/A

**Special notes for your reviewer**:

No behavior change.

**Release note**:

```release-note
"NONE"
```
2018-03-25 02:40:33 -07:00
Kubernetes Submit Queue
053a12aee9
Merge pull request #60107 from wangzhen127/cos-audit-placeholder
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update GCP fluentd configmap for COS audit logging on GKE node

**What this PR does / why we need it**:
This PR adds a placeholder in fluentd configmap for COS audit logging on GKE node.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
NONE

**Release note**:

```release-note
NONE
```
2018-03-25 00:51:52 -07:00
immutablet
d08799ca09 Enable AESGCM encryption of secrets in etcd by default. 2018-03-22 13:51:09 -07:00
Kubernetes Submit Queue
e81965d456
Merge pull request #61065 from freehan/fix-gcloud-dev
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix validation for dev gcloud

```release-note
NONE
```
2018-03-22 13:15:12 -07:00
Zhen Wang
d5c2cdcbbb Update GCP fluentd configmap for GKE node journal logging 2018-03-22 12:04:11 -07:00
Kubernetes Submit Queue
130caab7d5
Merge pull request #61235 from yguo0905/client-2
Automatic merge from submit-queue (batch tested with PRs 61124, 59537, 61235, 61258, 61114). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Support new NODE_OS_DISTRIBUTION 'custom' on GCE

**What this PR does / why we need it**:

This PR allows us to run e2e tests against arbitrary OS images on GCE.

It will be cherry picked into 1.8, 1.9 and 1.10.

**Release note**:

```
Support new NODE_OS_DISTRIBUTION 'custom' on GCE.
```

/assign @dashpole
2018-03-21 08:39:23 -07:00
Jing Ai
384868e570 Suppress error message from grep by removing in the end as it is wrongly interpreted as a file. 2018-03-16 18:12:39 -07:00
Kubernetes Submit Queue
c6d77ee656
Merge pull request #61119 from mtaufen/fix-cluster-autoscaler
Automatic merge from submit-queue (batch tested with PRs 61284, 61119, 61201). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add AUTOSCALER_ENV_VARS to kube-env to hotfix cluster autoscaler

This provides a temporary way for the cluster autoscaler to get at
values that were removed from kube-env in #60020. Ideally this
information will eventually be available via e.g. the Cluster API,
because kube-env is an internal interface that carries no stability
guarantees.

This is the first half of the fix; the other half is that cluster autoscaler
needs to be modified to read from AUTOSCALER_ENV_VARS, if it is
available.

Since cluster autoscaler was also reading KUBELET_TEST_ARGS for the
kube-reserved flag, and we don't want to resurrect KUBELET_TEST_ARGS in kube-env,
we opted to create AUTOSCALER_ENV_VARS instead of just adding back
the old env vars. This also makes it clear that we have an ugly dependency
on kube-env.

```release-note
NONE
```
2018-03-16 16:56:00 -07:00
Michael Taufen
8cf3dc103e Add AUTOSCALER_ENV_VARS to kube-env to hotfix cluster autoscaler
This provides a temporary way for the cluster autoscaler to get at
values that were removed from kube-env in #60020. Ideally this
information will eventually be available via e.g. the Cluster API,
because kube-env is an internal interface that carries no stability
guarantees.
2018-03-16 11:43:41 -07:00
Yang Guo
518c6c1a37 Support new NODE_OS_DISTRIBUTION 'custom' on GCE 2018-03-15 14:05:15 -07:00
Ryan Hitchman
68f5d44865 Fix deprecated gcloud compute networks --mode switches.
"create --mode" becomes "create --subnet-mode", and switch-mode has been
folded into "update".

Create --mode was deprecated in October and will be removed in the next
gcloud release. It is already failing in staging tests.
2018-03-14 15:00:59 -07:00
Minhan Xia
ec77fe97ec fix validation for dev gcloud 2018-03-12 14:10:35 -07:00
Kubernetes Submit Queue
31b4719066
Merge pull request #60859 from verult/remount-kube-env
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Setting REMOUNT_VOLUME_PLUGIN_DIR for COS images in kube-env

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #60725

**Special notes for your reviewer**: Not sure if it's the best place to set `REMOUNT_VOLUME_PLUGIN_DIR`.

/sig storage
/sig cluster-lifecycle
2018-03-12 10:54:31 -07:00
Mik Vyatskov
07905d6ee8 Make log audit backend configurable in GCE
Signed-off-by: Mik Vyatskov <vmik@google.com>
2018-03-08 14:09:32 +01:00
Cheng Xing
16ecc14017 Setting REMOUNT_VOLUME_PLUGIN_DIR for COS images in kube-env 2018-03-06 14:22:41 -08:00
Jing Ai
977252d4b2 Suppress error message from grep when checking whether a subnet has a secondary range or not. 2018-03-05 09:54:11 -08:00
Mike Danese
857690baf5 gce: add support for enabling TokenRequest feature 2018-02-27 18:54:03 -08:00
Robert Bailey
fe10c27ec0 Move kubelet flag generation from the node to the client, and
pass the kubelet flags through a new variable in kube-env
(KUBELET_ARGS).

Remove vars from kube-env that were only used for kubelet flags.

This will make it simpler to gradually migrate to dynamic kubelet
config, because we can gradually replace flags with config file
options in a single place without worrying about the plumbing to
move variables from the client onto the node.
2018-02-24 22:39:36 -08:00
Kubernetes Submit Queue
a85f7d9fff
Merge pull request #58090 from serathius/pass-location-to-event-exporter
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass location parameter to event exporter.

**What this PR does / why we need it**:
This PR makes event-exporter export cluster location together with events.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-23 23:15:43 -08:00
Marek Siarkowicz
bbfcd681b5 Pass location parameter to event exporter.
Location passed based on ZONE from kube-env.
2018-02-21 12:54:29 +01:00
Satyadeep Musuvathy
59b1ff820c fixes #54017, remove deprecated --mode flag 2018-02-20 14:53:19 -08:00
Karol Wychowaniec
443fd11bb9 Add cluster-location to GCE instance attributes 2018-02-19 10:48:25 +01:00
Robert Bailey
49cb1024b7 Move code only used by gce out of common.sh and into gce/util.sh. 2018-02-15 21:31:12 -08:00
Tim Hockin
3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Mike Danese
d6918bbbc0 cluster: remove kube-registry-proxy 2018-02-01 07:23:50 -08:00
Mike Danese
e420e0fca8 cluster: remove unused kubelet token 2018-02-01 07:23:50 -08:00
Mike Danese
02de75fb41 cluster: remove some cvm stuff 2018-02-01 07:23:50 -08:00
Mike Danese
4961065562 cluster: remove unused functions 2018-02-01 07:23:50 -08:00
Jesse Shieh
f9e43f3a6f
Fix master regex when running multiple clusters
I'm running two Kubernetes clusters on GCE. One for production and one for staging. The instance prefix I use for production is `kubernetes` and for staging it's `staging-kubernetes`. This caused a problem when running `kube-up.sh` for production because when it tries to find all instances which match `kubernetes(-...)?` it finds both the production and staging instances. This probably results in multiple problems, but the most noticeable one for me was that I`NITIAL_ETCD_CLUSTER` was incorrect and so etcd wouldn't start up correctly so the api server doesn't start up correctly so nothing else starts up. I tested this manually and it seems to work for me, but I didn't write an automated test.
2018-01-19 18:44:52 -08:00
Mike Danese
5f9735de53 cluster: delete saltbase
and don't build kubernetes-salt.tar.gz
2018-01-16 09:59:29 -08:00
Kubernetes Submit Queue
b6e12f1bb9
Merge pull request #58247 from mikedanese/rm-cvm
Automatic merge from submit-queue (batch tested with PRs 58246, 58247). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

cluster: remove support for cvm from gce kube-up

see #49213


```release-note
Remove deprecated ContainerVM support from GCE kube-up. 
```
2018-01-13 10:55:45 -08:00
Mike Danese
6387c7b5b3 cluster: remove support for cvm from gce kube-up 2018-01-13 08:12:18 -08:00
Mike Danese
9ac650c437 cluster: remove kube-push 2018-01-13 08:08:48 -08:00
Mike Danese
dca369dc84 remove support for container-linux in gce kube-up 2018-01-10 10:20:26 -08:00
Kubernetes Submit Queue
c7d52e27c8
Merge pull request #56888 from porridge/limit-curl-get
Automatic merge from submit-queue (batch tested with PRs 57670, 56888). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Limit number of pods listed as master liveness check.

**What this PR does / why we need it**:

Another step in making #55686 less likely.

**Release note**:
```release-note
NONE
```
2017-12-28 11:03:28 -08:00
Jake Sanders
c2c845f522
Use the regionless mirror alias 2017-12-27 15:47:09 -08:00
Tim Hockin
e9dd8a68f6 Revert k8s.gcr.io vanity domain
This reverts commit eba5b6092a.

Fixes https://github.com/kubernetes/kubernetes/issues/57526
2017-12-22 14:36:16 -08:00
Tim Hockin
eba5b6092a Use k8s.gcr.io vanity domain for container images 2017-12-18 09:18:34 -08:00
Marcin Owsiany
5a3c2d80aa Limit number of pods listed as master liveness check. 2017-12-06 15:17:30 +01:00
Kubernetes Submit Queue
fff99b72b9
Merge pull request #55690 from porridge/debug-curl
Automatic merge from submit-queue (batch tested with PRs 56208, 55690). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Dump last curl output if cluster fails to come up.

**What this PR does / why we need it**:
This is a step toward solving #55686

**Release note**:
```release-note
NONE
```
2017-11-23 08:20:32 -08:00
Marcin Owsiany
745db94e19 Dump last curl output if cluster fails to come up. 2017-11-22 13:45:43 +01:00
David Zhu
f780eefd39 Set up alternate mount point for SCSI/NVMe local SSD by UUID in /mnt/disks/by-uuid/, set up ability to have unformatted disk symlinked in /dev/disk/by-uuid/. Added tests. Preserved backwards compatibility. 2017-11-17 10:56:48 -08:00
Mike Danese
0117006a54
Revert "Add options for mounting SCSI or NVMe local SSD though Block or Filesystem and do all of that with UUID" 2017-11-16 07:51:38 -08:00
David Zhu
028258244c Set up alternate mount point for SCSI/NVMe local SSD by UUID in /mnt/disks/by-uuid/, set up ability to have unformatted disk symlinked in /dev/disk/by-uuid/. Added tests. Preserved backwards compatibility. 2017-11-14 17:14:41 -08:00
Kubernetes Submit Queue
ee3a08a772
Merge pull request #54001 from cblecker/sed-func
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Extract gnu-sed detection into a function

**What this PR does / why we need it**:
Moves gnu-sed detection into a reusable function across scripts (considering it's in multiple places).

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-10-31 03:09:46 -07:00
Kubernetes Submit Queue
3f59988907
Merge pull request #54623 from dims/fetch-kubeadm-gce-job-logs
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix log collection for kubeadm-gce tests

**What this PR does / why we need it**:

Separate out kuberenetes-anywhere provider under cluster/ but
delegate all the functionality to the "gce" one since the code
would be the same. Except for the name of the node, the
NODE_INSTANCE_PREFIX will be different, so account for that.


**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-10-28 08:13:19 -07:00
Maru Newby
adc338d330 Remove all traces of federation 2017-10-26 13:37:37 -07:00
Davanum Srinivas
9a217217c1 Fix log collection for kubeadm-gce tests
Separate out kuberenetes-anywhere provider under cluster/ but
delegate all the functionality to the "gce" one since the code
would be the same. Except for the name of the node, the
NODE_INSTANCE_PREFIX will be different, so account for that.
2017-10-26 07:57:42 -04:00
Kubernetes Submit Queue
a0ee4b9b44 Merge pull request #52868 from ihmccreery/node-service-account
Automatic merge from submit-queue (batch tested with PRs 52868, 53196, 54207). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Allow users to configure the service account made available on their nodes

**What this PR does / why we need it**: This allows users (and tests) to configure what GCP service account nodes are given when they are created, to allow users to grant fewer permissions to their nodes via IAM (instead of scopes).  Read more about service accounts and scopes here: https://cloud.google.com/compute/docs/access/service-accounts

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #53603

**Special notes for your reviewer**:

**Release note**:

```release-note
Allow GCE users to configure the service account made available on their nodes
```
2017-10-25 12:32:13 -07:00
Kubernetes Submit Queue
f816e75b53 Merge pull request #52489 from verb/e2e-gce-master
Automatic merge from submit-queue (batch tested with PRs 53051, 52489, 53920). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Test gcloud exit status when detecting master for GCE e2e test

e2e tests exit on error, so without testing the exit status of a command its scripted error message will never be printed.



**What this PR does / why we need it**: This prints the intended "could not detect Kubernetes master" error message instead of a stack trace from e2e test

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #52474

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-10-24 21:38:20 -07:00
Zihong Zheng
119b8012cd Add check-network-mode to support both new and deprecated network mode fields 2017-10-16 16:18:28 -07:00
Christoph Blecker
f5e8eca20b
Extract gnu-sed detection into a function 2017-10-16 13:21:09 -07:00
Jing Ai
8c55d71161 Rebase from https://github.com/kubernetes/kubernetes/pull/52377 2017-10-12 11:23:47 -07:00
Kubernetes Submit Queue
1d4e1d2dc2 Merge pull request #52377 from MrHohn/gce-kube-up-create-manual-network
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[GCE kube-up] Allow creating/deleting custom network

**What this PR does / why we need it**:
From https://github.com/kubernetes/test-infra/issues/4472.

This is the first step to make PR jobs use custom network instead of auto network (so that we will be less likely hitting subnetwork quota issue). 

The last commit is purely for testing out the changes on PR jobs. It will be removed after review.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #NONE.

**Special notes for your reviewer**:
/assign @bowei @nicksardo 

**Release note**:

```release-note
NONE
```
2017-10-12 02:55:28 -07:00
Jing Ai
a56241e13e Minior fix on getting subnet mode by gcloud 2017-10-11 14:32:25 -07:00
Zihong Zheng
80f57b7df7 GCE kube-down: Delete all remaining firewall rules when KUBE_DELETE_NETWORK is set 2017-10-10 14:49:08 -07:00
Zihong Zheng
496836b207 [GCE kube-up] Allow creating/deleting custom network 2017-10-06 10:50:18 -07:00
Isaac Hollander McCreery
ad05c8c444 Allow users to configure the service account made available on their nodes 2017-09-29 14:18:01 -07:00
Marian Lobur
a57ba588fb Fix mistype that causes breakage of e2e test. 2017-09-19 16:05:41 +02:00
Lee Verberne
83847eac79 Test gcloud exit when detecting master for e2e
e2e tests set errexit, so without testing the exit status of a command
its scripted error message will not be printed.
2017-09-14 12:00:29 +02:00
Jeff Grafton
b310b328d1 Convert deprecated gcloud --regexp flag into --filter 2017-09-07 14:39:38 -07:00
Nick Sardo
2bea4f7dd1 Fall back to network if subnet is unknown 2017-09-06 11:59:47 -07:00
Kubernetes Submit Queue
bcf22bcf6a Merge pull request #51951 from shyamjvs/heapster-node-with-ip-alias
Automatic merge from submit-queue

Make heapster VM creation work with IP aliases

Our experiment with 5k-node run today failed due to this :(
2017-09-05 15:46:10 -07:00
Shyam Jeedigunta
009f62ebad Make heapster VM creation work with IP aliases 2017-09-05 14:45:26 +02:00
Nick Sardo
8ac7bb5fb2 Use different project id for network ops & always set subnet 2017-09-04 09:55:56 -07:00
Kubernetes Submit Queue
9637f46122 Merge pull request #51725 from nicksardo/gce-plumb-netvars
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)

GCE: Plumb network & subnetwork to master

**Which issue this PR fixes** *
Fixes #51714

/assign @bowei 

**Release note**:
```release-note
NONE
```
2017-09-03 11:10:00 -07:00
Nick Sardo
506064376b Set NODE_NETWORK and NODE_SUBNETWORK in kube-up 2017-08-31 17:23:30 -07:00
Bowei Du
b145beb3da Create a secondary range for the services instead of a subnetwork
GCE now supports >1 secondary ranges / subnetwork.
2017-08-31 01:33:17 -07:00
Philip Ingrey
697f92a5d2
Switch away from gcloud deprecated flags in compute resource listings 2017-08-30 06:41:09 +01:00
Jeff Grafton
0d1055213d Fix gcloud compute instance-groups managed list call 2017-08-28 12:24:37 -07:00
Shyam Jeedigunta
2eafc562fa Block on master-creation step for large clusters (>50 nodes) in kube-up 2017-08-08 14:09:15 +02:00
Wojciech Tyczynski
5740c7afbf Fix ha_master tests 2017-07-24 12:10:22 +02:00
Kubernetes Submit Queue
ab40f526fd Merge pull request #49344 from shyamjvs/master-disk-size
Automatic merge from submit-queue

Auto-calculate master disk and root disk sizes in GCE

@gmarek PR https://github.com/kubernetes/kubernetes/pull/49282 didn't fix the issue because MASTER_DISK_SIZE was defaulting to 20GB in config-test.sh before being calculated inside get-master-disk-size() where you use pre-existing value if any.

It should be fixed by this now.
2017-07-21 06:24:39 -07:00
Kubernetes Submit Queue
ef98b80bd0 Merge pull request #49172 from ianchakeres/gce-local-ssd-fix
Automatic merge from submit-queue (batch tested with PRs 48565, 49172)

On GCE check whether NODE_LOCAL_SSDS=0 and handle this case appropriately

**What this PR does / why we need it**: Presently if you are using a mac and GCE and specify NODE_LOCAL_SSDS=0, or use the default, you end up with 2 local SSDs.

**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubernetes/issues/49171

**Special notes for your reviewer**:

I've discovered that this issue is due to b353792f9c/cluster/gce/util.sh (L579)

If NODE_LOCAL_SSDS=0, this evaluates to $(seq 0)

```
$ for i in $(seq 0); do echo $i; done
1
0
```

From man seq on mac osx

```
The seq utility prints a sequence of numbers, one per line (default), from first (default 1), 
to near last as possible, in increments of incr (default 1).When first is larger than last the 
default incr is -1.
```

This was run on mac with the seq manpage indicating it comes from BSD Feb 19 2010.

**Release note**:

```release-note
NONE
```
2017-07-21 01:13:19 -07:00
Shyam Jeedigunta
94b3cc21f9 Auto-calculate master disk and root disk sizes in GCE 2017-07-21 03:35:37 +02:00
gmarek
83d3018085 Fix master disk size variable usage 2017-07-20 14:15:06 +02:00
Ian Chakeres
a18a18367f Check whether NODE_LOCAL_SSDS=0 and handle this case appropriately.
Some versions of seq will count down from 1 if "seq 0" is specified
2017-07-19 17:06:21 -07:00
Shyam Jeedigunta
cc8bb857f9 Allow creating special node for heapster in GCE 2017-06-28 21:27:36 +02:00
gmarek
10ce8e2c0d Fix bug cluster-subnet logic 2017-06-28 14:27:52 +02:00
gmarek
536f48ef15 Fix test commands in cluster/gce/util.sh 2017-06-26 21:27:04 +02:00
gmarek
64f6606833 Make big clusters work again after introduction of subnets 2017-06-26 21:27:04 +02:00
Kubernetes Submit Queue
cdc9770346 Merge pull request #46792 from ianchakeres/avoid-redundant-copy-to-staging
Automatic merge from submit-queue (batch tested with PRs 47403, 46646, 46906, 46527, 46792)

Avoid redundant copying of tars during kube-up for gce if the same file already exists

**What this PR does / why we need it**: 

Whenever I execute cluster/kube-up.sh it copies my tar files to google cloud, even if the files haven't changed. This PR checks to see whether the files already exist, and avoids uploading them again. These files are large and can take a long time to upload.

**Which issue this PR fixes**: fixes #46791

**Special notes for your reviewer**:

Here is the new output:

cluster/kube-up.sh 
... Starting cluster in us-central1-b using provider gce
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
Project: PROJECT
Zone: us-central1-b
+++ Staging server tars to Google Storage: gs://kubernetes-staging-PROJECT/kubernetes-devel
+++ kubernetes-server-linux-amd64.tar.gz uploaded earlier, cloud and local file md5 match (md5 = 3a095kcf27267a71fe58f91f89fab1bc)


**Release note**:
```cluster/kube-up.sh on gce now avoids redundant copying of kubernetes tars if the local and cloud files' md5 hash match```
2017-06-23 02:59:31 -07:00
Marcin Wielgus
8d801d918d Set price expander in Cluster Autoscaler for gce 2017-06-19 23:52:47 +02:00
Ian Chakeres
b2450d2eb7 Moved gsutil_get_tar_md5 function before copy-to-staging function 2017-06-14 07:49:59 -07:00
Marcin Wielgus
5e390eff1a Allow autoscaler min at 0 in GCE 2017-06-14 07:36:18 +02:00
Ian Chakeres
14391d3eb8 Moved md5 comand to a separate function and added comments 2017-06-13 16:12:21 -07:00
Ian Chakeres
d0566faace Added logic to copy-to-staging to avoid copying if the same file already exists in gce 2017-06-01 11:06:48 -07:00
Bowei Du
3d8785648e gcloud command syntax changed between alpha and beta versions
syntax for secondary-ranges changed from:
  name=NAME,range=RANGE
to
  NAME=RANGE
2017-05-26 17:02:25 -07:00
Kubernetes Submit Queue
80171e5106 Merge pull request #46150 from bowei/ip-alias-service
Automatic merge from submit-queue (batch tested with PRs 46299, 46309, 46311, 46303, 46150)

Create a subnet for reserving the service cluster IP range

This will be done if IP aliases is enabled on GCP.

```release-note
NONE
```
2017-05-24 23:19:11 -07:00
Vishnu kannan
86b5edb79a Update COS version to m59
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-05-20 21:17:19 -07:00
Bowei Du
7c9b6e90e8 Create a subnet for reserving the service cluster IP range
This will be done if IP aliases is enabled on GCP.
2017-05-19 16:59:14 -07:00
Marcin Wielgus
2f4cb6bfe7 Use integer comparisons instead of string comparisons in autoscaler config validation 2017-05-19 14:50:55 +02:00
Bowei Du
7febdde22a Update cluster startup scripts to use gcloud beta for alias IP support
The feature has gone from alpha to beta.
2017-05-17 16:26:48 -07:00
Matt Liggett
5dd4a5d56b Add --quiet to instance-templates delete.
Otherwise it hangs waiting for confirmation.
2017-05-15 16:26:11 -07:00
Ryan Hitchman
62235c3bb8 Fix ip-alias testing.
IP aliases are an alpha feature, and node accelerators are a beta
feature. $gcloud determines which is appropriate.

Before, this would try to run "gcloud alpha beta", which is incoherent.
2017-05-10 12:10:17 -07:00
Vishnu kannan
e74d4a0d68 Adding support for Accelerators to GCE clusters.
Signed-off-by: Vishnu kannan <vishnuk@google.com>
2017-04-28 15:57:39 -07:00
Yang Guo
0d66fc97cd Makes cluster/gce/ubuntu to be a symlink to cluster/gce/gci and changes the gci's [master|node].yaml to enable kubernetes.target.
This enables Ubuntu to use the same provisioning scripts as GCI's. The
change for enabling kubernetes.target is needed for Ubuntu but a no-op
for GCI.
2017-04-25 16:19:00 -07:00
Chris Glass
cf23d4c939 Add Ubuntu as a platform for e2e tests on GCE
Using Ubuntu on GCE to run cluster e2e tests requires slightly different
node.yaml and master.yaml files than GCI, because Ubuntu uses systemd as
PID 1, wheras GCI uses upstart with a systemd delegate. Therefore the
e2e tests fail using those files since the kubernetes services are not
brought back up after a node/master reboot.
2017-04-20 17:40:51 -07:00
Bowei Du
c4a8ea2eb5 Missing --project in the cluster up scripts for gce
```release-note
NONE
```
2017-04-13 12:10:59 -07:00
Kubernetes Submit Queue
640c67792f Merge pull request #44363 from bowei/use-auto-net
Automatic merge from submit-queue

Use auto mode networks instead of legacy networks in GCP

Use of the --range flag creates legacy networks in GCP.

Legacy networks will not support new GCP features.

```release-note
NONE
```
2017-04-11 22:57:20 -07:00
Kubernetes Submit Queue
ceccd305ce Merge pull request #42147 from bowei/ip-alias-2
Automatic merge from submit-queue

Add support for IP aliases for pod IPs (GCP alpha feature)

```release-note
Adds support for allocation of pod IPs via IP aliases.

# Adds KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster up scripts (`kube-{up,down}.sh`).

KUBE_GCE_ENABLE_IP_ALIASES=true will enable allocation of PodCIDR ips
using the ip alias mechanism rather than using routes. This feature is currently
only available on GCE.

## Usage
$ CLUSTER_IP_RANGE=10.100.0.0/16 KUBE_GCE_ENABLE_IP_ALIASES=true bash -x cluster/kube-up.sh

# Adds CloudAllocator to the node CIDR allocator (kubernetes-controller manager).

If CIDRAllocatorType is set to `CloudCIDRAllocator`, then allocation
of CIDR allocation instead is done by the external cloud provider and
the node controller is only responsible for reflecting the allocation
into the node spec.

- Splits off the rangeAllocator from the cidr_allocator.go file.
- Adds cloudCIDRAllocator, which is used when the cloud provider allocates
  the CIDR ranges externally. (GCE support only)
- Updates RBAC permission for node controller to include PATCH
```
2017-04-11 22:09:24 -07:00
Bowei Du
079505023f Use auto mode networks instead of legacy networks in GCP
Use of the --range flag creates legacy networks in GCP.
2017-04-11 14:36:17 -07:00
Bowei Du
345c65847f Add KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster turn up scripts.
KUBE_GCE_ENABLE_IP_ALIASES=true will enable allocation of PodCIDR ips
using the ip alias mechanism rather than using routes.

NODE_IP_RANGE will control the node instance IP cidr
KUBE_GCE_IP_ALIAS_SIZE controls the size of each podCIDR
IP_ALIAS_SUBNETWORK controls the name of the subnet created for the cluster
2017-04-11 14:07:50 -07:00