Commit Graph

482 Commits

Author SHA1 Message Date
Stephen Augustus
f828821eea releng: Point debian-base image references to K8s Infra
Signed-off-by: Stephen Augustus <saugustus@vmware.com>
2020-05-02 22:51:40 -04:00
Di Xu
b3ae0f01e5 pass through KUBE_BUILD_PLATFORMS to enable build on specified platforms 2020-01-09 20:02:41 +08:00
Dan Winship
ffe93b3979 Update debian-iptables iptables-wrapper script
The debian-iptables v12.0.0 build didn't work because of another
previously-undiscovered iptables 1.8.x bug. Work around it for now and
bump the version to v12.0.1; we can revert back to the original
version of the script once iptables 1.8.4 is available in
buster-backports.
2019-11-14 09:57:57 -05:00
Tim Hockin
573a99ea51 Bump debian-base to buster 2019-11-14 09:57:56 -05:00
Jordan Liggitt
cd4474ae4f Revert "76093 restructure LICENSES file generation"
This reverts commit d39ac98cc5.
2019-11-13 10:24:32 -05:00
Ji Shan Xing
d39ac98cc5 76093 restructure LICENSES file generation 2019-11-12 20:38:57 -05:00
yuxiaobo
81e9f21f83 Correct spelling mistakes
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-11-06 20:25:19 +08:00
Odin Ugedal
cce1f32ea5
Fix shellcheck failures SC2034 2019-10-23 22:47:46 +02:00
Davanum Srinivas
adbd1800c8 Drop cloud-controller-manager artifacts from k/k release 2019-08-06 13:51:27 -04:00
hui luo
97087c66f7 Allow customize registry name of base and release images
default behavior does not change, it uses k8s.gcr.io by default
added two vars: KUBE_DOCKER_REGISTRY, KUBE_BASE_IMAGE_REGISTRY.
KUBE_BASE_IMAGE_REGISTRY is for base image registry of server binaries
KUBE_DOCKER_REGISTRY is for released images registry

user can interact with them by:
`KUBE_DOCKER_REGISTRY=### KUBE_BASE_IMAGE_REGISTRY=### make quick-release`

Signed-off-by: Hui Luo <luoh@vmware.com>
2019-07-25 17:43:01 -07:00
yankaiz
5549a330c2 Bump debian-iptables versions to v11.0.2. 2019-04-04 16:46:37 -07:00
Tim Allclair
1b9d0c1094 Bump debian-* base image versions to latest 2019-02-04 11:58:51 -08:00
war-turtle
b1b7d0dbae fixing shellcheck warning in build/common.sh 2019-01-26 19:21:17 +05:30
Jeff Grafton
f26048ecb1 Use debian-base instead of busybox as base image for server images 2018-10-25 10:19:40 -07:00
Jeff Grafton
003404685d Update to use debian-iptables v11.0 and debian-hyperkube-base 0.12.0 2018-10-24 16:18:53 -07:00
Davanum Srinivas
d9cfd77149
Ensure reproducible builds - support for SOURCE_DATE_EPOCH with dockerized builds
- Pass in SOURCE_DATE_EPOCH when we run the docker container
- Looks like cleaning up symbol table also helps
- Also trimming the path

Tips from
- https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/
- https://blog.filippo.io/shrink-your-go-binaries-with-this-one-weird-trick/

Change-Id: Iedba85d9c1a36790fb8814795f7c27c1371cff1b
2018-09-26 07:28:09 -04:00
Jeff Grafton
b4635a829a Update to use debian-iptables v10.2 and debian-hyperkube-base 0.10.2 2018-09-17 21:10:04 -07:00
Kubernetes Submit Queue
68d22a878d
Merge pull request #67971 from Katharine/coverage-instrumentation
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add ability to build with runtime coverage instrumentation

**What this PR does / why we need it**:

This PR adds the ability to instrument a subset of kubernetes binaries to report code coverage information. The specific use-case is to help determine coverage of our end-to-end Conformance tests, as well as provide data that can be used to help determine where to focus. This PR focuses on making it possible to build with instrumentation; collecting and using the generated coverage data will be done in later PRs. For more details as to the intent, see the [design doc](https://docs.google.com/document/d/1FKMBFxz7vtA-6ZgUkA47F8m6yR00fwqLcXMVJqsHt0g/edit?usp=sharing) (google doc; requires kubernetes-dev membership).

Specifically, this PR adds a new `KUBE_BUILD_WITH_COVERAGE` make variable, which when set will cause `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, `kube-proxy` and `kubelet` to be built with coverage instrumentation. These coverage-instrumented binaries will flush coverage information to disk every five seconds, defaulting to a temporary directory unless the `KUBE_COVERAGE_FILE` environment variable is set at launch, in which case it will write to that file instead.

The mechanism used to achieve coverage instrumentation is to build the targeted binaries as "unit tests" with coverage enabled, and then rigging the unit tests to just execute the binary's usual entry point. This is implemented only for the bash build system.

/sig testing

```release-note
NONE
```
2018-09-01 01:32:52 -07:00
Benjamin Elder
037fabd842 add make targets for building server images 2018-08-31 14:31:21 -07:00
Katharine Berry
254618ed61 Pass KUBE_BUILD_WITH_COVERAGE through to docker instance. 2018-08-29 14:48:24 -07:00
Kubernetes Submit Queue
f6eb545ad3
Merge pull request #67365 from ixdy/update-debian-container-images
Automatic merge from submit-queue (batch tested with PRs 67396, 67097, 67395, 67365, 67099). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update to debian-iptables v10.1 and hyperkube-base 0.10.1

**What this PR does / why we need it**: these images are based on the `debian-base` 0.3.2 images, which include CVE fixes (#67026) and permission fixes of the qemu-ARCH-static helper binary (#67026, #67283).

**Release note**:

```release-note
NONE
```
2018-08-15 05:45:24 -07:00
Jeff Grafton
37bb013943 Update to debian-iptables v10.1 and hyperkube-base 0.10.1 2018-08-13 22:32:55 -07:00
Davanum Srinivas
a66e1f089e
Drop kube-aggregator container image from release
First version of this was only standalone, now the code is being
vendored by anyone who wants to use it. So the standalone binary and
container are no longer useful.

Change-Id: Ib9369de66b4ecb3451f73ba2a252526d6615b96f
2018-08-08 15:57:52 -04:00
Maciej Pytel
6cdec82868 Unset CDPATH in build scripts 2018-07-17 15:06:32 +02:00
Matthias Bertschy
9b15af19b2 Update all script to use /usr/bin/env bash in shebang 2018-04-19 13:20:13 +02:00
Ismo Puustinen
231f79f89e build: fix building with spaces in directory names.
Fix kubernetes build scripts to work in case the source directory is in
a directory path which contains a space.

You can prepare such a directory like this:

    $ mkdir '/tmp/test dir/'
    $ cd '/tmp/test dir/'
    $ git clone https://github.com/kubernetes/kubernetes.git
    $ cd kubernetes

Then, without the fix:

$ KUBE_FASTBUILD=true KUBE_RELEASE_RUN_TESTS=n build/release.sh
cat: /tmp/test: No such file or directory
cat: dir/kubernetes/build/build-image/cross/VERSION: No such file or directory
cat: /tmp/test: No such file or directory
cat: dír/kubernetes/build/build-image/VERSION: No such file or directory
+++ [0307 18:10:33] Verifying Prerequisites....
cp: target '/tmp/test dir/kubernetes/_output/images/kube-build:build-7c7cd10a18--/Dockerfile' is not a directory
!!! [0307 18:10:33] Call tree:
!!! [0307 18:10:33]  1: build/release.sh:35
kube::build::build_image(...)
!!! Error in build/../build/common.sh:454
  Error in build/../build/common.sh:454. '((i<3-1))' exited with status 1
Call stack:
  1: build/../build/common.sh:454 kube::build::build_image(...)
  2: build/release.sh:35 main(...)
Exiting with status 1

With the fix the compilation succeeds. The fix is done adding double
quotes to required places (and also just in case to other places where
shellcheck recommended adding them).

Note that this fix doesn't help with the official make-based build:
it's tricky to make makefiles work with targets with spaces in their
names.
2018-03-12 09:12:47 +02:00
Kubernetes Submit Queue
5c86ef2150
Merge pull request #59207 from ipuustin/shell-bugfix
Automatic merge from submit-queue (batch tested with PRs 59705, 59207, 59677). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

build: fix a logic error in shell script.

**What this PR does / why we need it**: It looks like that there's a logic error in `build/common.sh`. The return value of a `docker inspect` command is not checked properly, since the value being assigned is actually the previous command's return value (a `0`, because `local` always returns `0` when used like this).

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**: 

```release-note
NONE
```
2018-02-13 12:29:31 -08:00
Tim Hockin
3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Ismo Puustinen
7a67246d47 build: fix a logic error in shell script.
The right place to assign the "docker inspect" return value is outside
of the subshell. The last return value was coming from something else
than the expected command.
2018-02-05 11:21:59 +02:00
Kubernetes Submit Queue
0d9e247aaa
Merge pull request #54071 from HubSpot/kube-build-parent-cgroup
Automatic merge from submit-queue (batch tested with PRs 54071, 58393). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Optionally launch build containers inside a specific parent cgroup

**What this PR does / why we need it**: Adds support to the build process for launching containers inside a specific parent cgroup via the `--cgroup-parent` docker arg. We needed this to enforce resource usage when building Kubernetes in our CI environment.

**Special notes for your reviewer**: None

**Release note**:

```release-note
NONE
```
2018-01-23 16:41:30 -08:00
tpetr
4ec6e1288e update if statement 2018-01-23 11:24:08 -05:00
stewart-yu
0601916d78 add KUBE_ROOT in directory 2018-01-12 18:12:37 +08:00
Tim Hockin
e9dd8a68f6 Revert k8s.gcr.io vanity domain
This reverts commit eba5b6092a.

Fixes https://github.com/kubernetes/kubernetes/issues/57526
2017-12-22 14:36:16 -08:00
Tim Hockin
eba5b6092a Use k8s.gcr.io vanity domain for container images 2017-12-18 09:18:34 -08:00
m1093782566
787a55bb67 install ipset in debian-iptables docker image and bump tag to v10 2017-11-22 15:37:15 +08:00
Christoph Blecker
555b4215d6
Fix .git rsync filter 2017-11-11 10:02:02 -08:00
Ryan Phillips
66965daf56 bump base images to debian stretch 2017-11-10 09:54:10 -06:00
tpetr
0df8838d15 allow setting --cgroup-parent for docker run commands 2017-10-17 10:58:39 -04:00
Dr. Stefan Schimanski
4156ec979b build/common.sh: silence kube::build::has_ip on Mac 2017-10-16 14:44:39 +02:00
Kubernetes Submit Queue
2e6ccc3f98 Merge pull request #52959 from mattjmcnaughton/mattjmcnaughton/dynamically-determine-default-docker-machine-memory
Automatic merge from submit-queue (batch tested with PRs 52959, 53790). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Dynamically determine default docker machine memory

**What this PR does / why we need it**:

Currently, if using `docker-machine` for k8s docker operations on a Mac,
we'll create a VM with 4096MB of memory. The machine's RAM will be the
same regardless of the memory available on the local machine. For
example, if the user has 16GB on their local machine, the VM will still
only have 4GB of RAM.

Update the method for defining the `kube-dev` VM with `docker-machine`,
so we give it access to 50% of the total RAM.

**Release note**:
```release-note
NONE
```
2017-10-13 19:19:29 -07:00
Ryan Phillips
de095385b9 common.sh fix to detect ip instead of ifconfig
* Some machines do not have ifconfig and rely on ip
2017-10-06 16:42:53 -05:00
mattjmcnaughton
abee38cb4a Dynamically determine default docker machine memory
Currently, if using `docker-machine` for k8s docker operations on a Mac,
we'll create a VM with 4096MB of memory. The machine's RAM will be the
same regardless of the memory available on the local machine. For
example, if the user has 16GB on their local machine, the VM will still
only have 4GB of RAM.

Update the method for defining the `kube-dev` VM with `docker-machine`,
so we give it access to 50% of the total RAM.
2017-10-03 09:10:31 -04:00
Kubernetes Submit Queue
e5fb3d7108 Merge pull request #50587 from CaoShuFeng/verify-staging-godeps
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

set proxy when build containers for users behind proxy

Fixes: https://github.com/kubernetes/kubernetes/issues/49921

**Release note**:

```
NONE
```
2017-09-24 22:32:39 -07:00
Christoph Blecker
b6f6419423
make clean will remove all gitignored files 2017-09-04 11:04:09 -07:00
Kubernetes Submit Queue
5d72d5c31d Merge pull request #50602 from dixudx/user_arm64v8_instead_aarch64
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)

Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50601

**Special notes for your reviewer**:
/assign @ixdy @jbeda @zmerlynn
**Release note**:

```release-note
Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations
```
2017-09-03 01:12:04 -07:00
Kubernetes Submit Queue
a244a55c96 Merge pull request #51004 from zhouhaibing089/rsync-fix
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)

add the caller ip into rsync hosts allow list

fixes https://github.com/kubernetes/kubernetes/issues/51003.

This patch allows running build in pods.

```release-note
NONE
```
2017-09-02 22:52:08 -07:00
Christoph Blecker
4f1106c8a5
Modify rsync filter to retain output across runs 2017-08-25 16:58:59 -07:00
zhouhaibing089
ca76002e89 add the caller ip into rsync hosts allow list 2017-08-23 09:58:32 +08:00
Di Xu
ec2fbbc85d bump new version due to base image changed 2017-08-21 14:51:44 +08:00
Di Xu
d4aa1611bd use more-specific arm64v8 instead of deprecated aarch64 organization 2017-08-21 10:18:19 +08:00
Di Xu
25a786f74d use more-specific arm32v7 instead of deprecated armhf organization 2017-08-21 10:17:43 +08:00
Cao Shufeng
6bc64dffe7 set proxy when build containers for users behind proxy
Fixes: https://github.com/kubernetes/kubernetes/issues/49921
2017-08-15 09:57:40 +08:00
Davanum Srinivas
827af63e72 Avoid rsync of .git directory
We don't really need the .git directory to be transferred. This
problem was introduced in the PR:
"Run the update-staging-godeps script inside a docker container"

Fixes #50272
2017-08-12 20:52:49 -04:00
Christoph Blecker
2ee8983daf
Dockerize update-staging-godeps 2017-07-25 12:26:57 -07:00
Christoph Blecker
d3b838b608
rsync git directories into kube-build 2017-07-25 12:11:06 -07:00
Dr. Stefan Schimanski
1f40e75aa0 Workaround docker-wait freeze with 17.06.0 2017-07-11 16:22:53 +02:00
Davanum Srinivas
733bb2c31d Add git branch to make build short hash unique
KUBE_ROOT_HASH depends just on the host name and
directory path. So when working with branches, it could lead
to some confusion since the hash is the same even when
you switch from branch to branch. Let us use the git branch
information when we compute the short hash

Fixes #1801
2017-06-02 21:18:00 -04:00
bruceauyeung
396695af15 fix make release/quick-release doesn't respect go build flags 2017-05-25 17:05:09 +08:00
Jeff Grafton
37eb292bc0 Update bazel build to dockerize cloud-controller-manager 2017-05-02 12:07:13 -07:00
Lucas Käldström
4ee9d047fd
Create a docker image for the cloud controller manager 2017-04-30 10:28:56 +03:00
StevenYCChou
7985e2c374 fix typo in build/common.sh 2017-04-16 19:22:30 -04:00
Clayton Coleman
7d413fb53b
protobuf generation modifies types.go, which needs to be copied out
This was broken when we moved to the build container, but no one
noticed. We also likely have another bug, which is that protobuf should
hard fail when we have fields that aren't assigned a tag.
2017-03-17 00:29:13 -04:00
Tim St. Clair
8a5314fb6d
Rebase kube-proxy and debian-iptables on debian-base 2017-02-27 15:48:23 -08:00
Kubernetes Submit Queue
3897fc00db Merge pull request #38926 from luxas/update_cross_go18
Automatic merge from submit-queue

Improve the multiarch situation; armel => armhf; reenable pcc64le; remove the patched golang

**What this PR does / why we need it**: 
 - Improves the multiarch situation as described in #38067 
 - Tries to bump to go1.8 for arm (and later enable ppc64le)
 - GOARM 6 => GOARM 7
 - Remove the golang 1.7 patch
 - armel => armhf
 - Bump QEMU version to v2.7.0

**Release note**:

```release-note
Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2
```

@kubernetes/sig-testing-misc @jessfraz @ixdy @jbeda @david-mcmahon @pwittrock
2017-01-28 03:51:19 -08:00
Lucas Käldström
04bc44acf9
Remove the patched golang from the kube-cross image and upgrade to go1.8rc1 for arm and ppc64le instead. Reenable the ppc64le builds 2017-01-27 20:03:37 +02:00
Robert Bailey
a6c65f3035 When calling chown, use : instead of . to separate the user and
group for cross platform compatibility.
2017-01-26 13:39:06 -08:00
Kubernetes Submit Queue
d5d9a594d8 Merge pull request #39515 from xilabao/fix-rsync-err
Automatic merge from submit-queue

fix rsync err when make release

fix #39513
2017-01-24 19:46:29 -08:00
xilabao
f6ae588207 fix rsync err when make release 2017-01-24 10:08:09 +08:00
Jeff Grafton
39217af9e9 Update amd64 kube-proxy base image to debian-iptables-amd64:v5 2017-01-11 00:49:13 -08:00
Jeff Grafton
9172bf63e5 Update to debian-iptables-*:v5 2017-01-10 16:23:01 -08:00
deads2k
453651cbfc rename kubernetes-discovery to kube-aggregator 2017-01-10 12:27:42 -05:00
deads2k
47073be356 create kuberentes-discovery image 2016-12-20 07:42:49 -05:00
Alejandro Escobar
7d9c06f82d local-up-cluster changes: added help option, added error message for why docker ps fails and how to recover, added test to check if etcd is in your path to fail fast when not found.
from etcd.sh split the start process into validate fucntion + start function so that the validate piece can be reused elsewhere. the up-cluster script has been changed to remove duplicate docker logic to the one used in buid-tools/common.sh and the validate etcd function is now used here.

moved docker daemon check function to util.sh and made function name changes and upstream changes.
2016-12-16 07:41:35 -08:00
xuxinkun
29dc75abc2 fix kube::log::error in start_rsyncd_container.
Signed-off-by: xuxinkun <xuxinkun@gmail.com>
2016-12-15 10:00:19 +08:00
Jeff Grafton
27d096d27d Rename build-tools/ back to build/ 2016-12-14 13:42:15 -08:00
Mike Danese
27116c6818 rename build/ to build-tools/ 2016-10-24 14:41:56 -07:00
Jeff Grafton
809e528d96 Increase build verbosity for rsync operations in verify-generated-{protobuf,runtime}.sh 2016-10-13 13:40:07 -07:00
Joe Beda
2582f1f368 Add --delete flag when rsyncing into container.
If you delete a source file, we want to reflect that in the build container.  We
only use --delete going that one way as we don't want to accidentally delete
files in the user's source tree.
2016-10-11 10:08:16 -07:00
Kubernetes Submit Queue
160adc3864 Merge pull request #34351 from jbeda/verify-rsync
Automatic merge from submit-queue

Check for rsync and give friendlier message

Fixes #34300.

Not sure if #34309 is the same issue.  Hopefully it is the same issue.
2016-10-08 02:48:14 -07:00
Kubernetes Submit Queue
bf1e7c8c69 Merge pull request #34288 from ixdy/build-image-rsync-sh
Automatic merge from submit-queue

Make sure rsync.sh is executable inside the build image

I kept having the build fail:
```console
$ make quick-release
+++ [1006 18:13:44] Verifying Prerequisites....
+++ [1006 18:13:44] Building Docker image kube-build:build-d3c60cf83f-3-v1.6.3-9
+++ [1006 18:13:54] Creating data container kube-build-data-d3c60cf83f-3-v1.6.3-9
+++ [1006 18:13:55] Syncing sources to container
!!! [1006 18:16:01] Could not connect to rsync container. See build/README.md for setting up remote Docker engine.
make: *** [quick-release] Error 1
```

`docker ps` revealed the issue:
```console
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
$ docker ps -a
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                       PORTS               NAMES
75c2a3c40cb3        kube-build:build-d3c60cf83f-3-v1.6.3-9   "/rsyncd.sh"             6 seconds ago       Exited (126) 5 seconds ago                       kube-rsync-d3c60cf83f-3-v1.6.3-9
3eb215e41f36        kube-build:build-d3c60cf83f-3-v1.6.3-9   "chown -R 85078.5000 "   8 seconds ago       Exited (0) 6 seconds ago                         kube-build-data-d3c60cf83f-3-v1.6.3-9
5a2707af2ccd        882577c54f67                             "/bin/sh -c 'cd ${K8S"   7 days ago          Exited (2) 7 days ago                            stupefied_goldberg
$ docker logs 75c2a3c40cb3
/bin/bash: /rsyncd.sh: Permission denied
```

I'm not sure why this works on Jenkins but not on my machine.
2016-10-07 13:28:58 -07:00
Joe Beda
48906e6295 Check for rsync and give friendlier message 2016-10-07 12:25:09 -07:00
Jonathan Boulle
69874a4dca build: fix typo in error message help
"hasn't been set of is set" --> "hasn't been set or is set"
2016-10-07 16:30:02 +02:00
Jeff Grafton
e4b16a5a18 Make sure rsync.sh is executable inside the build image
Also make sure /etc/localtime is readable

Also move BUILD_IMAGE_VERSION to build-image/VERSION to be consistent
with the way the cross-image works.
2016-10-06 18:56:41 -07:00
Joe Beda
add137c4d5 Use rsync to probe for rsync ready
We were using netcat to try and figure out when the rsync container is ready. Now we instead use rsync itself. I suspect that there was a race condition with some versions of Docker where it would accept connections and then close them during container start.

This fixes #34214 (I think)
2016-10-06 12:13:04 -07:00
Joe Beda
d955f54918 Remove unused bash variables 2016-10-03 19:42:22 -07:00
Joe Beda
4330560b6a Make update-generated-*.sh not hack around with docker images
make-generated-{protobuf,runtime}.sh was doing some really nasty stuff with how
the build container was managed in order to copy results out.  Since we have
more flexibility to grab results out of the build container, we can now avoid
all of this.  Ideally we wouldn't have `hack` calling `build` at all, but we
aren't there yet.
2016-10-03 19:42:21 -07:00
Joe Beda
f861163a15 Move release code (bash) out of build/common.sh 2016-10-03 19:42:21 -07:00
Joe Beda
dc586ea8f7 Use rsync to get source into build container
We also add "version" to all docker images and containers

This version is to be incremented manually when we change the shape of the build
image (like changing the golang version or the set of volumes in the data
container).  This will delete all older versions of images and containers when
the version is different.
2016-10-03 19:42:20 -07:00
Joe Beda
17454a04ac Remove support for boot2docker 2016-10-03 19:42:20 -07:00
deads2k
06b1a9636d promote contrib/mesos to incubator 2016-09-28 10:58:09 -04:00
Jeff Grafton
13a3313957 Bundle kubemark in test tarball instead of server tarball 2016-09-26 11:06:48 -07:00
Random-Liu
a67fec6419 Add separate build process for node test. 2016-09-16 11:17:49 -07:00
David McMahon
4e07b43d70 Deprecate kubernetes/kubenetes release infrastructure and doc. 2016-09-09 17:51:20 -07:00
gmarek
3e2c3a5db2 Make image-puller work on GCI nodes. 2016-09-07 15:23:39 +02:00
Minhan Xia
1a484d5585 bump debian-iptables version 2016-09-02 14:34:29 -07:00
Kubernetes Submit Queue
09a97203aa Merge pull request #30744 from madhusudancs/fed-split-build-to-develop-deploy
Automatic merge from submit-queue

Separate federation build.sh into development and deployment scripts.

The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.

Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.

This change also copies deploy/* files into the release distribution.

cc @kubernetes/sig-cluster-federation @colhom 

```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
2016-08-29 13:46:19 -07:00
Madhusudan.C.S
a6ff2364c5 Addressed review comments. 2016-08-25 14:34:19 -07:00
Madhusudan.C.S
d9cea93eb6 [Federation] Do not build separate binaries for federation control plane components, hyperkube should be sufficient.
Also build the hyperkube docker image on-the-fly.

This is only a temporary fix until the proposal in issue
https://github.com/kubernetes/kubernetes/issues/28630 is implemented.

Also, the new build/deployment method completely obviates this step.

We use debian image instead of busybox and do not build hyperkube as a
static binary yet. Wait until PR
https://github.com/kubernetes/kubernetes/pull/26028 is merged to build
static hyperkube binaries.
2016-08-25 14:29:32 -07:00
Madhusudan.C.S
9650a84f7b Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.

Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.

This change also copies deploy/* files into the release distribution.
2016-08-25 12:37:50 -07:00
Kubernetes Submit Queue
a24df66dc8 Merge pull request #30870 from piosz/rescheduler-setup
Automatic merge from submit-queue

Salt configuration for Rescheduler

ref #29023
2016-08-19 02:54:24 -07:00
Kubernetes Submit Queue
b1194ffb43 Merge pull request #30659 from ixdy/disable-kubemark-ppc
Automatic merge from submit-queue

Disable linux/ppc64le compilation by default

Work-around for #30384.

I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)

cc @Random-Liu @spxtr @david-mcmahon @luxas
2016-08-18 23:12:09 -07:00
Piotr Szczesniak
1f3fdab063 Salt configuration for Rescheduler 2016-08-18 12:24:09 +02:00
Madhusudan.C.S
2c32dc6ca4 Provide absolute path to cross build image VERSION file.
This allows invoking builds from non-root directories.
2016-08-16 22:51:04 -07:00
Jeff Grafton
dec4d1ea33 Disable linux/ppc64le compilation by default.
It can be reenabled with KUBE_BUILD_PPC64LE=y.
2016-08-16 17:05:42 -07:00
Tim Hockin
99a6a8ab16 Fix subtle build breakage
Repro case:
$ make clean generated_files
$ hack/update-generated-protobuf.sh

This would complain about not finding `fmt`, and it was indicating the wrong
GOROOT.  The problem was that the first step built binaries for generating
code, which *embeds* the value of GOROOT into the binary.  The whole tree was
bind-mounted into the build container and then JUST the dockerized dir was
mounted over it.  The in-container build tried to use the existing binaries,
but GOROOT is wrong.

This change whites-out the whole _output dir.

I first made just an anonymous volume for _output, but docker makes that as
root, which means I can't write to it from our non-root build.  So I just put
it in the data container.  This seems to work.  The biggest change this makes
is that the $GOPATH/bin/ and $GOPATH/pkg/ dirs will persist across dockerized
builds.
2016-08-12 21:20:38 -07:00
Michal Rostecki
5a85ed7646 Avoid duplicated contrib/ sources
This commit removes a part of common.sh script which copied
contrib/ sources for enabled contribs, which resulted in the
duplicated files inside tarball.

Fixes #30150
2016-08-10 07:39:37 +02:00
Tim Hockin
3b1407af70 Add goimports to the cross build 2016-08-05 22:58:57 -07:00
Kubernetes Submit Queue
3e5b40d9a5 Merge pull request #29947 from ixdy/go-bindata-cross-image
Automatic merge from submit-queue

Install go-bindata in cross-build image

Another follow-up to #25584.

We need `go-bindata` to create `test/e2e/generated`, and downloading it with `go get` at build time is painful for a variety of reasons. We can just include it in the cross-build image and not worry about it, especially as it updates very infrequently.

This fixes `hack/update-generated-protobuf.sh` as well.

cc @jayunit100 @soltysh
2016-08-04 06:03:29 -07:00
Jeff Grafton
b03784934d Add go-bindata to the cross-build image 2016-08-03 11:09:55 -07:00
Erick Fejta
add6feb53c Use format instead of awk to extract active account/project 2016-08-03 10:46:20 -07:00
Madhusudan.C.S
31f5525f43 Revert "[Federation] Downsize the release binary distribution." 2016-08-01 23:54:09 -07:00
Madhusudan.C.S
5a3a84968b [Federation] Do not build separate binaries for federation control plane components, hyperkube should be sufficient. 2016-08-01 11:45:21 -07:00
Jess Frazelle
a333cf429a update go to 1.6.3
Signed-off-by: Jess Frazelle <me@jessfraz.com>
2016-07-18 16:29:46 -04:00
Mike Danese
95e2e299a9 move kube-dns to the cluster/addons/ directory 2016-07-14 11:44:00 -07:00
Tim Hockin
7e5b59b08c Include almost the whole tree in the build img 2016-07-12 21:52:54 -07:00
Tim Hockin
881e21c2d5 Link _output/bin/ to real binaries for this arch
This makes followup commits easier wrt finding binaries during build.
2016-07-12 21:52:00 -07:00
Zach Loafman
3edbadc8a2 Migrate to kuberetes-release-dev
This turns off the kubernetes-release-dev mirror. #28193 should have
shifted all the references over, this should ensure that it's true.
2016-07-11 13:37:10 -07:00
Lorrin Nelson
85589b406b Fix unintentional grammatical double-negative in error message when Docker not found on OS X. 2016-07-07 11:26:30 -07:00
Lorrin Nelson
d0e36819a4 Fix build on OS X when GNU sed is present.
See #20147, which fixed it when system default BSD sed was present but did not account for possibiliyt of GNU sed on OS X.
2016-07-07 11:26:30 -07:00
k8s-merge-robot
e232ecd201 Merge pull request #28498 from endocode/kayrus/fix_kuberoot
Automatic merge from submit-queue

build: fixed ${KUBE_ROOT} prefix for build scripts

Running `./make-build-image.sh` command inside the `build` directory doesn't work:

```sh
$ cd build
$ ./make-build-image.sh 
./../build/common.sh: line 32: hack/lib/init.sh: No such file or directory
```

This PR adds `${KUBE_ROOT}` prefix for the `source` bash function. Also I added braces to unify the code style.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()
2016-07-07 07:33:25 -07:00
Brandon Weeks
d3208c6c3f
Test for certificate error and prompt to regenerate
If the docker-machine certificates get in a bad state, the current behavior
causes an infinite loop waiting for `docker-machine env` to return. Now it will
echo the certificate error and prompt the user to regenerate.
2016-07-06 10:27:16 -07:00
Anton Khramov
745d253d72 build: fixed ${KUBE_ROOT} prefix for build scripts 2016-07-05 14:47:31 +02:00
Tim Hockin
55eaa38036 Allow non-root build to write go code. 2016-07-03 00:39:59 -07:00
Tim Hockin
c3f4fb53ed Set hostname on build container 2016-07-03 00:38:32 -07:00
Tim Hockin
4959c2c76f Clean up logging in build container 2016-07-03 00:38:32 -07:00
Tim Hockin
ac75bd11cf Run builds as the calling user
This means no 'sudo' is needed to copy files out.
2016-07-03 00:36:20 -07:00
Tim Hockin
8fe8ec980e Cleanup a TODO from godeps -> vendor change
This logs a false "error" message, so it's time to go.  It was needed to ensure
nobody has stale build images laying around, but that was quite a while ago, so
it's probably safe now.
2016-07-02 15:15:39 -07:00
David McMahon
ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
k8s-merge-robot
3d22132a14 Merge pull request #28233 from zmerlynn/revert-revert-28172
Automatic merge from submit-queue

Revert revert of #28172

We're going to try reintroducing just the build changes in #28172, get signal off of that, and wait on re-introducing #28193. I also setup the `gke-updown` job for faster signal: https://github.com/kubernetes/test-infra/pull/239.

This reverts commit fe64293c6a.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()
2016-06-29 14:09:51 -07:00
k8s-merge-robot
3a6494e9ae Merge pull request #28132 from madhusudancs/fed-kubedns-flags-nodebootstrap
Automatic merge from submit-queue

Substitute federation_domain_map parameter with its value in node bootstrap scripts.

This PR also removes the substitution code we added to the build scripts.

**Release Note**

```release-note
If you use one of the kube-dns replication controller manifest in `cluster/saltbase/salt/kube-dns`, i.e. `cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}`, either substitute one of `__PILLAR__FEDERATIONS__DOMAIN__MAP__` or `{{ pillar['federations_domain_map'] }}` with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for `{{ pillar['federations_domain_map'] }`
pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
where `myfederation` is the name of the federation and `federation.test` is the domain name registered for the federation.
```

cc @erictune  @kubernetes/sig-cluster-federation @MikeSpreitzer @luxas 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()
2016-06-29 14:09:46 -07:00
Zach Loafman
ae313b6c8b Revert "Revert "Merge pull request #28172 from zmerlynn/push-alt-ci""
This reverts commit fe64293c6a.
2016-06-29 11:52:07 -07:00
Zach Loafman
fe64293c6a Revert "Merge pull request #28172 from zmerlynn/push-alt-ci"
This reverts commit 43437e4528, reversing
changes made to 532491aab6.
2016-06-29 07:56:48 -07:00
Zach Loafman
42827482b2 Build: Add KUBE_GCS_RELEASE_BUCKET_MIRROR option to push-ci-build.sh
And start pushing to new CI bucket as well.
2016-06-28 15:02:48 -07:00
k8s-merge-robot
f3359fe134 Merge pull request #27867 from johscheuer/add-upgrade-docker-vm
Automatic merge from submit-queue

Add upgrade Docker VM

Add an Error Message to upgarde your Docker VM if needed, example output:

```bash
+++ [0622 13:19:48] No docker host is set. Checking options for setting one...
+++ [0622 13:19:49] docker-machine was found.
+++ [0622 13:19:49] A Docker host using docker-machine named 'kube-dev' is ready to go!
Can't connect to 'docker' daemon.  please fix and retry.

Possible causes:
  - On Mac OS X, DOCKER_HOST hasn't been set. You may need to:
    - Create and start your VM using docker-machine or boot2docker:
      - docker-machine create -d virtualbox --virtualbox-memory 4096 --virtualbox-cpu-count -1 kube-dev
      - boot2docker init && boot2docker start
    - Set your environment variables using:
      - eval $(docker-machine env kube-dev)
      - $(boot2docker shellinit)
  - On Linux, user isn't in 'docker' group.  Add and relogin.
    - Something like 'sudo usermod -a -G docker jscheuermann'
    - RHEL7 bug and workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1119282#c8
  - On Linux, Docker daemon hasn't been started or has crashed.
!!! Error in hack/../hack/update-generated-protobuf.sh:53
  'return 1' exited with status 1
Call stack:
  1: hack/../hack/update-generated-protobuf.sh:53 main(...)
Exiting with status 1
Updating generated-protobuf FAILED

$docker info
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)
```

After running `docker-machine upgrade kube-dev` everything is fine again. So we should add a hint in the error message that this can also happen.
2016-06-28 04:35:20 -07:00
Madhusudan.C.S
816c4d9e2b Substitute federation_domain_map parameter with its value in node bootstrap scripts.
This also removes the substitution code we added to the build
scripts in one of the previous commits.
2016-06-27 14:54:49 -07:00
Matt Liggett
220da9949a Remove extra double quotes in --federations.
Fixes #28059
2016-06-24 16:43:52 -07:00
Madhusudan.C.S
3ee03a5584 Remove federation_domain_map entirely if it isn't set, seems to be not replaced by empty value. 2016-06-24 09:59:17 -07:00
Madhusudan.C.S
f980aa2ae1 Address review comments. 2016-06-23 17:47:09 -07:00
Madhusudan.C.S
c2e70a7c35 Parametrize the kube-dns --federations command line argument in the manifest.
This parameter is later substituted with the environment variable during
the build process.
2016-06-23 16:18:45 -07:00
Johannes Scheuermann
66e595b480 Add support for Docker for MacOS 2016-06-23 00:09:01 +02:00
Johannes Scheuermann
97ac3595d3 Add upgrade Docker VM 2016-06-22 23:27:41 +02:00
David McMahon
ef35e80604 Add sources to server tarballs. 2016-06-21 18:16:29 -07:00
k8s-merge-robot
637734f003 Merge pull request #27047 from luxas/enable-ppc64le
Automatic merge from submit-queue

Enable all ppc64le builds, except for hyperkube

Partially fixes: #25886 
Talked to @Pensu, and all other binaries seem to work fine

@david-mcmahon @ixdy @Pensu @smarterclayton
2016-06-18 16:32:19 -07:00
Lucas Käldström
9673ca16b2 Enable builds for all ppc64le binaries except hyperkube 2016-06-15 22:59:25 +03:00
nikhiljindal
ee478ec8c2 include federation binaries in the list of images we push during release 2016-06-14 14:30:51 -07:00
Daniel Smith
8faa88626d Revamp doc generation
Auto generated docs are **NO LONGER CHECKED IN**, only placeholders.

To generate them, e.g. before exporting docs, run hack/generate-docs.sh.

hack/verify-generated-docs.sh ensures that generated docs are merely the
placeholder text.

hack/update-generated-docs.sh puts the placeholder text in the proper
places.

The old munge behavior is moved into hack/{update|verify}-munge-docs.sh.
2016-06-08 17:22:12 -07:00
nikhiljindal
d2ea7a29ab Add federation controller manager to federation e2es 2016-06-02 00:17:11 -07:00
k8s-merge-robot
ee412efcef Merge pull request #26335 from girishkalele/kubedns-transition
Automatic merge from submit-queue

Switch DNS addons from skydns to kubedns

Change GCI and trusty cluster-helper scripts to use kubedns instead of skydns.
2016-05-31 16:14:48 -07:00
Girish Kalele
4c1047d359 Switch DNS addons from skydns to kubedns
Unified skydns templates using a simple underscore based template and
added transform sed scripts to transform into salt and sed yaml
templates

Moved all content out of cluster/addons/dns into build/kube-dns and
saltbase/salt/kube-dns
2016-05-31 10:14:14 -07:00
Prashanth Balasubramanian
c01a0583f7 Trusty deployment 2016-05-30 15:58:03 -07:00