Commit Graph

4391 Commits

Author SHA1 Message Date
Ian Campbell
87a6e19770 kubernetes: add "show-tags" target to image-cache
Apart from adding the recursive target itself this required:

- Unescaping the @ in the image names, this was confusing `make` into always
  rebuilding and wasn't necessary (I had previously thought I had seen oddities
  due to these being interpreted by the `patsubst`, but I think that was just the
  colons.
- Making the recursive rules silent (prepending an @), those command lines are
  not especially enlightening and they obscure the output in the show-tags case.

With this the output is like:

    $ make --no-print-directory -C image-cache/ show-tags
    linuxkitprojects/kubernetes-image-cache-common:94a0715c6b3604e909bc0da74260dc7f1142d90d-dirty
    linuxkitprojects/kubernetes-image-cache-control-plane:94a0715c6b3604e909bc0da74260dc7f1142d90d-dirty

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:36 +01:00
Ian Campbell
ee7d785b84 kubernetes: minimise size of image cache images
The total size of the images in the common and control-plane cache is 251M and
528M respectively.

This changes drops the size of the cache images from 353M to 274M and from 630M
to 530M, reducing the overhead from ~100M to ~20M.

The initrd images shrink from 273M to 246M and from 416M to 363M (the initrd's
are compressed).

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:36 +01:00
Ian Campbell
58227545b2 kubernetes: Only explictly include *.tar in images
No need to have Dockerfile etc.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:36 +01:00
Ian Campbell
4b030f2fc6 kubernetes: use digests for cached images
This ensures rebuilds will always get the same things.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:36 +01:00
Ian Campbell
56a8d2468a kubernetes: modernise image cache builds
This updates the build of the two image caches to use the `pkg/package.mk`
infrastructure, albeit in a slightly (ok, very) atypical way.

In order to share the bulk of the build code (including the `Dockerfile` and
the `Makefile` machinery to download the images) we arrange for the necessary
bits to be copied at build time into distinct subdirectories and for the
`pkg/package.mk` to be aware of this possibility.

Since pkg/package.mk is only set up to build a single package we use a single
`image-cache/Makefile` to drive the whole process and recurse into
`Makefile.pkg` to build individual packages.

One particular subtlety is that the package hash is based on the `image-cache`
directory (which is in `git`) rather than the generated subdirectories (which
are not in `git`). Since all the generators (and their inputs) are in the
`image-cache` directory this is what we want. This means that the two images
are given the same tag, but this is deliberate and desirable.

The generated directories are completely temporary to avoid picking up stale
versions of images when versions are updated. Images are hardlinked into place.

The images are moved to the linuxkitprojects org. Using a dev tag for now, will
update once everything is in place.

Also use "tag" rather than "build" where appropriate in the Makefile.

There is no point in the .dockerignore now, but add a .gitignore.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:32 +01:00
Ian Campbell
0d8c4789a1 kubernetes: update clean rule
The kernel has been -kernel rather than -bzImage for some time. Likewise the
disk moved the state dir.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:33:54 +01:00
Rolf Neugebauer
118816b4c3 Merge pull request #2231 from samoht/dhcp-split
[WIP] Split the DHCP client into three components communicating via RPCs
2017-07-18 20:05:02 +01:00
Rolf Neugebauer
c073b0ebd6 Merge pull request #2242 from rn/lcow-net
Fix networking in LCOW
2017-07-18 20:02:47 +01:00
Avi Deitcher
0f725c366e Merge pull request #2241 from deitch/fix-getty-background
flag in init and background only if not in init
2017-07-18 21:39:53 +03:00
Rolf Neugebauer
6295d0e425 Merge pull request #2240 from justincormack/authorsup
Update AUTHORS
2017-07-18 18:16:10 +01:00
Justin Cormack
09e9357499 Merge pull request #2235 from justincormack/use-library-docker
Use the upstream dind package to run docker
2017-07-18 17:27:45 +01:00
Rolf Neugebauer
26a324c35c blueprint: Update lcow blueprint
This should enable networking in LCOW containers.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-18 15:55:27 +01:00
Avi Deitcher
03244e3776 Change hash for getty in examples
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-18 17:47:36 +03:00
Rolf Neugebauer
4ff7916e10 pkg/init-lcow: Temporarily use a branch of my dev tree
The branch has fixes to the networking setup (see
https://github.com/Microsoft/opengcs/issues/56 and
https://github.com/Microsoft/opengcs/pull/63)

Also remove runc from the package as the hard-coded runc
path has been removed (see
https://github.com/Microsoft/opengcs/pull/58)

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-18 15:44:20 +01:00
Justin Cormack
85768e1b50 Update AUTHORS
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 15:28:14 +01:00
Justin Cormack
2f0933ba43 Merge pull request #2238 from justincormack/shutdown
Add onshutdown support
2017-07-18 15:23:27 +01:00
Justin Cormack
61e8b9a853 Merge pull request #2237 from dave-tucker/rtf-up
Makefile: Update rtf version
2017-07-18 14:45:09 +01:00
Avi Deitcher
af0331d382 flag in init and background only if not in init
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-18 16:37:05 +03:00
Justin Cormack
ae039ac141 update hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 14:22:16 +01:00
Justin Cormack
96e71324c6 Update moby tool for onshutdown support
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 14:20:59 +01:00
Dave Tucker
8a07c75e34 Makefile: Update rtf version
This update brings in linuxkit/rtf#24 that fixes panics in cases where
there are concurrent reads/writes of the logging backends.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-07-18 13:48:54 +01:00
Justin Cormack
c6aa7b9053 Add onshutdown support to runc
This will run any containers in `/containers/onshutdown` on a clean
shutdown.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:54:58 +01:00
Justin Cormack
fe8f979da5 Add support for shutdown scripts
These can be added by other packages if they need to do something on
clean shutdown.

Crash only software can ignore this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:54:58 +01:00
Justin Cormack
a4650b242f Use the upstream dind package to run docker
It is pretty close to our docker package, if we adjust the command
that is run to avoid the actual dind startup script. We can't use
the normal docker image as it does not have mkfs and so on.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:15:38 +01:00
Rolf Neugebauer
90d1866bdc Merge pull request #2234 from deitch/kernel-tpm-modules
[WIP] Add TPM module builder options to kernel
2017-07-18 09:59:47 +01:00
Avi Deitcher
19ce7e18a2 Add TPM module builder options to kernel
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-18 11:01:40 +03:00
Riyaz Faizullabhoy
4c5e202d28 Merge pull request #2232 from justincormack/metadata
Update moby to support metadata
2017-07-17 11:13:56 -07:00
Riyaz Faizullabhoy
3dabd36d7c Merge pull request #2230 from ijc/getty-test-with-hello-world
Use hello-world instead of redis for getty test
2017-07-17 11:13:10 -07:00
Thomas Gazagnaire
f5716ce985 Split the DHCP client into three components communicating via named pipes
3 components:

- network: read eht0 and proxy only DHCP traffic
- engine: read DHCP traffic, handle DHCP client state machine, and call the
  host actuator to change the host config when a lease is obtained
  host system configuration.
- actuator: perform the acutall net syscalls, read and write host configuration
  files, etc

These three components can either be linked together in a single binary
(see src/dhcp-client/main.ml) or can be used as 3 binaries communicating
over cap-n-proto.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-07-17 17:46:17 +02:00
Justin Cormack
7886e61e3d Update moby to support metadata
Add the yaml config to the default `linuxkit.yml`. You can check
this with `cat /proc/1/root/etc/linuxkit-config`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-17 15:58:11 +01:00
Justin Cormack
f1345eefb9 Merge pull request #2228 from ijc/kubernetes
kubernetes project updates
2017-07-17 15:27:35 +01:00
Ian Campbell
39c8b1b5e4 Use hello-world instead of redis for getty test
It is a far smaller download. Also pin to a specific version to protect against
the possibility of the output string changing.

The new image name is a bit of a mouthful, put it (and the expected output)
into a variable.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 15:01:54 +01:00
Justin Cormack
7aacc2b9bc Merge pull request #2220 from thebsdbox/VMwareDocs
Added VMware documentation
2017-07-17 14:29:07 +01:00
Justin Cormack
6079dfeff7 Merge pull request #2224 from deitch/updated-kernel-instructions
Update kernel build docs
2017-07-17 14:28:11 +01:00
Ian Campbell
c495ccb8fb kubernetes: Update yml files.
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:45:55 +01:00
Ian Campbell
d48cc7998b kubernetes: Use pkg/metadata
Still a flat/unstructured config space, but at least uses the mounting
machinery.

`boot.sh` continues to just work without modification.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
23d6649de0 kubernetes: Format scripts to be more readable.
These were originally generated by some box builder runes and then taken
wholesale here. Format them to be more readable.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
f18ed90d4f kubernetes: Add local scripts after downloads
Caches much better.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
a08a312d1f kubernetes: Use image labels for image-cache bind mounts
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
20c0d080df kubernetes: use image labels to simplify yaml requirements
Also `net: host` is the default, so drop.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
f853fab1f9 kubernetes: disable content trust for gcr.io
It doesn't support it. This makes "make cache-images" work. Previously it would
fail with various:

    Error: remote trust data does not exist for gcr.io/google_containers/pause-amd64: gcr.io does not have trust data for gcr.io/google_containers/pause-amd64

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:11 +01:00
Ian Campbell
cd43b44e7c kubernetes: Build image using Dockerfile and standard build system
This is a pretty straight port of the previous box stuff, without much attempt
to clean things up.

Image label is a placeholder, will update once a batch of changes are complete.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:42:09 +01:00
Ian Campbell
d208078aa0 kubernetes: cmdline drop page_poison and put console=ttyS0 last
Following the trend started in a5b9464a4e.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:41:05 +01:00
Ian Campbell
4a27008397 kubernetes: Update to latest pkg/mount.
I somehow managed to miss these in #2209

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-17 13:41:05 +01:00
Ian Campbell
32506bd7f7 kubernetes: Drop custom mounts image and use standard one
Apart from the /var/lib mount itself the custom package:

- Made host /etc/cni and /opt/cni rshared. This has been handled by init make /
  rshared since 3c326bebdf ("Make / rshared").
- Make /var/lib/kubeadm after mount. For now handle this with a dedicated start
  of day container instead.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-17 13:41:03 +01:00
Ian Campbell
2cfac0e8bb kubernetes: Expose each node's ssh port on the host starting from 2222
Port base is configurable (via $KUBE_PORT_BASE envvar). Master uses this and
nodes use subsequent ports.

Check that the node number is numeric so we can add them to things, but avoid
worker node 0 since the port will clash with master.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-17 13:37:46 +01:00
Rolf Neugebauer
ce35fea83e Merge pull request #2225 from rn/kernup
Update kernels to 4.11.11/4.9.38/4.4.77
2017-07-17 13:23:06 +01:00
Avi Deitcher
c6b7b3c00a Update kernel build docs
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-17 14:03:53 +03:00
thebsdbox
0c1776303b Added VMware documentation
Signed-off-by: Dan Finneran <daniel.finneran@gmail.com>
2017-07-17 11:53:28 +01:00
Rolf Neugebauer
fcac29681b Update kernels in YAML files
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-17 11:04:18 +01:00