Commit Graph

4882 Commits

Author SHA1 Message Date
Rolf Neugebauer
719bba5486 docs: Update kernel docs with new build customisation
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 18:26:46 +01:00
Rolf Neugebauer
7209afb758 kernel: Enable custom kernel config additions
For some use cases, we may want to add additional kernel
configuration options (e.g. when adding AUFS). This commit
enables it by:
- renaming DEBUG to EXTRA
- append kernel_config${EXTRA} to the kernel config
- allowing passing in an EXTRA argument to the make file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 18:26:46 +01:00
Rolf Neugebauer
e75cf2f611 kernel: Use -dbg instead of _dbg as suffix for debug kernels
Otherwise we have a eird mixture of _ and - in the names.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 18:26:46 +01:00
Rolf Neugebauer
15d3ce25aa kernel: Add contents of ${DEBUG} to the LOCALVERSION
This way 'uname -r' will report something like
4.9.44-linuxkit_dbg on debug kernels.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 15:10:53 +01:00
Rolf Neugebauer
8b84baf226 kernel: Allow disabling content trust
specifying NOTRUST=1 on the make command line disables
content trust just like with packages.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 15:10:53 +01:00
Rolf Neugebauer
677bf96dff kernel: Fix warning about empty continuation
Without this change, recent Docker build produce this warning:
[WARNING]: Empty continuation line found in:
    RUN apk add     xz     xz-dev     zlib-dev &&     if [ $(uname -m) == x86_64 ]; then         apk add libunwind-dev;
    fi
[WARNING]: Empty continuation lines will become errors in a future release.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-24 11:06:28 +01:00
Justin Cormack
54518101f0 Merge pull request #2451 from justincormack/runtime-mount
Add support for mount in runtime config
2017-08-23 19:31:26 +01:00
Justin Cormack
8e652b4656 Merge pull request #2456 from justincormack/missing-case
Add missing case when running non existent file in hyperkit
2017-08-23 17:57:42 +01:00
Justin Cormack
24e10defc5 Update moby tool
This has corresponding changes to support runtime mount changes.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:44:51 +01:00
Justin Cormack
baf76d5e0c update hashes for init
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:43:18 +01:00
Justin Cormack
5773dfd844 Cleanup cleanup code
Only remove main mount at rootfs, as we do not really know
about the state of the rest.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
49cf74f4f6 Switch to having the moby tool specify how to do mounts
This removes all the code that had knowledge of how to do read only
and read write container mounts, and just uses the runtime config.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
c677b391fc Add support for mount in runtime config
This could be used in LinuxKit now, as there are some examples, eg
https://github.com/linuxkit/linuxkit/blob/master/blueprints/docker-for-mac/base.yml#L33
which are creating containers to do a mount.

The main reason though is to in future change the ad hoc code that generates
overlay mounts for writeable containers with a runtime config which does
the same thing; this code needs to create both tmpfs and overlay mounts.

See https://github.com/moby/tool/pull/145

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Rolf Neugebauer
dfc1068e32 Merge pull request #2455 from rn/4.12
Initial support for 4.12 kernels (and assorted fixes)
2017-08-23 16:38:20 +01:00
Justin Cormack
4a374eb175 Add missing case when running non existent file in hyperkit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:08:30 +01:00
Rolf Neugebauer
13e5a05a4c tests: Add tests for 4.12.x kernels
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
cc67c08667 kernel: Add kernel config and build support for 4.12 kernels
- The x86_64 kernel config was derived from our 4.11 config
  and then adjusted with the recent changes
- The arm64 kernel config was derived from the 4.9 config

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
6fc9a32c15 kernel: Add patches for the 4.12 kernel
The VMBus/Hyper-V socket patches were partly taken from the now
defunct 4.11 tree and partly form the WIP 4.12 tree at:
https://github.com/dcui/linux/commits/decui/msft-4.12.y

From the 4.11 tree:
- 0001-tools-build-Add-test-for-sched_getcpu.patch
  Does not apply, may not be needed anymore to compile perf
- 0002-vmbus-vmbus_open-reset-onchannel_callback-on-error.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0003-vmbus-add-the-matching-tasklet_enable-in-vmbus_close.patch
  Already upstream: 5116f5e2e05cf("vmbus: re-enable channel tasklet")
- 0004-vmbus-remove-goto-error_clean_msglist-in-vmbus_open
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0005-vmbus-dynamically-enqueue-dequeue-a-channel-on-vmbus.patch
  From the 4.11 patches
- 0006-hv_sock-implements-Hyper-V-transport-for-Virtual-Soc.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0007-VMCI-only-try-to-load-on-VMware-hypervisor.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0008-hv_sock-add-the-support-of-auto-loading.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0009-tools-hv_sock-2-simple-test-cases.patch
  Dropped, this was just test code
- 0010-vmbus-introduce-in-place-packet-iterator.patch
  Already upstream: f3dd3f4797652("vmbus: introduce in-place packet iterator")
- 0011-hvsock-fix-a-race-in-hvs_stream_dequeue.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0012-hvsock-fix-vsock_dequeue-enqueue_accept-race.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0013-Drivers-hv-vmbus-Fix-rescind-handling.patch
  From the 4.11 patches
- 0014-vmbus-fix-hv_percpu_channel_deq-enq-race.patch
  From the 4.11 patches
- 0015-vmbus-add-vmbus-onoffer-onoffer_rescind-sync.patch
  From the 4.11 patches
- 0016-hv-sock-a-temporary-workaround-for-the-pending_send_.patch
  DROPPED. Does not apply at all anymore. Was a hack anyway
- 0017-vmbus-fix-the-missed-signaling-in-hv_signal_on_read.patch
  Applied manually from the 4.11 patches
- 0018-hv-sock-avoid-double-FINs-if-shutdown-is-called.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0019-Added-vsock-transport-support-to-9pfs.patch
  From the 4.11 patches
- 0020-NVDIMM-reducded-ND_MIN_NAMESPACE_SIZE-from-4MB-to-4K.patch
  From the 4.11 patches

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
0f5825fc32 kernel: Improve kconfig Docker file
Be a bit more verbose and error on error.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
aec0d5a702 kernel: Disable VHOST_VSOCK on 4.4 kernels
The host side VSOCK implementation introduced with
0009-VSOCK-Introduce-vhost_vsock.ko.patch
does not compile due to vhost_vq_init_access not being defined.

VHOST support (including VHOST_VSOCK) was enabled with
86deeaff ("kernel: Bring 4.4 x86_64 kernel config more in line
with 4.9") but not compile tested. Having VHOST support in
itself is fine, it's just the VHOST_VSOCK portion which is not
avail.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
20cc4b3ddf kernel: Don't check kernel config for debug kernels
The kernel config for debug kernels is created by concatenating
config files, so we can't use diff to check it.

This fixes a regression introduced by:
9362de0a ("kernel: Verify kernel config")

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Riyaz Faizullabhoy
d9869fb727 Merge pull request #2453 from riyazdf/okernel-video
sig-security: add okernel video and forum post
2017-08-22 12:39:40 -10:00
Riyaz Faizullabhoy
a5e98db3da sig-security: add okernel video and forum post
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-08-22 14:09:03 -07:00
Justin Cormack
2129d1de21 Merge pull request #2450 from justincormack/improve-wireguard-example
Improve wireguard example
2017-08-22 17:44:01 +01:00
Justin Cormack
9ec58e12bb Merge pull request #2443 from djs55/fix-metadata
init: mkdir /etc/ssl/certs as required by the metadata package
2017-08-22 13:31:24 +01:00
Justin Cormack
3656a7b03e Merge pull request #2448 from justincormack/dockerup
Update Docker CE image to 17.06.1
2017-08-22 13:29:14 +01:00
Justin Cormack
2ed8b2997c Add a test for Wireguard
This is based on the example, but modified so that it can work as a test.

It is slightly less convenient running services as tests as the output is
sent to log files, so we have an `onshutdown` container that checks to see
if the test passed.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 13:23:27 +01:00
Justin Cormack
6e2e9d7049 Improve Wireguard example with more realistic setup
This creates two containers connected over a Wireguard tunnel.

From the `getty` container that comes up you can connect to the
Nginx server via an encrypted tunnel with
```
wget -O - http://192.168.2.1
```

You can use `nsenter` to check out the other namespaces. There is no
external connectivity in either the `getty` or `nginx` containers,
so the only available networking is through the wireguard tunnel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:57:45 +01:00
Justin Cormack
584613cddb Update ip package to latest Alpine base
There was a wireguard mismatch with the kernel module in the recent kernel
builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:50:20 +01:00
Justin Cormack
1c4e8c3ae2 Update Docker CE image to 17.06.1
Bug fix update.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 10:54:26 +01:00
Rolf Neugebauer
329cdb7c7f Merge pull request #2445 from rn/kconfig
Kernel config improvements
2017-08-21 22:18:28 +01:00
Rolf Neugebauer
1d8790fb76 docs: Update kernel docs
- Update section on how to change the kernel config
- Reword kernel module section. It was messy
- General tidying up:
  - empty lines around quoted areas
  - two empty line above and one empty line below section headings
  - Limit line length.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
c3b9972b32 scripts: Add script to split kernel config files
This script is slightly modified from the ChromiumOS splitconfig

It takes a number of kernel config files and prints the common
on specific kernel config options to seperate files.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
119cc56fcf kernel: Align arm64 4.4 kernel config with 4.9
Note, vhost vsock is disabled on arm64 because it failed to compile.
'vhost_vq_init_access' was not defined, but with a quick check
I could not find where it was supposed to be defined.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
86deeaff90 kernel: Bring 4.4 x86_64 kernel config more in line with 4.9
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
ba2e6a5bb8 kernel: Add a Dockerfile to make it easier to configure kernels
The new Dockerfile.kconfig can be used, via the 'kconfig' make target
to build a 'linuxkit/kconfig' images. This images contains the patched
source and default kernel configs for all supported kernels.

It's useful to updating the kernel config files.

While at it, also update the alpine base.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Justin Cormack
6f1e4add2e Merge pull request #2444 from justincormack/cni-persistence
Kubernetes improvements
2017-08-21 20:03:30 +01:00
Justin Cormack
4fec826d22 Update kube hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 17:57:08 +01:00
Justin Cormack
c857b91e64 Update Kubernetes to 1.7.4
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 17:54:21 +01:00
David Scott
da86df1ad6 init: add /etc/ssl/certs as required by the metadata package
The metadata package has binds
  - /dev
  - /var
  - /sys
  - /etc/resolv.conf
  - /etc/ssl/certs

but unfortunately `/etc/ssl/certs` doesn't exist and this causes the
following commands:

  cd blueprints/docker-for-mac # easy example
  moby build -name docker-for-mac base.yml docker-17.06-ce.yml
  linuxkit run hyperkit -networking=vpnkit -vsock-ports=2376 -disk size=500M docker-for-mac

to produce the following error on the VM console:

  container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/etc/ssl/certs\\\" to rootfs \\\"/containers/onboot/000-metadata/rootfs\\\" at \\\"/etc/ssl/certs\\\" caused \\\"stat /etc/ssl/certs: no such file or directory\\\"\""
  2017/08/21 16:39:40 Error creating 000-metadata: exit status 1

This patch creates /etc/ssl/certs in the `init` package. The metadata package
will now say things like

  2017/08/21 16:44:39 No metadata/userdata found. Bye

Signed-off-by: David Scott <dave.scott@docker.com>
2017-08-21 17:46:29 +01:00
Justin Cormack
498d5a1966 Merge pull request #2442 from justincormack/tools-additions
Add some more tools to base image and only clone containerd once
2017-08-21 17:00:23 +01:00
Justin Cormack
6264912489 Restrict kubernetes to amd64
- currently no arm64 binary downloads available

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 16:07:49 +01:00
Justin Cormack
1597ddb72f Update weave yaml to k8s 1.7
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:58:13 +01:00
Justin Cormack
ea87b6df79 Use standard LinuxKit base image to build Kubernetes container
- now has everything we need in it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:51:26 +01:00
Justin Cormack
424203cd9c Move Kubernetes CNI paths to /var so they persist
Currently they will be on a read only partition so broken;
previously this would have been a non persistent read write partition
in an initramfs but this no longer works.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:46:18 +01:00
Justin Cormack
0cda27d4ef Update hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:40:17 +01:00
Justin Cormack
90f905a799 Move containerd clone to tools/alpine
As discussed before, as we use this in three places, cloning in
base makes more sense.

Update base image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:16:09 +01:00
Justin Cormack
2acb7151fb Merge pull request #2441 from justincormack/kubernetes-mkdir
Use new runtime mkdir instead of mkdir container
2017-08-21 13:35:15 +01:00
Justin Cormack
92d31d733d Add ebtables, socat, libc6-compat to alpine base
These are needed by Kubernetes which is currently not using the standard abse image.

Also sort it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 13:34:02 +01:00
Justin Cormack
534026e3ea Use new runtime mkdir instead of mkdir container
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 12:01:18 +01:00