Commit Graph

4616 Commits

Author SHA1 Message Date
Rolf Neugebauer
cc67c08667 kernel: Add kernel config and build support for 4.12 kernels
- The x86_64 kernel config was derived from our 4.11 config
  and then adjusted with the recent changes
- The arm64 kernel config was derived from the 4.9 config

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
6fc9a32c15 kernel: Add patches for the 4.12 kernel
The VMBus/Hyper-V socket patches were partly taken from the now
defunct 4.11 tree and partly form the WIP 4.12 tree at:
https://github.com/dcui/linux/commits/decui/msft-4.12.y

From the 4.11 tree:
- 0001-tools-build-Add-test-for-sched_getcpu.patch
  Does not apply, may not be needed anymore to compile perf
- 0002-vmbus-vmbus_open-reset-onchannel_callback-on-error.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0003-vmbus-add-the-matching-tasklet_enable-in-vmbus_close.patch
  Already upstream: 5116f5e2e05cf("vmbus: re-enable channel tasklet")
- 0004-vmbus-remove-goto-error_clean_msglist-in-vmbus_open
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0005-vmbus-dynamically-enqueue-dequeue-a-channel-on-vmbus.patch
  From the 4.11 patches
- 0006-hv_sock-implements-Hyper-V-transport-for-Virtual-Soc.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0007-VMCI-only-try-to-load-on-VMware-hypervisor.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0008-hv_sock-add-the-support-of-auto-loading.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0009-tools-hv_sock-2-simple-test-cases.patch
  Dropped, this was just test code
- 0010-vmbus-introduce-in-place-packet-iterator.patch
  Already upstream: f3dd3f4797652("vmbus: introduce in-place packet iterator")
- 0011-hvsock-fix-a-race-in-hvs_stream_dequeue.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0012-hvsock-fix-vsock_dequeue-enqueue_accept-race.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0013-Drivers-hv-vmbus-Fix-rescind-handling.patch
  From the 4.11 patches
- 0014-vmbus-fix-hv_percpu_channel_deq-enq-race.patch
  From the 4.11 patches
- 0015-vmbus-add-vmbus-onoffer-onoffer_rescind-sync.patch
  From the 4.11 patches
- 0016-hv-sock-a-temporary-workaround-for-the-pending_send_.patch
  DROPPED. Does not apply at all anymore. Was a hack anyway
- 0017-vmbus-fix-the-missed-signaling-in-hv_signal_on_read.patch
  Applied manually from the 4.11 patches
- 0018-hv-sock-avoid-double-FINs-if-shutdown-is-called.patch
  From https://github.com/dcui/linux/commits/decui/msft-4.12.y
- 0019-Added-vsock-transport-support-to-9pfs.patch
  From the 4.11 patches
- 0020-NVDIMM-reducded-ND_MIN_NAMESPACE_SIZE-from-4MB-to-4K.patch
  From the 4.11 patches

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
0f5825fc32 kernel: Improve kconfig Docker file
Be a bit more verbose and error on error.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
aec0d5a702 kernel: Disable VHOST_VSOCK on 4.4 kernels
The host side VSOCK implementation introduced with
0009-VSOCK-Introduce-vhost_vsock.ko.patch
does not compile due to vhost_vq_init_access not being defined.

VHOST support (including VHOST_VSOCK) was enabled with
86deeaff ("kernel: Bring 4.4 x86_64 kernel config more in line
with 4.9") but not compile tested. Having VHOST support in
itself is fine, it's just the VHOST_VSOCK portion which is not
avail.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Rolf Neugebauer
20cc4b3ddf kernel: Don't check kernel config for debug kernels
The kernel config for debug kernels is created by concatenating
config files, so we can't use diff to check it.

This fixes a regression introduced by:
9362de0a ("kernel: Verify kernel config")

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-23 14:33:25 +01:00
Riyaz Faizullabhoy
d9869fb727 Merge pull request #2453 from riyazdf/okernel-video
sig-security: add okernel video and forum post
2017-08-22 12:39:40 -10:00
Riyaz Faizullabhoy
a5e98db3da sig-security: add okernel video and forum post
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-08-22 14:09:03 -07:00
Justin Cormack
2129d1de21 Merge pull request #2450 from justincormack/improve-wireguard-example
Improve wireguard example
2017-08-22 17:44:01 +01:00
Justin Cormack
9ec58e12bb Merge pull request #2443 from djs55/fix-metadata
init: mkdir /etc/ssl/certs as required by the metadata package
2017-08-22 13:31:24 +01:00
Justin Cormack
3656a7b03e Merge pull request #2448 from justincormack/dockerup
Update Docker CE image to 17.06.1
2017-08-22 13:29:14 +01:00
Justin Cormack
2ed8b2997c Add a test for Wireguard
This is based on the example, but modified so that it can work as a test.

It is slightly less convenient running services as tests as the output is
sent to log files, so we have an `onshutdown` container that checks to see
if the test passed.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 13:23:27 +01:00
Justin Cormack
6e2e9d7049 Improve Wireguard example with more realistic setup
This creates two containers connected over a Wireguard tunnel.

From the `getty` container that comes up you can connect to the
Nginx server via an encrypted tunnel with
```
wget -O - http://192.168.2.1
```

You can use `nsenter` to check out the other namespaces. There is no
external connectivity in either the `getty` or `nginx` containers,
so the only available networking is through the wireguard tunnel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:57:45 +01:00
Justin Cormack
584613cddb Update ip package to latest Alpine base
There was a wireguard mismatch with the kernel module in the recent kernel
builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:50:20 +01:00
Justin Cormack
1c4e8c3ae2 Update Docker CE image to 17.06.1
Bug fix update.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 10:54:26 +01:00
Rolf Neugebauer
329cdb7c7f Merge pull request #2445 from rn/kconfig
Kernel config improvements
2017-08-21 22:18:28 +01:00
Rolf Neugebauer
1d8790fb76 docs: Update kernel docs
- Update section on how to change the kernel config
- Reword kernel module section. It was messy
- General tidying up:
  - empty lines around quoted areas
  - two empty line above and one empty line below section headings
  - Limit line length.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
c3b9972b32 scripts: Add script to split kernel config files
This script is slightly modified from the ChromiumOS splitconfig

It takes a number of kernel config files and prints the common
on specific kernel config options to seperate files.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
119cc56fcf kernel: Align arm64 4.4 kernel config with 4.9
Note, vhost vsock is disabled on arm64 because it failed to compile.
'vhost_vq_init_access' was not defined, but with a quick check
I could not find where it was supposed to be defined.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
86deeaff90 kernel: Bring 4.4 x86_64 kernel config more in line with 4.9
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Rolf Neugebauer
ba2e6a5bb8 kernel: Add a Dockerfile to make it easier to configure kernels
The new Dockerfile.kconfig can be used, via the 'kconfig' make target
to build a 'linuxkit/kconfig' images. This images contains the patched
source and default kernel configs for all supported kernels.

It's useful to updating the kernel config files.

While at it, also update the alpine base.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-21 20:43:30 +01:00
Justin Cormack
6f1e4add2e Merge pull request #2444 from justincormack/cni-persistence
Kubernetes improvements
2017-08-21 20:03:30 +01:00
Justin Cormack
4fec826d22 Update kube hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 17:57:08 +01:00
Justin Cormack
c857b91e64 Update Kubernetes to 1.7.4
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 17:54:21 +01:00
David Scott
da86df1ad6 init: add /etc/ssl/certs as required by the metadata package
The metadata package has binds
  - /dev
  - /var
  - /sys
  - /etc/resolv.conf
  - /etc/ssl/certs

but unfortunately `/etc/ssl/certs` doesn't exist and this causes the
following commands:

  cd blueprints/docker-for-mac # easy example
  moby build -name docker-for-mac base.yml docker-17.06-ce.yml
  linuxkit run hyperkit -networking=vpnkit -vsock-ports=2376 -disk size=500M docker-for-mac

to produce the following error on the VM console:

  container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/etc/ssl/certs\\\" to rootfs \\\"/containers/onboot/000-metadata/rootfs\\\" at \\\"/etc/ssl/certs\\\" caused \\\"stat /etc/ssl/certs: no such file or directory\\\"\""
  2017/08/21 16:39:40 Error creating 000-metadata: exit status 1

This patch creates /etc/ssl/certs in the `init` package. The metadata package
will now say things like

  2017/08/21 16:44:39 No metadata/userdata found. Bye

Signed-off-by: David Scott <dave.scott@docker.com>
2017-08-21 17:46:29 +01:00
Justin Cormack
498d5a1966 Merge pull request #2442 from justincormack/tools-additions
Add some more tools to base image and only clone containerd once
2017-08-21 17:00:23 +01:00
Justin Cormack
6264912489 Restrict kubernetes to amd64
- currently no arm64 binary downloads available

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 16:07:49 +01:00
Justin Cormack
1597ddb72f Update weave yaml to k8s 1.7
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:58:13 +01:00
Justin Cormack
ea87b6df79 Use standard LinuxKit base image to build Kubernetes container
- now has everything we need in it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:51:26 +01:00
Justin Cormack
424203cd9c Move Kubernetes CNI paths to /var so they persist
Currently they will be on a read only partition so broken;
previously this would have been a non persistent read write partition
in an initramfs but this no longer works.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:46:18 +01:00
Justin Cormack
0cda27d4ef Update hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:40:17 +01:00
Justin Cormack
90f905a799 Move containerd clone to tools/alpine
As discussed before, as we use this in three places, cloning in
base makes more sense.

Update base image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:16:09 +01:00
Justin Cormack
2acb7151fb Merge pull request #2441 from justincormack/kubernetes-mkdir
Use new runtime mkdir instead of mkdir container
2017-08-21 13:35:15 +01:00
Justin Cormack
92d31d733d Add ebtables, socat, libc6-compat to alpine base
These are needed by Kubernetes which is currently not using the standard abse image.

Also sort it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 13:34:02 +01:00
Justin Cormack
534026e3ea Use new runtime mkdir instead of mkdir container
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 12:01:18 +01:00
Rolf Neugebauer
79109df372 Merge pull request #2440 from rn/kbuild
Improvements to the kernel build
2017-08-20 22:28:50 +01:00
Rolf Neugebauer
ffcf5db6ee kernel: Use local kernel source if available
The kernel build currently downloads the source tar ball every
time, which is a little tedious when experimenting with kernel
configs or when compiling the kernel multiple times.

This commit adds a new 'fetch' make target which downloads the
kernel sources into ./sources. Then in the Dockerfile we add
the directory and only download the source if it is not present.

The tarballs signature is till checked on each build.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-20 11:41:59 +01:00
Rolf Neugebauer
9362de0adb kernel: Verify kernel config
Since we supply a full .config file we can check that after
make defconfig/oldconfig it hasn't changed. This should catch
cases where a config option has changed between releases.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-20 11:05:35 +01:00
Rolf Neugebauer
72ed2b3a06 kernel: Rename kernel_config-4.x.x to kernel_config-4.x.x-x86_64
Consistently arch suffixes for kernel config.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-20 10:46:28 +01:00
Justin Cormack
ca35c06723 Merge pull request #2439 from martencassel/patch-1
Fix typo in docs/faq.md
2017-08-19 22:07:03 +01:00
Rolf Neugebauer
f0609113d8 Merge pull request #2438 from justincormack/packet-metadata
Add Packet metadata and bonding support
2017-08-19 20:59:35 +01:00
marten
c09dc5190a Fix typo in faq
Signed-off-by: Marten <mcpop28@hotmail.com>
2017-08-19 21:46:46 +02:00
Justin Cormack
3e494f7762 Update hashes for metadata
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:30:04 +01:00
Justin Cormack
ada924b527 Update packet example
- use one shot dhcp
- add a one shot rngd as needs entropy
- add metadata
- remove ssh key from host as we have in metadata

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:27:37 +01:00
Justin Cormack
a808808ce1 Add vendoring needed for netlink, packet
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:21 +01:00
Justin Cormack
92451cf2e4 Add packet metadata support, with bonding
Use the packet metadata to set up bonding, also get ssh keys and hostname.

This does not yet do anything with disk metadata.

Userdata is not used if it has been used for ipxe, but is otherwise available.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:14 +01:00
Justin Cormack
d01438acd4 Merge pull request #2436 from rn/kern-up
Update Kernel to 4.9.44/4.4.83
2017-08-18 22:35:04 +01:00
Rolf Neugebauer
efe8292e25 Update YAML files to new kernels
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-18 18:08:33 +01:00
Rolf Neugebauer
5898bc9f0d kernel: Update to 4.9.44/4.4.83
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-18 18:08:32 +01:00
Rolf Neugebauer
6f14a2acda Merge pull request #2435 from rn/karm64
Converge arm64 and x86 kernel configs
2017-08-18 17:56:59 +01:00
Justin Cormack
9506d45d49 Merge pull request #2413 from justincormack/runtime-config
Add a runtime config
2017-08-18 17:45:10 +01:00