Commit Graph

34 Commits

Author SHA1 Message Date
Justin Cormack
3aff6b503c Simplify extracting from kernel build
Makes build faster

Improve #691

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-19 17:20:18 +00:00
Justin Cormack
99648a82d1 Remove aufs 4.4 support
This only shipped for one rc and we are not going to ship it any more,
and CI is disabled.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-18 11:44:31 +00:00
Justin Cormack
3fbd615f9a Add an LTS 4.4 kernel
This is the same as aufs variant, but without AUFS patches. Looks like
GCP may need this, at least initially.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:47:32 +00:00
Justin Cormack
26e24a760a Reinstate Linux 4.4 LTS series support
- Add back Linux kernel 4.4.x support, only for AUFS at present.
- Add back config options that are different for 4.4 series

See #923 for discussion on whether we need to do this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-11 16:08:14 +00:00
Justin Cormack
3930ad787b Add a kernel-dev tarball with kernel headers
These headers are needed for defining kernel probes etc, tested with
eBPF. Could also be used for perf, building kernel modules etc. Saved
to the media tarball at present, may add to base image or container.

Also rationalise the paths in the headers tarball a little to match.

Will add an eBPF container using these later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-04 12:10:30 +00:00
Justin Cormack
f953ad780c Update AUFS kernel to 4.9
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-21 21:59:07 +00:00
Justin Cormack
0b7ffbd135 Use exact build context for kernel build
Saves passing too much context, less error prone and should
mean builds are faster if not clean, consistent with elsewhere.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 11:27:41 -08:00
Justin Cormack
75078a67bb Create all directories for kernel
Non AUFS kernels do not create `sbin/` and `/usr` directories as they
do not provide the AUFS directories. Just create empty directories to
avoid a warning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 10:26:00 -08:00
Justin Cormack
3b774e3657 Make AUFS optional and ship choice of kernels
This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-13 18:23:45 -08:00
Justin Cormack
212e6f8545 Kernel scanning now fixed, only needs vmlinuz64
- The scanning process was not ignoring the kernel extraversion before,
so was only sometimes picking up issues.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 09:54:02 -08:00
Justin Cormack
5fb7594217 Keep kernel headers from build
Also upload some kernel artefacts to a private repo at mobylinux/kernel on Hub for scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-08 17:32:59 -08:00
Justin Cormack
6132c69857 Keep vmlinux in build media for kernel debugging
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:45:37 +00:00
Justin Cormack
cfbdc7d067 do not remove build containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-11 16:12:40 +01:00
Justin Cormack
fa19d3259b Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
72a2e4b90f No log driver for large copies
Takes 75% of the time without logs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-14 09:05:36 +01:00
Justin Cormack
6c0a0bf2fe fix kernel build missing arch
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 22:30:26 +01:00
Justin Cormack
a0b1153d93 Rework kernel Makefile
Do not delete build container on `make clean`, as it should be fine
to reuse cached layers.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 21:49:14 +01:00
Justin Cormack
b392bf0a2e Simplify kernel output and main Dockerfile
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
69f290d28f Remove support for arm kernel build
See #449. Plan is to use upstream Alpine kernel for Arm, as
does not need vsock, hvsock or aufs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 14:23:40 +01:00
David Sheets
661c19649a kernel: add a Makefile DEBUG variable and a kernel_config for debugging
Signed-off-by: David Sheets <dsheets@docker.com>
2016-08-10 17:07:48 +01:00
Natanael Copa
66ba586cac Move built kernel to arch specific subdir
We want be able to build kernels for different archs without that they
clash with each other so we but the generated files into an $arch subdir.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-25 17:18:02 +02:00
David Sheets
25f3d06b7b kernel: add vmlinux target to make kernel debugging easier
With this patch, it is easy to get an unstripped kernel object to use with
gdb.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 15:37:04 +01:00
Natanael Copa
907d1f8c0c improve kernel build makefile
- create an empty dummy file to indicate that docker image is built
- reuse same make rule to extract the different files from docker image
- make sure that we remove empty files on failure

This makes build more robust and improves parallelism.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-13 18:03:16 +02:00
Ian Campbell
7378c6e282 Populate /lib/modules/uname -r
This is less to do with installing modules (which we generally don't expect to
use in Moby) but to populate /lib/modules/`uname -r`/modules.builtin which
turns:

    moby:~# modprobe ip_vs
    modprobe: FATAL: Module ip_vs not found in directory /lib/modules/4.4.14-moby
    moby:~# modprobe nf_nat
    modprobe: FATAL: Module nf_nat not found in directory /lib/modules/4.4.14-moby
    moby:~#

into:

    moby:~# modprobe ip_vs
    moby:~# modprobe nf_nat
    moby:~#

which reduces the amount noise in the logs, e.g. in docker.log:

time="2016-07-04T11:21:58Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.14-moby`, error: exit status 1"

A fair number of these appear in the logs.

This also stops various tools logging about /lib/modules/`uname -r` not
existing (there was one in the boot log until recently I think)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 15:44:48 +01:00
Ian Campbell
5165a6c795 Add || true to the docker rmi invocations in the clean target
Linux xargs calls the command with no arguments if it gets no inputs, which
`docker rmi` complains about. It provides -r / --no-run-if-empty to prevent
this but unfortunately this isn't supported on OSX.

Ignore errors from `docker rmi` so that `make clean` will keep going and clean
up later stuff.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-08 13:59:47 +01:00
Justin Cormack
6621512ed6 fix arm kernel build see #96
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-04-18 10:46:57 +01:00
Ian Campbell
eb221b4d4f Add VSOCK patches
These are WIP taken from git@github.com:stefanha/linux.git#vsock
(==4c9d2a6be1c6, using "cherry-pick -x") and correspond to RFC v5 of the
frontend patches posted in
http://thread.gmane.org/gmane.linux.kernel.virtualization/27455

There is no corresponding spec proposal update yet, but this set of patches
correspond (roughly) to addressing the feedback on v4 of the spec proposal
http://thread.gmane.org/gmane.comp.emulators.virtio.devel/1062.

kernel_config.arm modifications copied from x86, not tested.

Added /etc/kernel-patches/ directory to the image to be consumed by the
licensing.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-04-05 13:40:36 +01:00
Justin Cormack
cbd7af5e1d store source info for kernel and kernel patches in image for #45
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-03-18 13:53:26 +00:00
Justin Cormack
91cafe12f5 fix arm build, dtb files now checked in
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-03-11 13:48:59 +00:00
Justin Cormack
6ee7b22152 remove temporary containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-02-10 14:31:37 -08:00
Mindy Preston
554eb6ba26 fix qemu invocation for arm kernel
Download the correct dtb from debian when making a kernel and use it
when invoking qemu-system-arm.  Also, provide additional serial
parameters.
2016-01-14 14:23:57 +00:00
Justin Cormack
87dae6079b Initial arm support
Not yet working, some parts also not yet cross compiled

Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2016-01-13 15:49:38 +00:00
Justin Cormack
c9b6eb3dad clean up build containers on make clean; rename for consistency
Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2016-01-07 10:56:02 +00:00
Justin Cormack
70d8143e48 move kernel under rest of build and pull through aufs tools
Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2015-12-17 15:26:42 +00:00