Commit Graph

32 Commits

Author SHA1 Message Date
Justin Cormack
c73d6f1721 Add an LTS 4.4 kernel
This is the same as aufs variant, but without AUFS patches. Looks like
GCP may need this, at least initially.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:47:32 +00:00
Justin Cormack
3358f57891 Reinstate Linux 4.4 LTS series support
- Add back Linux kernel 4.4.x support, only for AUFS at present.
- Add back config options that are different for 4.4 series

See #923 for discussion on whether we need to do this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-11 16:08:14 +00:00
Justin Cormack
e955457195 Add a kernel-dev tarball with kernel headers
These headers are needed for defining kernel probes etc, tested with
eBPF. Could also be used for perf, building kernel modules etc. Saved
to the media tarball at present, may add to base image or container.

Also rationalise the paths in the headers tarball a little to match.

Will add an eBPF container using these later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-04 12:10:30 +00:00
Justin Cormack
90c375a044 Update AUFS kernel to 4.9
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-21 21:59:07 +00:00
Justin Cormack
a7f6bba295 Use exact build context for kernel build
Saves passing too much context, less error prone and should
mean builds are faster if not clean, consistent with elsewhere.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 11:27:41 -08:00
Justin Cormack
64ae794897 Create all directories for kernel
Non AUFS kernels do not create `sbin/` and `/usr` directories as they
do not provide the AUFS directories. Just create empty directories to
avoid a warning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 10:26:00 -08:00
Justin Cormack
5567606fd0 Make AUFS optional and ship choice of kernels
This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-13 18:23:45 -08:00
Justin Cormack
2f558885f0 Kernel scanning now fixed, only needs vmlinuz64
- The scanning process was not ignoring the kernel extraversion before,
so was only sometimes picking up issues.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 09:54:02 -08:00
Justin Cormack
15522201ea Keep kernel headers from build
Also upload some kernel artefacts to a private repo at mobylinux/kernel on Hub for scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-08 17:32:59 -08:00
Justin Cormack
5256e9e4e7 Keep vmlinux in build media for kernel debugging
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:45:37 +00:00
Justin Cormack
47124e8282 do not remove build containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-11 16:12:40 +01:00
Justin Cormack
652d5eddf3 Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
c522ca1d31 No log driver for large copies
Takes 75% of the time without logs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-14 09:05:36 +01:00
Justin Cormack
a6fc00e41b fix kernel build missing arch
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 22:30:26 +01:00
Justin Cormack
99dcbdf322 Rework kernel Makefile
Do not delete build container on `make clean`, as it should be fine
to reuse cached layers.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 21:49:14 +01:00
Justin Cormack
c51a0b831e Simplify kernel output and main Dockerfile
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
c1e2418a16 Remove support for arm kernel build
See #449. Plan is to use upstream Alpine kernel for Arm, as
does not need vsock, hvsock or aufs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 14:23:40 +01:00
David Sheets
9e5c86f006 kernel: add a Makefile DEBUG variable and a kernel_config for debugging
Signed-off-by: David Sheets <dsheets@docker.com>
2016-08-10 17:07:48 +01:00
Natanael Copa
0bb80957bc Move built kernel to arch specific subdir
We want be able to build kernels for different archs without that they
clash with each other so we but the generated files into an $arch subdir.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-25 17:18:02 +02:00
David Sheets
cdac4fde34 kernel: add vmlinux target to make kernel debugging easier
With this patch, it is easy to get an unstripped kernel object to use with
gdb.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 15:37:04 +01:00
Natanael Copa
47545dce55 improve kernel build makefile
- create an empty dummy file to indicate that docker image is built
- reuse same make rule to extract the different files from docker image
- make sure that we remove empty files on failure

This makes build more robust and improves parallelism.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-13 18:03:16 +02:00
Ian Campbell
209dd754bc Populate /lib/modules/uname -r
This is less to do with installing modules (which we generally don't expect to
use in Moby) but to populate /lib/modules/`uname -r`/modules.builtin which
turns:

    moby:~# modprobe ip_vs
    modprobe: FATAL: Module ip_vs not found in directory /lib/modules/4.4.14-moby
    moby:~# modprobe nf_nat
    modprobe: FATAL: Module nf_nat not found in directory /lib/modules/4.4.14-moby
    moby:~#

into:

    moby:~# modprobe ip_vs
    moby:~# modprobe nf_nat
    moby:~#

which reduces the amount noise in the logs, e.g. in docker.log:

time="2016-07-04T11:21:58Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.14-moby`, error: exit status 1"

A fair number of these appear in the logs.

This also stops various tools logging about /lib/modules/`uname -r` not
existing (there was one in the boot log until recently I think)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 15:44:48 +01:00
Ian Campbell
b8a7032161 Add || true to the docker rmi invocations in the clean target
Linux xargs calls the command with no arguments if it gets no inputs, which
`docker rmi` complains about. It provides -r / --no-run-if-empty to prevent
this but unfortunately this isn't supported on OSX.

Ignore errors from `docker rmi` so that `make clean` will keep going and clean
up later stuff.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-08 13:59:47 +01:00
Justin Cormack
4989d24c6b fix arm kernel build see #96
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-04-18 10:46:57 +01:00
Ian Campbell
a887fb110c Add VSOCK patches
These are WIP taken from git@github.com:stefanha/linux.git#vsock
(==4c9d2a6be1c6, using "cherry-pick -x") and correspond to RFC v5 of the
frontend patches posted in
http://thread.gmane.org/gmane.linux.kernel.virtualization/27455

There is no corresponding spec proposal update yet, but this set of patches
correspond (roughly) to addressing the feedback on v4 of the spec proposal
http://thread.gmane.org/gmane.comp.emulators.virtio.devel/1062.

kernel_config.arm modifications copied from x86, not tested.

Added /etc/kernel-patches/ directory to the image to be consumed by the
licensing.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-04-05 13:40:36 +01:00
Justin Cormack
af7777c832 store source info for kernel and kernel patches in image for #45
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-03-18 13:53:26 +00:00
Justin Cormack
373375569a fix arm build, dtb files now checked in
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-03-11 13:48:59 +00:00
Justin Cormack
4564a44b8d remove temporary containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-02-10 14:31:37 -08:00
Mindy Preston
55bfabc8f2 fix qemu invocation for arm kernel
Download the correct dtb from debian when making a kernel and use it
when invoking qemu-system-arm.  Also, provide additional serial
parameters.
2016-01-14 14:23:57 +00:00
Justin Cormack
658ac6145f Initial arm support
Not yet working, some parts also not yet cross compiled

Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2016-01-13 15:49:38 +00:00
Justin Cormack
f4b04272a7 clean up build containers on make clean; rename for consistency
Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2016-01-07 10:56:02 +00:00
Justin Cormack
fca32a3b72 move kernel under rest of build and pull through aufs tools
Signed-off-by: Justin Cormack <justin.cormack@unikernel.com>
2015-12-17 15:26:42 +00:00