Commit Graph

115 Commits

Author SHA1 Message Date
Justin Cormack
e579b09a5c Remove aufs4.4 reference from make media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-18 16:28:43 +00:00
Justin Cormack
0a1ffd9203 make get compatibility with older builds
Old builds did not have kernel headers or kernel dev headers, so do not fail

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 14:35:44 +00:00
Justin Cormack
262fb03750 Merge pull request #987 from justincormack/reinstate-4.4
Reinstate 4.4 support
2017-01-12 11:18:14 +00:00
Justin Cormack
3fbd615f9a Add an LTS 4.4 kernel
This is the same as aufs variant, but without AUFS patches. Looks like
GCP may need this, at least initially.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:47:32 +00:00
Justin Cormack
a4c545b87c Add prefix for make get
Allows `make AUFS=1 get` to pull the aufs build binaries.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:14:01 +00:00
Justin Cormack
26e24a760a Reinstate Linux 4.4 LTS series support
- Add back Linux kernel 4.4.x support, only for AUFS at present.
- Add back config options that are different for 4.4 series

See #923 for discussion on whether we need to do this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-11 16:08:14 +00:00
Justin Cormack
98a98bcd5c Add datakit CI control file
This controls build with AUFS=1 so we can build in parallel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-06 15:10:05 +00:00
Justin Cormack
f6c2143197 Add a development eBPF container build
See `docs/ebpf.md` for how to use. This is built by CI or you can build
manually if you customise the kernel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-05 11:59:28 +00:00
Justin Cormack
3930ad787b Add a kernel-dev tarball with kernel headers
These headers are needed for defining kernel probes etc, tested with
eBPF. Could also be used for perf, building kernel modules etc. Saved
to the media tarball at present, may add to base image or container.

Also rationalise the paths in the headers tarball a little to match.

Will add an eBPF container using these later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-04 12:10:30 +00:00
Justin Cormack
0ba24fa482 make media pushing wrong image
The base image for scanning the initrd was being pushed over
the the media image for pinata.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 13:05:15 -08:00
Justin Cormack
9f64a9df00 Add new Makefile targets for the CI
This builds both the standard and AUFS versions of Moby,
and does `make clean` carefully.

- `make ci` for branches (pushes artifacts)
- `make pr` for PRs

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-14 16:05:54 -08:00
Justin Cormack
75b0814221 Merge pull request #880 from justincormack/kernel-4.8-4.9
Make AUFS optional and ship choice of kernels
2016-12-14 15:33:39 -08:00
Justin Cormack
3b774e3657 Make AUFS optional and ship choice of kernels
This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-13 18:23:45 -08:00
Justin Cormack
9245dd656f Build a GCE image
- this is a raw 1GB filesystem image with syslinux for booting
- built with libguestfs so does not need any privileges
- need not be built on GCE
- there is a target that runs the image in qemu for local tests

Does not yet have a script to upload the image to cloud storage or create image from it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 19:17:24 -08:00
Justin Cormack
212e6f8545 Kernel scanning now fixed, only needs vmlinuz64
- The scanning process was not ignoring the kernel extraversion before,
so was only sometimes picking up issues.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 09:54:02 -08:00
Justin Cormack
a7f57526a5 Containerize test suite
- run test suite under containerd
- in future this should be converted to Go not shell see #860
- test suite is now in its own initrd, can be run on any platform not just qemu

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-09 17:13:16 -08:00
Justin Cormack
5fb7594217 Keep kernel headers from build
Also upload some kernel artefacts to a private repo at mobylinux/kernel on Hub for scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-08 17:32:59 -08:00
Justin Cormack
ba50adfc36 Push a copy of the Moby base image to mobylinux/mobylinux
This will be used for ongoing security scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-07 12:10:39 -08:00
Justin Cormack
08de1b1477 Remove more references to EXPERIMENTAL
can use MEDIA_PREFIX to set similar rpefix

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-24 15:02:41 +00:00
Justin Cormack
d45bf3d89b upgrade Docker to 1.13.0-rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 16:48:26 +00:00
Justin Cormack
0d2e507be2 Also pull the efi file when doing make get
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 15:41:52 +00:00
Justin Cormack
d0f7337014 Add a script to get current binaries
If these have been pushed you can fetch them rather than doing a build. For
use in CI.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 15:22:04 +00:00
Justin Cormack
f8a66ec47e Merge pull request #692 from justincormack/vmlinux
Keep vmlinux in build media for kernel debugging
2016-11-07 15:55:31 +00:00
Justin Cormack
6132c69857 Keep vmlinux in build media for kernel debugging
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:45:37 +00:00
Justin Cormack
020f0be85e Do not include non-efi ISO in media
We are not using this for editions.

We should be able to make a dual boot ISO, or you can create from the other
artefacts anyway.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:37:24 +00:00
Justin Cormack
473d0e37c7 improve hyperkit build to do sudo in the right place
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-31 16:04:25 +00:00
Justin Cormack
b6c6ec8905 Add hyperkit build
Useful for local performance tests

Fix #443

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-31 15:34:56 +00:00
Justin Cormack
052116d7b9 do not push a latest tag on make media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-27 14:53:29 +01:00
Justin Cormack
cfbdc7d067 do not remove build containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-11 16:12:40 +01:00
Justin Cormack
fa19d3259b Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
2f99a71e4f Run test in foreground
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-26 12:03:56 +01:00
Justin Cormack
36524eef6c fix media prefix
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-14 12:28:25 +01:00
Justin Cormack
a12411b794 allow overriding the media prefix, so can make custom modifications eg for custom Docker versions
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:37:24 +01:00
Justin Cormack
9b2e1367f9 add experimental- prefix for experimental Docker versions in media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
145dead110 do not need to remove images in make clean
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
4ac419e663 Add mobylinux.efi to media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
6fdcb8716a Add a media target to push final artefacts to tagged by commit id
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
7d08c3f9c1 we never use uncompressed initrd, so always compress
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
5c79c1caf8 for now, make sure rebuild on make test
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
c0fe2b959d fix Makefile dependencies
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
18912fa3b5 do not force rebuild from top level makefile
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
c498f7ac23 remove broken arm functionality - will rework
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
ce4561b1cb Merge pull request #468 from justincormack/alpine-iso
Use alpine to build bios
2016-09-06 23:57:49 +01:00
Justin Cormack
407b4419f5 Faster build via less context sent
See #470. Still more to do.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-06 09:45:37 +01:00
Justin Cormack
75d5b2b1eb fix iso build
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-06 00:11:48 +01:00
Justin Cormack
210aef2a50 Remove xhyve
See #443

We now use qemu for testing, having a copy of xhyve in the tree is no
longer useful. Should add documentation for how to run in hyperkit instead.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 14:30:17 +01:00
Justin Cormack
dde7f73985 hide any messages from tee about short writes when pipe aborted
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-09 17:40:32 +01:00
Justin Cormack
da7448e6ef make sure test.log exists, and append, otherwise racy
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-29 16:33:01 +01:00
Justin Cormack
40d6e10924 Improve make test output
Incrementally output log, and exit as soon as tests pass
or fail. Will still timeout on a kernel panic or suchlike
though...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-18 16:05:01 +01:00
Justin Cormack
6e8b504efd make test should not need a tty
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-18 11:08:26 +01:00