Commit Graph

60 Commits

Author SHA1 Message Date
Justin Cormack
3358f57891 Reinstate Linux 4.4 LTS series support
- Add back Linux kernel 4.4.x support, only for AUFS at present.
- Add back config options that are different for 4.4 series

See #923 for discussion on whether we need to do this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-11 16:08:14 +00:00
Justin Cormack
285266dd93 Add datakit CI control file
This controls build with AUFS=1 so we can build in parallel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-06 15:10:05 +00:00
Justin Cormack
ee7f24defd Add a development eBPF container build
See `docs/ebpf.md` for how to use. This is built by CI or you can build
manually if you customise the kernel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-05 11:59:28 +00:00
Justin Cormack
e955457195 Add a kernel-dev tarball with kernel headers
These headers are needed for defining kernel probes etc, tested with
eBPF. Could also be used for perf, building kernel modules etc. Saved
to the media tarball at present, may add to base image or container.

Also rationalise the paths in the headers tarball a little to match.

Will add an eBPF container using these later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-04 12:10:30 +00:00
Justin Cormack
bc3823c387 make media pushing wrong image
The base image for scanning the initrd was being pushed over
the the media image for pinata.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 13:05:15 -08:00
Justin Cormack
7e8115ce89 Add new Makefile targets for the CI
This builds both the standard and AUFS versions of Moby,
and does `make clean` carefully.

- `make ci` for branches (pushes artifacts)
- `make pr` for PRs

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-14 16:05:54 -08:00
Justin Cormack
be8a1aaeb5 Merge pull request #880 from justincormack/kernel-4.8-4.9
Make AUFS optional and ship choice of kernels
2016-12-14 15:33:39 -08:00
Justin Cormack
5567606fd0 Make AUFS optional and ship choice of kernels
This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-13 18:23:45 -08:00
Justin Cormack
f1f2f1b222 Build a GCE image
- this is a raw 1GB filesystem image with syslinux for booting
- built with libguestfs so does not need any privileges
- need not be built on GCE
- there is a target that runs the image in qemu for local tests

Does not yet have a script to upload the image to cloud storage or create image from it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 19:17:24 -08:00
Justin Cormack
2f558885f0 Kernel scanning now fixed, only needs vmlinuz64
- The scanning process was not ignoring the kernel extraversion before,
so was only sometimes picking up issues.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 09:54:02 -08:00
Justin Cormack
f3e3a4029b Containerize test suite
- run test suite under containerd
- in future this should be converted to Go not shell see #860
- test suite is now in its own initrd, can be run on any platform not just qemu

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-09 17:13:16 -08:00
Justin Cormack
15522201ea Keep kernel headers from build
Also upload some kernel artefacts to a private repo at mobylinux/kernel on Hub for scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-08 17:32:59 -08:00
Justin Cormack
2897b808bb Push a copy of the Moby base image to mobylinux/mobylinux
This will be used for ongoing security scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-07 12:10:39 -08:00
Justin Cormack
a1c2442270 Remove more references to EXPERIMENTAL
can use MEDIA_PREFIX to set similar rpefix

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-24 15:02:41 +00:00
Justin Cormack
040d6faeab upgrade Docker to 1.13.0-rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 16:48:26 +00:00
Justin Cormack
6fa0e3a2d4 Also pull the efi file when doing make get
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 15:41:52 +00:00
Justin Cormack
7d22f18e7a Add a script to get current binaries
If these have been pushed you can fetch them rather than doing a build. For
use in CI.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 15:22:04 +00:00
Justin Cormack
3505253d84 Merge pull request #692 from justincormack/vmlinux
Keep vmlinux in build media for kernel debugging
2016-11-07 15:55:31 +00:00
Justin Cormack
5256e9e4e7 Keep vmlinux in build media for kernel debugging
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:45:37 +00:00
Justin Cormack
255cebca11 Do not include non-efi ISO in media
We are not using this for editions.

We should be able to make a dual boot ISO, or you can create from the other
artefacts anyway.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-07 14:37:24 +00:00
Justin Cormack
f9563cff68 improve hyperkit build to do sudo in the right place
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-31 16:04:25 +00:00
Justin Cormack
bd839a3cf8 Add hyperkit build
Useful for local performance tests

Fix #443

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-31 15:34:56 +00:00
Justin Cormack
b4bee2840b do not push a latest tag on make media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-27 14:53:29 +01:00
Justin Cormack
47124e8282 do not remove build containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-11 16:12:40 +01:00
Justin Cormack
652d5eddf3 Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
9ade1be504 Run test in foreground
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-26 12:03:56 +01:00
Justin Cormack
26159df061 fix media prefix
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-14 12:28:25 +01:00
Justin Cormack
1664cebcf9 allow overriding the media prefix, so can make custom modifications eg for custom Docker versions
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:37:24 +01:00
Justin Cormack
8acb1211c6 add experimental- prefix for experimental Docker versions in media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
b03d08ef4d do not need to remove images in make clean
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
9f7e9ca168 Add mobylinux.efi to media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
ef2812a70a Add a media target to push final artefacts to tagged by commit id
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
3f665f5e2a we never use uncompressed initrd, so always compress
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
59ad769b00 for now, make sure rebuild on make test
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
a1edc6cec2 fix Makefile dependencies
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
3a2a755f58 do not force rebuild from top level makefile
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
4a4e755c91 remove broken arm functionality - will rework
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
6268a1ace0 Merge pull request #468 from justincormack/alpine-iso
Use alpine to build bios
2016-09-06 23:57:49 +01:00
Justin Cormack
5fe4445c88 Faster build via less context sent
See #470. Still more to do.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-06 09:45:37 +01:00
Justin Cormack
feb141d324 fix iso build
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-06 00:11:48 +01:00
Justin Cormack
5333a5c796 Remove xhyve
See #443

We now use qemu for testing, having a copy of xhyve in the tree is no
longer useful. Should add documentation for how to run in hyperkit instead.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 14:30:17 +01:00
Justin Cormack
3899ca9cb7 hide any messages from tee about short writes when pipe aborted
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-09 17:40:32 +01:00
Justin Cormack
b9532e2a89 make sure test.log exists, and append, otherwise racy
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-29 16:33:01 +01:00
Justin Cormack
626ad90e41 Improve make test output
Incrementally output log, and exit as soon as tests pass
or fail. Will still timeout on a kernel panic or suchlike
though...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-18 16:05:01 +01:00
Justin Cormack
07052d1875 make test should not need a tty
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-18 11:08:26 +01:00
Justin Cormack
ede5a0ea6f Initial test hooks
Beginning of a proper test suite, using qemu. Test just runs docker
for now, will add further integration tests.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 15:44:04 +01:00
Justin Cormack
8777d775ed Make the qemu containers interactive so the console works
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-18 21:47:31 +01:00
Ian Campbell
b8a7032161 Add || true to the docker rmi invocations in the clean target
Linux xargs calls the command with no arguments if it gets no inputs, which
`docker rmi` complains about. It provides -r / --no-run-if-empty to prevent
this but unfortunately this isn't supported on OSX.

Ignore errors from `docker rmi` so that `make clean` will keep going and clean
up later stuff.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-08 13:59:47 +01:00
Justin Cormack
3c97c540be remove more kexec code
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-04-06 10:28:15 +01:00
Justin Cormack
e7d2e0c4cd add a qemu iso target
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-02-19 16:11:23 +00:00