Commit Graph

97 Commits

Author SHA1 Message Date
Justin Cormack
8427af09c5 Make the hyperkit script get the kernel command line if just passed a name
This is all slightly annoying, maybe we should make a file for the CLI for
hyperkit, but this is better and fixes a bug that the test CLI was coming from moby,
and is easier to use with custom builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-15 15:01:47 +00:00
Justin Cormack
c7d1438128 Merge pull request #1264 from tych0/hoist-kernel-cmdline
Hoist kernel cmdline
2017-03-09 22:34:30 +00:00
Tycho Andersen
a383e6bb46 hoist kernel cmdline into moby.yaml
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 14:29:40 -08:00
Justin Cormack
154c3516b2 Strip Go files to make them a bit smaller
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-09 15:30:15 +00:00
Justin Cormack
5ab5b32413 Cleanup after merge
- update README
- script to start qemu
- fixes to hyperkit

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-07 12:23:53 +00:00
Rolf Neugebauer
9e2d7df258 build: Make some top-level targets .PHONY
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 10:54:10 +00:00
Justin Cormack
159202416c Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-06 22:28:41 +00:00
French Ben
9b23adbcea Rebased cloud clean
Signed-off-by: French Ben <frenchben@docker.com>
2017-03-02 13:50:18 -08:00
French Ben
e3d1349d3d Removed extra cloud make
Signed-off-by: French Ben <frenchben@docker.com>
2017-03-02 13:50:01 -08:00
Riyaz Faizullabhoy
4502f3b8a8 Fix old slirp references to vpnkit
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-01 13:54:08 -08:00
Justin Cormack
4ffd42e4f0 slirp binary renamed to vpnkit
Recently com.docker.slirp was renamed.

Please update Docker for Mac before using this script...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-28 17:52:38 -08:00
Robb Kistler
2fa112b1b7 Delegated make targets should be phony targets
Make won't evaluate actual dependencies of targets called with
`$(MAKE) -C` if the target file already exists.  Marking the
top-level targets `.PHONY` will enable incremental builds.

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-28 10:18:50 -08:00
Riyaz Faizullabhoy
945f826c8b Fix alpine/mobylinux-efi.iso target
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-27 16:18:37 -08:00
Justin Cormack
7fdb89d22f Add a CI tag Makefile target
The CI will set `CI_TAG` to the tag name, which will be used for the
uploaded images instead of the git commit ID.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-21 10:36:25 +00:00
Justin Cormack
f4a112dc21 Change kernel upload for new tooling
The mobylinux/kernel image now has the bzImage (no longer named vmlinuz64)
and a tarball of the files needed for the initrd, ie modules etc.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-20 15:54:02 +00:00
Justin Cormack
0b2251212a Generate media Dockerfile
It is trivial no need to have committed in repo.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-13 14:01:32 +00:00
Justin Cormack
53399b98be Move containers directory to alpine directory
Not using this with new tool, so leave for editions.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-13 11:10:13 +00:00
Justin Cormack
65e2c49ca2 Merge pull request #1156 from justincormack/top-level-dockerfiles
Simpler media Dockerfiles
2017-02-06 16:23:23 +00:00
Justin Cormack
38ef228b7b Simpler media Dockerfiles
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-06 15:50:40 +00:00
Justin Cormack
4aa0444c92 Move hyperkit script to scripts directory
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-06 15:28:32 +00:00
Justin Cormack
3f81edee37 Move kernel directory to top level
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-04 09:17:16 +01:00
Justin Cormack
aff2b5f514 Use hyperkit and slirp from Docker for Mac install if available
Save some downloading as the user probably has these installed
already; still fall back if not found to downloading from CI.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-02 08:48:50 +00:00
Robb Kistler
57ee2eae95 OS X CI test without docker installed
OS X CI endpoints won't need to do a build, the artifacts can be pulled
from mobylinux/media.  This requires docker, which may not be available.

- 'get-regextract' make target pulls media without Docker using the
  regextract utility, which it will fetch from CI if needed.
- Add initrd-test.img to mobylinux/media image

Addresses #1046

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-01 19:47:53 -08:00
Justin Cormack
7b7e672270 Move test directory to top level
This is temporary, it should be under `containers/` just as soon
as we have a manifest setup.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-01 23:34:21 +00:00
Justin Cormack
2db23f8d18 Move containers directory to top level
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-01 22:56:55 +00:00
Justin Cormack
3e020abdb2 Use bin dir and clean up hyperkit test
- make a `bin/` directory
- make hyperkit-test pass, as it writes to pty so redirect was not working

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-01 14:15:52 +00:00
Riyaz Faizullabhoy
56899b318d reformat tags before digests
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-31 09:33:06 -08:00
Justin Cormack
0fca0245c4 Cleanup BIOS ISO builds
- do not require specific filenames for input
- change EFI build to use `docker run`

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-30 15:18:06 +00:00
Justin Cormack
007a42ad41 Use docker run to run tests and interactive containers
Using docker build is slower and needs lots of Dockerfiles,
while a single image with a careful script can accept any type
of image, either with `-v` to share into `/tmp` for interactive
use (where you need the input and a tty, or by adding a tarball
for cases where there is no login such as running tests, so you
can still use a remote daemon in these cases.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-24 23:41:06 +00:00
Robb Kistler
5caa1838d9 Don't fail hyperkit-test after clean
Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-01-23 11:12:42 -08:00
Justin Cormack
5f29f59618 Merge pull request #1039 from RobbKistler/hyperkit-test
Improve hyperkit/osx testing
2017-01-18 18:18:36 +00:00
Robb Kistler
29e3d74c2a Improve hyperkit/osx testing
Address #954:
- Pull hyperkit and vpnkit binaries from CI
- Use vpnkit instead of vmnet
- Add hyperkit-test test target

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-01-18 09:56:24 -08:00
Justin Cormack
e579b09a5c Remove aufs4.4 reference from make media
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-18 16:28:43 +00:00
Justin Cormack
0a1ffd9203 make get compatibility with older builds
Old builds did not have kernel headers or kernel dev headers, so do not fail

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 14:35:44 +00:00
Justin Cormack
262fb03750 Merge pull request #987 from justincormack/reinstate-4.4
Reinstate 4.4 support
2017-01-12 11:18:14 +00:00
Justin Cormack
3fbd615f9a Add an LTS 4.4 kernel
This is the same as aufs variant, but without AUFS patches. Looks like
GCP may need this, at least initially.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:47:32 +00:00
Justin Cormack
a4c545b87c Add prefix for make get
Allows `make AUFS=1 get` to pull the aufs build binaries.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-12 09:14:01 +00:00
Justin Cormack
26e24a760a Reinstate Linux 4.4 LTS series support
- Add back Linux kernel 4.4.x support, only for AUFS at present.
- Add back config options that are different for 4.4 series

See #923 for discussion on whether we need to do this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-11 16:08:14 +00:00
Justin Cormack
98a98bcd5c Add datakit CI control file
This controls build with AUFS=1 so we can build in parallel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-06 15:10:05 +00:00
Justin Cormack
f6c2143197 Add a development eBPF container build
See `docs/ebpf.md` for how to use. This is built by CI or you can build
manually if you customise the kernel.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-05 11:59:28 +00:00
Justin Cormack
3930ad787b Add a kernel-dev tarball with kernel headers
These headers are needed for defining kernel probes etc, tested with
eBPF. Could also be used for perf, building kernel modules etc. Saved
to the media tarball at present, may add to base image or container.

Also rationalise the paths in the headers tarball a little to match.

Will add an eBPF container using these later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-04 12:10:30 +00:00
Justin Cormack
0ba24fa482 make media pushing wrong image
The base image for scanning the initrd was being pushed over
the the media image for pinata.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-15 13:05:15 -08:00
Justin Cormack
9f64a9df00 Add new Makefile targets for the CI
This builds both the standard and AUFS versions of Moby,
and does `make clean` carefully.

- `make ci` for branches (pushes artifacts)
- `make pr` for PRs

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-14 16:05:54 -08:00
Justin Cormack
75b0814221 Merge pull request #880 from justincormack/kernel-4.8-4.9
Make AUFS optional and ship choice of kernels
2016-12-14 15:33:39 -08:00
Justin Cormack
3b774e3657 Make AUFS optional and ship choice of kernels
This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-13 18:23:45 -08:00
Justin Cormack
9245dd656f Build a GCE image
- this is a raw 1GB filesystem image with syslinux for booting
- built with libguestfs so does not need any privileges
- need not be built on GCE
- there is a target that runs the image in qemu for local tests

Does not yet have a script to upload the image to cloud storage or create image from it.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 19:17:24 -08:00
Justin Cormack
212e6f8545 Kernel scanning now fixed, only needs vmlinuz64
- The scanning process was not ignoring the kernel extraversion before,
so was only sometimes picking up issues.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-12 09:54:02 -08:00
Justin Cormack
a7f57526a5 Containerize test suite
- run test suite under containerd
- in future this should be converted to Go not shell see #860
- test suite is now in its own initrd, can be run on any platform not just qemu

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-09 17:13:16 -08:00
Justin Cormack
5fb7594217 Keep kernel headers from build
Also upload some kernel artefacts to a private repo at mobylinux/kernel on Hub for scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-08 17:32:59 -08:00
Justin Cormack
ba50adfc36 Push a copy of the Moby base image to mobylinux/mobylinux
This will be used for ongoing security scanning.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-07 12:10:39 -08:00