Commit Graph

417 Commits

Author SHA1 Message Date
Justin Cormack
47e483fea3 Do not set ambient caps for now
Will make it explicit later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 18:39:16 +01:00
Ian Campbell
709402d780 Update to runc ef9a4b315558d31eae520725ff67383c2f79c3cb
This is compatible with containerd 8353da59c6ae7e1933aac2228df23541ef8b163f
which was picked up by d2caae4c1a.

This required jiggering with riddler output some more to update to new OCI
config.json format for capabilities.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-03 16:47:14 +01:00
Dave Tucker
6726746484 Add a runner for UEFI ISOs
Fixes #480

Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-03 14:13:37 +01:00
Rolf Neugebauer
be16cf67b4 tools: Fix make-efi script
mkfs.vfat prints output (even without the -v option) which ends up
in front of the ISO, which is cat'ed to stdout. This made the generated
ISO unbootable. Redirect stdout of mkfs.vfat to /dev/null.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-01 16:22:35 +01:00
Rolf Neugebauer
b4a3e2e0da Merge pull request #1411 from rneugeba/cleanup
more repository cleanup
2017-03-29 16:52:32 +01:00
Rolf Neugebauer
557f039747 base: Move guestfs to tools.
Guestfs is not really a base package and is only used
byt other images in the ./tools directory. Move it there.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:37:59 +01:00
Rolf Neugebauer
2c49714ef9 projects: Move AWS bits to projects until they are usable again
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:28:21 +01:00
Rolf Neugebauer
6a29d153f5 infrakit: Move the hyperkit instance plugin into the source directory
- The tools directory ideally should not contain source code
- Removes double vendoring of packagages
- Makes it easer to hook the build into the top-level Makefile

Eventually, the plugin should be moved to the infrakit repo.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-25 13:02:45 +01:00
Justin Cormack
ab5cb52823 Update hyperkit Go bindings
Fixes location of hyperkit executable with recent builds of Docker for Mac

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-24 16:31:08 +00:00
Justin Cormack
eac7ab7db0 Merge pull request #1378 from justincormack/riddler-files
Use Go code to extract rootfs from system containers
2017-03-24 13:36:51 +00:00
Justin Cormack
2be31831d8 Use Go code to extract rootfs from system containers
- this removes the use of riddler to extract the rootfs, use code
  we were using for rootfs. riddler now just geenrates the config,
  next stage is to generate this ourselves
- change the naming of the daemons so no longer include number as we
  do not guarantee ordering as they start up simultaneously

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-24 13:05:28 +00:00
Justin Cormack
6bba73ec0b Merge pull request #1357 from thebsdbox/vmdk
added the capability to produce a vmdk file through Moby
2017-03-23 11:25:58 +00:00
dan
a75989f447 Added the capability to output moby images as vmdk files for use with VMware Workstation/Fusion
Corrected naming from vmware->vmdk and fixed Makfile

Fixed mistake outputting a vhd instead of a vmdk in output.go

Build vmdk image and added to Docker Hub, corrected link in output.go

Modified directories to confirm to standard mkimage-<imgType>

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2017-03-23 11:16:40 +00:00
Justin Cormack
c76074ff66 Remove docker2tar docker image, use Docker directly from Go
Removing the left over indirect creates that use the Docker socket
and run in containers not directly.

See #1347

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-22 22:26:13 +00:00
Justin Cormack
7ca40d7801 We no longer directly convert to initrd, we use Go libraries for this
Removed unused tooling.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-21 11:47:16 +00:00
Justin Cormack
9976d6e31d Fix accidental removal of /etc/mtab from images
This was breaking some containers that use it, was a mistake.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 23:27:37 +00:00
Justin Cormack
5746d3d2bf Move test related code to the test directory
This does not yet move the `make test` options in the Makefile, will
probably move those too later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 16:14:06 +00:00
Justin Cormack
9208496d82 Output kernel command line to a file in the kernel+initrd output case
Trying to find the relevant yaml file was an issue as we now support
`--name` and it might be in a different directory, so although it is
a bit verbose outputing a whole file at least it is more consistent.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 13:43:09 +00:00
Rolf Neugebauer
dcb6f8e5f6 infrakit: Use the new HyperKit go package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-18 20:38:02 +00:00
Rolf Neugebauer
a6dab6e6ca infrakit: Update vendored packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-18 19:29:33 +00:00
Justin Cormack
73e30a757f Add a metadata handler for GCP which allows ssh login
- this needs improvements to make it more "platform native", in
  particular GCP supports multiple users and more ssh key mangement
  options.
- at present you can login as root with any platform ssh key
- add support for uts=host and ipc=host
- set the hostname from the metadata as well

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-16 18:45:44 +00:00
Justin Cormack
b0ceaf2d0b Merge pull request #1306 from riyazdf/separate-container-logs
logs: separate daemon container logs into separate files
2017-03-15 14:22:41 +00:00
Justin Cormack
d72d5c6d0d Merge pull request #1307 from ijc25/isoefi-autosize
tools: Calculate EFI ESP FAT FS size based on contents.
2017-03-15 14:22:20 +00:00
Ian Campbell
fb242fad9d tools: Calculate EFI ESP FAT FS size based on contents.
This requires switching to the dosfstools from alpine:edge since neither the
busybox nor alpine:3.5 dosfstools supports the -C option (in fact alpine:3.5
only has mkfs.fat and not mkfs.vfat).

The 511k slack seems like a lot to me, but 256k was somehow not enough.

Fixes #1304.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-15 13:30:20 +00:00
Riyaz Faizullabhoy
8d3a7ff351 logs: separate daemon container logs into separate files
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-15 12:24:10 +00:00
Justin Cormack
6d72dcee5f Merge pull request #1301 from riyazdf/readonly-check-container
Read-only check container for testing, clean up error output
2017-03-15 11:15:53 +00:00
Riyaz Faizullabhoy
7c322c0e81 read-only check container for testing, clean up error output
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-14 17:08:07 +00:00
Rolf Neugebauer
3874a37824 kernel: Update YAML files with new kernel images
Use the 4.9.x (aka latest) in all files

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-14 10:54:44 +00:00
Rolf Neugebauer
76a02c15fc tools: Add YAML file for running LTP
LTP does need disk access, so this needs to be added once it
is supported again.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Rolf Neugebauer
7362202478 tools: Add initial Linux Test Project container
LTP is compiled in on debian container and the binaries
then copied into another container, which is pushed to hub.

LTP does not compile on Alpine as it uses glibc specific
pthread extensions.

I tried to link LTP statically to not require a glibc based
base image but that failed too.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Justin Cormack
1a80011a4c Merge pull request #1296 from ijc25/q35-and-kvm
qemu: Enable KVM support and switch to Q35
2017-03-13 15:55:16 +00:00
Ian Campbell
0dbb1e6e97 qemu: Enable KVM support and switch to Q35
Passing accel=kvm:tcg causes it to try KVM first if available with a fallback
to TCG (emulated/JIT mode) if it is not available. With this the boot logs gain:

+Hypervisor detected: KVM

and also

-Booting paravirtualized kernel on bare hardware
+Booting paravirtualized kernel on KVM

Among various other noise.

If I rename my host /dev/kvm then this is reversed, although with the following
message:

  Could not access KVM kernel module: No such file or directory
  failed to initialize KVM: No such file or directory
  Back to tcg accelerator.

Q35 is a more modern emulated platform based on the ICH9 host chipset rather
than the default "pc" I440FX (Pentium Pro / Pentium II era) emulation. See
http://wiki.qemu-project.org/Features/Q35 for more info. Switching to Q35 is
not a requirement for enabling KVM but seemed like a reasonable change.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-13 15:39:08 +00:00
Riyaz Faizullabhoy
5c1662b1c1 Remove unused test bits
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-13 12:06:24 +00:00
Justin Cormack
054c3b08a3 Add qcow2 output support
- outputs compressed qcow2 image
- 1GB underlying size, may need changing

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:38:37 +00:00
Justin Cormack
5c19da6fa3 Merge pull request #1277 from justincormack/vhd
Add VHD and GCE output formats
2017-03-10 16:13:23 +00:00
Justin Cormack
718d45bdf5 Add VHD and GCE output formats
- VHD is uncompressed VHD. Currently hard coded at 1GB, which may need to change. Use `format: vhd`
- GCE is the GCE compressed tarred raw image. Use `format: gce-img` - reserving `gce` for actually
  uploading the image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:10:48 +00:00
Rolf Neugebauer
562bc65dec infrakit: Attach ISO file to VM if it exists
The file can me mounted in the VM as /dev/sr0

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:55:23 +00:00
Rolf Neugebauer
cde6fb9309 infrakit: Write ISO file from instance init contents
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:24:18 +00:00
Justin Cormack
c7d1438128 Merge pull request #1264 from tych0/hoist-kernel-cmdline
Hoist kernel cmdline
2017-03-09 22:34:30 +00:00
Tycho Andersen
a383e6bb46 hoist kernel cmdline into moby.yaml
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 14:29:40 -08:00
Justin Cormack
57355aa813 Merge pull request #1271 from rneugeba/infra-up
infrakit: Fix detection of running proceses
2017-03-09 22:13:26 +00:00
Rolf Neugebauer
0f7807e48d infrakit: Fix detection of running proceses
os.FindProcess() does not return an error when then process does not
exist. It even returns a dummy process object.

Use the go-ps package to find out if the hyperkit process is actually
running.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-09 22:08:44 +00:00
Tycho Andersen
8a3ae66f64 hoist kernel command line out of qemu docker image
we want to fiddle with this for selinux, and probably for other things too.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 11:13:49 -08:00
Justin Cormack
154c3516b2 Strip Go files to make them a bit smaller
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-09 15:30:15 +00:00
Rolf Neugebauer
ba7a162603 infrakit: Use the moby tool naming convention to select moby builds
the `moby` tool uses a prefix for the build output. Specify the prefix
(and optional path) in the `Moby` property in the JSON instead of assuming
a different path for each initrd/kernel.

Also, adopt the new name for the kernel image (vmlinuz64 -> <prefix>-bzImage).

This should make it simpler to boot up hyperkit with Moby images
build from by the moby tool.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 16:09:26 +00:00
Rolf Neugebauer
118309b03f Merge pull request #1255 from rneugeba/infra-docs
infrakit: Update docs
2017-03-07 15:58:56 +00:00
Rolf Neugebauer
198599912d infrakit: Update docs
While at it, also simplify the JSON file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 15:03:03 +00:00
Justin Cormack
7d6de92700 More cleanup
- remove more editions code
- remove unused tool pad4
- add back whale to test output

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-07 13:05:53 +00:00
Justin Cormack
159202416c Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-06 22:28:41 +00:00
Justin Cormack
e35c4c9b37 Add output types for BIOS and EFI ISOs
Note that the EFI ISO is not yet automatically sized, and the
kernel command lines are currently hard coded in the builders.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-03 18:44:47 -08:00
Rolf Neugebauer
1c3f4b4865 infrakit: Make Destroy() usable elsewhere
Don't return if the process does not exist. Instead proceed and delete
the state directory.

This allows us to use the Destroy() method elsewhere to clean up the state.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:41:18 -08:00
Rolf Neugebauer
d1ad02e079 infrakit: Move default VM directory to .infrakit/hyperkit-vms
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:13:17 -08:00
Rolf Neugebauer
95fa38f879 infrakit: removed vm-lib from arguments
Instead pass the directory where to find initrd/vmlinuz via the
"Moby" entry in the instance JSON config.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:52:37 -08:00
Rolf Neugebauer
af2ee9e023 infrakit: Use template based instance configuratio
This change now allows setting the Disk size, memory, and number of CPUs
for an instance via the JSON config file.

We now also write the hyperkit command line to the VM directory.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:09:30 -08:00
Rolf Neugebauer
175daffe71 tools: Add initial HyperKit/Moby InfraKit instance plugin
This is the first cut of a InfraKit instance plugin which
primarily allows one to manage Moby instances (though, other Linux's
may work too).

It's very rough on the edges and needs work and update to handle
templates and better configuration etc.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-02-28 18:51:31 -08:00
Robb Kistler
b2b585ac99 Allow docker engine to be pulled from s3 bucket
Use aws cli to pull docker engine binaries from s3 bucket.  Conditional and
happens if DOCKER_BIN_URL begins with 's3://'

Example usage:
```
  export AWS_ACCESS_KEY_ID=key
  export AWS_SECRET_ACCESS_KEY=secret
  make DOCKER_BIN_URL=s3://path-to-engine-binaries
```
Signed-off-by: Robb Kistler <robb.kistler@docker.com>

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-23 20:32:00 -08:00
Riyaz Faizullabhoy
447ee3b6c0 Merge pull request #1202 from justincormack/go-up
Update to Go 1.8
2017-02-17 09:19:53 -08:00
Riyaz Faizullabhoy
c643f19434 Merge pull request #1197 from justincormack/testup
Update kernel config tests
2017-02-17 09:19:44 -08:00
Justin Cormack
fe64819020 Update to Go 1.8
This includes most things except the containerd and runc builds
which I will do next time I update them.

Note that all golang:alpine images now for 1.8 are ALpine 3.5 based.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 16:31:19 +00:00
Justin Cormack
74111017f6 Merge pull request #1201 from justincormack/newtools
Add new minimal init prototype and new tools
2017-02-17 15:56:42 +00:00
Justin Cormack
d29be01b0d Add new minimal init prototype and new tools
This is part of the new moby tool work to get a minimal system
up that can be expanded.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 15:31:00 +00:00
Justin Cormack
844c1c39df Update kernel config tests
- `CONFIG_BUG_ON_DATA_CORRUPTION` is a 4.10 kernel only flag, so gate this.
- some config was duplicated on gated and normal config
- some 4.4 issues with the new config

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 14:52:32 +00:00
Riyaz Faizullabhoy
4daa3b5c88 Bump kernel config test and revert IA32_EMULATION
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-13 11:38:14 -08:00
Riyaz Faizullabhoy
25b2d47f1d Add page_poison=1 to boot args
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-13 10:16:46 -08:00
Justin Cormack
66c92081fc Update kernel config check for suggested values
This should be done with #1175

Config not updated as may still need changes

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-13 10:14:53 -08:00
Justin Cormack
9fe9e95349 Replace mkinitrd.sh with a tool to convert base image to initrd
Better split between content and tools.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-01 22:32:34 +00:00
Riyaz Faizullabhoy
d66bc2d501 Split base/ into base/ and tools/
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-31 09:52:30 -08:00