Commit Graph

58 Commits

Author SHA1 Message Date
Justin Cormack
8b677a54d9 Merge pull request #1378 from justincormack/riddler-files
Use Go code to extract rootfs from system containers
2017-03-24 13:36:51 +00:00
Justin Cormack
d8fc863e98 Use Go code to extract rootfs from system containers
- this removes the use of riddler to extract the rootfs, use code
  we were using for rootfs. riddler now just geenrates the config,
  next stage is to generate this ourselves
- change the naming of the daemons so no longer include number as we
  do not guarantee ordering as they start up simultaneously

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-24 13:05:28 +00:00
Justin Cormack
f756188a21 Merge pull request #1357 from thebsdbox/vmdk
added the capability to produce a vmdk file through Moby
2017-03-23 11:25:58 +00:00
dan
8e6b2cfd46 Added the capability to output moby images as vmdk files for use with VMware Workstation/Fusion
Corrected naming from vmware->vmdk and fixed Makfile

Fixed mistake outputting a vhd instead of a vmdk in output.go

Build vmdk image and added to Docker Hub, corrected link in output.go

Modified directories to confirm to standard mkimage-<imgType>

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2017-03-23 11:16:40 +00:00
Justin Cormack
40d007aac8 Remove docker2tar docker image, use Docker directly from Go
Removing the left over indirect creates that use the Docker socket
and run in containers not directly.

See #1347

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-22 22:26:13 +00:00
Justin Cormack
3092d2ca03 We no longer directly convert to initrd, we use Go libraries for this
Removed unused tooling.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-21 11:47:16 +00:00
Justin Cormack
77285b1bef Fix accidental removal of /etc/mtab from images
This was breaking some containers that use it, was a mistake.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 23:27:37 +00:00
Justin Cormack
d881c6ff75 Move test related code to the test directory
This does not yet move the `make test` options in the Makefile, will
probably move those too later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 16:14:06 +00:00
Justin Cormack
ea4462357c Output kernel command line to a file in the kernel+initrd output case
Trying to find the relevant yaml file was an issue as we now support
`--name` and it might be in a different directory, so although it is
a bit verbose outputing a whole file at least it is more consistent.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-20 13:43:09 +00:00
Rolf Neugebauer
1b6fa97c44 infrakit: Use the new HyperKit go package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-18 20:38:02 +00:00
Rolf Neugebauer
a39e543e42 infrakit: Update vendored packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-18 19:29:33 +00:00
Justin Cormack
a66a537164 Add a metadata handler for GCP which allows ssh login
- this needs improvements to make it more "platform native", in
  particular GCP supports multiple users and more ssh key mangement
  options.
- at present you can login as root with any platform ssh key
- add support for uts=host and ipc=host
- set the hostname from the metadata as well

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-16 18:45:44 +00:00
Justin Cormack
dc02e9588b Merge pull request #1306 from riyazdf/separate-container-logs
logs: separate daemon container logs into separate files
2017-03-15 14:22:41 +00:00
Justin Cormack
d8d6a851cd Merge pull request #1307 from ijc25/isoefi-autosize
tools: Calculate EFI ESP FAT FS size based on contents.
2017-03-15 14:22:20 +00:00
Ian Campbell
f16ee76f57 tools: Calculate EFI ESP FAT FS size based on contents.
This requires switching to the dosfstools from alpine:edge since neither the
busybox nor alpine:3.5 dosfstools supports the -C option (in fact alpine:3.5
only has mkfs.fat and not mkfs.vfat).

The 511k slack seems like a lot to me, but 256k was somehow not enough.

Fixes #1304.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-15 13:30:20 +00:00
Riyaz Faizullabhoy
f72a5693b5 logs: separate daemon container logs into separate files
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-15 12:24:10 +00:00
Justin Cormack
73abdeffc1 Merge pull request #1301 from riyazdf/readonly-check-container
Read-only check container for testing, clean up error output
2017-03-15 11:15:53 +00:00
Riyaz Faizullabhoy
7af8fefd40 read-only check container for testing, clean up error output
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-14 17:08:07 +00:00
Rolf Neugebauer
ba7f6c7a84 kernel: Update YAML files with new kernel images
Use the 4.9.x (aka latest) in all files

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-14 10:54:44 +00:00
Rolf Neugebauer
c50bcb6611 tools: Add YAML file for running LTP
LTP does need disk access, so this needs to be added once it
is supported again.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Rolf Neugebauer
c792c76bae tools: Add initial Linux Test Project container
LTP is compiled in on debian container and the binaries
then copied into another container, which is pushed to hub.

LTP does not compile on Alpine as it uses glibc specific
pthread extensions.

I tried to link LTP statically to not require a glibc based
base image but that failed too.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Justin Cormack
e3aad24067 Merge pull request #1296 from ijc25/q35-and-kvm
qemu: Enable KVM support and switch to Q35
2017-03-13 15:55:16 +00:00
Ian Campbell
6788566a62 qemu: Enable KVM support and switch to Q35
Passing accel=kvm:tcg causes it to try KVM first if available with a fallback
to TCG (emulated/JIT mode) if it is not available. With this the boot logs gain:

+Hypervisor detected: KVM

and also

-Booting paravirtualized kernel on bare hardware
+Booting paravirtualized kernel on KVM

Among various other noise.

If I rename my host /dev/kvm then this is reversed, although with the following
message:

  Could not access KVM kernel module: No such file or directory
  failed to initialize KVM: No such file or directory
  Back to tcg accelerator.

Q35 is a more modern emulated platform based on the ICH9 host chipset rather
than the default "pc" I440FX (Pentium Pro / Pentium II era) emulation. See
http://wiki.qemu-project.org/Features/Q35 for more info. Switching to Q35 is
not a requirement for enabling KVM but seemed like a reasonable change.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-13 15:39:08 +00:00
Riyaz Faizullabhoy
eeae1d9630 Remove unused test bits
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-13 12:06:24 +00:00
Justin Cormack
e30489fb77 Add qcow2 output support
- outputs compressed qcow2 image
- 1GB underlying size, may need changing

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:38:37 +00:00
Justin Cormack
9ebc11034a Merge pull request #1277 from justincormack/vhd
Add VHD and GCE output formats
2017-03-10 16:13:23 +00:00
Justin Cormack
41fac88bf6 Add VHD and GCE output formats
- VHD is uncompressed VHD. Currently hard coded at 1GB, which may need to change. Use `format: vhd`
- GCE is the GCE compressed tarred raw image. Use `format: gce-img` - reserving `gce` for actually
  uploading the image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:10:48 +00:00
Rolf Neugebauer
7d03bd8c3e infrakit: Attach ISO file to VM if it exists
The file can me mounted in the VM as /dev/sr0

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:55:23 +00:00
Rolf Neugebauer
4bfbe8ad5d infrakit: Write ISO file from instance init contents
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:24:18 +00:00
Justin Cormack
6122a483dc Merge pull request #1264 from tych0/hoist-kernel-cmdline
Hoist kernel cmdline
2017-03-09 22:34:30 +00:00
Tycho Andersen
e8319f6761 hoist kernel cmdline into moby.yaml
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 14:29:40 -08:00
Justin Cormack
9d3eaefd93 Merge pull request #1271 from rneugeba/infra-up
infrakit: Fix detection of running proceses
2017-03-09 22:13:26 +00:00
Rolf Neugebauer
d6d2162999 infrakit: Fix detection of running proceses
os.FindProcess() does not return an error when then process does not
exist. It even returns a dummy process object.

Use the go-ps package to find out if the hyperkit process is actually
running.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-09 22:08:44 +00:00
Tycho Andersen
81a439751b hoist kernel command line out of qemu docker image
we want to fiddle with this for selinux, and probably for other things too.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 11:13:49 -08:00
Justin Cormack
4170897db0 Strip Go files to make them a bit smaller
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-09 15:30:15 +00:00
Rolf Neugebauer
848afc03e4 infrakit: Use the moby tool naming convention to select moby builds
the `moby` tool uses a prefix for the build output. Specify the prefix
(and optional path) in the `Moby` property in the JSON instead of assuming
a different path for each initrd/kernel.

Also, adopt the new name for the kernel image (vmlinuz64 -> <prefix>-bzImage).

This should make it simpler to boot up hyperkit with Moby images
build from by the moby tool.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 16:09:26 +00:00
Rolf Neugebauer
d76e0c1dd3 Merge pull request #1255 from rneugeba/infra-docs
infrakit: Update docs
2017-03-07 15:58:56 +00:00
Rolf Neugebauer
d78c402c11 infrakit: Update docs
While at it, also simplify the JSON file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 15:03:03 +00:00
Justin Cormack
fc6c5e5c66 More cleanup
- remove more editions code
- remove unused tool pad4
- add back whale to test output

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-07 13:05:53 +00:00
Justin Cormack
cd08084738 Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-06 22:28:41 +00:00
Justin Cormack
2ab099f349 Add output types for BIOS and EFI ISOs
Note that the EFI ISO is not yet automatically sized, and the
kernel command lines are currently hard coded in the builders.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-03 18:44:47 -08:00
Rolf Neugebauer
64c0c441ec infrakit: Make Destroy() usable elsewhere
Don't return if the process does not exist. Instead proceed and delete
the state directory.

This allows us to use the Destroy() method elsewhere to clean up the state.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:41:18 -08:00
Rolf Neugebauer
0c1d9fd624 infrakit: Move default VM directory to .infrakit/hyperkit-vms
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:13:17 -08:00
Rolf Neugebauer
3aa4bedab8 infrakit: removed vm-lib from arguments
Instead pass the directory where to find initrd/vmlinuz via the
"Moby" entry in the instance JSON config.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:52:37 -08:00
Rolf Neugebauer
1d5cc2f59b infrakit: Use template based instance configuratio
This change now allows setting the Disk size, memory, and number of CPUs
for an instance via the JSON config file.

We now also write the hyperkit command line to the VM directory.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:09:30 -08:00
Rolf Neugebauer
2bcaea5322 tools: Add initial HyperKit/Moby InfraKit instance plugin
This is the first cut of a InfraKit instance plugin which
primarily allows one to manage Moby instances (though, other Linux's
may work too).

It's very rough on the edges and needs work and update to handle
templates and better configuration etc.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-02-28 18:51:31 -08:00
Robb Kistler
e0269e0978 Allow docker engine to be pulled from s3 bucket
Use aws cli to pull docker engine binaries from s3 bucket.  Conditional and
happens if DOCKER_BIN_URL begins with 's3://'

Example usage:
```
  export AWS_ACCESS_KEY_ID=key
  export AWS_SECRET_ACCESS_KEY=secret
  make DOCKER_BIN_URL=s3://path-to-engine-binaries
```
Signed-off-by: Robb Kistler <robb.kistler@docker.com>

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-23 20:32:00 -08:00
Riyaz Faizullabhoy
3053ac1725 Merge pull request #1202 from justincormack/go-up
Update to Go 1.8
2017-02-17 09:19:53 -08:00
Riyaz Faizullabhoy
5ed7744538 Merge pull request #1197 from justincormack/testup
Update kernel config tests
2017-02-17 09:19:44 -08:00
Justin Cormack
d7e6d42963 Update to Go 1.8
This includes most things except the containerd and runc builds
which I will do next time I update them.

Note that all golang:alpine images now for 1.8 are ALpine 3.5 based.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 16:31:19 +00:00