Commit Graph

246 Commits

Author SHA1 Message Date
Justin Cormack
b0ceaf2d0b Merge pull request #1306 from riyazdf/separate-container-logs
logs: separate daemon container logs into separate files
2017-03-15 14:22:41 +00:00
Justin Cormack
d72d5c6d0d Merge pull request #1307 from ijc25/isoefi-autosize
tools: Calculate EFI ESP FAT FS size based on contents.
2017-03-15 14:22:20 +00:00
Ian Campbell
fb242fad9d tools: Calculate EFI ESP FAT FS size based on contents.
This requires switching to the dosfstools from alpine:edge since neither the
busybox nor alpine:3.5 dosfstools supports the -C option (in fact alpine:3.5
only has mkfs.fat and not mkfs.vfat).

The 511k slack seems like a lot to me, but 256k was somehow not enough.

Fixes #1304.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-15 13:30:20 +00:00
Riyaz Faizullabhoy
8d3a7ff351 logs: separate daemon container logs into separate files
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-15 12:24:10 +00:00
Justin Cormack
6d72dcee5f Merge pull request #1301 from riyazdf/readonly-check-container
Read-only check container for testing, clean up error output
2017-03-15 11:15:53 +00:00
Riyaz Faizullabhoy
7c322c0e81 read-only check container for testing, clean up error output
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-14 17:08:07 +00:00
Rolf Neugebauer
3874a37824 kernel: Update YAML files with new kernel images
Use the 4.9.x (aka latest) in all files

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-14 10:54:44 +00:00
Rolf Neugebauer
76a02c15fc tools: Add YAML file for running LTP
LTP does need disk access, so this needs to be added once it
is supported again.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Rolf Neugebauer
7362202478 tools: Add initial Linux Test Project container
LTP is compiled in on debian container and the binaries
then copied into another container, which is pushed to hub.

LTP does not compile on Alpine as it uses glibc specific
pthread extensions.

I tried to link LTP statically to not require a glibc based
base image but that failed too.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-13 17:43:10 +00:00
Justin Cormack
1a80011a4c Merge pull request #1296 from ijc25/q35-and-kvm
qemu: Enable KVM support and switch to Q35
2017-03-13 15:55:16 +00:00
Ian Campbell
0dbb1e6e97 qemu: Enable KVM support and switch to Q35
Passing accel=kvm:tcg causes it to try KVM first if available with a fallback
to TCG (emulated/JIT mode) if it is not available. With this the boot logs gain:

+Hypervisor detected: KVM

and also

-Booting paravirtualized kernel on bare hardware
+Booting paravirtualized kernel on KVM

Among various other noise.

If I rename my host /dev/kvm then this is reversed, although with the following
message:

  Could not access KVM kernel module: No such file or directory
  failed to initialize KVM: No such file or directory
  Back to tcg accelerator.

Q35 is a more modern emulated platform based on the ICH9 host chipset rather
than the default "pc" I440FX (Pentium Pro / Pentium II era) emulation. See
http://wiki.qemu-project.org/Features/Q35 for more info. Switching to Q35 is
not a requirement for enabling KVM but seemed like a reasonable change.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-13 15:39:08 +00:00
Riyaz Faizullabhoy
5c1662b1c1 Remove unused test bits
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-13 12:06:24 +00:00
Justin Cormack
054c3b08a3 Add qcow2 output support
- outputs compressed qcow2 image
- 1GB underlying size, may need changing

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:38:37 +00:00
Justin Cormack
5c19da6fa3 Merge pull request #1277 from justincormack/vhd
Add VHD and GCE output formats
2017-03-10 16:13:23 +00:00
Justin Cormack
718d45bdf5 Add VHD and GCE output formats
- VHD is uncompressed VHD. Currently hard coded at 1GB, which may need to change. Use `format: vhd`
- GCE is the GCE compressed tarred raw image. Use `format: gce-img` - reserving `gce` for actually
  uploading the image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-10 16:10:48 +00:00
Rolf Neugebauer
562bc65dec infrakit: Attach ISO file to VM if it exists
The file can me mounted in the VM as /dev/sr0

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:55:23 +00:00
Rolf Neugebauer
cde6fb9309 infrakit: Write ISO file from instance init contents
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-10 14:24:18 +00:00
Justin Cormack
c7d1438128 Merge pull request #1264 from tych0/hoist-kernel-cmdline
Hoist kernel cmdline
2017-03-09 22:34:30 +00:00
Tycho Andersen
a383e6bb46 hoist kernel cmdline into moby.yaml
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 14:29:40 -08:00
Justin Cormack
57355aa813 Merge pull request #1271 from rneugeba/infra-up
infrakit: Fix detection of running proceses
2017-03-09 22:13:26 +00:00
Rolf Neugebauer
0f7807e48d infrakit: Fix detection of running proceses
os.FindProcess() does not return an error when then process does not
exist. It even returns a dummy process object.

Use the go-ps package to find out if the hyperkit process is actually
running.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-09 22:08:44 +00:00
Tycho Andersen
8a3ae66f64 hoist kernel command line out of qemu docker image
we want to fiddle with this for selinux, and probably for other things too.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-09 11:13:49 -08:00
Justin Cormack
154c3516b2 Strip Go files to make them a bit smaller
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-09 15:30:15 +00:00
Rolf Neugebauer
ba7a162603 infrakit: Use the moby tool naming convention to select moby builds
the `moby` tool uses a prefix for the build output. Specify the prefix
(and optional path) in the `Moby` property in the JSON instead of assuming
a different path for each initrd/kernel.

Also, adopt the new name for the kernel image (vmlinuz64 -> <prefix>-bzImage).

This should make it simpler to boot up hyperkit with Moby images
build from by the moby tool.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 16:09:26 +00:00
Rolf Neugebauer
118309b03f Merge pull request #1255 from rneugeba/infra-docs
infrakit: Update docs
2017-03-07 15:58:56 +00:00
Rolf Neugebauer
198599912d infrakit: Update docs
While at it, also simplify the JSON file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-07 15:03:03 +00:00
Justin Cormack
7d6de92700 More cleanup
- remove more editions code
- remove unused tool pad4
- add back whale to test output

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-07 13:05:53 +00:00
Justin Cormack
159202416c Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-06 22:28:41 +00:00
Justin Cormack
e35c4c9b37 Add output types for BIOS and EFI ISOs
Note that the EFI ISO is not yet automatically sized, and the
kernel command lines are currently hard coded in the builders.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-03 18:44:47 -08:00
Rolf Neugebauer
1c3f4b4865 infrakit: Make Destroy() usable elsewhere
Don't return if the process does not exist. Instead proceed and delete
the state directory.

This allows us to use the Destroy() method elsewhere to clean up the state.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:41:18 -08:00
Rolf Neugebauer
d1ad02e079 infrakit: Move default VM directory to .infrakit/hyperkit-vms
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 17:13:17 -08:00
Rolf Neugebauer
95fa38f879 infrakit: removed vm-lib from arguments
Instead pass the directory where to find initrd/vmlinuz via the
"Moby" entry in the instance JSON config.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:52:37 -08:00
Rolf Neugebauer
af2ee9e023 infrakit: Use template based instance configuratio
This change now allows setting the Disk size, memory, and number of CPUs
for an instance via the JSON config file.

We now also write the hyperkit command line to the VM directory.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-01 16:09:30 -08:00
Rolf Neugebauer
175daffe71 tools: Add initial HyperKit/Moby InfraKit instance plugin
This is the first cut of a InfraKit instance plugin which
primarily allows one to manage Moby instances (though, other Linux's
may work too).

It's very rough on the edges and needs work and update to handle
templates and better configuration etc.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-02-28 18:51:31 -08:00
Robb Kistler
b2b585ac99 Allow docker engine to be pulled from s3 bucket
Use aws cli to pull docker engine binaries from s3 bucket.  Conditional and
happens if DOCKER_BIN_URL begins with 's3://'

Example usage:
```
  export AWS_ACCESS_KEY_ID=key
  export AWS_SECRET_ACCESS_KEY=secret
  make DOCKER_BIN_URL=s3://path-to-engine-binaries
```
Signed-off-by: Robb Kistler <robb.kistler@docker.com>

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-23 20:32:00 -08:00
Riyaz Faizullabhoy
447ee3b6c0 Merge pull request #1202 from justincormack/go-up
Update to Go 1.8
2017-02-17 09:19:53 -08:00
Riyaz Faizullabhoy
c643f19434 Merge pull request #1197 from justincormack/testup
Update kernel config tests
2017-02-17 09:19:44 -08:00
Justin Cormack
fe64819020 Update to Go 1.8
This includes most things except the containerd and runc builds
which I will do next time I update them.

Note that all golang:alpine images now for 1.8 are ALpine 3.5 based.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 16:31:19 +00:00
Justin Cormack
74111017f6 Merge pull request #1201 from justincormack/newtools
Add new minimal init prototype and new tools
2017-02-17 15:56:42 +00:00
Justin Cormack
d29be01b0d Add new minimal init prototype and new tools
This is part of the new moby tool work to get a minimal system
up that can be expanded.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 15:31:00 +00:00
Justin Cormack
844c1c39df Update kernel config tests
- `CONFIG_BUG_ON_DATA_CORRUPTION` is a 4.10 kernel only flag, so gate this.
- some config was duplicated on gated and normal config
- some 4.4 issues with the new config

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 14:52:32 +00:00
Riyaz Faizullabhoy
4daa3b5c88 Bump kernel config test and revert IA32_EMULATION
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-13 11:38:14 -08:00
Riyaz Faizullabhoy
25b2d47f1d Add page_poison=1 to boot args
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-13 10:16:46 -08:00
Justin Cormack
66c92081fc Update kernel config check for suggested values
This should be done with #1175

Config not updated as may still need changes

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-13 10:14:53 -08:00
Justin Cormack
9fe9e95349 Replace mkinitrd.sh with a tool to convert base image to initrd
Better split between content and tools.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-01 22:32:34 +00:00
Riyaz Faizullabhoy
d66bc2d501 Split base/ into base/ and tools/
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-31 09:52:30 -08:00