Commit Graph

745 Commits

Author SHA1 Message Date
Rolf Neugebauer
148b51347f vendor: add ssh agent package
This is needed for ssh into the serial console on packet.net

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
c804ff0dc1 cmd: Enable always-pxe for packet.net
This option configures the machine to always PXE boot. By
default it would only PXE boot for the first time.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
bdc06ee30a cmd: Add packet.net env variable names to help message
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
71a4cd2f3c cmd: Prefix packet.net host name with username
If a packet account is shared between users, this makes it
clearer who created the machine.

Also, switch "moby" to "linuxkit"

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
a837312cc7 cmd: Switch packet.net API to github.com/bzub/packngo
This fork has initial support for always-pxe and was
suggested to use be used as an interim solution here:
https://github.com/packethost/packngo/issues/22#issuecomment-319973502

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Justin Cormack
7b36db8358 Output EFI ISO as for BIOS ISO with filesystem on.
The image is multi-arch and should work on arm64 as it has no boot loader...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-07 16:44:13 +01:00
Justin Cormack
73b516f09b Merge pull request #2302 from dave-tucker/mdgcp
linuxkit: Better GCP disk handling
2017-08-07 16:01:53 +01:00
Ian Campbell
97da92dc65 linuxkit: Bump to logrus 1.0.2 (with case change).
To enable this bump github.com/Azure/go-ansiterm and github.com/docker/docker
to their latest version which have switched to lower case.

This in turn requires bumping golang.org/x/sys since github.com/docker/docker/pkg/term
now uses `unix.IoctlGetTermios`. I picked the revision from docker/docker's vendor.conf.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-07 12:11:43 +01:00
Zachery Hostens
9173d88456 save mac address within state
Signed-off-by: Zachery Hostens <zacheryph@gmail.com>
2017-08-06 22:39:34 -05:00
Rolf Neugebauer
7c3f9690d5 cmd/qemu: Better handling of arch
- When executing on aarch64, use it as the default arch
- When selecting aarch64 on a non aarch64 system set the
  CPU flag to a default value (not 'host').

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-04 15:45:00 +01:00
Justin Cormack
d7caf92708 Merge pull request #2355 from justincormack/bootrom
Support UEFI ISO boot on hyperkit
2017-08-04 12:45:27 +01:00
Justin Cormack
317ddb09df Do not try to autodetect UEFI mode
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 11:42:30 +01:00
Justin Cormack
8aa811540f Support UEFI ISO boot on hyperkit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 11:34:51 +01:00
Eric Briand
5e53f548a1 Fix AWS push
Signed-off-by: Eric Briand <eric.briand@gmail.com>
2017-08-04 10:36:00 +02:00
Rolf Neugebauer
684ad6778d cmd: Use qemu multi-arch image for qemu runner
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Nick Jones
f65a4e803d Update OpenStack run and push support
This commit updates the support for pushing images into OpenStack by
inheriting environment variables for endpoint and authentication
information, when available.

It also attempts to make the `openstack run` support more consistent
with other providers (specifically GCP and AWS), i.e just take the name
of the image as the argument and launch an instance using that.

Finally, it also updates the relevant documentation for OpenStack
support.

Signed-off-by: Nick Jones <nick@dischord.org>
2017-08-02 22:45:38 +01:00
Justin Cormack
550bf41d0b Merge pull request #2337 from zacheryph/qemu-pidfile
save qemu pid in state directory
2017-08-02 15:53:17 +01:00
Nick Jones
a6f3f99d72 Initial support for launching instances on OpenStack
This commit provides rudimentary support for launching instances on an
OpenStack platform.

Signed-off-by: Nick Jones <nick@dischord.org>
2017-08-02 14:40:38 +01:00
Justin Cormack
2d930bc8e5 Use new style ISO BIOS mkimage from Linuxkit which unpacks filesystem
Rather than using an initrd, unpack full filesystem for ISO BIOS.

Stream docker output direct to file rather than via a buffer, to save
memory.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-02 14:24:24 +01:00
Zachery Hostens
aa48795203 save qemu pid in state directory
Signed-off-by: Zachery Hostens <zacheryph@gmail.com>
2017-08-01 11:23:43 -05:00
Justin Cormack
48e42540d2 Fix hardlinks in tar output
When we converted these to cpio we were not noticing that they
were invalid as they had incorrect paths as we converted the
path to a symlink anyway. Only the busybox images have hard links
in, the Alpine ones are symlinks anyway, which is why it was
less visible too.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-01 17:04:57 +01:00
Justin Cormack
3dec6855e0 Fix read only output when defined in a label
Also do some code cleanup.

Related to #131 we need to read the OCI config to find if the container
is read only, not rely on the yaml, as it may just be set in the label.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-28 14:34:18 +01:00
Justin Cormack
7010eae061 Update mkimage used for building to lastest hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-28 13:35:59 +01:00
Justin Cormack
e388c0ff14 Rework setup of container for read only/read write
To work with truly immutable filesystems, rather than ones
we sneakily remount `rw`, we are going to use overlay for
writeable containers. To leave the final mount as `rootfs`,
in the writeable case we make a new `lower` path for the read
only filesystem, and leave `rootfs` as a mount point for an
overlay, with the writable layer and workdir mounted as a tmpfs
on `tmp`.

See https://github.com/linuxkit/linuxkit/issues/2288

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-27 14:52:13 +01:00
Dave Tucker
3dd45d41aa linuxkit: Better GCP disk handling
This commit allows the GCP backend to use the familiar `-disk` behaviour
that the local hypervisors use. The `file` attribute is used as the disk
name in GCP. The size is converted to GB and is always > 1GB.
This has the benefit of allowing multiple disks to be used with GCP
instances.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-07-26 20:53:50 +01:00
Rolf Neugebauer
896c3574ab cmd: Change logic for using EFI FW when qemu is run in a container
By default we want qemu to use the EFI firmware image in the qemu
container. However the logic in the code would always bind mount
the FW image into the container.

This commit changes the logic to only bind mount the FW image if
it was specified on the commandline.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-26 18:14:28 +01:00
Rolf Neugebauer
4b25e00c7d cmd: Update qmeu container image hash
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-26 18:14:28 +01:00
Ian Campbell
096aec0a19 qemu: Add -networking option, with various new alternatives
This follows the model in the hyperkit runner, although the options are
different.

The options are:

- `user`: the existing user mode networking (the default).
- `tap,«device»`: replaces the previous `-tap-device «device»` option.
- `bridge,«name»`: tap device on (preexisting) named bridge.
- `none`: No networking at all.

If not running as root then `bridge` mode requires host configuration
http://wiki.qemu.org/Features/HelperNetworking. TL;DR: you need to `chmod u+s`
the `qemu-bridge-helper` and to whitelist specific bridges in
`/etc/qemu/bridge.conf`.

Pass an explicit virtio nic and configure a random MAC since QEMU seems to use
the same one by default.

In the hyperkit runner the various `networking*` constants become
`hyperkitNetworking*` to avoid namespace clashes (e.g. for `None`). The QEMU
equivalents are `qemuNetworking*`.

Both hyperkit and qemu now support an explicit `-networking default` or
`-networking ''` to make scripting easier.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-25 16:17:22 +01:00
Ian Campbell
cb86cdb027 qemu: Generate a random uuid and pass via -uuid
This is the same behaviour as the LinuxKit backend.

This populates /sys/class/dmi/id/product_uuid, which newer version of weave-net
appears to require.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-25 16:17:22 +01:00
Justin Cormack
6403215635 In the init section use a symlink for /etc/resolv.conf
Unfortunately there are a lot of issues with resolv.conf as we
cannot actually write it into the image from any docker image, as docker will
always have something bind mounted in.

In addition, normally we expect the filesystem to br read only for images
that moby generates, so the actual etc/resolv.conf is likely not to be writeable.

Previously we were adding in a default resolv.conf into every image pointing at
Google's name servers but that is really a bad idea.

Instead, normal images now get an empty default, while images in the `init`
section will get a symlink, currently hard coded to `/run/resolvconf/resolv.conf`
but you can override this with the `files` section to be static or a different
link.

In future, if we have an easy way to build and extract images with user control
of this, we can drop this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-25 14:40:40 +01:00
Justin Cormack
2c4d567781 Merge pull request #125 from justincormack/tmpdir
Add a tmp directory for each container to be used as a mount point
2017-07-24 17:19:32 +01:00
Nick Jones
7c65860a5c Refactor OpenStack push support to use Gophercloud
This commit refactors the OpenStack push support to make use of the
Gophercloud library in order to handle authentication and talking to the
right image service as defined in the service catalogue.

Signed-off-by: Nick Jones <nick@dischord.org>
2017-07-24 13:47:18 +01:00
Rolf Neugebauer
3219daa674 Merge pull request #2264 from tych0/support-tap-qemu
support tap devices in qemu backend
2017-07-21 10:56:24 +01:00
Tycho Andersen
d237c92273 support tap devices in qemu backend
The motivation for this is networking out (in particular, testing NFS
support) from the VM.

We could be a lot more user friendly (a la libvirt) by creating the tap
device for users and allowing them to specify a bridge instead, but then
we'd need root to create this tap device. For now, let's make people do
their own tap devices, and just use them. A tap device can be created for a
bridge as follows:

    # ip tuntap add linuxkit0 mode tap user `whoami`
    # ip link set linuxkit0 up
    # ip link set linuxkit0 master $bridge_name

and then used by:

    $ ./bin/linuxkit run qemu -tap-device linuxkit0 linuxkit

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-07-20 12:40:07 -06:00
thebsdbox
41f2d2c256 OpenStack Push support
Signed-off-by: Dan Finneran <daniel.finneran@gmail.com>
2017-07-20 16:24:48 +01:00
Ian Campbell
e8d5005b0e Synchronise vendoring with containerd v1.0.0-alpha1
Some of these are arbitrary and just syncing for the sake of it, however the
image- and runtime-spec are relevant. Interesting changes:

- runtime spec:
  - LinuxRLimit is now POSIXRLimit.
  - Specs.Config is now a pointer.
  - LinuxResources.DisableOOMKiller moved to
    LinuxResources.LinuxMemory.DisableOOMKiller
- image spec:
  - Platform.Features is removed (unused here).

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-20 12:03:22 +01:00
Justin Cormack
4720490c92 Add a tmp directory for each container to be used as a mount point
This is to clean up how we do writeable containers in LinuxKit.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-19 11:29:03 +01:00
Justin Cormack
6eb74b2f2b Add support for onshutdown
This is a list of images to run on a clean shutdown. Note that you must not rely on these
being run at all, as machines may be be powered off or shut down without having time to run
these scripts. If you add anything here you should test both in the case where they are
run and when they are not. Most systems are likely to be "crash only" and not have any setup here,
but you can attempt to deregister cleanly from a network service here, rather than relying
on timeouts, for example.

Fix https://github.com/linuxkit/linuxkit/issues/1988

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 13:55:52 +01:00
Justin Cormack
55f7df577a Add json config for top level config
Otherwise JSON output as metadata cannot actually be used as it
has capitalised field names.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 13:16:58 +01:00
Justin Cormack
12439d947d Do not modify data structure while building
This will give odd effects if we output the yaml structure into the image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-17 15:42:26 +01:00
Justin Cormack
040c500648 Add more omitempty to json spec
The metadata config was rather verbose.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-17 15:31:31 +01:00
Justin Cormack
389dd8c0fa Add a "metadata" file contents flag
Currently this supports "yaml" as the only option, which will output
the yaml config (as JSON) into the file specified in the image.

Fix #107

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-17 15:31:24 +01:00
Riyaz Faizullabhoy
c3b2c7d575 trust: do not call x509.SystemCertPool as nil will use this as default
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-07-14 14:39:39 -07:00
Justin Cormack
cc33e67492 Allow uid, gid fields to be numeric or names
Previously I was forcing them to be strings, which is horrible. Now you
can either specify a numeric uid or the name of a service to use the
allocated id for that service.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 16:57:12 +01:00
Justin Cormack
830be373da Allow specifying the kernel and tarball names, or omitting tarball
fix #113

Use `tar: none` or `tar: ""` to omit the tarball.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 14:09:42 +01:00
Justin Cormack
7bc763a628 Move Contributing to top level
- accidentally committed to wrong directory

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 11:50:51 +01:00
Dennis Chen
3b1dfb69c7 ARM64: specify external firware binary for containerized qemu
Current implementation uses a fixed firmware(bios) binary
installed by the build process of the qemu container image,
which will prevent us from providing an external firmware binary
outside the container. This patch removes this limitation, thus we
can assign a firware binary image file with "-fw" option.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2017-07-14 09:10:42 +00:00
Justin Cormack
944793312a Merge pull request #2197 from ijc/qemu-envvar-overrides
qemu: allow kvm and containerized options to be overridden by the env…
2017-07-13 16:25:51 +01:00
Ian Campbell
cc8bd94960 qemu: allow kvm and containerized options to be overridden by the environment
This is useful in the case where Moby is shelling out to LinuxKit for certain
image types (currently raw and qcow2). Currently to experiment with different
options (e.g. when comparing performance to CI) you have to edit either the
moby or linuxkit tool to change the options used.

The environment variables take precedence over any explict command line options
given.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 15:23:14 +01:00
Ian Campbell
f20828bf7e Update LinuxKit components used for build
This updates to the current set of images as of
https://github.com/linuxkit/linuxkit/pull/2159

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 14:43:34 +01:00