Commit Graph

791 Commits

Author SHA1 Message Date
Justin Cormack
cc823286f4 Merge pull request #153 from ijc/subassign-runtime
More granular merge of Runtime from labels & yaml
2017-10-02 11:40:14 +01:00
Justin Cormack
1fc16c4580 Update to use new common iso EFI for amd64 and arm64
See https://github.com/linuxkit/linuxkit/pull/2524

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-25 11:02:37 -07:00
Avi Deitcher
ff889ef763 Better error message when running qemu on macOS when --fw not supplied
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-09-24 11:18:19 +03:00
Ian Campbell
a5a3b778c0 More granular merge of Runtime from labels & yaml
Previously any Runtime specified in yml would completely override anything from
the image label, even if they set distinct fields. This pushes the merging down
to the next layer, and in the case of BindNS down two layers.

Most of the fields involved needed to become pointers to support this, which
required a smattering of other changes to cope. As well as the local test suite
this has been put through the linuxkit test suite (as of cc200d296a).

I also tested in the scenario which caused me to file #152.

Fixes #152.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-22 13:18:36 +01:00
Justin Cormack
63eae8abee Merge pull request #149 from justincormack/shared-init
Used shared init with LinuxKit
2017-09-18 18:10:08 -07:00
Nick Jones
0556812b63 [OpenStack] Support specifying an SSH key name when creating an instance
This commit introduces a new option - `keyname` - to the OpenStack
runner, which allows the user to specify the name of a keypair they want
to associate with the instance at the time of creation.

Signed-off-by: Nick Jones <nick@dischord.org>
2017-09-18 21:36:04 +01:00
Justin Cormack
dff6dc2876 Used shared init with LinuxKit
With https://github.com/linuxkit/linuxkit/pull/2500 the init script
can be used for both bare metal and in Docker.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-18 13:11:37 -07:00
Ian Campbell
411a21ae07 Make "metadata: yaml" actually output yaml
The existing code made json...

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-18 15:13:32 +01:00
Ian Campbell
16147a7591 qemu: restore persistent mac address support
If the state dir is preserved then the VM should come up with the same mac
address on subsequent reboots.

Fixes regression in #2498.

Rather than fixing the typo in the associated comments just remove them since
they don't really convey any useful information.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-12 12:12:03 +01:00
Dennis Chen
147964b241 qemu: Fix the networking issue in 'tap' mode on arm64
This PR is used to fix the issue #2488.

Currently we use '-net' the old way to initialize a host nic
interface, this method will not work on arm64 platform(#2488 issue),
so we use the '-netdev' method which will work on both arm64 and amd64.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2017-09-11 05:33:06 +00:00
Rolf Neugebauer
57394620b8 cmd: On KVM/aarch64 add gic_version=host
This picks the correct GIC version to be emulated.
Without it you get an error on Thunder-X (and possibly others):

KVM_SET_DEVICE_ATTR failed: No such device

See also https://bugzilla.redhat.com/show_bug.cgi?id=1375571#c6

Fixes #2360

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-08 16:34:33 +01:00
Magnus Skjegstad
c8ba942a80 Update to new Hyperkit API / VPNKit protocol
This adds support for the updated Hyperkit API, which is needed to
request a specific IP address in new versions of VPNKit / Docker for
Mac. IPs encoded in the UUID (the old method) will now be ignored by
VPNKit.

A preferred IPv4 address can be requested directly via the new API. The
IP is then associated with the VPNKit UUID identifying the connection.
The UUID is either user specified or randomly assigned if left empty.
VMs launched with the same VPNKit UUID it will get the same IP address.

To avoid having to copy the assigned UUID manually, a file `uuid.vpnkit`
is now saved in the state directory when the UUID is generated.  The UUID
from this file is reused automatically if it exists, unless a different
VPNKit UUID is specified on the command line. This also means that VMs
that use dynamically assigned IPs will by default get the same IP each
time they are started, as long as the state directory exists.

This change is incompatible with earlier versions of VPNKit / Hyperkit
and a recent version of Docker for Mac has to be installed. If the
feature is unsupported using the `--ip` parameter will exit with an
error message.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-09-07 12:15:49 +02:00
Magnus Skjegstad
241136e910 Update Hyperkit to latest version
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-09-07 10:49:45 +02:00
Justin Cormack
11b573c6fb Rename "output" to "format"
This was confusing as there is an option to output to a file as well.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-01 23:45:25 +01:00
Ian Campbell
4bdca70c5d qemu: Support ISO boot at the same time as metadata
Tested with:
 - boot from kernel + initrd with metadata (appears as sr0)
 - boot from iso with no metadata (root is sr0)
 - boot from iso with metadata (root is sr0, metadata is sr1)
 - boot from iso with metadata using containerized qemu

In all cases where it was present the metadata was correctly expanded to
/var/config.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-29 21:16:10 +01:00
Justin Cormack
4a374eb175 Add missing case when running non existent file in hyperkit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:08:30 +01:00
Justin Cormack
cfa5d273b7 De-dup container root filesystems
With the mount framework we can de-dup containers that share the same image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 17:27:08 +01:00
Justin Cormack
6b98aff58b Use the runtime config to specify how overlay should be created
This puts the build side in charge of the runtime layout, which enables
additional optimisations later, like sharing the rootfs if it is
used multiple times.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 16:21:56 +01:00
Justin Cormack
ee0901762f Add support for mount in runtime config
This could be used in LinuxKit now, as there are some examples, eg
https://github.com/linuxkit/linuxkit/blob/master/blueprints/docker-for-mac/base.yml#L33
which are creating containers to do a mount.

The main reason though is to in future change the ad hoc code that generates
overlay mounts for writeable containers with a runtime config which does
the same thing; this code needs to create both tmpfs and overlay mounts.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 15:17:54 +01:00
Rolf Neugebauer
c6ab35f721 cmd: Use the main packet.net API repo again
The local changes were merged, so switching back to the original
repository. A minor change in API was needed.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-18 10:21:36 +01:00
Justin Cormack
ea60eff557 Add support for runtime configuration
This adds a `runtime` section in the config that can be used
to move network interfaces into a container, create directories,
and bind mount container namespaces into the filesystem.

See also https://github.com/linuxkit/linuxkit/pull/2413

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 17:36:49 +01:00
Rolf Neugebauer
e28d960635 Merge pull request #2428 from justincormack/packet-fork
Vendor fork correctly
2017-08-17 15:36:43 +01:00
Justin Cormack
ad4131ad4b Merge pull request #2426 from arm64b/firmware-file-format
qemu: Specify the firmware file format explicitly
2017-08-17 14:56:35 +01:00
Justin Cormack
63f521fd42 Vendor fork correctly
`vndr` has support for forks!

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 14:16:48 +01:00
Dennis Chen
023dba65c9 qemu: Specify the firmware file format explicitly
Currently we depend on the qemu to detect the firmware file format
automatically, which is dangerous. This patch specify the 'raw'
format explicitly to remove the kind of restrictions.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2017-08-17 09:27:28 +00:00
Rolf Neugebauer
56ae6d5055 cmd: Allow re-using an existing packet.net machine
This add a -device option which enables the re-use of an existing
device. The device state is update to use a potentially new base-url
ro PXE boot from. This maybe faster than provisioning a new server.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-16 16:27:50 +01:00
Rolf Neugebauer
d246ea9130 cmd: Temporarily use my fork of the packet.net API
This adds a new call to update an existing device.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-16 16:27:50 +01:00
Rolf Neugebauer
4d5bf9d90c Merge pull request #2416 from rn/p4
iPXE boot of arm64 on packet.net
2017-08-15 23:24:11 +01:00
Rolf Neugebauer
a6e25b0825 cmd: Print the packet machine ID when booting
This is useful if the connection to the SOS fails and you
want to debug manually after.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 21:36:10 +01:00
Rolf Neugebauer
f8c45539cc cmd: Add iPXE support for arm64 on packet.net
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 17:54:38 +01:00
Justin Cormack
b7fbabe2ba Simpler known_hosts parsing
Note it cannot deal with hashed host keys still.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-15 17:22:43 +01:00
Rolf Neugebauer
7ae9fab6b6 cmd: Unify Logging for packet backend
Be less verbose when running.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 14:31:44 +01:00
Rolf Neugebauer
2590c66511 cmd: Remove the packet machine when done
Adds a new option, '-keep' which will keep the machine around,
but by default remove the machine when done.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 14:31:44 +01:00
Rolf Neugebauer
fced67dab8 cmd: Add a -serve command to the packet.net backend
The -serve command starts a local HTTP server which serves
the kernel and initrd files.  The server can then easily be
made available via ngrok or other means.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 14:31:44 +01:00
Rolf Neugebauer
33e8afb292 cmd: Use the proper packet.net go bindings again
The support for always PXE has been merged.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-15 14:31:44 +01:00
Justin Cormack
4da3c09e19 Do not use filepath.Join if constructing LinuxKit paths
This will do the wrong thing on Windows, and construct paths with \.

fix #142

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-14 15:38:58 +01:00
Justin Cormack
a3de645048 Merge pull request #2397 from dave-tucker/aws-panic
aws: Fix panic when there is no console output
2017-08-10 21:59:54 +01:00
Dave Tucker
90bf0cc2f6 aws: Fix panic when there is no console output
When an AWS machine is stopped and there is no console output the CLI
panics due to a nil pointer dereference. This commit checks that the
console output is not null before attempting to decode it.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-08-10 17:28:09 +01:00
Dave Tucker
f574ada213 aws: Honour the zone variable when creating an instance
Instances were being created in a different AZ to the one specified in
the zone variable. This could lead to situations where the disks and the
instance were in different AZs and would result in an error.

This commit adds placement information to the API call used to create
the instance.

Fixes: #2388

Signed-off-by: Dave Tucker <dt@docker.com>
2017-08-10 16:49:56 +01:00
Justin Cormack
13b1eb7d0c Do not use rm as may not be in root filesystem
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-10 14:59:32 +01:00
Justin Cormack
13a1ffdd44 Add Cgroup resource limits into Yaml spec
This adds all the cgroup restrictions into the spec.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-09 16:39:10 +01:00
Rolf Neugebauer
1a9d736f75 Merge pull request #2392 from rn/p2
Better kernel cmdline handling on packet.net
2017-08-09 14:20:55 +01:00
Rolf Neugebauer
a885eb3304 cmd: Append the kernel command line args when booting on Packet
The iPXE needs the hard coded value of 'kernel-params' but
we should append the kernel command line from the YAML in case
it sets other parameters.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-09 13:18:41 +01:00
Rolf Neugebauer
934d818bf1 cmd: Restructure iPXE script code for packet backend
Break it into multiple instructions. The single line was
super hard to parse.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-09 12:19:31 +01:00
Justin Cormack
34dc65561b Robustify ISO/kernel detection on hyperkit
Much easier to follow and more correct; also ignores directories.

fix #2385

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-09 12:00:00 +01:00
Rolf Neugebauer
04a689fcc4 cmd: Enable console access for packet.net
On 'linuxkit run packet' ssh into the SOS for the newly created
machine. This requires that you have set up your ssh keys and
have at least once logged into the ssh so that the host keys
are present in the 'known_hosts' file.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
148b51347f vendor: add ssh agent package
This is needed for ssh into the serial console on packet.net

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
c804ff0dc1 cmd: Enable always-pxe for packet.net
This option configures the machine to always PXE boot. By
default it would only PXE boot for the first time.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
bdc06ee30a cmd: Add packet.net env variable names to help message
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00
Rolf Neugebauer
71a4cd2f3c cmd: Prefix packet.net host name with username
If a packet account is shared between users, this makes it
clearer who created the machine.

Also, switch "moby" to "linuxkit"

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-07 21:35:41 +01:00