Commit Graph

791 Commits

Author SHA1 Message Date
Rolf Neugebauer
47f139eb08 cmd: Update vendored hyperkit go bindings
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-24 16:06:09 +00:00
Justin Cormack
99fd71dc7f Update moby vendoring
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-18 12:40:56 +00:00
Justin Cormack
2dd65d27b6 Add support for setting containerd namespace in runtime config
This adds a namespace field to override the LinuxKit containerd
default namespace, in case you want to run a container in another
namespace.

Needs a patch in LinuxKit to implement this that I will open soon.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 14:12:15 +00:00
Rolf Neugebauer
0470dc89a8 cmd/gcp: Add -nest-virt option to push and run
If the flag is set on push, then the image will have the license
set to enabled the currently experimental netsted virtualisation
feature. The flag currently also needs to be set on run to make
sure the right CPU model (at least Haswell) is selected.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-17 14:07:27 +00:00
Justin Cormack
929a837d65 Add support for OCI annotations
Annotations do not do anything by default but get passed through to the runtime,
which can be useful. I never metadata I didn't like...

Also fix sysctl to be a map in the validation, not an array. I can't see any
examples using this in LinuxKit, but this matches OCI so is correct.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 12:26:08 +00:00
Rolf Neugebauer
358a020b1a cmd: Update vendored google cloud API
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-17 11:30:44 +00:00
Rolf Neugebauer
63ec7f58c0 vendor: Update moby tool to latest
This includes support for CPU microcode handling

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-16 10:27:58 +00:00
Ian Campbell
6483a2e6dc linuxkit: print response status, not the response object when validating a URL
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:

    error: arg resp for printf verb %s of wrong type: *net/http.Response (vet)

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-15 17:24:36 +00:00
Rolf Neugebauer
5d5a13526b output: Add support for CPU ucode to the kernel+initrd format
This prepends 'ucode.cpio' to the initrd if present. Padding
should not be necessary as the ucode.cpio should be padded
to the right size.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:43 +00:00
Rolf Neugebauer
1d24454438 output: Add ucode.cpio to the tar-kernel-initrd format
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
2b1a611bab output: Extract ucode if present
For now the backends for the different formats do not yet
use the extracted ucode cpio archive, but '// TODO' are
placed for the backends which should eventually handle it.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
0c1b1def13 build: Add ucode to the intermediate tar ball if specified
This extends the kernel filter to also look for the CPU microcode
file if specified in the YAML. If found, the ucode cpio archive
is placed into the intermediate tar file as '/boot/ucode.cpio'.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
b4704b8ba1 config: Add 'ucode' to the kernel schema
This optional option will allow users to specify a CPU
microcode cpio archive to be prepended to the initrd file.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Avi Deitcher
6c015141f5 Split linuxkit run -data options into string and file
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-01-15 16:57:03 +02:00
Ian Campbell
57cf8df3b2 Revendor moby tool to f816553d2f
Pulls in:

f816553 Merge pull request #195 from ijc/prepend-rootfs-mounts
0a6d69d Prepend the rootfs mounts to the user specified runtime.mounts.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-11 12:22:34 +00:00
Rolf Neugebauer
cd1a472678 Merge pull request #2852 from rn/pktpxe
Improve packet.net support
2018-01-11 12:12:47 +00:00
Ian Campbell
0a6d69d2d3 Prepend the rootfs mounts to the user specified runtime.mounts.
User specified mounts should be able to rely on the rootfs being mounted, in
particular for a writeable container they should expect the writeable overlay
to already be in place.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-11 11:42:25 +00:00
thebsdbox
8b9eddf6f6 Added capability to find a host if -hostname isn't specified
Will return the list of all hosts that are part of a vCenter DataCenter
and choose one at random to use.

Signed-off-by: Dan Finneran <daniel.finneran@gmail.com>
2018-01-10 19:44:18 +00:00
Rolf Neugebauer
1cef947ee1 cmd/serve: Add a new 'linuxkit serve' command
This simply starts a web server serving the specified directory.
It's useful for PXE booting.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:48:47 +00:00
Rolf Neugebauer
16ae50b593 cmd/packet: Add linuxkit push packet support
This currently just copies the kernel/initrd to a destination
and writes the iPXE script there as well. The scheme is flexible
enough to support scp or other means for pushing in the future.

The kernel/initrd are conditionally decompressed (default for arm64).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:35 +00:00
Rolf Neugebauer
2431dd9950 cmd/packet: Use IPXEScriptURL instead of passing script via Userdata
Using the userdat to pass the iPXE script precludes us from passing
actual userdata to the instance. Instead pass a URL to the iPXE script.
The script is expected to be a <base-url>/<name>-packet.ipxe.

When starting a server also serve the iPXE script from memory (not
the file system). The http server needed extending to differentiate
between the serving files and the iPXE script and ServeMux is used
to differentiate based on path.

A subsequent commit will provide an option to write the iPXE script
to disk.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:34 +00:00
Rolf Neugebauer
8ae4f5bc86 cmd/packet: Factor out the iPXE script creation
Subsequent commits will use it in other places.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:24 +00:00
Rolf Neugebauer
90510777ad cmd/packet: Tidy up the code
prefix packet.net specific functions with 'packet' and make
non-packet specific functions, such as validateHTTPURL()
more generic so that they can easily be moved to 'utils.go'
should they be needed elsewhere.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:24 +00:00
Rolf Neugebauer
50c4eb5461 cmd/packet: Actually use the -serve argument
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:24 +00:00
Rolf Neugebauer
007854f85e cmd/packet: Fix API changes in the new version of the go bindings
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:23 +00:00
Rolf Neugebauer
b272819540 cmd/packet: Update the vendored packet.net go binding
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-10 17:45:23 +00:00
Justin Cormack
baac3e98c9 Update linuxkit deps in line with moby
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-04 13:55:42 +00:00
Justin Cormack
b0ea58bd90 Update Moby tool
- containerd 1.0 vendoring
- yaml omitempty
- null pointer dereference

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-04 12:14:51 +00:00
Justin Cormack
b5615a3714 Merge pull request #192 from w9n/fix_empty_container_config
allow ImageInspect.Config to be nil when parsing
2018-01-04 11:27:42 +00:00
Justin Cormack
9e65276160 Merge pull request #2821 from rgl/master
add the -gui flag to vbox run
2018-01-02 11:06:22 +00:00
Justin Cormack
1e97929ec5 Fix sriov flag on AWS
This needs to be set to nil if empty.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2017-12-27 18:17:45 +00:00
Robin Winkelewski
175fadb3d4 add omitempty for yml config
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-27 03:08:44 +01:00
Rui Lopes
f36b4a7e59 add the -gui flag to vbox run
Signed-off-by: Rui Lopes <rgl@ruilopes.com>
2017-12-24 10:15:02 +00:00
Ilya Dmitrichenko
142ebf6f82 Improve error message for new packages
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2017-12-18 16:46:00 +00:00
Ian Campbell
dd799eb230 Bump moby tool to d9d2a91780
656bd87fd2...d9d2a91780

d9d2a91 Merge pull request #193 from ijc/bugfix-191
307f13b Defer dockerRm until we are finished with the contents
ebd7228 Merge pull request #191 from ijc/reduce-memory-via-tempfiles
3045a80 Stream `docker export` directly to consumer
9f44acf Generate intermediate image into a temp file
9558740 Add cpu and mem profiling options

Reduces the memory usage substantially.

While here make some notes about the need to update src/cmd/linuxkit/build.go
where people might see them.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-14 10:34:23 +00:00
Robin Winkelewski
1b06de3136 fix inspect.Config usage
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-14 07:11:17 +01:00
Ian Campbell
307f13b129 Defer dockerRm until we are finished with the contents
This was introduced by #191 but somehow did not trigger either for me in local
testing or in CI.

It did trigger in initial CI of https://github.com/linuxkit/linuxkit/pull/2811
which can be seen at https://linuxkit.datakit.ci/linuxkit/linuxkit/pr/2811?history=1637690296123e9a15307b3a41b290da6e27e7cc
The error is:

    Failed to docker rm container «...»: «...»: aufs: unmount error after retries: «...»: device or resource busy

No doubt because we were still holding an open fd while trying to remove the
container.

Unclear why this didn't repro for me (docker 17.11.0-ce with overlay2) or
whatever CI uses.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-13 11:44:32 +00:00
Rolf Neugebauer
0e1133281b Update all YAML files with new packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-08 16:25:13 +00:00
Rolf Neugebauer
9f175e1e6c Update the use of linuxkit/qemu to the alpine:3.7 base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-08 16:25:13 +00:00
Rolf Neugebauer
86f12e15f0 Merge pull request #2796 from pcfdev-forks/master
Ensure VPNKit process is properly killed when errors occur
2017-12-08 11:26:49 +00:00
Ian Campbell
3045a80c85 Stream docker export directly to consumer
Rather than queueing up into a `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) compared with the previous patch from 2.8G to 110M. The
tar output case goes from 2.1G to 110M also. Overall allocations are ~715M in
both cases.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:41:42 +00:00
Ian Campbell
9f44acf8e3 Generate intermediate image into a temp file
All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:07:46 +00:00
Rolf Neugebauer
4a5e06a033 Merge pull request #2797 from pcfdev-forks/vpnkit-path
Allow vpnkit executable path to be specified
2017-12-06 13:23:45 +00:00
Justin Cormack
241e3a1bdb Connect cable on virtualbox networking
fix #2798

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-12-05 16:03:47 -06:00
Dave Protasowski
2d6f4c5484 Allow vpnkit executable path to be specified
Signed-off-by: Steve Hiehn <shiehn@pivotal.io>
Signed-off-by: Dave Protasowski <dprotaso@gmail.com>
2017-12-05 10:06:58 -05:00
Steve Hiehn
ec45349638 Register vpnkit publish port clean up function as a logrus.ExitHandler
Signed-off-by: Dave Protasowski <dprotaso@gmail.com>
2017-12-05 10:05:23 -05:00
Dave Protasowski
c9b4f8306a Ensure VPNKit process is properly killed when errors occur
The log.Fatal* calls will leak the vpnkit process since
defer functions are not invoked when os.Exit(int) is invoked

We register an ExitHandler with logrus - that'll be invoke when
log.Fatal* is called

Signed-off-by: Steve Hiehn <shiehn@pivotal.io>
Signed-off-by: Dave Protasowski <dprotaso@gmail.com>
2017-12-05 09:48:09 -05:00
Ian Campbell
ce639e8080 linuxkit: implement docker image dependencies for pkg build.
This allows the `linuxkit/kubernetes` "image-cache" packages to use a standard
`linuxkit pkg build` based flow rather than requiring surrounding scaffolding.

Fixes #2766. Compared with the original (actually, the second) proposal made in
issue #2766, the field is `docker-images` rather than `images` to allow for
future inclusion of e.g. `containerd-images`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-04 14:14:45 +00:00
Ian Campbell
98d46945d1 linuxkit: Use docker image «foo» consistently
In preference to the `docker «foo»` shortcut/aliases.

Better to be more specific.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-04 12:12:41 +00:00
Ian Campbell
8eed6a4b38 linuxkit: Rename Pkg.pkgPath to Pkg.path
Previous name a bit stuttery.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-04 12:12:41 +00:00