Commit Graph

429 Commits

Author SHA1 Message Date
Krisztian Horvath
c68f52c76f log: log the actual error message when creating the Docker API client
Signed-off-by: Krisztian Horvath <keyki.kk@gmail.com>
2019-01-02 17:12:40 +01:00
Rolf Neugebauer
78281af751 build: Make list of mounts deterministic
Sort the list of mount points by destination. This makes the list
deterministic for reproducible builds and also ensures that, e.g.,
the mount for /dev happens before the mount for /dev/pts.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-12-29 14:10:38 +00:00
Rolf Neugebauer
2fec949cd9 build: Make list of capabilities deterministic
Sort list of capabilities that go into config.json.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-12-29 14:10:38 +00:00
Rolf Neugebauer
93596225da build: Fix timestamp of files created by 'docker export'
Currently 'docker export' is used to convert a linuxkit entry
in the YAML file to a tar file of the root filesystem. This
process creates a number of files and directories which have
the timestamp of when the 'docker export' is run. Fix 'em up.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-12-29 14:10:38 +00:00
Rolf Neugebauer
5cbc156faf build: Improve debug output
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-12-29 14:10:38 +00:00
Rolf Neugebauer
02579b22e5 build: Fix the ModTime for files created during build
When creating files for the "intermediate" tar ball,
fix the ModTime. This reduces the difference between
LinuxKit images build from identical inputs.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-12-29 14:10:38 +00:00
Omar Ramadan
71808a004d Add qemu USB devices
Signed-off-by: Omar Ramadan <omar.ramadan93@gmail.com>
2018-11-27 09:39:12 -08:00
Rolf Neugebauer
f635cad7a6 build: Add support for decompressing bzLinux kernels
Support plain gzip'ed files, as used on arm64, and bzImage with
embedded gzip'ed kernel, as used on x86.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-11-22 23:03:26 +00:00
Rolf Neugebauer
09fbcb59d7 cmd: Add scaffolding to decompress the kernel
Add the '-vmlinux' flag to build and pass it all
the way to the kernel filter.

Note, this commit only adds the flag but does not
yet perform the decompression. This will be added
with the next commit.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-11-20 21:22:49 +00:00
Rolf Neugebauer
4f0cec5c14 build: Restructure the kernel filter
Stash the kernel image in a local buffer and
flush it out once done.

This is preparation work for supporting uncompressed
kernels in the next commit.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-11-20 21:22:49 +00:00
Ian Campbell
ab8524d3d1 Bump yml to pickup containerd v1.2.0
Also pick a newer version of runc.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-30 11:08:50 +00:00
Ian Campbell
e78b25062c scripts: support credentials helpers on Linux
On Linux a key in `~/.docker/config.json` indicates if a credentials helper is
in use (and which), if one is then the method is identical to the Darwin case
so refactor to support that.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-30 09:29:19 +00:00
Rolf Neugebauer
1ec59f9aeb build: Don't create empty kernel/cmdline files
If the YAML does not specify a kernel, kernel commandline
or any containers, don't create empty files. Note, an
initrd file is still created if the kernel image contains
CPU ucode.

This only applies to kernel+initrd and tar-kernel-initrd
output formats.

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-10-29 21:03:47 +00:00
Eric Briand
4fb5ecd066 gcp: output error message when client could not be created
Signed-off-by: Eric Briand <eric.briand@gmail.com>
2018-10-16 11:57:30 +02:00
Eric Briand
0c6a059f71 gcp: check if bucket exists after client creation
Signed-off-by: Eric Briand <eric.briand@gmail.com>
2018-10-16 11:57:30 +02:00
Eric Briand
b15d0963f4 gcp: validate if project name is filled
Signed-off-by: Eric Briand <eric.briand@gmail.com>
2018-10-16 11:57:30 +02:00
Ian Campbell
c60cef7be2 Bump yml to pickup containerd v1.1.4
Also pick a newer version of runc.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-25 16:28:56 +01:00
Mathieu Champlon
54868329e6 Fix path separator on Windows
On Windows os.PathSeparator is \ but here it's all unix file paths being manipulated.

Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
2018-09-10 21:53:09 +02:00
Mathieu Champlon
e17b603be8 Fix relative path under Windows
Using filepath primitives instead of manipulating file paths manually takes care of platform specific formats.

Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
2018-09-10 14:41:43 +02:00
Ian Campbell
610bfd3e11 Bump yml to pickup containerd v1.1.3
Also pick a newer version of runc.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 13:58:12 +01:00
Brice Figureau
02e5f70589 Allow to specify more than one networking adapter for vbox
Note: this patch introduces an incompatibility in the
`linuxkit run vbox` arguments.

It wasn't impossible to specify more than one network adapter
to the `linuxkit run vbox` command.

This patch allows to specify more than one `-networking` argument to specify
different network adapters.

For instance:
~~~sh
linuxkit run vbox -networking type=nat -networking type=hostonly,adapter=vboxnet0
~~~
will setup the VM with 2 NICs.

It is also possible to get rid of the `type` argument.

Signed-off-by: Brice Figureau <brice@daysofwonder.com>
2018-08-02 10:34:26 +02:00
Brice Figureau
f1f128c6dd Allow linuxkit run vbox to use multiple drives
VirtualBox hardware (like physical hardware) has only a limited number
of IDE device on an IDE Controller.
Unfortunately when using an additional drive, it was given the port
value of 2, which doesn't exists in VirtualBox IDE controllers (as
only 0 and 1 are permitted).

This change makes use of the SATA Controller which can host much
more drives, to hook the additional drives.

Signed-off-by: Brice Figureau <brice@daysofwonder.com>
2018-07-31 18:27:54 +02:00
Justin Cormack
4968390a92 Merge pull request #3141 from masterzen/fix/3091-enforce-PAX-for-all-files
Fix #3091 enforce all outgoing tar files to use PAX
2018-07-29 11:35:48 -07:00
Brice Figureau
7a7f5f66c2 Fix #3091 enforce all outgoing tar files to use PAX
While processing the content of a tar image, linuxkit's moby tool is
blindly reusing the original tar format.

Moreover it locates the files under a new prefix, so if the original
file was stored as USTAR in the original archive, the filename length
and new prefix could be greater than the USTAR name limit leading
to a fatal error.

The fix is to always enforce PAX format on all copied files from the
original image archive.

Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
2018-07-29 11:03:44 +02:00
Rolf Neugebauer
23555494fe cmd/pkg: Fix symlink handling in pkg build context
When building the build context, symlink need special
treatment as the link name needs to be added when
building the tar.FileInfoHeader. This code does that.

We may also need to add a special case for hard links
as the moby/moby package 'archive' does, but this
should for now
fixes #3142

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-27 18:39:08 +01:00
Rolf Neugebauer
d5d4d6fe0d Update package tags to v0.6
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 20:57:30 +01:00
Rolf Neugebauer
27bb10a14c Update use of test packages to latest
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 20:02:12 +01:00
Rolf Neugebauer
169e97fe52 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 19:49:18 +01:00
Rolf Neugebauer
bebde3a2ea cmd/pkg: Build a build context from 'sources'
This is the final piece. If 'sources' are defined, tar up
the sources and rewrite them accordingly. Pass it as build
build context to 'docker'.

This allows building from something like this:

  ├── etc
  │   ├── foo
  └── foo
      ├── Dockerfile
          ├── build.yml
          └── main.go

With 'build.yml':

  image: foo
  extra-sources:
    - ../etc:etc

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:35 +01:00
Rolf Neugebauer
b03288f5b4 cmd/pkg: Add the ability to pass a build context to docker
This commit adds the ability to add a build context to
docker for the package build. The build context is passed
on stdin to the docker process.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:34 +01:00
Rolf Neugebauer
ba4d1c79b0 cmd/pkg: Extract 'extra-sources' and adjust hash calculation
If the build.yml specifies 'extra-sources', ie sources
outside the package directory, calculate the hash based on
the tree hash of all source directories and the package
directory.

Note, this requires the source directories to be under
git revision control.

Also clean up the src and dst of the path and stash the
result in the Pkg structure.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:34 +01:00
Rolf Neugebauer
75149c56c2 cmd/pkg: Add 'extra-source' field to the pkg schema
And 'sources' to the Pkg structure

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 14:27:54 +01:00
Rolf Neugebauer
69d913c5a7 cmd: Add 'kernel+iso' output format
This output format create a kernel + and straight data ISO.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-25 11:34:36 +01:00
Rolf Neugebauer
66b7417819 cmd: Update mkimage-* hashes
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-24 22:30:53 +01:00
Rolf Neugebauer
615b122767 cmd: Add support for private repositories and registries
This commit adds support for authentication for image pulls for
'linuxkit build'. For each image reference we look up credentials
via the docker CLI configuration and use it if defined for
a given registry server. The code caches credentials to avoid
lookups for every image.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:48:26 +01:00
Rolf Neugebauer
1679668121 cmd: Vendor 'github.com/docker/cli' and its deps
This vendors the bits from 'github.com/docker/cli' needed
to pull private images.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:48:26 +01:00
Rolf Neugebauer
e410070401 cmd: Use "context" instead of "x/net/context"
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:11:20 +01:00
Rolf Neugebauer
c4f8af23ff cmd: Update vendoring to for docker/docker and dependencies
A subsequent commit will enable support for private repositories.
This requires some functions from 'github.com/docker/cli' which
in turn relies on some newer versions of some of the vendored
packages here.

In this commit, update all packages used here to the versions
used by 'github.com/docker/cli' release 18.06 (the latest stable).

This requires vendoring a bunch of additional packages, such
as prometheus

Also run 'sort' over 'vendor.conf' to keep things in order.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:11:20 +01:00
Justin Cormack
cf5cec204d Update LinuxKit for merge of moby tool
- use the mkimage hashes that we had in LinuxKit as more up to date than tool.
- update docs
- move the code from moby under src/cmd/linuxkit

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-07-16 14:22:15 +01:00
Rolf Neugebauer
a0a4ec5449 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 15:13:58 +01:00
Rolf Neugebauer
5c8df3837d Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-07 14:00:07 +01:00
Patrik Cyvoct
a6783261f3 Add Scaleway support for linuxkit command line tool
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-06-14 11:51:04 +02:00
Patrik Cyvoct
f8d399490e Add new dependencies
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-06-14 11:51:04 +02:00
Justin Cormack
068df79031 Add support for aws metadata and security groups in linuxkit run
- set userdata for AWS
- set security group

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-13 19:07:12 -07:00
Justin Cormack
42397f7f1d Add support for setting different name on GCP
Previously name and image were always the same so running two hosts
from one image was not possible!

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-06 16:22:20 -07:00
Justin Cormack
c9db3f0625 Add support for userdata on GCP
This was missing in the linuxkit CLI, even though we support it in the
metadata package.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-06 16:05:51 -07:00
Rolf Neugebauer
5de0a86293 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-05-12 12:38:05 +01:00
Justin Cormack
415d0d3c32 Update to allow the new locations for Docker for Mac socket paths
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-05-10 13:32:38 +01:00
Rolf Neugebauer
0b5ea3fcc3 cmd: Add support for kernel+squashfs to the qemu runner
Unlike the hyperkit runner, the qemu runner already had better
support for auto-detecting the boot method so the changes
are less invasive (and backward compatible).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:32 +01:00
Rolf Neugebauer
0e3c88d47c cmd: Add support for squashfs root filesystem to hyperkit runnner
Previous commits added support for building 'kernel+squashfs'.
This commit adds support for booting this build format on hyperkit.

The changes are a little bigger because some restructuring of the
code was required to support a third (after kernel+initrd and EFI
ISO) boot method.

To keep the code simpler this commit also removes some auto-detection
code for ISO booting. Users now have to specify '-iso -uefi' on the
command line to boot an EFI ISO. Previously, only '-uefi' was
required.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:31 +01:00