Commit Graph

613 Commits

Author SHA1 Message Date
Rolf Neugebauer
23555494fe cmd/pkg: Fix symlink handling in pkg build context
When building the build context, symlink need special
treatment as the link name needs to be added when
building the tar.FileInfoHeader. This code does that.

We may also need to add a special case for hard links
as the moby/moby package 'archive' does, but this
should for now
fixes #3142

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-27 18:39:08 +01:00
Rolf Neugebauer
d5d4d6fe0d Update package tags to v0.6
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 20:57:30 +01:00
Rolf Neugebauer
27bb10a14c Update use of test packages to latest
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 20:02:12 +01:00
Rolf Neugebauer
169e97fe52 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 19:49:18 +01:00
Rolf Neugebauer
bebde3a2ea cmd/pkg: Build a build context from 'sources'
This is the final piece. If 'sources' are defined, tar up
the sources and rewrite them accordingly. Pass it as build
build context to 'docker'.

This allows building from something like this:

  ├── etc
  │   ├── foo
  └── foo
      ├── Dockerfile
          ├── build.yml
          └── main.go

With 'build.yml':

  image: foo
  extra-sources:
    - ../etc:etc

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:35 +01:00
Rolf Neugebauer
b03288f5b4 cmd/pkg: Add the ability to pass a build context to docker
This commit adds the ability to add a build context to
docker for the package build. The build context is passed
on stdin to the docker process.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:34 +01:00
Rolf Neugebauer
ba4d1c79b0 cmd/pkg: Extract 'extra-sources' and adjust hash calculation
If the build.yml specifies 'extra-sources', ie sources
outside the package directory, calculate the hash based on
the tree hash of all source directories and the package
directory.

Note, this requires the source directories to be under
git revision control.

Also clean up the src and dst of the path and stash the
result in the Pkg structure.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 16:23:34 +01:00
Rolf Neugebauer
75149c56c2 cmd/pkg: Add 'extra-source' field to the pkg schema
And 'sources' to the Pkg structure

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-26 14:27:54 +01:00
Rolf Neugebauer
69d913c5a7 cmd: Add 'kernel+iso' output format
This output format create a kernel + and straight data ISO.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-25 11:34:36 +01:00
Rolf Neugebauer
66b7417819 cmd: Update mkimage-* hashes
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-24 22:30:53 +01:00
Rolf Neugebauer
615b122767 cmd: Add support for private repositories and registries
This commit adds support for authentication for image pulls for
'linuxkit build'. For each image reference we look up credentials
via the docker CLI configuration and use it if defined for
a given registry server. The code caches credentials to avoid
lookups for every image.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:48:26 +01:00
Rolf Neugebauer
1679668121 cmd: Vendor 'github.com/docker/cli' and its deps
This vendors the bits from 'github.com/docker/cli' needed
to pull private images.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:48:26 +01:00
Rolf Neugebauer
e410070401 cmd: Use "context" instead of "x/net/context"
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:11:20 +01:00
Rolf Neugebauer
c4f8af23ff cmd: Update vendoring to for docker/docker and dependencies
A subsequent commit will enable support for private repositories.
This requires some functions from 'github.com/docker/cli' which
in turn relies on some newer versions of some of the vendored
packages here.

In this commit, update all packages used here to the versions
used by 'github.com/docker/cli' release 18.06 (the latest stable).

This requires vendoring a bunch of additional packages, such
as prometheus

Also run 'sort' over 'vendor.conf' to keep things in order.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-20 13:11:20 +01:00
Justin Cormack
cf5cec204d Update LinuxKit for merge of moby tool
- use the mkimage hashes that we had in LinuxKit as more up to date than tool.
- update docs
- move the code from moby under src/cmd/linuxkit

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-07-16 14:22:15 +01:00
Justin Cormack
021b5718f8 Merge moby/tool into LinuxKit
Note these ended up with unrelated histories in the export process.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-07-14 11:26:02 +01:00
Rolf Neugebauer
a0a4ec5449 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 15:13:58 +01:00
Rolf Neugebauer
5c8df3837d Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-07 14:00:07 +01:00
Patrik Cyvoct
a6783261f3 Add Scaleway support for linuxkit command line tool
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-06-14 11:51:04 +02:00
Patrik Cyvoct
f8d399490e Add new dependencies
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-06-14 11:51:04 +02:00
Justin Cormack
068df79031 Add support for aws metadata and security groups in linuxkit run
- set userdata for AWS
- set security group

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-13 19:07:12 -07:00
Justin Cormack
42397f7f1d Add support for setting different name on GCP
Previously name and image were always the same so running two hosts
from one image was not possible!

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-06 16:22:20 -07:00
Justin Cormack
c9db3f0625 Add support for userdata on GCP
This was missing in the linuxkit CLI, even though we support it in the
metadata package.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-06 16:05:51 -07:00
Rolf Neugebauer
5de0a86293 Update use of tools to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-05-12 12:38:05 +01:00
Justin Cormack
415d0d3c32 Update to allow the new locations for Docker for Mac socket paths
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-05-10 13:32:38 +01:00
Justin Cormack
d3a1de393f Use PAX tar format
The default Go tar has restrictions on filename length for example.

PAX is recommended over GNU.

Requires Go 1.10

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-04-24 16:46:26 +01:00
Rolf Neugebauer
0b5ea3fcc3 cmd: Add support for kernel+squashfs to the qemu runner
Unlike the hyperkit runner, the qemu runner already had better
support for auto-detecting the boot method so the changes
are less invasive (and backward compatible).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:32 +01:00
Rolf Neugebauer
0e3c88d47c cmd: Add support for squashfs root filesystem to hyperkit runnner
Previous commits added support for building 'kernel+squashfs'.
This commit adds support for booting this build format on hyperkit.

The changes are a little bigger because some restructuring of the
code was required to support a third (after kernel+initrd and EFI
ISO) boot method.

To keep the code simpler this commit also removes some auto-detection
code for ISO booting. Users now have to specify '-iso -uefi' on the
command line to boot an EFI ISO. Previously, only '-uefi' was
required.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:31 +01:00
Rolf Neugebauer
c26a83011d vendor: Update hyperkit go bindings
This contains a small fix to the disk binadings and allows
booting with a kernel alone (no initrd).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:31 +01:00
Rolf Neugebauer
ccb0007947 cmd: Register local mkimage- images with the vendored moby tool
This allows us to update tags for these locally without having
to re-vendor.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:31 +01:00
Rolf Neugebauer
69679d1db7 vendor: Update moby/tool vendoring
This includes two improvements:
- being able to specify the packages used for building images
- support for building squashfs images.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 18:39:31 +01:00
Rolf Neugebauer
a39cee3f18 initrd: Skip rest of ./boot
For the initrd we only want to extract kernel, cmdline, and
the ucode CPIO archive. Skip whatever is left in ./boot

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 14:42:14 +01:00
Rolf Neugebauer
ad11be6b83 moby: Add 'kernel+squashfs' output
This output produces a kernel and a root filesystem
in squashfs format. squashfs is a read-only, compressed
filesystem.

The 'kernel+squashfs' output can be used in a similar way as
the default 'kernel+initrd' output format with the benefit
that the rootfs does not consume any memory.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 14:35:38 +01:00
Rolf Neugebauer
52cb0e1cfe moby: Improve debug output for 'docker run'
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-16 14:12:19 +01:00
Rolf Neugebauer
4cae8c95e3 Add ability to overwrite images used for outputs
We currently hardcode the Linuxkit/mkimage- images. This has the
unfortunate consequence that, if we update the LinuxKit image used
to generate the output, we have to update the Moby tool and then
vendor it back into the LinuxKit repository.

This commit introduces UpdateOutputImages() which allows a client
of the Moby tools package to selectively overwrite the packages
used to generate the outputs.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-04-13 13:50:33 +01:00
Ian Campbell
3be156da13 pkg: build: do not infer a release if not pushing
If you happen to be exactly on a tag then:

    $ linuxkit pkg build --dev pkg/init
    Building "ijc/init:dev"
    Cannot release "v0.3" if not pushing

Do not try and infer a release if not pushing so this is possible again.

The subsequent check for `bo.release != "" && !bo.push` remains since the
caller could have used `WithRelease` but not `WithPush`. Our CLI never does
this, but a hypothetical other user of the library might.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-06 14:13:08 +01:00
Rolf Neugebauer
db663d30a6 Update YAMLs to released packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-05 15:23:20 +01:00
Rolf Neugebauer
c4f613e047 cmd: Update the moby tool vendoring
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-05 14:00:10 +01:00
Rolf Neugebauer
f848bc9a3c Update mkimage images based on linuxkit/alpine to latest.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-05 11:07:53 +01:00
Rolf Neugebauer
4a0813ec2e cmd: Update linuxkit/qemu to the latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-05 10:59:12 +01:00
Justin Cormack
7be989363b Update moby tool vendoring
This only includes https://github.com/moby/tool/pull/210 which
makes bind mounts into containers rshared not rprivate by default,
which makes debug easier as you can see them from the host.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-04-04 19:00:48 +01:00
Rolf Neugebauer
d7d072b2a9 cmd: Update manifest_push_script.go
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-03 16:08:00 +01:00
Justin Cormack
25cfac5463 Default bind mounts to rshared not rprivate if not specified
It is quite confusing that from the host or another container that
binds `/containers` you cannot see the bind mounts, you have to enter
the container namespace. I think `rshared` is a better default. You
can always be explicit and add `private` if you want a private bind mount.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-03-26 13:54:49 -07:00
Rolf Neugebauer
0840aff18e cmd: Update moby/tool to latest
This picks up the updated mkimage tools

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-03-23 20:15:32 +00:00
Rolf Neugebauer
efeb0ac185 Update mkimage packages
They have been update in linuxkit/linuxkit in prep for
s390x support.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-03-16 09:53:35 +00:00
Justin Ko
df660d35c7 Subcommand serve should parse flags.
Signed-off-by: Justin Ko <justin.ko@oracle.com>
2018-03-07 14:59:14 -08:00
Justin Cormack
ee6d320706 Update moby tool
Also update its notary vendor as that was part of update.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-03-07 09:17:41 +00:00
Avi Deitcher
5d30b45cbc Merge pull request #2942 from deitch/onboot-stream-logs
Add logging to /var/log for onboot containers
2018-03-02 16:47:54 +02:00
Avi Deitcher
6f524eb616 Update hashes for linuxkit/init
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-03-02 15:51:09 +02:00
Justin Cormack
adebf0377c Merge pull request #2933 from pcfdev-forks/master
Configure guest with 9P mount and vsock port when using existing VPNKit instance
2018-03-01 11:22:41 +00:00