Commit Graph

479 Commits

Author SHA1 Message Date
Justin Cormack
f1ae82c9eb
Merge pull request #210 from justincormack/bind-shared
Default bind mounts to rshared not rprivate if not specified
2018-04-04 18:54:44 +01:00
Justin Cormack
25cfac5463
Default bind mounts to rshared not rprivate if not specified
It is quite confusing that from the host or another container that
binds `/containers` you cannot see the bind mounts, you have to enter
the container namespace. I think `rshared` is a better default. You
can always be explicit and add `private` if you want a private bind mount.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-03-26 13:54:49 -07:00
Justin Cormack
749585dd13
Merge pull request #208 from rn/up
Update mkimage packages
2018-03-19 21:23:35 -07:00
Rolf Neugebauer
efeb0ac185 Update mkimage packages
They have been update in linuxkit/linuxkit in prep for
s390x support.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-03-16 09:53:35 +00:00
Justin Cormack
ed49ca3aa0
Merge pull request #207 from nogoegst/fix-dup-string-readme
Reword the README to remove repeating statements
2018-03-12 14:17:34 +00:00
Ivan Markin
47986c53e9 Reword the README to remove repeating statements
Signed-off-by: Ivan Markin <sw@nogoegst.net>
2018-03-11 20:28:52 +00:00
Justin Cormack
c9d52b5787
Merge pull request #206 from justincormack/notaryup
Update notary vendor to 0.6.0
2018-03-01 11:23:24 +00:00
Justin Cormack
8682dd213f
Merge pull request #205 from garthy/qcow2-efi-support
Add qcow2 EFI image building support
2018-03-01 11:23:18 +00:00
Justin Cormack
d3919d0a79
Update notary vendor to 0.6.0
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-03-01 11:15:50 +00:00
Garth Bushell
e1b4f769cb Add qcow2 EFI image building support
Signed-off-by: Garth Bushell <garth.bushell@oracle.com>
2018-03-01 11:11:23 +00:00
Justin Cormack
830794f3c0
Merge pull request #204 from deitch/fix-doc-typos
Fix typos
2018-02-16 11:48:36 +00:00
Avi Deitcher
07878aa631 Fix typos
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-02-15 21:37:38 +02:00
Justin Cormack
4a0d9e1247
Merge pull request #203 from deitch/docs-for-private-repos
Add documentation about using images from private registries
2018-02-15 13:24:21 +00:00
Avi Deitcher
01ea437bd1 Add documentation about using images from private registries
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-02-15 14:43:17 +02:00
Justin Cormack
bd556c86f7
Merge pull request #201 from rn/fix
The vmdk and dynamic-vhd packages had a bug in them
2018-01-25 19:35:07 +00:00
Rolf Neugebauer
8fb1a88757 The vmdk and dynamic-vhd packages had a bug in them
This picks up a newer version

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-25 18:04:07 +00:00
Rolf Neugebauer
7fb28974b6
Merge pull request #200 from rn/up
Update LinuxKit images to the latest
2018-01-25 14:48:36 +00:00
Rolf Neugebauer
5c1ebdbaa2 Update LinuxKit images to the latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-25 14:15:52 +00:00
Justin Cormack
caca03c097
Merge pull request #199 from justincormack/namespace
Add support for setting containerd namespace in runtime config
2018-01-17 14:19:45 +00:00
Justin Cormack
2dd65d27b6 Add support for setting containerd namespace in runtime config
This adds a namespace field to override the LinuxKit containerd
default namespace, in case you want to run a container in another
namespace.

Needs a patch in LinuxKit to implement this that I will open soon.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 14:12:15 +00:00
Justin Cormack
978deceed0
Merge pull request #198 from justincormack/annotations
Add support for OCI annotations in config
2018-01-17 12:40:31 +00:00
Justin Cormack
929a837d65 Add support for OCI annotations
Annotations do not do anything by default but get passed through to the runtime,
which can be useful. I never metadata I didn't like...

Also fix sysctl to be a map in the validation, not an array. I can't see any
examples using this in LinuxKit, but this matches OCI so is correct.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 12:26:08 +00:00
Justin Cormack
7918437b72 Make test should be phony
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 12:02:32 +00:00
Rolf Neugebauer
57b6e2ab94
Merge pull request #197 from rn/ucode
Add support for Intel ucode
2018-01-15 17:00:03 +00:00
Rolf Neugebauer
1d995e198a docs: Document the 'ucode' kernel option.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:43 +00:00
Rolf Neugebauer
5d5a13526b output: Add support for CPU ucode to the kernel+initrd format
This prepends 'ucode.cpio' to the initrd if present. Padding
should not be necessary as the ucode.cpio should be padded
to the right size.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:43 +00:00
Rolf Neugebauer
1d24454438 output: Add ucode.cpio to the tar-kernel-initrd format
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
2b1a611bab output: Extract ucode if present
For now the backends for the different formats do not yet
use the extracted ucode cpio archive, but '// TODO' are
placed for the backends which should eventually handle it.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
0c1b1def13 build: Add ucode to the intermediate tar ball if specified
This extends the kernel filter to also look for the CPU microcode
file if specified in the YAML. If found, the ucode cpio archive
is placed into the intermediate tar file as '/boot/ucode.cpio'.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
b4704b8ba1 config: Add 'ucode' to the kernel schema
This optional option will allow users to specify a CPU
microcode cpio archive to be prepended to the initrd file.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Justin Cormack
f816553d2f
Merge pull request #195 from ijc/prepend-rootfs-mounts
Prepend the rootfs mounts to the user specified runtime.mounts.
2018-01-11 12:15:18 +00:00
Ian Campbell
0a6d69d2d3 Prepend the rootfs mounts to the user specified runtime.mounts.
User specified mounts should be able to rely on the rootfs being mounted, in
particular for a writeable container they should expect the writeable overlay
to already be in place.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-11 11:42:25 +00:00
Justin Cormack
1aca4eefa5
Merge pull request #190 from ijc/containerd-v1.0.0-rc.0
Bump to containerd v1.0.0
2018-01-04 11:28:10 +00:00
Justin Cormack
b5615a3714
Merge pull request #192 from w9n/fix_empty_container_config
allow ImageInspect.Config to be nil when parsing
2018-01-04 11:27:42 +00:00
Justin Cormack
41d67e3bf3
Merge pull request #194 from w9n/yml_omitempty
add omitempty for yml config
2018-01-04 11:26:54 +00:00
Robin Winkelewski
175fadb3d4 add omitempty for yml config
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-27 03:08:44 +01:00
Robin Winkelewski
1b06de3136 fix inspect.Config usage
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-14 07:11:17 +01:00
Justin Cormack
d9d2a91780
Merge pull request #193 from ijc/bugfix-191
Defer dockerRm until we are finished with the contents
2017-12-13 09:35:13 -08:00
Ian Campbell
307f13b129 Defer dockerRm until we are finished with the contents
This was introduced by #191 but somehow did not trigger either for me in local
testing or in CI.

It did trigger in initial CI of https://github.com/linuxkit/linuxkit/pull/2811
which can be seen at https://linuxkit.datakit.ci/linuxkit/linuxkit/pr/2811?history=1637690296123e9a15307b3a41b290da6e27e7cc
The error is:

    Failed to docker rm container «...»: «...»: aufs: unmount error after retries: «...»: device or resource busy

No doubt because we were still holding an open fd while trying to remove the
container.

Unclear why this didn't repro for me (docker 17.11.0-ce with overlay2) or
whatever CI uses.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-13 11:44:32 +00:00
Ian Campbell
12629fcb96 Bump to containerd v1.0.0
Since that bumps to gogo protobuf v0.5 too do the same.

Note that there are no actual containerd changes here, although there are some
gogo proto ones.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-13 10:12:42 +00:00
Justin Cormack
ebd7228a44
Merge pull request #191 from ijc/reduce-memory-via-tempfiles
Reduce maximum memory usage via tempfiles
2017-12-12 13:27:32 -08:00
Ian Campbell
3045a80c85 Stream docker export directly to consumer
Rather than queueing up into a `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) compared with the previous patch from 2.8G to 110M. The
tar output case goes from 2.1G to 110M also. Overall allocations are ~715M in
both cases.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:41:42 +00:00
Ian Campbell
9f44acf8e3 Generate intermediate image into a temp file
All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:07:46 +00:00
Ian Campbell
9558740c11 Add cpu and mem profiling options
Following https://golang.org/pkg/runtime/pprof/. When attempting to build
images in https://github.com/linuxkit/kubernetes CI the process is mysteriously
being SIGKILL'd, which I think might be down to OOMing due to the resource
limits placed on the build container.

I haven't done so yet but I'm intending to use these options to investigate and
they seem potentially useful in any case, even if this turns out to be a
red-herring.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 15:54:48 +00:00
Justin Cormack
656bd87fd2
Merge pull request #189 from ijc/image-config-substruct
Split config-related fields of Image into a substruct.
2017-11-28 11:59:25 +00:00
Ian Campbell
f9893d9a9b Fixup test after Image → ImageConfig split.
The syntax used for the yaml definitions is changed by the need to include the
substruct in the struct literal.

For the label switch to `ImageConfig` directly, which is actually more correct
in that it avoids spurious `name` and `image` fields in the label.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-27 14:47:21 +00:00
Ian Campbell
aec82c4cdf Split config-related fields of Image into a substruct.
Where "config-related" here means "ones you might find in the
"org.mobyproject.config" label on an image.

By making this new struct an anonymous member of the existing Image struct the
Go json parser does the right thing (i.e. inlines into the parent) when parsing
a complete image (from a yml assembly) by default. The Go yaml library which we
use requires a tag on the anonymous field to achieve the same.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-27 14:16:59 +00:00
Justin Cormack
63a5dedd28
Merge pull request #188 from justincormack/make-tmp-dir
Create tmp dir in case required
2017-11-20 16:59:27 +00:00
Justin Cormack
3389f89c44 Create tmp dir in case required
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-20 16:50:58 +00:00
Justin Cormack
ad2fda5769
Merge pull request #187 from justincormack/build-improvements
Make easier to use as a library
2017-11-20 14:05:25 +00:00