Commit Graph

460 Commits

Author SHA1 Message Date
Justin Cormack
2dd65d27b6 Add support for setting containerd namespace in runtime config
This adds a namespace field to override the LinuxKit containerd
default namespace, in case you want to run a container in another
namespace.

Needs a patch in LinuxKit to implement this that I will open soon.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 14:12:15 +00:00
Justin Cormack
978deceed0
Merge pull request #198 from justincormack/annotations
Add support for OCI annotations in config
2018-01-17 12:40:31 +00:00
Justin Cormack
929a837d65 Add support for OCI annotations
Annotations do not do anything by default but get passed through to the runtime,
which can be useful. I never metadata I didn't like...

Also fix sysctl to be a map in the validation, not an array. I can't see any
examples using this in LinuxKit, but this matches OCI so is correct.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 12:26:08 +00:00
Justin Cormack
7918437b72 Make test should be phony
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-17 12:02:32 +00:00
Rolf Neugebauer
57b6e2ab94
Merge pull request #197 from rn/ucode
Add support for Intel ucode
2018-01-15 17:00:03 +00:00
Rolf Neugebauer
1d995e198a docs: Document the 'ucode' kernel option.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:43 +00:00
Rolf Neugebauer
5d5a13526b output: Add support for CPU ucode to the kernel+initrd format
This prepends 'ucode.cpio' to the initrd if present. Padding
should not be necessary as the ucode.cpio should be padded
to the right size.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:43 +00:00
Rolf Neugebauer
1d24454438 output: Add ucode.cpio to the tar-kernel-initrd format
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
2b1a611bab output: Extract ucode if present
For now the backends for the different formats do not yet
use the extracted ucode cpio archive, but '// TODO' are
placed for the backends which should eventually handle it.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
0c1b1def13 build: Add ucode to the intermediate tar ball if specified
This extends the kernel filter to also look for the CPU microcode
file if specified in the YAML. If found, the ucode cpio archive
is placed into the intermediate tar file as '/boot/ucode.cpio'.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Rolf Neugebauer
b4704b8ba1 config: Add 'ucode' to the kernel schema
This optional option will allow users to specify a CPU
microcode cpio archive to be prepended to the initrd file.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-15 16:49:42 +00:00
Justin Cormack
f816553d2f
Merge pull request #195 from ijc/prepend-rootfs-mounts
Prepend the rootfs mounts to the user specified runtime.mounts.
2018-01-11 12:15:18 +00:00
Ian Campbell
0a6d69d2d3 Prepend the rootfs mounts to the user specified runtime.mounts.
User specified mounts should be able to rely on the rootfs being mounted, in
particular for a writeable container they should expect the writeable overlay
to already be in place.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-11 11:42:25 +00:00
Justin Cormack
1aca4eefa5
Merge pull request #190 from ijc/containerd-v1.0.0-rc.0
Bump to containerd v1.0.0
2018-01-04 11:28:10 +00:00
Justin Cormack
b5615a3714
Merge pull request #192 from w9n/fix_empty_container_config
allow ImageInspect.Config to be nil when parsing
2018-01-04 11:27:42 +00:00
Justin Cormack
41d67e3bf3
Merge pull request #194 from w9n/yml_omitempty
add omitempty for yml config
2018-01-04 11:26:54 +00:00
Robin Winkelewski
175fadb3d4 add omitempty for yml config
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-27 03:08:44 +01:00
Robin Winkelewski
1b06de3136 fix inspect.Config usage
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2017-12-14 07:11:17 +01:00
Justin Cormack
d9d2a91780
Merge pull request #193 from ijc/bugfix-191
Defer dockerRm until we are finished with the contents
2017-12-13 09:35:13 -08:00
Ian Campbell
307f13b129 Defer dockerRm until we are finished with the contents
This was introduced by #191 but somehow did not trigger either for me in local
testing or in CI.

It did trigger in initial CI of https://github.com/linuxkit/linuxkit/pull/2811
which can be seen at https://linuxkit.datakit.ci/linuxkit/linuxkit/pr/2811?history=1637690296123e9a15307b3a41b290da6e27e7cc
The error is:

    Failed to docker rm container «...»: «...»: aufs: unmount error after retries: «...»: device or resource busy

No doubt because we were still holding an open fd while trying to remove the
container.

Unclear why this didn't repro for me (docker 17.11.0-ce with overlay2) or
whatever CI uses.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-13 11:44:32 +00:00
Ian Campbell
12629fcb96 Bump to containerd v1.0.0
Since that bumps to gogo protobuf v0.5 too do the same.

Note that there are no actual containerd changes here, although there are some
gogo proto ones.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-13 10:12:42 +00:00
Justin Cormack
ebd7228a44
Merge pull request #191 from ijc/reduce-memory-via-tempfiles
Reduce maximum memory usage via tempfiles
2017-12-12 13:27:32 -08:00
Ian Campbell
3045a80c85 Stream docker export directly to consumer
Rather than queueing up into a `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) compared with the previous patch from 2.8G to 110M. The
tar output case goes from 2.1G to 110M also. Overall allocations are ~715M in
both cases.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:41:42 +00:00
Ian Campbell
9f44acf8e3 Generate intermediate image into a temp file
All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 16:07:46 +00:00
Ian Campbell
9558740c11 Add cpu and mem profiling options
Following https://golang.org/pkg/runtime/pprof/. When attempting to build
images in https://github.com/linuxkit/kubernetes CI the process is mysteriously
being SIGKILL'd, which I think might be down to OOMing due to the resource
limits placed on the build container.

I haven't done so yet but I'm intending to use these options to investigate and
they seem potentially useful in any case, even if this turns out to be a
red-herring.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-06 15:54:48 +00:00
Justin Cormack
656bd87fd2
Merge pull request #189 from ijc/image-config-substruct
Split config-related fields of Image into a substruct.
2017-11-28 11:59:25 +00:00
Ian Campbell
f9893d9a9b Fixup test after Image → ImageConfig split.
The syntax used for the yaml definitions is changed by the need to include the
substruct in the struct literal.

For the label switch to `ImageConfig` directly, which is actually more correct
in that it avoids spurious `name` and `image` fields in the label.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-27 14:47:21 +00:00
Ian Campbell
aec82c4cdf Split config-related fields of Image into a substruct.
Where "config-related" here means "ones you might find in the
"org.mobyproject.config" label on an image.

By making this new struct an anonymous member of the existing Image struct the
Go json parser does the right thing (i.e. inlines into the parent) when parsing
a complete image (from a yml assembly) by default. The Go yaml library which we
use requires a tag on the anonymous field to achieve the same.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-27 14:16:59 +00:00
Justin Cormack
63a5dedd28
Merge pull request #188 from justincormack/make-tmp-dir
Create tmp dir in case required
2017-11-20 16:59:27 +00:00
Justin Cormack
3389f89c44 Create tmp dir in case required
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-20 16:50:58 +00:00
Justin Cormack
ad2fda5769
Merge pull request #187 from justincormack/build-improvements
Make easier to use as a library
2017-11-20 14:05:25 +00:00
Justin Cormack
6395e339b9 Do not error if cache/config directory not specified
Just use the default. Easier to use as an external library.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-20 13:52:53 +00:00
Justin Cormack
e3a7739f42 Make easier to embed in LinuxKit
Split out util file in same way.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-20 12:04:08 +00:00
Justin Cormack
6ba3288963
Merge pull request #186 from gesellix/fix-docker-build-cmd
Fix docker format build example
2017-11-19 10:11:16 +00:00
Tobias Gesellchen
5aebd207d0 Fix docker format build example
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
2017-11-18 22:22:33 +01:00
Justin Cormack
315e5f80e8
Merge pull request #185 from justincormack/fix-make
Stop make test from removing binary
2017-11-17 15:21:08 +00:00
Justin Cormack
f85e019d2e
Merge pull request #183 from ijc/static-builds
Build a static binary
2017-11-17 14:43:08 +00:00
Justin Cormack
8f75b635ec Stop make test from removing binary
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-17 14:40:49 +00:00
Ian Campbell
532b41ff8b Disable CGO so as to build a static binary
https://github.com/linuxkit/kubernetes/pull/1#issuecomment-345253075 lead to
https://circleci.com/gh/linuxkit/kubernetes/15

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-17 14:39:19 +00:00
Justin Cormack
eceb6d11f8
Merge pull request #181 from justincormack/runtime-cgroups
Add Cgroups to runtime config
2017-11-17 14:12:08 +00:00
Justin Cormack
c9bb1ecdf1 Add Cgroups to runtime config
This will create the list of cgroups specified

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-17 13:22:38 +00:00
Justin Cormack
91311b9de8
Merge pull request #178 from tristanls/master
Prob: Files section does not work for /var dir
2017-11-16 17:20:33 +00:00
Justin Cormack
7c3e0e87b2
Merge pull request #182 from ijc/circleci-artefacts
Tweak circleci config to try and fix artifact upload
2017-11-16 17:19:58 +00:00
Ian Campbell
c180946fb4 Publish a checksum of the binaries.
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-16 17:15:06 +00:00
Ian Campbell
fff4f0ce79 Try and put dist artifacts at the top level.
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-16 17:13:41 +00:00
Ian Campbell
e6bdd72158 Tweak circleci config to try and fix artifact upload
It appears that the `$GOPATH` in `working_directory` is being treated as a literal
`GOPATH` at least when processing the `state_artifacts.path`. Inlining it seems
to have worked, at the cost of some duplication.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-16 17:13:11 +00:00
Tristan Slominski
11c90c4cdb Prob: Files section does not work for /var dir
Solv: Updated documentation to point out limits of
files section regarding /var, /run, and /tmp dirs.

Signed-off-by: Tristan Slominski <tristan.slominski@gmail.com>
2017-11-14 15:08:07 -06:00
Justin Cormack
a14db39956
Merge pull request #180 from ehazlett/poule-config
Initial poule config
2017-11-14 17:20:38 +00:00
Evan Hazlett
36c5bf55ed
add initial poule config
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2017-11-14 12:16:54 -05:00
Justin Cormack
b262aaa058
Merge pull request #177 from arm64b/docker-efi-images
EFI image hash value update
2017-11-14 16:04:34 +00:00