All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.
In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.
Signed-off-by: Ian Campbell <ijc@docker.com>
Following https://golang.org/pkg/runtime/pprof/. When attempting to build
images in https://github.com/linuxkit/kubernetes CI the process is mysteriously
being SIGKILL'd, which I think might be down to OOMing due to the resource
limits placed on the build container.
I haven't done so yet but I'm intending to use these options to investigate and
they seem potentially useful in any case, even if this turns out to be a
red-herring.
Signed-off-by: Ian Campbell <ijc@docker.com>
We are going to phase out the LinuxKit build option, in favour of keeping Docker
or a native Linux build option for CI use cases, as it is faster. So the
hyperkit option that only worked in one very limited use case is not needed.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Continue to allow onboot to have duplicates as we do not run simultaneously
so that is ok (and we number them anyway), but services are run together
so we will get a runtime error if duplicated as this is the containerd/runc
id.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- this is pretty much the smallest change to split this out and it
exposes a few things that can be improved later
- no change to logging yet
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- enable the hyperkit option by default on MacOS
- use it for creating raw disk images
fix#68
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This disables the code in LinuxKit's `/bin/rc.init` which attempts to detect an
unconfigured hostname and generate a unique (ish) version from the MAC address.
Anyone who wants a specific fallback hostname can populate `etc/hostname`
through the `files` stanza in their `yml` file.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Also keep track of directory creation there, so you can explicitly
set directory permissions if required, and to avoid duplicates.
We should really keep track of files created elsewhere in the build
as well as we still might create some extras, but at least you can
set the write permisisons.
We can add uid, gid support too if required...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This will add a Dockerfile which will build the contents into an
image and then call `tinit` to start it.
This is fairly experimental, but is a prototype for other non
LinuxKit outputs. The container will need to run as `privileged`
as `runc` needs quite a few capabilities and `containerd` needs to
mount.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- generally people refer to a plain disk image as `raw`
- `gcp` is shorter and it is the only image type supported
- remove `img-gz` as it is not needed. It does not really save space
as you have to build the full image and compress it anyway. On
many platforms the `raw` image will be a sparse file anyway,
even on the Mac soon.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This is a little ugly in terms of the validation now, but it is a move towards
splitting "build" and "package".
The "tar" output (and soon others) can output direct to a file or to stdout.
Obviously you can only build a single output format like this.
The LinuxKit output formats that build disk images cannot stream as they
have to build whole images. These allow multiple outputs.
In future we will probably change to
```
moby build | moby package
```
or similar, but that is a bit ugly, so currently have a compromise where
there are essentially two output types.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
GCP does not recognise the images, even though they appear identical to those made
by libguestfs and work on qemu fine. Their validation code does not like them for some
reason.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Each section will be appended in order of the CLI, other then
kernel where last specified one wins.
This is useful if you eg want to have a base version for (say)
AWS and GCP and then add your own image on top.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- does not require docker if user has qemu natively, will still fall back to docker
- allow specifying size for fixed size disk images
- add a raw disk output format
- more dogfooding
- marginally slower, but can be improved later
The images used to do the build are cached to make the process quicker.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Default to sharing net, ipc, uts namespaces between containers in config.
This makes most sense, as this is normal other than if we want to specifically
isolate system containers, in which case we will specify in config.
- explicitly support the value "new" if you want to isolate
- support the synonym "root" for "host" as in non LinuxKit setups it may
not actually be the host, it will be the current namespace.
- only support "none" as a synonym for "new" for network namespace where it is
carried over from Docker.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
