All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.
In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.
Signed-off-by: Ian Campbell <ijc@docker.com>
We are going to phase out the LinuxKit build option, in favour of keeping Docker
or a native Linux build option for CI use cases, as it is faster. So the
hyperkit option that only worked in one very limited use case is not needed.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Continue to allow onboot to have duplicates as we do not run simultaneously
so that is ok (and we number them anyway), but services are run together
so we will get a runtime error if duplicated as this is the containerd/runc
id.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- this is pretty much the smallest change to split this out and it
exposes a few things that can be improved later
- no change to logging yet
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- enable the hyperkit option by default on MacOS
- use it for creating raw disk images
fix#68
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Also keep track of directory creation there, so you can explicitly
set directory permissions if required, and to avoid duplicates.
We should really keep track of files created elsewhere in the build
as well as we still might create some extras, but at least you can
set the write permisisons.
We can add uid, gid support too if required...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This will add a Dockerfile which will build the contents into an
image and then call `tinit` to start it.
This is fairly experimental, but is a prototype for other non
LinuxKit outputs. The container will need to run as `privileged`
as `runc` needs quite a few capabilities and `containerd` needs to
mount.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This is a little ugly in terms of the validation now, but it is a move towards
splitting "build" and "package".
The "tar" output (and soon others) can output direct to a file or to stdout.
Obviously you can only build a single output format like this.
The LinuxKit output formats that build disk images cannot stream as they
have to build whole images. These allow multiple outputs.
In future we will probably change to
```
moby build | moby package
```
or similar, but that is a bit ugly, so currently have a compromise where
there are essentially two output types.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Each section will be appended in order of the CLI, other then
kernel where last specified one wins.
This is useful if you eg want to have a base version for (say)
AWS and GCP and then add your own image on top.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- does not require docker if user has qemu natively, will still fall back to docker
- allow specifying size for fixed size disk images
- add a raw disk output format
- more dogfooding
- marginally slower, but can be improved later
The images used to do the build are cached to make the process quicker.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This removes outputs from yaml, instead you can do
```
moby build -output tar -output qcow2 file.yaml
```
or alternative syntax
```
moby build -output tar,qcow2 file.yaml
```
In future we may change this to be available in a `moby package`
step, but lets try this for now.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Using the label `org.mobyproject.config` will use that JSON
(or yaml, but it is very hard to get yaml into a label as newlines are
not respected) for parameters that are not explicitly set in the yaml file.
Had to change parameter definitions so override behaves as expected.
fix#16
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Add a canonical single tarball output format. This
adds kernel and cmdline to `/boot` where LinuxKit output
formats will find them.
Make the other output formats use that as a base.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
`docker create` will not pull an image so we need an additional fallback.
Rework the pull and trust code so it is in one place to facilitate this.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Also do not require `tar` to be in container, use the standard
image export code that we already have and find the files we
want.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
