We use dedicated docker container as builder and we are able to clean
data inside only by re-creating of it. Let's add disk usage and clean
commands for builder.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We cannot build for another arch after building for one arch because of
setting skipBuild to true if one arch found. In other words "linuxkit
pkg build --platforms linux/riscv64,linux/amd64 ..." after "linuxkit pkg
build --platforms linux/amd64 ..." will not build for linux/riscv64
which is not expected.
In general when we check for available images and able to found part of
platforms we do not want to rebuild all of them. So this PR includes
platformsToBuild slice which we fill with platforms we want to build for
.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
It is not easy to use cross-platform build with CGO enabled so lets
allow build without cgo for darwin and use virtualization framework only
if we built with CGO.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
If we cannot open file for some reason it is better to skip it instead
of exit. Also we should skip symlinks and directories.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We should expand the list of supported arches to be able to build them if we want. Without this we will stuck on sending tarball during build for riscv64.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
To be able to identify successive file changes without commit, we should
use their hash in tag alongside with dirty flag
(<ls-tree>-dirty-<content hash>).
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We pull all arches for the image which is suboptimal in terms of storage
consumption. Let's pull only required platforms.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We check only for existence of builder container and do not start it in
case of not running state. We should start it for example after reboot
of node to be able to build something.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We noticed that we use host arch when we want to use previously build
image in oci-layout. Let's use fix on buildkit side and improve test.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We should check if we have args in "FROM" and replace them:
ARG IMAGE=linuxkit/img
FROM ${IMAGE} as src
will be parsed as
FROM linuxkit/img as src
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
We do not allow to load into docker images that are targets another
platform differ from current arch. Assume this is because of no support
of manifest. But we can keep all images in place by adding arch suffix
and using tag without arch suffix to point onto current system arch. It
will help to use images from docker for another arch.
Signed-off-by: Petr Fedchenkov <giggsoff@gmail.com>
This option was previously not available and required postprocessing of a `tar-kernel-initrd` output.
Comparison with `iso-efi`:
`iso-efi` only loads the kernel at boot, and the root filesystem is mounted from the actual boot media (eg, a CD-ROM - physical or emulated). This can often cause trouble (it has for us) for multiple reasons:
- the linuxkit kernel might not have the correct drivers built-in for the hardware (see #3154)
- especially with virtual or emulated CD-ROMs, performance can be abysmal: we saw the case where the server IPMI allowed using a ISO stored in AWS S3 over HTTP...you can imagine what happens when you start doing random I/O on the root fs in that case.
- The ISO image has the root device name baked in (ie, `/dev/sr0`) which fails if for some reason the CD-ROM we're running from doesn't end up using that device, so manual tweaking is required (see #2375)
`iso-efi-initrd`, on the other hand, packs the root filesystem as an initramfs (ie similar to what the raw output does, except that in this case we're preparing an ISO image), so both the kernel and the initramfs are loaded in memory by the boot loader and, once running, we don't need to worry about root devices or kernel drivers (and the speed is good, as everything runs in RAM).
Also, the generated ISO can be copied verbatim (eg with `dd`) onto a USB media and it still works.
Finally, the image size is much smaller compared to `iso-efi`.
IMHO, `iso-efi-initrd` could be used almost anywhere `iso-efi` would be used, or might even supersede it. I can't think of a scenario where one might explicitly want to use `iso-efi`.
Points to consider:
- Not tested under aarch64 as I don't have access to that arch. If the automated CI tests also test that, then it should be fine.
- I'm not sure what to put inside `images.yaml` for the `iso-efi-initrd` image. As it is it works of course (my personal image on docker hub), but I guess it'll have to be some more "official" image. However, that cannot be until this PR is merged, so it's kind of a chicken and egg situation. Please advise.
- I can look into adding the corresponding `iso-bios-initrd` builder if there is interest.
Signed-off-by: Davide Brini <waldner@katamail.com>
This allows multiple build flavors for a single codebase, without
sacrificing reproducible builds. The build-args are set in build.yml,
which is typically under the source control (if it is not, then no
reproducible builds are possible anyways). Meaning that mutating
build-args would result in setting "dirty" flag.
Intended use of this commit is to switch between build flavors by
specifying a different yaml file (presumably also under the version
control) by `-build-yml` option.
Because it is impossible to build a final image from packages in
cache, the test for this feature relies on the `RUN echo $build-arg`
output during the `pkg build` process.
Signed-off-by: Yuri Volchkov <yuri@zededa.com>
