Commit Graph

330 Commits

Author SHA1 Message Date
Zvonko Kaiser
428cc5d586 gpu: Chroot Cleanup
With the newest NVRC we do not need the supported GPUs
anymore.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-01-17 19:27:24 +01:00
Fabiano Fidêncio
33b1f0786e Revert "arm64: Do not use DAX with the rootfs image"
This reverts commit 2acb94ef2d, as we have
a kernel patch approved fixing the issue.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-01-17 19:15:53 +01:00
Zvonko Kaiser
adce41c432 gpu: Bump NVRC Version
The new NVRC version works for CC and non-CC use cases,
no --feature confidential needed anymore.

Bump versions.yaml and adjust deployment instructions.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-01-15 01:51:10 +00:00
Zvonko Kaiser
ffc8725164 gpu: rootfs update decoupling
Remove all the driver build instructions,
sicne those are now done in the kernel target.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2026-01-14 20:45:54 +01:00
Fabiano Fidêncio
2acb94ef2d arm64: Do not use DAX with the rootfs image
Kernel 6.18.x has an issue with DAX, which is not yet fixed upstream:
```
[    0.737679] EXT4-fs (pmem0p1): mounted filesystem 79676804-7c8b-491a-b2a6-9bae3c72af70 ro with ordered data mode. Quota mode: disabled.
[    0.737891] VFS: Mounted root (ext4 filesystem) readonly on device 259:1.
[    0.739119] devtmpfs: mounted
[    0.739476] Freeing unused kernel memory: 1920K
[    0.740156] Run /sbin/init as init process
[    0.740229]   with arguments:
[    0.740286]     /sbin/init
[    0.740321]   with environment:
[    0.740369]     HOME=/
[    0.740400]     TERM=linux
[    0.743162] Unable to handle kernel paging request at virtual address fffffdffbf000008
[    0.743285] Mem abort info:
[    0.743316]   ESR = 0x0000000096000006
[    0.743371]   EC = 0x25: DABT (current EL), IL = 32 bits
[    0.743444]   SET = 0, FnV = 0
[    0.743489]   EA = 0, S1PTW = 0
[    0.743545]   FSC = 0x06: level 2 translation fault
[    0.743610] Data abort info:
[    0.743656]   ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000
[    0.743720]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[    0.743785]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[    0.743848] swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000b9d17000
[    0.743931] [fffffdffbf000008] pgd=10000000bfa3d403, p4d=10000000bfa3d403, pud=1000000040bfe403, pmd=0000000000000000
[    0.744070] Internal error: Oops: 0000000096000006 [#1]  SMP
[    0.748888] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 6.18.4 #1 NONE
[    0.749421] pstate: 004000c5 (nzcv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    0.749969] pc : dax_disassociate_entry.constprop.0+0x20/0x50
[    0.750444] lr : dax_insert_entry+0xcc/0x408
[    0.750802] sp : ffff80008000b9e0
[    0.751083] x29: ffff80008000b9e0 x28: 0000000000000000 x27: 0000000000000000
[    0.751682] x26: 0000000001963d01 x25: ffff0000004f7d90 x24: 0000000000000000
[    0.752264] x23: 0000000000000000 x22: ffff80008000bcc8 x21: 0000000000000011
[    0.752836] x20: ffff80008000ba90 x19: 0000000001963d01 x18: 0000000000000000
[    0.753407] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[    0.753970] x14: ffffbf3154b9ae70 x13: 0000000000000000 x12: ffffbf3154b9ae70
[    0.754548] x11: ffffffffffffffff x10: 0000000000000000 x9 : 0000000000000000
[    0.755122] x8 : 000000000000000d x7 : 000000000000001f x6 : 0000000000000000
[    0.755707] x5 : 0000000000000000 x4 : 0000000000000000 x3 : fffffdffc0000000
[    0.756287] x2 : 0000000000000008 x1 : 0000000040000000 x0 : fffffdffbf000000
[    0.756871] Call trace:
[    0.757107]  dax_disassociate_entry.constprop.0+0x20/0x50 (P)
[    0.757592]  dax_iomap_pte_fault+0x4fc/0x808
[    0.757951]  dax_iomap_fault+0x28/0x30
[    0.758258]  ext4_dax_huge_fault+0x80/0x2dc
[    0.758594]  ext4_dax_fault+0x10/0x3c
[    0.758892]  __do_fault+0x38/0x12c
[    0.759175]  __handle_mm_fault+0x530/0xcf0
[    0.759518]  handle_mm_fault+0xe4/0x230
[    0.759833]  do_page_fault+0x17c/0x4dc
[    0.760144]  do_translation_fault+0x30/0x38
[    0.760483]  do_mem_abort+0x40/0x8c
[    0.760771]  el0_ia+0x4c/0x170
[    0.761032]  el0t_64_sync_handler+0xd8/0xdc
[    0.761371]  el0t_64_sync+0x168/0x16c
[    0.761677] Code: f9453021 f2dfbfe3 cb813080 8b001860 (f9400401)
[    0.762168] ---[ end trace 0000000000000000 ]---
[    0.762550] note: init[1] exited with irqs disabled
[    0.762631] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
```

For now, we limit the rootfs that we ship to ARM64 to not use DAX, in
the future we'll re-enable it as soon as the patch lands on mainstream
kernel.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2026-01-14 11:46:40 +01:00
Mikko Ylinen
99bc0f49cc use-cases: drop Intel QuickAssist instructions
While the use-case of Intel QuickAssist (QAT) accelerated crypto
and/or compression with k8s and Kata Containers is still valid,
the setup instructions are outdated:

Starting with Intel Xeon Gen4 (Sapphire Rapids), QAT driver
stack moved to in-tree drivers without a separete SR-IOV VF
driver.

Drop all the setup instructions but keep the use-cases doc
for reference. Users wanting to enable the use-case, should consult
with Intel QAT Device plugins or Intel QAT DRA driver authors.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2026-01-02 12:14:04 +02:00
Fabiano Fidêncio
923f97bc66 rootfs: Temporarily revert "gpu: Handle root_hash.txt correctly"
This reverts commit e4a13b9a4a, as it
caused some issues with the GPU workflows.

Reverting it is better, as it unblocks other PRs.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2025-12-05 11:47:37 +01:00
Zvonko Kaiser
e4a13b9a4a gpu: Handle root_hash.txt correctly
Updates to the shim-v2 build and the binaries.sh script.
Makeing sure that both variants "confidential" AND
"nvidia-gpu-confidential" are handled.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-02 19:56:19 +01:00
Fabiano Fidêncio
776e08dbba build: Add nvidia image rootfs builds
So far we've only been building the initrd for the nvidia rootfs.
However, we're also interested on having the image beind used for a few
use-cases.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2025-11-27 22:46:07 +01:00
Manuel Huber
3966864376 gpu: introduce devkit build flag
Introduce a new devkit parameter which will produce a rootfs
without chisselling. This results in a larger rootfs with various
packages and binaries being included, for instance, enabling the
use of the debug console.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2025-11-19 15:50:03 +01:00
Manuel Huber
2c9e0f9f4f gpu: add signed-by to package sources
Pin to specific key. CUDA package sources in
/etc/apt/sources.list.d already use a specific key.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2025-11-19 15:50:03 +01:00
Zvonko Kaiser
94abe4fc00 osbuilder: nvrc: Consume NVRC release instead of building it
Let's ensure that we consume NVRC releases straight from GitHub instead
of building the binaries ourselves.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2025-10-31 12:10:20 +01:00
Zvonko Kaiser
5ff218823c gpu: Remove unneeded libraries
The libs in question were added when moving to developer.nvidia.com
but switching back to ubuntu only based builds they are not needed.
Remove them to keep the rootfs as minimal as possible.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-29 08:03:36 +01:00
Zvonko Kaiser
6d9b4059f5 gpu: Add libs for CC
In the case of CC we need additional libraries in the rootfs.
Add them conditionally if type == confidential.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-29 08:03:36 +01:00
Zvonko Kaiser
39848e0983 gpu: rootfs fixes
Build only from Ubuntu repositories do not mix with developer.nvidia.com

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>

Update tools/osbuilder/rootfs-builder/nvidia/nvidia_chroot.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-10-26 19:36:55 +01:00
Fabiano Fidêncio
12a515826d tools: Install Golang from a reliable mirror (follow-up)
Aurélien has moved to a reliable mirror for our tests, but we missed
that our tools Dockerfiles could benefit from the same change, which is
added now.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
2025-10-23 11:15:13 +02:00
Manuel Huber
af34308c83 gpu: remove version suffixes for imex and nscq
This change ensures that the NVIDIA package repository for nvidia-imex
and libnvidia-nspc is being used as source.

The NVIDIA repository does not publish these packages with a -580
version suffix, which made us fall back to the packages from the
Ubuntu repository.

These two packages were recently updated by Ubuntu to depend on
nvidia-kernel-common-580-server (this happened from version
580.82.07-0ubuntu1 to version 580.95.05-0ubuntu1). This conflicts
with nvidia-kernel-common-580 which gets installed by
nvidia-headless-no-dkms-580-open, thus causing a build failure.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2025-10-21 15:42:51 +02:00
Manuel Huber
4ad8c31b5a gpu: build nv rootfs with guest pull support
While the local-build's folder's Makefile dependencies for the
confidential nvidia rootfs targets already declare the pause image
and coco-guest-components dependencies, the actual rootfs
composition does not contain the pause image bundle and relevant
certificates for guest pull. This change ensure the rootfs gets
composed with the relevant files.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2025-10-16 09:20:49 -07:00
Manuel Huber
8221361915 gpu: Use variable to differentiate rootfs variants
With this change we namespace the stage one rootfs tarball name
and use the same name across all uses. This will help overcome
several subtle local build problems.

Signed-off-by: Manuel Huber <manuelh@nvidia.com>
2025-10-15 12:39:44 +02:00
Zvonko Kaiser
b00013c717 kernel: Add KBUILD_SIGN_PIN pass through
This is needed to the kernel setup picks up the correct
config values from our fragments directories.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-10 15:45:34 -04:00
Zvonko Kaiser
37bd5e3c9d gpu: Add kernel CONFIG check
We need to make sure that the kernel we're using has the
correct configs set, otherwise the module signing will not work.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-10 15:45:34 -04:00
Zvonko Kaiser
91739d4425 gpu: PPCIE support DGX like systems
For DGX like systems we need additional binaries and libraries,
enable the Kata AND CoCo use-case.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>

Update tools/osbuilder/rootfs-builder/nvidia/nvidia_rootfs.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-10-09 00:00:12 +00:00
Zvonko Kaiser
7061f64db5 gpu: Fix confidential build
NVRC introduced the confidential feature flag and we
haven't updated the rootfs build to accomodate.
If rootfs_type==confidential user --feature=confidential

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-08 10:01:27 +02:00
Zvonko Kaiser
2260f66339 gpu: Some fixes regarding the rootfs v580
With the 580 driver version we need new dependencies
in the rootfs.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-08 10:01:27 +02:00
Zvonko Kaiser
2693daf503 gpu: Install dcgm export from the CUDA repo
Do not use the repo to install the exporter,
we rely on the version tested with Ubuntu <version>

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-02 18:05:13 +02:00
Zvonko Kaiser
56c6512781 gpu: Bump to noble and rearrange repos
Moving the CUDA repo to the top for all essential packages
and adding a repo priority favouring NVIDIA based repos.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-10-02 18:05:13 +02:00
Zvonko Kaiser
3743eb4cea gpu: Add ligcc for RUST libc=gnul builds
Since we cannot build all components with libc=musl and
static RUSTFLAG we still need to ship libcc for AA or other guest
components.

Without this change the guest components do not work and we see

/usr/local/bin/attestation-agent: error while loading shared
libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-09-26 15:08:58 -04:00
Zvonko Kaiser
e6f12d8f86 gpu: Add latest driver per default
Lets make sure that we use latest driver for CI and release.
There was a sort step missing.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-09-20 23:50:35 +00:00
Fabiano Fidêncio
ad240a39e6 kata-deploy: tools: tests: Use zstd instead of xz
Although the compress ratio is not as optimal as using xz, it's way
faster to compress / uncompress, and it's "good enough".

This change is not small, but it's still self-contained, and has to get
in at once, in order to help bisects in the future.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2025-08-21 19:53:55 +02:00
Fabiano Fidêncio
c32fc409ec rootfs-builder: Bump alpine to 3.22
As we were using a very old non-supported version.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-08-21 19:53:55 +02:00
Zvonko Kaiser
8422411d91 gpu: Add coco guest components
The second stage needs to consider the coco guest components

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-31 17:11:21 +00:00
Zvonko Kaiser
da17b06d28 gpu: Pin toolkit version
New versions have incompatibilites, pin toolkit to a working
version

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-14 22:07:21 +00:00
Zvonko Kaiser
97a4a1574e gpu: Remove gpu-admin-tools
NVRC got a new feature reading the CC mode directly from register

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-14 21:59:31 +00:00
Zvonko Kaiser
c3b2d69452 gpu: NVRC static build
We had the proper config.toml configuration for static builds
but were building the glibc  target and not the musl target.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-03 15:31:00 +00:00
stevenhorsman
d9defd5102 osbuilder: Update image-builder base to f42
Fedora 40 is EoL, and I've seen the registry pull fail
a few times recently, so let's bump to fedora 42 which
has 10 months of support left.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-06-20 20:52:30 +01:00
Hyounggyu Choi
4be261f248 rootfs: Bump rootfs-{image,initrd} to 24.04
Since #11197 was merged, all confidential k8s e2e tests for s390x
have been failing with the following errors:

```
attestation-agent: error while loading shared libraries:
libcurl.so.4: cannot open shared object file
libnghttp2.so.14: cannot open shared object file
```

In line with the update on x86_64, we need to upgrade the OS used
in rootfs-{image,initrd} on s390x.
This commit also bumps all 22.04 to 24.04 for all architectures.
For s390x, this ensures the missing packages listed above are
installed.

Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
2025-06-17 22:03:26 +02:00
Xynnn007
7420194ea8 build: abandon PULL_TYPE build env
Now kata-agent by default supports both guest pull and host pull
abilities, thus we do not need to specify the PULL_TYPE env when
building kata-agent.

Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
2025-06-16 13:53:55 +08:00
Steve Horsman
c8fcda0d73 Merge pull request #11407 from Champ-Goblem/fix/nvidia-rootfs-only-copy-opa-when-agent-policy-enabled
nvidia-rootfs: only copy `kata-opa` if `AGENT_POLICY` is enabled
2025-06-11 13:39:07 +01:00
Champ-Goblem
d6c45027f5 nvidia-rootfs: only copy kata-opa if AGENT_POLICY is enabled
In the nvidia rootfs build, only copy in `kata-opa` if `AGENT_POLICY` is enabled. This fixes
builds when `AGENT_POLICY` is disabled and opa is not built.

Signed-off-by: Champ-Goblem <cameron@northflank.com>
2025-06-11 11:25:10 +02:00
Aurélien Bombo
004c1a4595 Revert "ci: Fix Mariner rootfs build failure"
This reverts commit dfa25a42ff.

The original issue was fixed:

https://github.com/microsoft/azurelinux/issues/13971#issuecomment-2956384627
2025-06-09 14:06:07 -05:00
Aurélien Bombo
dfa25a42ff ci: Fix Mariner rootfs build failure
This implements a workaround for microsoft/azurelinux#13971 to unblock
the CI.

Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
2025-06-09 10:56:10 -05:00
Dan Mihai
65385a5bf9 image: custom guest rootfs image file size alignment
The Guest rootfs image file size is aligned up to 128M boundary,
since commmit 2b0d5b2. This change allows users to use a custom
alignment value - e.g., to align up to 2M, users will be able to
specify IMAGE_SIZE_ALIGNMENT_MB=2 for image_builder.sh.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-06-02 16:15:17 +00:00
Steve Horsman
91f2e97aae Merge pull request #11267 from Rtoax/p001-fix-osbuilder-lib.sh-indent
osbuilder: lib.sh: Fix indent
2025-05-22 09:54:18 +01:00
Jacek Tomasiak
91fb4353f6 osbuilder: ubuntu: Add REPO_COMPONENTS setting
Added variable REPO_COMPONENTS (default: "main") which sets components
used by mmdebstrap for rootfs building.
This is useful for custom image builders who want to include EXTRA_PKGS
from components other than the default "main" (e.g. "universe").

Fixes: #11278
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
2025-05-20 14:01:48 +02:00
Jacek Tomasiak
da6860a632 osbuilder: ubuntu: Expose REPO_URL variables
This exposes REPO_URL and adds REPO_URL_X86_64 which can be set to use
custom Ubuntu repo for building rootfs.
If only one architecture is built, REPO_URL can be set. Otherwise,
REPO_URL_X86_64 is used for x86_64 arch and REPO_URL for others.

Fixes: #11276
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
2025-05-19 12:41:49 +02:00
Jacek Tomasiak
e20fb377fc osbuilder: ubuntu: Switch from multistrap to mmdebstrap
Multistrap requires usrmerge package which was dropped in Ubuntu 24.04
(Noble). Based on details from [0], the rootfs build process was switched
to mmdebstrap.

Some additional minor tweaks were needed around chrony as the version
from Noble has very strict systemd sandboxing configured and it doesn't
work with readonly root by default.

[0] https://lists.debian.org/debian-dpkg/2023/05/msg00080.html

Fixes: #11245
Signed-off-by: Jacek Tomasiak <jtomasiak@arista.com>
Signed-off-by: Jacek Tomasiak <jacek.tomasiak@gmail.com>
2025-05-14 11:46:19 +02:00
Rong Tao
37a16c19d1 osbuilder: lib.sh: Fix indent
Replace 4 spaces to [tab].

Signed-off-by: Rong Tao <rongtao@cestc.cn>
2025-05-13 16:56:54 +08:00
Zvonko Kaiser
445cad7754 gpu: Set the ARCH explicilty for driver builds
Kernel Makefiles changed how to deduce the right arch
lets set it explicilty to enable arm and amd builds.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-05-01 17:13:20 +00:00
Greg Kurz
734e7e8c54 rootfs: Don't remove files from the rootfs by default
Recent PR #10732 moved the deletion of systemd files and units that were
deemed uneccessary by 02b3b3b977 from `image_builder.sh` to `rootfs.sh`.
This unfortunately broke `rootfs.sh centos` and `rootfs.sh -r` as used by
some other downstream users like fedora and RHEL, with the following error :

  Warning  FailedCreatePodSandBox  1s (x5 over 63s)  kubelet
   Failed to create pod sandbox: rpc error: code = Unknown
     desc = CreateContainer failed: Establishing a D-Bus connection

Caused by:
    0: I/O error: Connection reset by peer (os error 104)
    1: Connection reset by peer (os error 104)

This is because the aforementioned distros use dbus-broker [1] that requires
systemd-journald to be present.

It is questionable that systemd units or files should be deemed unnecessary
for _all_ distros but this has been around since 2019. There's now also a
long-standing expectation from CI that `make rootfs && make image` does
remove these files.

In order to accomodate all the expectations, add a `-d` flag to `rootfs.sh`
to delete the systemd files and have `make rootfs` to use it.

[1] https://github.com/bus1/dbus-broker

Reported-by: Niteesh Dubey <niteesh@us.ibm.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
2025-04-17 16:53:05 +02:00
Zvonko Kaiser
497ab9faaf Merge pull request #10999 from zvonkok/rootfs-updates
gpu: Update creation permissions
2025-04-16 10:15:38 -04:00