Commit Graph

16539 Commits

Author SHA1 Message Date
Zvonko Kaiser
8768e08258 gpu: Add embeding service
For a simple RAG pipeline add a embeding service

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
254dbd9b45 gpu: Add Pod spec for NIM llama
Pod spec for the NIM inferencing service

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
568b13400a gpu: Add NIM bats test
We're running a simple NIM container to test if the GPUs
are working properly

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
6188b7f79f gpu: Add run_kubernetes_nv_tests.sh
Replicate what we have for run_tests and run .bats files

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
9a829107ba gpu: Add selector for k8s tests
We want to reuse the current run_tests with GPUs, introduce a var
that will define what to run.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
7669f1fbd1 gpu: Add NVIDIA GPU test block for amd64
Once we have the amd64 artifacts we can run some arm64 k8s tests.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:59 +00:00
Zvonko Kaiser
97d7575d41 gpu: Disable metrics tests
We are not running the metrics tests anyway for now
lets make room to run the GPU tests.

Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
2025-07-30 13:45:58 +00:00
Saul Paredes
1aaaef2134
Merge pull request #11553 from microsoft/danmihai1/genpolicy-cleanup
genpolicy: reduce complexity
2025-07-28 14:32:59 -07:00
Dan Mihai
c11c972465 genpolicy: config layer logging clean-up
Use a simple debug!() for logging the config_layer string, instead of
transcoding, etc.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-07-28 18:30:13 +00:00
Dan Mihai
30bfa2dfcc genpolicy: use CoCo settings by default
- "confidential_emptyDir" becomes "emptyDir" in the settings file.
- "confidential_configMap" becomes "configMap" in settings.
- "mount_source_cpath" becomes "cpath".
- The new "root_path" gets used instead of the old "cpath" to point to
  the container root path..
- "confidential_guest" is no longer used. By default it gets replaced
  by "enable_configmap_secret_storages"=false, because CoCo is using
  CopyFileRequest instead of the Storage data structures for ConfigMap
  and/or Secret volume mounts during CreateContainerRequest.
- The value of "guest_pull" becomes true by default.
- "image_layer_verification" is no longer used - just CoCo's guest pull
  is supported.
- The Request input files from unit tests are changing to reflect the
  new default settings values described above.
- tests/integration/kubernetes/tests_common.sh adjusts the settings for
  platforms that are not set-up for CoCo during CI (i.e., platforms
  other than SNP, TDX, and CoCo Dev).

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-07-28 18:30:13 +00:00
Dan Mihai
94995d7102 genpolicy: skip pulling layers for guest-pull
Skip pulling container image layers when guest-pull=true. The contents
of these layers were ignored due to:
- #11162, and
- tarfs snapshotter support having been removed from genpolicy.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-07-28 18:30:13 +00:00
Dan Mihai
f6016f4f36 genpolicy: remove tarfs snapshotter support
AKS Confidential Containers are using the tarfs snapshotter. CoCo
upstream doesn't use this snapshotter, so remove this Policy complexity
from upstream.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2025-07-28 18:30:10 +00:00
Steve Horsman
077c59dd1f
Merge pull request #11385 from wainersm/ci_make_coco_nontee_required
ci/gatekeeper: make run-k8s-tests-coco-nontee job required
2025-07-28 14:16:23 +01:00
Steve Horsman
74fba9c736
Merge pull request #11619 from kata-containers/install-dependencies-gh-cli
ci: Try passing api token into githubh api call
2025-07-28 13:35:12 +01:00
Xuewei Niu
2a3c8b04df
Merge pull request #11613 from RuoqingHe/clippy-fix-for-libs-20250721
mem-agent: Ignore Cargo.lock
2025-07-28 17:45:29 +08:00
RuoqingHe
3f46347dc5
Merge pull request #11618 from RuoqingHe/fix-dragonball-default-build
dragonball: Fix warnings in default build
2025-07-28 11:24:46 +08:00
Xuewei Niu
e5d5768c75
Merge pull request #11626 from RuoqingHe/bump-cloud-hypervisor-v47
versions: Upgrade to Cloud Hypervisor v47.0
2025-07-28 10:34:45 +08:00
Ruoqing He
4ca6c2d917 mem-agent: Ignore Cargo.lock
`mem-agent` here is now a library and do not contain examples, ignore
Cargo.lock to get rid of untracked file noise produced by `cargo run` or
`cargo test`.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-28 10:32:46 +08:00
Ruoqing He
3ec10b3721 runtime: clh: Re-generate client code against v47.0
Re-generates the client code against Cloud Hypervisor v47.0.

Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 20:44:14 +02:00
Ruoqing He
14e9d2c815 versions: Upgrade to Cloud Hypervisor v47.0
Details of v47.0 release can be found in our roadmap project as
iteration v47.0: https://github.com/orgs/cloud-hypervisor/projects/6.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 20:42:24 +02:00
Xuewei Niu
6f6d64604f
Merge pull request #11598 from justxuewei/cgroups 2025-07-25 17:53:03 +08:00
Hyounggyu Choi
860779c4d9
Merge pull request #11621 from Apokleos/enhance-copyfile
runtime-rs: Some extra work to enhance copyfile with sharedfs disabled
2025-07-25 11:27:03 +02:00
Ruoqing He
639273366a dragonball: Gate MmapRegion behind virtio-fs
`MmapRegion` is only used while `virtio-fs` is enabled during testing
dragonball, gate the import behind `virtio-fs` feature.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 09:09:35 +00:00
Ruoqing He
2e81ac463a dragonball: Allow unused to suppress warnings
Some variables went unused if certain features are not enabled, use
`#[allow(unused)]` to suppress those warnings at the time being.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 09:07:19 +00:00
Ruoqing He
5f7da1ccaa dragonball: Silence never read fields
Some fields in structures used for testing purpose are never read,
rename to send out the message.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 09:07:19 +00:00
Ruoqing He
225e6fffbc dragonball: Gate VcpuManagerError behind host-device
`VcpuManagerError` is only needed when `host-device` feature is enabled,
gate the import behind that feature.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 09:07:19 +00:00
Ruoqing He
0502b05718 dragonball: Remove with-serde feature assertion
Code inside `test_mac_addr_serialization_and_deserialization` test does
not actually require this `with-serde` feature to test, removing the
assertion here to enable this test.

Signed-off-by: Ruoqing He <heruoqing@iscas.ac.cn>
2025-07-25 09:05:55 +00:00
Xuewei Niu
60e3679eb7 runtime-rs: Add full cgroups support on host
Add full cgroups support on host. Cgroups are managed by `FsManager` and
`SystemdManager`. As the names impies, the `FsManager` manages cgroups
through cgroupfs, while the `SystemdManager` manages cgroups through
systemd. The two manages support cgroup v1 and cgroup v2.

Two types of cgroups path are supported:

1. For colon paths, for example "foo.slice:bar:baz", the runtime manages
cgroups by `SystemdManager`;
2. For relative/absolute paths, the runtime manages cgroups by
`FsManager`.

vCPU threads are added into the sandbox cgroups in cgroup v1 + cgroupfs,
others, cgroup v1 + systemd, cgroup v2 + cgroupfs, cgroup v2 + systemd, VMM
process is added into the cgroups.

The systemd doesn't provide a way to add thread to a unit. `add_thread()`
in `SystemdManager` is equivalent to `add_process()`.

Cgroup v2 supports threaded mode. However, we should enable threaded mode
from leaf node to the root node (`/`) iteratively [1]. This means the
runtime needs to modify the cgroups created by container runtime (e.g.
containerd). Considering cgroupfs + cgroup v2 is not a common combination,
its behavior is aligned with systemd + cgroup v2, which is not allowed to
manage process at the thread level.

1: https://www.kernel.org/doc/html/v4.18/admin-guide/cgroup-v2.html#threads

Fixes: #11356

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2025-07-25 14:52:55 +08:00
alex.lyn
613dba6f1f runtime-rs: Some extra work to enhance copyfile with sharedfs disabled
As some reasons, it first should make it align with runtime-go, this
commit  will do this work.

Fixes #11543

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-25 11:39:20 +08:00
Xuewei Niu
6aa3517393 tests: Prevent the shim from being killed in k8s-oom test
The actual memory usage on the host is equal to the hypervisor memory usage
plus the user memory usage. An OOM killer might kill the shim when the
memory limit on host is same with that of container and the container
consumes all available memory. In this case, the containerd will never
receive OOM event, but get "task exit" event. That makes the `k8s-oom.bats`
test fail.

The fix is to add a new container to increase the sandbox memory limit.
When the container "oom-test" is killed by OOM killer, there is still
available memory for the shim, so it will not be killed.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2025-07-24 23:44:21 +08:00
Steve Horsman
c762a3dd4f
Merge pull request #11372 from kata-containers/dependabot/cargo/src/dragonball/openssl-af8515b6e0
build(deps): bump the openssl group across 4 directories with 1 update
2025-07-24 13:27:24 +01:00
Fupan Li
fdbe549368
Merge pull request #11547 from Apokleos/virtio-scsi
runtime-rs: support block device driver virtio-scsi within qemu-rs
2025-07-24 18:02:11 +08:00
Xuewei Niu
635272f3e8 runtime-rs: Ignore SIGTERM signal in shim
When enabling systemd cgroup driver and sandbox cgroup only, the shim is
under a systemd unit. When the unit is stopping, systemd sends SIGTERM to
the shim. The shim can't exit immediately, as there are some cleanups to
do. Therefore, ignoring SIGTERM is required here. The shim should complete
the work within a period (Kata sets it to 300s by default). Once a timeout
occurs, systemd will send SIGKILL.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2025-07-24 17:15:15 +08:00
Xuewei Niu
79f29bc523 runtime-rs: QEMU get_thread_ids() returns real vCPU's tids
The information is obtained through QMP query_cpus_fast.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2025-07-24 17:15:15 +08:00
stevenhorsman
475baf95ad ci: Try passing api token into githubh api call
Our CI keeps on getting
```
jq: error (at <stdin>:1): Cannot index string with string "tag_name"
```
during the install dependencies phase, which I suspect
might be due to github rate limits being reduced, so try
to pass through the `GH_TOKEN` env and use it in the auth header.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2025-07-24 08:49:32 +01:00
alex.lyn
b40d65bc1b runtime-rs: support block device driver virtio-scsi within qemu-rs
It is important that we continue to support VirtIO-SCSI. While
VirtIO-BLK is a common choice, virtio-scsi offers significant
performance advantages in specific scenarios, particularly when
utilizing iothreads and with NVMe Fabrics.

Maintaining Flexibility and Choice by supporting both virtio-blk and
virtio-scsi, we provide greater flexibility for users to choose the
optimal storage(virtio-blk, virtio-scsi) interface based on their
specific workload requirements and hardware configurations.

As virtio-scsi controller has been created when qemu vm starts with
block device driver is set to `virtio-scsi`. This commit is for blockdev_add
the backend block device and device_add frondend virtio-scsi device via qmp.

Fixes #11516

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 14:00:02 +08:00
alex.lyn
e683a7fd37 runtime-rs: Change the device_id with block device index
As block device index is an very important unique id of a block device
and can indicate a block device which is equivalent to device_id.
In case of index is required in calculating scsi LUN and reduce
useless arguments within reusing `hotplug_block_device`, we'd better
change the device_id with block device index.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
4521cae0c0 runtime-rs: Support AIO for hotplugging block device within qemu
In this commit, block device aio are introduced within hotplug_block_device
within qemu via qmp and the "iouring" is set the default.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
b4d276bc2b runtime-rs: Handle virtio-scsi within device manager
It should be correctly handled within the device manager when do
create_block_device if the driver_option is virtio-scsi.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
fbd84fd3f4 runtime-rs: Support virtio-scsi device within handle_block_volume
It supports handling scsi device when block device driver is `scsi`.
And it will ensure a correct storage source with LUN.

Fixes #11516

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
57645c0786 runtime-rs: Add support for block device AIO
In this commit, three block device aio modes are introduced and the
"iouring" is set the default.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
40e6aacc34 runtime-rs: Introduce scsi_addr within BlockConfig for SCSI devices
It's used to help discover scsi devices inside guest and also add a
new const value `KATA_SCSI_DEV_TYPE` to help pass information.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:57:00 +08:00
alex.lyn
125383e53c runtime-rs: Add support for configurable block device aio
AIO is the I/O mechanism used by qemu with options:
- threads
  Pthread based disk I/O.
- native
  Native Linux I/O.
- io_uring (default mode)
  Linux io_uring API. This provides the fastest I/O operations on

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-24 11:56:52 +08:00
dependabot[bot]
ef9d960763
build(deps): bump the openssl group across 4 directories with 1 update
Bumps the openssl group with 1 update in the /src/dragonball directory: [openssl](https://github.com/sfackler/rust-openssl).
Bumps the openssl group with 1 update in the /src/runtime-rs directory: [openssl](https://github.com/sfackler/rust-openssl).
Bumps the openssl group with 1 update in the /src/tools/genpolicy directory: [openssl](https://github.com/sfackler/rust-openssl).
Bumps the openssl group with 1 update in the /src/tools/kata-ctl directory: [openssl](https://github.com/sfackler/rust-openssl).


Updates `openssl` from 0.10.72 to 0.10.73
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.72...openssl-v0.10.73)

Updates `openssl` from 0.10.72 to 0.10.73
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.72...openssl-v0.10.73)

Updates `openssl` from 0.10.72 to 0.10.73
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.72...openssl-v0.10.73)

Updates `openssl` from 0.10.72 to 0.10.73
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.72...openssl-v0.10.73)

---
updated-dependencies:
- dependency-name: openssl
  dependency-version: 0.10.73
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: openssl
- dependency-name: openssl
  dependency-version: 0.10.73
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: openssl
- dependency-name: openssl
  dependency-version: 0.10.73
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: openssl
- dependency-name: openssl
  dependency-version: 0.10.73
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: openssl
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-23 15:17:12 +00:00
Fabiano Fidêncio
58925714d2
Merge pull request #11579 from Apokleos/fix-hotplug-blk
runtime-rs: Support hotplugging host block devices within qemu-rs
2025-07-23 11:10:04 +02:00
alex.lyn
a12ae58431 runtime-rs: Support hotplugging host block devices within qemu-rs
Although Previous implementation of hotplugging block device via QMP
can successfully hot-plug the regular file based block device, but it
fails when the backend is /dev/xxx(e.g. /dev/loop0). With analysis about
it, we can know that it lacks the ablility to hotplug host block devices.

This commit will fill the gap, and make it work well for host block
devices.

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2025-07-22 15:40:03 +08:00
Fabiano Fidêncio
acae4480ac
Merge pull request #11604 from fidencio/release/3.19.1
release: Bump version to 3.19.1
2025-07-22 09:00:15 +02:00
Fabiano Fidêncio
0220b4d661 release: Bump version to 3.19.1
As there were a few moderate security vulnerability fixes missed as part
of the 3.19.0 release.

Signed-off-by: Fabiano Fidêncio <fidencio@northflank.com>
2025-07-21 20:09:21 +02:00
Steve Horsman
09efcfbd86
Merge pull request #11606 from kata-containers/dependabot/cargo/src/tools/genpolicy/zerocopy-0.6.6
build(deps): bump zerocopy from 0.6.1 to 0.6.6 in /src/tools/genpolicy
2025-07-21 18:58:56 +01:00
Steve Horsman
9f04d8e121
Merge pull request #11605 from kata-containers/dependabot/cargo/src/tools/kata-ctl/unsafe-libyaml-0.2.11
build(deps): bump unsafe-libyaml from 0.2.9 to 0.2.11 in /src/tools/kata-ctl
2025-07-21 18:50:01 +01:00