Commit Graph

14877 Commits

Author SHA1 Message Date
Aurélien Bombo
614e21ccfb
Merge pull request #10415 from GabyCT/topic/egreptim
tools/osbuilder/tests: Remove egrep in test images script
2024-10-11 13:47:30 -05:00
Gabriela Cervantes
aae654be80 tools/osbuilder/tests: Remove egrep in test images script
This PR removes egrep command as it has been deprecated and it replaces by
grep in the test images script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-11 17:23:35 +00:00
Dan Mihai
3622b5e8b4 ci: static_sandbox_resource_mgmt for cbl-mariner
Use the configuration used by AKS (static_sandbox_resource_mgmt=true)
for CI testing on Mariner hosts.

Hopefully pod startup will become more predictable on these hosts -
e.g., by avoiding the occasional hotplug timeouts described by #10413.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-10 22:17:39 +00:00
Fabiano Fidêncio
02f5fd94bd
Merge pull request #10409 from fidencio/topic/ci-add-ita_image-and-ita_image_tag
kbs: ita: Ensure the proper image / image_tag is used for ITA
2024-10-10 11:46:26 +02:00
Fabiano Fidêncio
cf5d3ed0d4 kbs: ita: Ensure the proper image / image_tag is used for ITA
When dealing with a specific release, it was easier to just do some
adjustments on the image that has to be used for ITA without actually
adding a new entry in the versions.yaml.

However, it's been proven to be more complicated than that when it comes
to dealing with staged images, and we better explicitly add (and
update) those versions altogether to avoid CI issues.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-10 10:01:33 +02:00
Steve Horsman
0c4a7c8771
Merge pull request #10406 from ChengyuZhu6/fix-unit
agent:cdh: fix unit tests about sealed secret
2024-10-10 08:57:28 +01:00
Fabiano Fidêncio
3f7ce1d620
Merge pull request #10401 from stevenhorsman/kbs-deploy-overlays-update
Kbs deploy overlays update
2024-10-10 09:50:19 +02:00
Fabiano Fidêncio
036b04094e
Merge pull request #10397 from fidencio/topic/build-remove-initrd-mariner-target
build: mariner: Remove the ability to build the marine initrd
2024-10-10 09:44:36 +02:00
ChengyuZhu6
65ecac5777 agent:cdh: fix unit tests about sealed secret
The root cause is that the CDH client is a global variable, and unit tests `test_unseal_env` and `test_unseal_file`
share this lock-free global variable, leading to resource contention and destruction.
Merging the two unit tests into one test_sealed_secret will resolve this issue.

Fixes: #10403

Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
2024-10-10 08:38:06 +08:00
ChengyuZhu6
a992feb7f3 Revert "Revert "agent:cdh: unittest for sealed secret as file""
This reverts commit b5142c94b9.

Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
2024-10-10 08:37:06 +08:00
GabyCT
0cda92c6d8
Merge pull request #10407 from GabyCT/topic/fixbuildk
packaging: Remove unused variable in build kernel script
2024-10-09 16:53:45 -06:00
Gabriela Cervantes
616eb8b19b packaging: Remove unused variable in build kernel script
This PR removes an unused variable in the build kernel script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-09 20:02:56 +00:00
Fabiano Fidêncio
652ba30d4a
build: mariner: Remove the ability to build the marine initrd
As mariner has switched to using an image instead of an initrd, let's
just drop the abiliy to build the initrd and avoid keeping something in
the tree that won't be used.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 21:42:55 +02:00
Fabiano Fidêncio
59e3ab07e4
Merge pull request #10396 from fidencio/topic/ci-mariner-test-using-mariner-image-instead-of-initrd
ci: mariner: Use the image instead of the initrd
2024-10-09 21:39:44 +02:00
stevenhorsman
b2fb19f8f8 versions: Bump KBS version
Bump to the commit that had the overlays changes we want
to adapt to.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-09 17:49:21 +01:00
Fabiano Fidêncio
01a957f7e1
ci: mariner: Stop building mariner initrd
As the mariner image is already in place, and the tests were modified to
use them (as part of this series), let's just stop building it as part
of the CI.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 18:23:35 +02:00
Fabiano Fidêncio
091ad2a1b2
ci: mariner: Ensure kernel_params can be set
The reason we're doing this is because mariner image uses, by default,
cgroups default-hierarchy as `unified` (aka, cgroupsv2).

In order to keep the same initrd behaviour for mariner, let's enforce
that `SYSTEMD_CGROUP_ENABLE_LEGACY_FORCE=1
systemd.legacy_systemd_cgroup_controller=yes
systemd.unified_cgroup_hierarchy=0` is passed to the kernel cmdline, at
least for now.

Other tests that are setting `kernel_params` are not running on mariner,
then we're safe taking this path as it's done as part of this PR.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 18:23:35 +02:00
Fabiano Fidêncio
3bbf3c81c2
ci: mariner: Use the image instead of the initrd
As an image has been added for mariner as part of the commit 63c1f81c2,
let's start using it in the CI, instead of using the initrd.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 18:23:32 +02:00
Fabiano Fidêncio
9c0c159b25
Merge pull request #10404 from fidencio/topic/rever-sealed-secrets-tests
Revert "agent:cdh: unittest for sealed secret as file"
2024-10-09 18:09:09 +02:00
GabyCT
2035d638df
Merge pull request #10388 from GabyCT/topic/testimtemp
tools/osbuilder/tests: Add trap statement in test images script
2024-10-09 09:49:45 -06:00
Fabiano Fidêncio
b5142c94b9
Revert "agent:cdh: unittest for sealed secret as file"
This reverts commit 31e09058af, as it's
breaking the agent unit tests CI.

This is a stop gap till Chengyu Zhu finds the time to properly address
the issue, avoiding the CI to be blocked for now.

Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
2024-10-09 16:06:09 +02:00
stevenhorsman
8763880e93 tests/k8s: kbs: Update overlays logic
In https://github.com/confidential-containers/trustee/pull/521
the overlays logic was modified to add non-SE
s390x support and simplify non-ibm-se platforms.
We need to update the logic in `kbs_k8s_deploy`
to match and can remove the dummying of `IBM_SE_CREDS_DIR`
for non-SE now

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2024-10-09 09:39:41 +01:00
Gabriela Cervantes
e08749ce58 tools/osbuilder/tests: Add trap statement in test images script
This PR adds the trap statement in the test images script to clean up
tmp files.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-08 19:54:23 +00:00
Fabiano Fidêncio
80196c06ad
Merge pull request #10390 from microsoft/danmihai1/new-rootfs-image-mariner
local-build: add ability to build rootfs-image-mariner
2024-10-08 21:40:43 +02:00
Fabiano Fidêncio
083b2f24d8
Merge pull request #10363 from ChengyuZhu6/secret-as-volume
Support Confidential Sealed Secrets (as volume)
2024-10-08 19:23:40 +02:00
Dan Mihai
63c1f81c23 local-build: add rootfs-image-mariner
Kata CI will start testing the new rootfs-image-mariner instead of the
older rootfs-initrd-mariner image.

The "official" AKS images are moving from a rootfs-initrd-mariner
format to the rootfs-image-mariner format. Making the same change in
Kata CI is useful to keep this testing in sync with the AKS settings.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-08 17:15:56 +00:00
GabyCT
7a38cce73c
Merge pull request #10383 from kata-containers/topic/imagevar
image-builder: Remove unused variable
2024-10-08 10:27:03 -06:00
Aurélien Bombo
e56af7a370
Merge pull request #10389 from emanuellima1/fix-agent-policy
build: Fix RPM build fail due to AGENT_POLICY
2024-10-08 09:59:21 -05:00
ChengyuZhu6
a94024aedc tests: add test for sealed file secrets
add a test for sealed file secrets.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-10-08 16:01:48 +08:00
ChengyuZhu6
fe307303c8 agent:rpc: Refactor CDH-related operations
Refactor CDH-related operations into the cdh_handler function to make the `create_container` code clearer.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-10-08 16:01:48 +08:00
ChengyuZhu6
31e09058af agent:cdh: unittest for sealed secret as file
add unittest for sealed secret as file.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
2024-10-08 16:01:48 +08:00
ChengyuZhu6
974d6b0736 agent:cdh: initialize cdhclient with the input cdh socket uri
Refactor cdh code to initialize cdhclient with the input cdh socket uri.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
2024-10-08 14:58:07 +08:00
ChengyuZhu6
1f33fd4cd4 agent:rpc: handle the sealed secret in createcontainer
Users must set the mount path to `/sealed/<path>` for kata agent to detect the sealed secret mount
and handle it in createcontainer stage.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
2024-10-08 14:58:07 +08:00
ChengyuZhu6
da281b4444 agent:cdh: support to unseal secret as file
Introduced `unseal_file` function to unseal secret as files:
- Implemented logic to handle symlinks and regular files within the sealed secret directory.
- For each entry, call CDH to unseal secrets and the unsealed contents are written to a new file, and a symlink is created to replace the sealed symlink.

Fixes: #8123

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
2024-10-08 14:58:07 +08:00
Fabiano Fidêncio
71d0c46e0a
Merge pull request #10384 from microsoft/danmihai1/virtio-fs-policy
tests: k8s: AUTO_GENERATE_POLICY=yes for local testing
2024-10-07 21:25:52 +02:00
Emanuel Lima
e989e7ee4e
build: Fix RPM build fail due to AGENT_POLICY
By checking for AGENT_POLICY we ensure we only try to read
allow-all.rego if AGENT_POLICY is set to "yes"

Signed-off-by: Emanuel Lima <emlima@redhat.com>
2024-10-07 15:43:23 -03:00
Dan Mihai
6d5fc898b8 tests: k8s: AUTO_GENERATE_POLICY=yes for local testing
The behavior of Kata CI doesn't change.

For local testing using kubernetes/gha-run.sh and AUTO_GENERATE_POLICY=yes:

1. Before these changes users were forced to use:
- SEV, SNP, or TDX guests, or
- KATA_HOST_OS=cbl-mariner

2. After these changes users can also use other platforms that are
configured with "shared_fs = virtio-fs" - e.g.,
- KATA_HOST_OS=ubuntu + KATA_HYPERVISOR=qemu

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2024-10-04 18:26:00 +00:00
Dan Mihai
5aaef8e6eb
Merge pull request #10376 from microsoft/danmihai1/auto-generate-just-for-ci
gha: enable AUTO_GENERATE_POLICY where needed
2024-10-04 10:52:31 -07:00
Gabriela Cervantes
4cd737d9fd image-builder: Remove unused variable
This PR removes an unused variable in the image builder script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-04 15:56:28 +00:00
Greg Kurz
77c5db6267
Merge pull request #9637 from ldoktor/selective-ci
CI: Select jobs by touched code
2024-10-04 11:29:05 +02:00
GabyCT
2d089d9695
Merge pull request #10381 from GabyCT/topic/archrootfs
osbuilder: Remove duplicated arch variable definition
2024-10-03 14:48:08 -06:00
Wainer Moschetta
b9025462fb
Merge pull request #10134 from ldoktor/ci-sort-range
ci.ocp: Sort images according to git
2024-10-03 15:08:41 -03:00
Chelsea Mafrica
9138f55757
Merge pull request #10375 from GabyCT/topic/mktempkbs
k8s:kbs: Add trap statement to clean up tmp files
2024-10-03 12:32:30 -04:00
Gabriela Cervantes
d7c2b7d13c osbuilder: Remove duplicated arch variable definition
This PR removes duplicated arch variable definition in the rootfs script
as this variable and its value is already defined at the top of the
script.

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2024-10-03 16:22:27 +00:00
Greg Kurz
96336d141b
Merge pull request #10165 from pmores/add-network-device-hotplugging
runtime-rs: add network device hotplugging to qemu-rs
2024-10-03 17:44:50 +02:00
Pavel Mores
23927d8a94 runtime-rs: plug in netdev hotplugging functionality and actually call it
add_device() now checks if QEMU is running already by checking if we have
a QMP connection.  If we do a new function hotplug_device() is called
which hotplugs the device if it's a network one.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-03 11:23:10 +02:00
Pavel Mores
ac393f6316 runtime-rs: implement netdev hotplugging for qemu-rs
With the helpers from previous commit, the actual hotplugging
implementation, though lengthy, is mostly just assembling a QMP command
to hotplug the network device backend and then doing the same for the
corresponding frontend.

Note that hotplug_network_device() takes cmdline_generator types Netdev
and DeviceVirtioNet.  This is intentional and aims to take advantage of
the similarity between parameter sets needed to coldplug and hotplug
devices reuse and simplify our code.  To enable using the types from qmp,
accessors were added as needed.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-03 11:20:02 +02:00
Pavel Mores
4eb7e2966c runtime-rs: add netdev hotplugging helpers to qemu-rs
Before adding network device hotplugging functionality itself we add
a couple of helpers in a separate commit since their functionality is
non-trivial.

To hotplug a device we need a free PCI slot.  We add find_free_slot()
which can be called to obtain one.  It looks for PCI bridges connected
to the root bridge and looks for an unoccupied slot on each of them.  The
first found is returned to the caller.  The algorithm explicitly doesn't
support any more complex bridge hierarchies since those are never produced
when coldplugging PCI bridges.

Sending netdev queue and vhost file descriptors to QEMU is slightly
involved and implemented in pass_fd().  The actual socket has to be passed
in an SCM_RIGHTS socket control message (also called ancillary data, see
man 3 cmsg) so we have to use the msghdr structure and sendmsg() call
(see man 2 sendmsg) to send the message.  Since qapi-rs doesn't support
sending messages with ancillary data we have to do the sending sort of
"under it", manually, by retrieving qapi-rs's socket and using it directly.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-03 11:15:31 +02:00
Pavel Mores
3f46dfcf2f runtime-rs: don't treat NetworkConfig::index as unique in qemu-rs
NetworkConfig::index has been used to generate an id for a network device
backend.  However, it turns out that it's not unique (it's always zero
as confirmed by a comment at its definition) so it's not suitable to
generate an id that needs to be unique.

Use the host device name instead.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-03 11:12:37 +02:00
Pavel Mores
cda04fa539 runtime-rs: factor setup of network device out of QemuCmdLine
Network device hotplugging will use the same infrastructure (Netdev,
DeviceVirtioNet) as coldplugging, i.e. QemuCmdLine.  To make the code
of network device setup visible outside of QemuCmdLine we factor it out
to a non-member function `get_network_device()` and make QemuCmdLine just
delegate to it.

Signed-off-by: Pavel Mores <pmores@redhat.com>
2024-10-03 11:03:32 +02:00