Canonical TDX release is not needed for vanilla Ubuntu 25.10 but
GRUB_CMDLINE_LINUX_DEFAULT needs to contain `nohibernate` and
`kvm_intel.tdx=1`
Signed-off-by: Szymon Klimek <szymon.klimek@intel.com>
Use grep_pod_exec_output to retry possible failing "kubectl exec"
commands. Other tests have been hitting such errors during CI in
the past.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This adds an integration test to verify that privileged containers work
properly when deploying Kata with kata-deploy.
This is a follow-up to #11878.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Let's rely on kata-deploy setting up the nydus snapshotter for us,
instead of doing this with external code.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
This allows us to stop setting up the snapshotter ourselves, and just
rely con kata-deploy to do so.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's expose the EXPERIMENTAL_SETUP_SNAPSHOTTER script environment
variable to our chart, allowing then users of our helm chart to take
advantage of this experimental feature.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
We may deploy in scenarios where we want to have both snapshotters set
up, sometimes even for simple test on which one behaves better.
With this in mind, let's allow EXTERNAL_SETUP_SNAPSHOTTER to receive a
comma separated list of snapshotters, such as:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="erofs,nydus"
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Similarly to what's been done for the nydus-snapshotter, let's allow
users to have erofs-snapshotter set up by simply passing:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="erofs".
```
Mind that erofs, although a built-in containerd snapshotter, has system
depdencies that we will *NOT* install and it's up to the admin to do so.
These dependencies are:
* erofs-utils
* fsverity
* erofs module loaded
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
In the previous commit we added the assumption that the
nydus-snapshotter version should be the same in two different places.
Now, with this test, we ensure those will always be in sync.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Let's introduce a new EXPERIMENTAL_SETUP_SNAPSHOTTER environemnt
variable that, when set, allows kata-deploy to put the nydus snapshotter
in the correct place, and configure containerd accordingly.
Mind, this is a stop gap till the nydus-snapshotter helm chart is ready
to be used and behaving well enough to become a weak dependency of our
helm chart. When that happens this code can be deleted entirely.
Users can have nydus-snapshotter deployed and configured for the
guest-pull use case by simply passing:
```
EXPERIMENTAL_SETUP_SNAPSHOTTER="nydus"
```
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Otherwise we'd end up adding a the file several times, which could lead
to problems when removing the entry, leading to containerd not being
able to start due to an import file not being present.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
The code, how it was, would lead to the following broke command:
`--header "Authorization: Bearer: "`
Let's only expand that part of the command if ${GH_TOKEN} is passed,
otherwise we don't even bother adding it.
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Document that privileged containers with
privileged_without_host_devices=false are not generally supported.
When you try the above, the runtime will pass all the host devices to Kata
in the OCI spec, and Kata will fail to create the container for various
reasons depending on the setup, e.g.:
- Attempting to hotplug uninitialized loop devices.
- Attempting to remount /dev devices on themselves when the agent had
already created them as default devices (e.g. /dev/full).
- "Conflicting device updates" errors.
- And more...
privileged_without_host_devices was originally created to support
Kata [1][2] and lots of people are having issues when it's set to
false [3].
[1] https://github.com/kata-containers/runtime/issues/1568
[2] https://github.com/containerd/cri/pull/1225
[3] https://github.com/kata-containers/kata-containers/issues?q=is%3Aissue%20%20in%3Atitle%20privileged
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We have noticed in the CI that the `gen_init_cpio ...` was returning 255
and breaking the build. Why? I am not sure.
When chatting with Steve, he suggested to split the command, so it'd be
easier to see what's actually breaking. But guess what? There's no
breakage when we split the command.
So, let's try it out and see whether the CI passes after it.
If someone is willing to educate us on this one, please, that would be
helpful! :-)
Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>
Moving the CUDA repo to the top for all essential packages
and adding a repo priority favouring NVIDIA based repos.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Introduce new test case which verifies that openvpn clients and servers
can run as Kata pods and can successfully establish a connection.
Volatile certificates and keys are generated by an initialization
container and injected into the client and server containers.
This scenario requires TUN/TAP support for the UVM kernel.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Co-authored-by: Manuel Huber <manuelh@nvidia.com>
No need to die when a Kind that does not require a policy annotation is
found in a pod manifest. Print an informational message instead.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Currently, use of openvpn clients/servers is not possible in Kata UVMs.
Following error message can be expected:
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such device (errno=19)
To support opevpn scenarios using bridging and TAP, we enable various
kernel networking config options.
Signed-off-by: Manuel Huber <mahuber@microsoft.com>
Manually added "hostPath" to main.txt then regenerated the dictionary
with `./kata-spell-check.sh make-dict`.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This change crystallizes and simplifies the current handling of /dev
hostPath mounts with virtually no functional change.
Before this change:
- If a mount DESTINATION is in /dev and it is a non-regular file on the HOST,
the shim passes the OCI bind mount as is to the guest (e.g.
/dev/kmsg:/dev/kmsg). The container rightfully sees the GUEST device.
- If the mount DESTINATION does not exist on the host, the shim relies on
k8s/containerd to automatically create a directory (ie. non-regular file) on
the HOST. The shim then also passes the OCI bind mount as is to the guest. The
container rightfully sees the GUEST device.
- For other /dev mounts, the shim passes the device major/minor to the guest
over virtio-fs. The container rightfully sees the GUEST device.
After this change:
- If a mount SOURCE is in /dev and it is a non-regular file on the HOST,
the shim passes the OCI bind mount as is to the guest. The container
rightfully sees the GUEST device.
- The shim does not anymore rely on k8s/containerd to create missing mount
directories. Instead it explicitely handles missing mount SOURCES, and
treats them like the previous bullet point.
- The shim no longer uses virtio-fs to pass /dev device major/minor to the
guest, instead it passes the OCI bind mount as is.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
test_add_one_arp_neighbor modifies the root network namespace, so we
should ensure that it does not interfere with normal network setup.
Adding an IP to a device results in automatic routes, which may affect
routing to non-test endpoints. Thus, we change the addresses used in the
test to come from TEST-NET-1, which is designated for tests and usually
not routable.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
list_routes and test_add_one_arp_neighbor have been flaky in the past
(#10856), but it's been hard to tell what exactly is going wrong.
This commit adds debug information for the most likely problem in
list_routes: devices being added/removed/modified concurrently.
Furthermore, it adds the exit code and stderr of the ip command, in case
it failed to list the ARP neighborhood.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The previous code only checked the result of with_nix_path(), not statfs(),
thus leading to an uninitialized memory read if statfs() failed.
No functional change otherwise.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
85f3391bc added the support for TDX QGS port=0 but missed
defaultQgsPort in the default config. defaultQgsPort overrides
user provided tdx_quote_generation_service_socket_port=0.
After this change, defaultQgsPort is not needed anymore since
there's no default: any positive integer is OK and negative or
unset value becomes a parse error.
QEMUTDXQUOTEGENERATIONSERVICESOCKETPORT in the Makefile is used
to provide a sane default when tdx_quote_generation_service_socket_port
gets set in the configuration.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
genpolicy is a developer tool that should be usable on MacOS. Adding it
to the darwin CI job ensures that it can still be built after changes.
On an Apple M2, the output of `uname -m` is `arm64`, which is why a new
case is needed in the arch_to_* functions.
We're not going to cross-compile binaries on darwin, so don't install
any additional Rust targets.
Fixes: #11635
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Most of the kata-types code is reusable across platforms. However, some
functions in the mount module require safe-path, which is Linux-specific
and can't be used on other platforms, notably darwin.
This commit adds a new feature `safe-path` to kata-types, which enables
the functions that use safe-path. The Linux-only callers kata-ctl and
runtime-rs enable this feature, whereas genpolicy only needs initdata
and does not need the functions from the mount module. Using a feature
instead of a target_os restriction ensures that the developer experience
for genpolicy remains the same.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
This commit adds changes to enable fs sharing between host/guest
using virtio-fs when booting a pod VM for testing. This primarily
enables sharing container rootfs for testing container lifecycle
commands.
Summary of changes is as below:
- adds minimal virtiofsd code to start userspace daemon (based on
`runtime-rs/crates/resource/src/share_fs`)
- adds the virtiofs device to the test vm
- prepares and mounts the container rootfs on host
- modifies container storage & oci specs
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Fixing the shellcheck issues first so that they are not coupled to the
subsequent commit introducing Darwin support to the script.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
Auto-generate policy for nginx-deployment pods, instead of hard-coding
the "allow all" policy.
Note that the `busybox_pod` - created using `kubectl run` - still
doesn't have an Init Data annotation, so it is using the default policy
built into the Kata Guest rootfs image file.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Auto-generate agent policy in k8s-liveness-probes.bats, instead of using
the non-confidential "allow all" policy.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
