This will help us to debug issues in the future (and would have helped
in the past as well). :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed, as b1710ee2c0 made the
default agent shipped the one with policy support. However, we simply
didn't update the rootfs to reflect that, causing then an issue to start
the agent as shown by the strace below:
```
open("/etc/kata-opa/default-policy.rego", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
futex(0x7f401eba0c28, FUTEX_WAKE_PRIVATE, 1) = 1
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
tkill(553681, SIGABRT) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=553681, si_uid=1000} ---
+++ killed by SIGABRT (core dumped) +++
```
This happens as the default policy **must** be set when the agent is
built with policy support, but the code path that copies that into the
rootfs is only triggered if the rootfs itself is built with
AGENT_POLICY=yes, which we're now doing for both confidential and
non-confidential cases.
Sadly this was not caught by CI till we the cache was not used for
rootfs, which should be solved by the previous commit.
Fixes: #9630, #9631
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is to add an info for files at `tools/packaging/kata-deploy/local-build/*
to a version of the components and ensure that the cached artefacts are not used
when the files of interest are updated.
Fixes: #9630
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
the `tdx_not_supported_warning` function does not exists, the
`tdx_not_supported` should be called instead.
Fixes: #9628
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Using a debugger with the kata runtime is complicated, but it can be done
and can be very useful.
This commits provides a helper script that simplifies it, and updates
the developper's documentation to explain how to use it.
Signed-off-by: Julien Ropé <jrope@redhat.com>
kube-router decided to use :8080 for its metrics, and this seems to be a
change that affected k0s 1.30.0+, leading to kube-router pod crashing
all the time and anything can actually be started after that.
Due to this issue, let's simply use a different port (:9999) and move on
with our tests.
Fixes: #9623
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Whenever we count on having the headers tarball, we must unpack the
cached content into the expected directory, otherwise we'd simply fail,
as we've been failing in our CI, at the end of the process where we
generate the tarball from the cached components.
It's weird to me, sincerely, that the headers tarball end up in such
weird place (build/kernel-nvidia-gpu/builddir/), but I'll leave that to
Zvonko to figure out whether something better can be done, as the intuit
of this PR is simply unblock Kata Containers CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
RTC was being built in a wrong fashion on commit #2bc5e3c6e2ab0145fa9e8be95df0d5086c07a517
RTC was being constructed inside the QemuCmdLine struct,
but it should've been built inside the devices vector.
Signed-off-by: Emanuel Lima <emlima@redhat.com>
This was needed when we were using an old (and not maintained anymore)
host stack. Considering what we have as part of the distros, Today,
this can simply be dropped, as I cannot find any reference of this one
being needed in any up-to-date documentation.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit b7cccfa019.
The `private=on` bit has never made its way upstream, and was removed
from the latest iteration that we're using. With that in mind, let's
revert its usage in the code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 582b5b6b19.
The `private=on` bit has never made its way upstream, and was removed
from the latest iteration that we're using. With that in mind, let's
revert its addition, and later on its usage in the code.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Here we're checking the distro's `/etc/os-release` or
`/usr/lib/os-release` in order to get which distro we're deploying the
Kata Containers artefacts to, and then to properly adjust the QEMU and
OVMF with TDX support that's been shipped with the distros.
Together with that, we're also printing the instructions provided by the
distro on how to enable and use TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the PLACEHOLDER_FOR_DISTRO_{QEMU,OVMF}_WITH_TDX_SUPPORT
variables instead of actually setting a path, so we can easily replace
those as part of our deployment scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll need to have access to the host os-release file (either under
`/etc/os-release` or under `/usr/lib/os-release`), and the simplest
approach that comes to my mind to do is doing what a debug pod would do,
mounting `/` as `/host` and then allowing us to have access to those
files, and then corectly set the TDX specific QEMU and OVMF (TDVF) paths
for the tdx available configurations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We haven't been using nor testing with td-shim, as Cloud Hypervisor does
not officially support TDX yet, and TDVF is supposed to be used with
QEMU, instead of td-shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's remove everything related to the TDX specific QEMU building /
shipping from our repo, as we'll be relying on the one coming from the
distros.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's remove everything related to the TDVF building / shipping from our
repo, as we'll be relying on the one coming from the distro.
Later on, we may need to re-add TDVF logic, as we're already using
upstream edk2 repo / content, but when that's needed we'll simply revert
this commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's skip those tests on TEEs as we've been facing a reasonable amount
of issues, most likely on the containerd side, related to pulling the
image on the guest.
Once we're able to fix the issues on containerd, we can get back and
re-enable those by reverting this commit.
The decision of disabling the tests for TEEs is because the machines may
end up in a state where human intervention is necessary to get them back
to a functional state, and that's really not optimal for our CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the first step of the work to start relying on the artefacts
coming from the distros (CentOS 9 Stream, and Ubuntu) themselves.
Let's have this first one merged, as this will not run the CI due to the
changes being on the yaml itself, and then follow-up with the changes
needed on other parts of the project (kata-deploy, runtime, etc).
Fixes: #9590 -- part I
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Everytime I create contianer on arm64 machine, containerd/kata logs a redundant warning
as follows:
``` shell
time="2024-05-07" level=warning msg="<nil>" arch=arm64 name=containerd-shim-v2
pid=xxx sandbox=fdd1f05 source=virtcontainers/hypervisor
```
I added an error statement so that the error would be logged when it occurs.
Signed-off-by: cncal <flycalvin@qq.com>
This PR removes oci information from versions file as this is not
longer being used in kata containers repository.
Fixes#9599
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We should init and asign the runtime instance to runtime
handler, otherwise, if the pause container failed to start,
which means the runtime instance failed to start, then the
following delete & shutdown request wouldn't be run, thus
the dead shim would be left.
Fixes: #9597
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
dragonball reserves 2048G of mmio space for the pci root bus by default
on physical addresses greater than 4G. However, for some machines with
smaller physical address widths, such as 39-bit wide physical addresses,
dragonball reserves the mmio space when initializing the memory. It is
less than 2048G, so this commit dynamically calculates and allocates the
mmio size of each pci root bus.
Fixes: #9509
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
The CH v39 upgrade in #9575 is currently blocked because of a bug in the
Mariner host kernel. To address this, we temporarily tweak the Mariner
CI to use an Ubuntu host and the Kata guest kernel, while retaining the
Mariner initrd. This is tracked in #9594.
Importantly, this allows us to preserve CI for genpolicy. We had to
tweak the default rules.rego however, as the OCI version is now
different in the Ubuntu host. This is tracked in #9593.
This change has been tested together with CH v39 in #9588.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
The build message shows that yq was not found when I tried to build
runtime binaries, but I've actually installed yq by yum install.
Signed-off-by: cncal <flycalvin@qq.com>
- The testVersionString logic use regex to check that the ociVersion is
displayed correctly, but with the new go module that version has a
`+` in, so we need to quote this to escape special characters
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
Dependabot doesn't follow all our commit format guidelines,
so add a check and skip these if the author is `dependabot[bot]`
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
With the addition of the 'qemu-coco-dev' runtimeClass we no longer need
to run CoCo tests on non-TEE environments with 'qemu'. As a result the
tests also no longer need to set the "io.katacontainers.config.hypervisor.image"
annotation to pods.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created the runtimeclasses/kata-qemu-coco-dev.yaml file and updated the list
of SHIMS.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created a new configuration to configure Kata for CoCo without requiring TEE
hardware so to allow developers implement/test/debug platform agnostic code
on their workstations. It will also ease testing of CoCo features on CI with
non-TEE supported VMs.
This is based off qemu configuration. The following differences applied:
- switched to confidential guest image/initrd
- switched to confidential kernel
- switched to 9p shared_fs
Fixes#9487
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Now that the `kata-agent` is being built with policy support, let's stop
building the `kata-opa-agent`, reducing the amount of things we need to
test and maintain.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the OPA binary is not required anymore, let's start shipping
the agent with the policy enabled by default.
The agent *without* policy enabled has 30MB, while it's 34MB *with* the
policy enabled.
This 4MB (~10%) increase is, IMHO, worth it in order to reduce the
amount of components we have to maintain and test, including the
possibility to also reduce the amount of possible rootfs / initrd
images.
Whoever wants to use the agent without policy enabled can simply do that
by building their own agent. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since OPA binary was replaced by the regorus crate, we can finally stop
building and shipping the binary.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we have an issue with a golang version for `run-cri-containerd`,
it is required to bump the language.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
iommu_platform support was already added on initial DeviceVhostUserFs
introduction, however it incorrectly enabled iommu_platform also on
non-CCW (e.g. PCI) systems.
Signed-off-by: Pavel Mores <pmores@redhat.com>
iommu_platform is only turned on for CCW systems.
PartialEq is added to VirtioBusType to enable the '==' operator.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The adding itself is done by a new function add_iommu() that conforms with
the add_*() convention. Note though that this function is called
internally, by the QemuCmdLine constructor, simply because there's nothing
to trigger its invocation from QemuInner (unlike the other add_*()
functions so far).
Signed-off-by: Pavel Mores <pmores@redhat.com>
Remove reminder to initialize Policy earlier, because currently there
are no plans to initialize earlier.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
When docker is installed on the host system using script from https://get.docker.com/ it automatically creates a docker group with gid=999.
Then during docker build process of tarball, eg. make qemu-tdx-experimental-tarball docker is also installed inside the image with the same
script, which also automatically adds docker group with gid=999.
Then, the build tries to add a new group docker_on_host with gid=999, which already exists, which breaks the build.
Signed-off-by: Jakub Ledworowski <jakub.ledworowski@intel.com>
Avoid auto-generating Policy on platforms that haven't been tested
yet with auto-generated Policy.
Support for auto-generated Policy on these additional platforms is
coming up in future PRs, so the tests being fixed here were
prematurely enabled.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This should only be done once, and if CRI-O restarts, there's a big
chance kata-deploy will also restart and the user would end up with a
file that looks like:
```
[crio]
log_level = "debug"
[crio]
log_level = "debug"
[crio]
log_level = "debug"
...
```
And that would simply cause CRI-O to not start.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Implement Agent Policy using the regorus crate instead of the OPA
daemon.
The OPA daemon will be removed from the Guest rootfs in a future PR.
Fixes: #9388
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Remove k8s-policy-set-keys.bats in preparation for using the regorus
crate instead of the OPA daemon for evaluating the Agent Policy. This
test depended on sending HTTP requests to OPA.
Fixes: #9388
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Lock anyhow version to 1.0.58 because:
- Versions between 1.0.59 - 1.0.76 have not been tested yet using
Kata CI. However, those versions pass "make test" for the
Kata Agent.
- Versions 1.0.77 or newer fail during "make test" - see
https://github.com/kata-containers/kata-containers/issues/9538.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
In case of block devices using virtio-block, we need to pass the
pci-path as the storage source field to the agent.
Current the virt-path is being passed which works just for mmio block
devices.
In the future when support is added for scsi, block-ccw and pmem
devices, the storage source would need to be handled accordingly.
Fixes: #9034
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit is to add a new CI job to run-k8s-tests-on-zvsi.yaml.
Why the job is not configured in run-kata-coco-tests.yaml by having it
integrated with `run-k8s-tests-coco-nontee` is:
- It uses k3s instead of AKS
- It runs on a self-hosted runner
These differences make the integrated job not easy to read and maintain
when it comes to incorporating other platforms in the near future.
Fixes: #9467
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In do_create_container and do_exec_process, we should create the proc_io first,
in case there's some error occur below, thus we can make sure
the io stream closed when error occur.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Fixes: #9334
In linux, when a FIFO is opened and there are no writers, the reader
will continuously receive the HUP event. This can be problematic.
To avoid this problem, we open stdin in write mode and keep the stdin-writer
We need to open the stdout/stderr as the read mode and keep the open endpoint
until the process is delete. otherwise,
the process would exit before the containerd side open and read
the stdout fifo, thus runD would write all of the stdout contents into
the stdout fifo and then closed the write endpoint. Then, containerd
open the stdout fifo and try to read, since the write side had closed,
thus containerd would block on the read forever.
Here we keep the stdout/stderr read endpoint File in the common_process,
which would be destroied when containerd send the delete rpc call,
at this time the containerd had waited the stdout read return, thus it
can make sure the contents in the stdout/stderr fifo wouldn't be lost.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In scenario passfd-io, we should wait for stdin to close itself
instead of manually intervening in it.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This patch adds a default implementation for the use_sandbox_pidns
and updates the structs that implement the K8sResource trait to use
the default.
Fixes: #8960
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
- Provide default implementation for get_sandbox_name in K8sResource trait
- Remove default implementation from structs implementing the trait K8sResource
Fixes: #8960
Signed-off-by: Archana Choudhary <archana1@microsoft.com>
concurrently with itself
Based on 3a1461b0a5186a92afedaaea33ff2bd120d1cea0
Previously the tool would use the layers_cache folder for all instances
and hence delete the cache when it was done, interfereing with other
instances. This change makes it so that each instance of the tool will
have its own temp folder to use.
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
The new run-k8s-tests-coco-nontee job should be the home of attestation
tests.
Changed run-k8s-tests-coco-nontee to get KBS installed and by the time the
KBS variable is exported in the environment then the attestation tests
will kick in (likewise they will skip in run-k8s-tests-on-aks).
Fixes#9455
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This is to update a document `how-to-run-kata-containers-with-SE-VMs`
on using confidential artifacts to build a secure image.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make `boot-image-se-tarball` use confidential kernel and
initrd instead of vanilla version of artifacts.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
updates golang.org/x/net to newer version that closes some reported
vulnerabilities and security issues
Fixes#9486
Signed-off-by: Adil Sadik <sparky.005@gmail.com>
1. EPOLLHUP events also need to be read and will be got len 0.
2. We should kill the connection when EPOLLERR events are received.
Signed-off-by: Tim Zhang <tim@hyper.sh>
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This is to make a workflow `run-k8s-tests` and `run-cri-containerd`
(s390x and zvsi) run only on the runners labeled by `s390x-large`.
Fixes: #9507
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
- Bump the stalebot action version to v9 as that fixes the
```
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/stale@v8.
```
warning.
Fixes: #9512
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This commit is simply to remove a CI workflow `k8s-cri-containerd-rhel9-e2e-tests`.
Fixes: #9504
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As documented in https://github.com/actions/stale?tab=readme-ov-file#start-date
> The start date is used to ignore the issues and pull requests created before the start date.
> Particularly useful when you wish to add this stale workflow on an existing repository
> and only wish to stale the new issues and pull requests.
As we don't want need to treat PRs older than May 2023 as a special case, then remove this option.
Fixes: #9502
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
We've discussed this over and over. Let's try to get to an agreement here.
I will use this issue to remove the mandatory Issue - PR dependency.
Fixes: #9500
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Configuration file for qemu with runtime-rs was recently renamed.
Doc contains name for old file. This was somehow not caught in the CI
earlier.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Document describes the steps needed to pass an entire Intel Discrete GPU
as well a GPU SR-IOV interface to a Kata Container.
Fixes: #9083
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Pods. Additional policy failures
are injected during CI using other types of K8s resources - e.g.,
using Jobs and Replication Controllers - from separate PRs.
Fixes: #9491
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The commit is to make the OPA build from source working in `ubuntu-rootfs-osbuilder`.
To achieve the goal, the configuration is changed as follows:
- Switch the make target to `ci-build-linux-static` not triggering docker-in-docker build
- Install go in the builder image for s390x and ppc64le
Fixes: #9466
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The new CoCo non-tee job introduced on commit 0d5399ba92 need to read secrets
like AZ_TENANT_ID, so run-kata-coco-tests workflow should inherit the secrets from
the caller workflow.
Fixes#9477
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fixes: #9472
For initrd and image, the related NVIDIA will not use the default targets and we will pin them to a specific release.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In order to build a coco {image,initrd}, it is required to
specify its name and version in versions.yaml. This commit
is to add the configuration for them, respectively.
Fixes: #9470
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`CONFIG_TN3270_TTY` and `CONFIG_S390_AP_IOMMU` are dropped for s390x
in 6.7.x which is used for a confidential kernel.
But they are still used for a vanilla kernel. So we need to add them
to the whitelist.
Fixes: #9465
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This commit expands the VMM matrix for run-cri-containerd,
adding a new item `qemu-runtime-rs` for a test scenario where
the VMM is QEMU and runtime-rs is employed.
This expansion affects the workflows for both x86_64 and s390x platforms.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To make `qemu-runtime-rs` working for CI, we have to rename a configuration
template file and `CONFIG_FILE_QEMU` in Makefile.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`qemu-runtime-rs` will be utilized to handle a test scenario where
the VMM is QEMU and runtime-rs is employed.
Note: Some of the tests are skipped. They are going to be reintegrated in
the follow-up PR (Check out #9375).
Fixes: #9371
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Linux kernel generates a panic when the init process exits.
The kernel is booted with panic=1, hence this leads to a
vm reboot.
When used as a service the kata-agent service has an ExecStop
option which does a full sync and shuts down the vm.
This patch mimicks this behavior when kata-agent is used as
the init process.
Fixes: #9429
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
isClhRunning uses signal 0 to test whether the process is
still alive or not. This doesn't work because the process is a
direct child of the shim. Once it is dead the process becomes
zombie.
Since no one waits for it the process lingers until
its parent dies and init reaps it. Hence sending signal 0 in
isClhRunning will always return success whether the process is
dead or not.
This patch calls wait to reap the process, if it succeeds that
means it is our child process, if not we send the signal.
Fixes: #9431
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Replication Controllers. Additional
policy failures will be injected using other types of K8s resources
- e.g., using Pods and/or Jobs - in separate PRs.
Fixes: #9463
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
genpolicy is a handy tool to use in CI systems, to prepare workloads
before applying them to the Kubernetes API server. However, many modern
build systems like Bazel or Nix restrict network access, and rightfully
so, so any registry interaction must take place on localhost.
Configuring certificates for localhost is tricky at best, and since
there are no privacy concerns for localhost traffic, genpolicy should
allow to contact some registries insecurely. As this is a runtime
environment detail, not a target environment detail, configuring
insecure registries does not belong into the JSON settings, so it's
implemented as command line flags.
Fixes: #9008
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
By passing --overwrite-existing to `aks get-credentials` it will stop
asking if I want to overwrite the existing credentials. This is handy
for running the scripts locally.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
When running on non-TEE environments (e.g. KATA_HYPERVISOR=qemu) the tests should
be stressing the CoCo image (/opt/kata/share/kata-containers/kata-containers-confidential.img)
although currently the default image/initrd is built to be able to do guest-pull as well.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Enabled guest-pull tests on non-TEE environment. It know requires the SNAPSHOTTER environment
variable to avoid it running on jobs where nydus-snapshotter is not installed
Fixes: #9410
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Created the new run-k8s-tests-coco-nontee jobs for running CoCo tests on
non-TEE. It currently generates the run-k8s-tests-coco-nontee(qemu, nydus, guest-pull)
job only to run the guest-pull tests.
Fixes: #9410
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the function to uninstall kbs client command function
specially when we are running with baremetal devices.
Fixes#9460
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
CRIs don't always use a pause container, but even if they do the
concrete container choice is not specified. Even if the CRI config can
be tweaked, it's not guaranteed that registries in the public internet
can be reached. To be portable across CRI implementations and
configurations, the genpolicy user needs to be able to configure the
container the tool should append to the policy.
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
Decouple initialization of the Settings struct from creating the
AgentPolicy struct, so that the settings are available for evaluating,
extending or overriding command line arguments.
Signed-off-by: Markus Rudy <webmaster@burgerdev.de>
This patch introduces a one-time cpath to mitigate the cgroup residuals. It
might break the device cgroup merging rules when the cgroup has children.
Fixes: #9456
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR defines the PULL_TYPE variable to avoid failures of unbound
variable when this is being test it locally.
Fixes#9453
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Auto-generate the policy and then simulate attacks from the K8s
control plane by modifying the test yaml files. The policy then
detects and blocks those changes.
These test cases are using K8s Jobs. Additional policy failures
will be injected using other types of K8s resources - e.g., using
Pods and/or Replication Controllers - in future PRs.
Fixes: #9406
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR adds onednn test to exercise additional ML benchmarks.
Onednn is an Intel-optimized library for Deep Neural Networks.
Fixes: #9390
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds openvino test in order to exercise additional ML
benchmarks.
OpenVino bench used to optimize and deploy deep learning models.
Fixes: #9389
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Include HTTP and HTTPS env variables in the building docker
images because they are required to download packages
such as Phoronix.
Added a restriction that verifies that docker building images
is performed as root.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Adds a function that receives as a single parameter the name of
a valid Kata configuration file which will be established as
the default kata configuration to start kata containers.
Adds a second function that returns the path to the current
kata configuration file.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Add an extra function that updates kata config
to use the max num. of vcpus available and
to use the available memory in the system.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This is actually a first attempt to document our CI, and all this
content was based on the document created by Fabiano Fidencio (kudos to
him). We are just moving the content and discussion from Google Docs to
here.
I used the "poetic license" to add some notes on what I believe our CI
will look like in the future.
Fixes#9006
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Beraldo Leal <bleal@redhat.com>
- Add v1 image test case
- Install protobuf-compiler in build check
- Reset containerd config to default in kubernetes test if we are testing genpolicy
- Update docker_credential crate
- Add test that uses default pull method
- Use GENPOLICY_PULL_METHOD in test
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR improves the kbs_k8s_delete function to verify that the
resources were properly deleted for baremetal environments.
Fixes#9379
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add optional toggle to use existing containerd installation to pull and manage container images.
This adds support to a wider set of images that are currently not supported by standard pull method,
such as those that use v1 manifest.
Fixes: #9144
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
- Set KBC_PROVIDER and ATTESTER rather than TEE_PLATFORM
to avoid tss build issues for vTPM attester(s)
- There are future plans to make a matching TEE_PLATFORM, so this can be simplified once that is available
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- At the moment we aren't supporting ppc64le or
aarch64 for
CoCo, so filter out these tests from running
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Switch to Ubuntu 20.04 for building guest-components as
The rootfs is based on 20.04, so we need matching GLIBC versions.
See #8955
- Add dependencies needed by TDX verifier as we want to build for all platforms
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Add basic test case to check that a ruuning
pod can use the api-server-rest (and attestation-agent
and confidential-data-hub indirectly) to get a resource
from a remote KBS
Fixes#9057
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Co-authored-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Support different pause images in the guest for guest-pull, such as k8s
pause image (registry.k8s.io/pause) and openshift pause image (quay.io/bpradipt/okd-pause).
Fixes: #9225 -- part III
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This commit is a mess, but I'm not exactly sure what's the best way to
make it less messy, as we're getting QEMU TDX to work while partially
reverting 1e34220c41.
With that said, let me cover the content of this commit.
Firstly, we're reverting all the changes related to
"memory-backend-memfd-private", as that's what was used with the
previous host stack, but it seems it
didn't fly upstream.
Secondly, in order to get QEMU to properly work with TDX, we need to
enforce the 'private=on' knob and use the "memory-backend-ram", and
we're doing so, and also making sure to test the `private=on` newly
added knob.
I'm sorry for the confusion, I understand this is not optimal, I just
don't see an easy path to do changes without leaving the code broken
during those changes.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
This fixes all remaining sites.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
As explained at [1] :
> The contents of an Artifact are uploaded together into an immutable
> archive. They cannot be altered by subsequent jobs. Both of these
> factors help reduce the possibility of accidentally corrupting
> Artifact files.
This means that artifacts cannot have the same name.
Adapt the `run-k8s-tests-on-garm` workflow accordingly by embedding all
the other `${{ vmm.* }}` fields and `${{ inputs.tag }}` in the artifact
names that would otherwise collide.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
`Node.js 19` is deprecated. Bump to a new version based on `Node.js 20`.
As explained at [1] :
> The contents of an Artifact are uploaded together into an immutable
> archive. They cannot be altered by subsequent jobs. Both of these
> factors help reduce the possibility of accidentally corrupting
> Artifact files.
This means that artifacts cannot have the same name.
Adapt all `build-kata-static-tarball` workflows accordingly by
embedding `${{ matrix.asset }}` in the artifact names that would
otherwise collide.
Fixes#9245
Signed-off-by: Greg Kurz <groug@kaod.org>
Occasionally, the removal of GITHUB_WORKSPACE fails for self-hosted runners
because one of the subdirectories is not empty. This is likely due to another
process occupying the directory at the time.
Implementing a secondary cleanup resolves this issue.
This commit focuses on the implementation for the secondary cleanup.
Fixes: #9317
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This reverts commit d1b54ede29.
Conflicts:
src/runtime/virtcontainers/qemu.go
This commit was a hack that was needed in order to get QEMU + TDX to
work atop of the stack our CI was running on. As we're moving to "the
officially supported by distros" host OS, we need to get rid of this.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The private=on|off knob is required in order to properly lauunch a TDX
guest VM.
This is a brand new property that is part of the still in-flight patches
adding TDX support on QEMU.
Please, see:
3fdd8072da
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the QEMU to the one that's officially maintained by Intel
till all the TDX patches make their way upstream.
We've had to also update python to explicitly use python3 and add
python3-venv as part of the dependencies.
Fixes: #8810
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR improves the latency test cleanup in order to avoid random
failures of leaving the pods.
Fixes#9418
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There's an rg name duplication situation that got introduced by #9385
where 2 different test runs might have same rg name.
Add back uniqueness by including the first letter of GENPOLICY_PULL_METHOD to
cluster name.
Fixes: #9412
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
We used to utilize go runtime's "NumCPUs()", which will give the number
of cores available to the Go runtime, which may be a subset of physical
cores if the shim is started from within a cpuset. From the function's
description:
"NumCPU returns the number of logical CPUs usable by the current
process."
As an example, if containerd is run from within a smaller CPUset, the
maximum size of a pod will be dictated by this CPUset, instead of what
will be available on the rest of the system.
Since the shim will be moved into its own cgroup that may have a
different CPUset, let's stick with checking physical cores. This also
aligns with what we have documented for maxVCPU handling.
In the event we fail to read /proc/cpuinfo, let's use the goruntime.
Fixes: #9327
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
This PR defines the GH_PR_NUMBER variable in gha run k8s common
script to avoid failures like unbound variable when running
locally the scripts just like the GHA CI.
Fixes#9408
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
updating the machine config takes even longer than 1200s, use 60m to be
sure everything is updated.
Fixes: #9338
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Try to reduce duplicated code in decrease_attach_count with public
new function do_decrease_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Try to reduce duplicated code in increase_attach_count with public
new function do_increase_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Introduce a dedicated public function do_decrease_count to
reduce duplicated code in drivers' decrease_attach_count.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Since there are many implementations of reference counting in the
drivers, all of which have the same implementation, we should try
to reduce such duplicated code as much as possible. Therefore, a
new function is introduced to solve the problem of duplicated code.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Use the "allow all" policy for k8s-sandbox-vcpus-allocation.bats,
instead of relying on the Kata Guest image to use the same policy
as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-nginx-connectivity.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-volume.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-sysctls.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-security-context.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-seccomp.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-projected-volume.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-pod-quota.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-optional-empty-secret.bats,
instead of relying on the Kata Guest image to use the same policy as
its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-measured-rootfs.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-liveness-probes.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-inotify.bats, instead of relying
on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-guest-pull-image.bats, instead of
relying on the Kata Guest image to use the same policy as its default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-footloose.bats, instead of
relying on the Kata Guest image to use the same policy as its
default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Use the "allow all" policy for k8s-empty-dirs.bats, instead of
relying on the Kata Guest image to use the same policy as its
default.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Check from:
- k8s-exec-rejected.bats
- k8s-policy-set-keys.bats
if policy testing is enabled or not, to reduce the complexity of
run_kubernetes_tests.sh. After these changes, there are no policy
specific commands left in run_kubernetes_tests.sh.
add_allow_all_policy_to_yaml() is moving out of run_kubernetes_tests.sh
too, but it not used yet. It will be used in future commits.
Fixes: #9395
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add GENPOLICY_PULL_METHOD that will be used to test pulling
container images in genpolicy using the oci-distribution crate
and/or the containerd interface.
GENPOLICY_PULL_METHOD will start being used in a future PR.
Fixes: #9384
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This PR removes conmon information from versions.yaml as this is not
longer being used in kata containers repository.
Fixes#9396
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Don't add the "allow all" policy to all the test YAML files anymore.
After this change, the k8s tests assume that all the Kata CI Guest
rootfs image files either:
- Don't support Agent Policy at all, or
- Include an "allow all" default policy.
This relience/assumption will be addressed in a future commit.
Fixes: #9395
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Configure the system to mount cgroups-v2 by default during system boot
by the systemd system, We must add systemd.unified_cgroup_hierarchy=1
parameter to kernel cmdline, which will be passed by kernel_params in
configuration.toml.
To enable cgroup-v2, just add systemd.unified_cgroup_hierarchy=true[1]
to kernel_params.
Fixes: #9336
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
When there's a pod with multiple containers, there may be case that
attach point more than 2, we should not return Err in that case when
we are doing attach ops, but just return Ok.
Fixes: #8738
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This reverts commit 1c5693be86.
Avoid apparent infinite loop when ReadStreamRequest is blocked by
policy - for some of the pods.
When running the k8s-limit-range.bats test with Policy enabled,
the Shim + VMM never get terminated on my cluster. Not sure why
the sandbox clean-up works better for other tests, but the
k8s-limit-range test pod gets stuck in an infinite loop:
stdout io stream copy error happens: error = %wrpc error: code =
PermissionDenied desc = \"ReadStreamRequest is blocked by policy
...
policy check: ReadStreamRequest
...
stdout io stream copy error happens: error = %wrpc error: code =
PermissionDenied desc = \"ReadStreamRequest is blocked by policy
...
policy check: ReadStreamRequest
...
Fixes: #9380
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The 3.3.0 release installs the `kata-manager` script with overly restrictive
permissions (see #9373), so add details to help users handle the situation.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR removes the runc version information as this is not longer being used
in the kata containers scripts.
Fixes#9364
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Since we're removing the unused service_offload parameter,
don't set it in any of the packaging scripts.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Since we no longer use the service_offload configuration,
remove the ServiceOffload field from the image struct.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
These experimental options were added 2 years ago
in anticipation of features that would be added
in CoCo. These do not match the features that were
eventually added and will soon be ported to main.
Fixes: #8047
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
This is the report from `make check`:
```
error: this loop never actually loops
--> src/signal.rs:147:9
|
147 | / loop {
148 | | select! {
149 | | _ = handle => {
150 | | println!("INFO: task completed");
... |
156 | | }
157 | | }
| |_________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#never_loop
= note: `#[deny(clippy::never_loop)]` on by default
```
There is only one option: you get something or a timeout. You never retry, so
the report is correct.
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
The lint report is the following:
```
error: `flatten()` will run forever if the iterator repeatedly produces an `Err`
--> src/rpc.rs:1754:10
|
1754 | .flatten()
| ^^^^^^^^^ help: replace with: `map_while(Result::ok)`
|
note: this expression returning a `std::io::Lines` may produce an infinite number of `Err` in case of a read error
--> src/rpc.rs:1752:5
|
1752 | / reader
1753 | | .lines()
| |________________^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#lines_filter_map_ok
= note: `-D clippy::lines-filter-map-ok` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::lines_filter_map_ok)]`
```
This commit simply applies the suggestion.
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Running `make check` in the `src/agent` directory gives:
```
error: you seem to use `.enumerate()` and immediately discard the index
--> rustjail/src/mount.rs:572:27
|
572 | for (_index, line) in reader.lines().enumerate() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unused_enumerate_index
= note: `-D clippy::unused-enumerate-index` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unused_enumerate_index)]`
help: remove the `.enumerate()` call
|
572 | for line in reader.lines() {
| ~~~~ ~~~~~~~~~~~~~~
Checking tokio-native-tls v0.3.1
Checking hyper-tls v0.5.0
Checking reqwest v0.11.18
error: could not compile `rustjail` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101
```
Fixes: #9342
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
support to configure CreateContainerRequestTimeout in the
configurations.
e.g.:
[runtime]
...
create_container_timeout = 300
Note: The effective timeout is determined by the lesser of two values: runtime-request-timeout from kubelet config
(https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout) and create_container_timeout.
In essence, the timeout used for guest pull=runtime-request-timeout<create_container_timeout?runtime-request-timeout:create_container_timeout.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Most of the content of `docs/Stable-Branch-Strategy.md` got de-facto
deprecated by the re-design of the release process described in #9064.
Remove this file and all its references in the repo.
The `## Versioning` section has some useful information though. It is
moved to `docs/Release-Process.md`. The documentation of the `PATCH`
field is adapted according to new workflow.
Fixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
Support to configure CreateContainerRequestTimeout in the annotations.
e.g.:
annotations:
"io.katacontainers.config.runtime.create_container_timeout": "300"
Note: The effective timeout is determined by the lesser of two values: runtime-request-timeout from kubelet config
(https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#:~:text=runtime%2Drequest%2Dtimeout) and create_container_timeout.
In essence, the timeout used for guest pull=runtime-request-timeout<create_container_timeout?runtime-request-timeout:create_container_timeout.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
In the situation to pull images in the guest #8484, it’s important to account for pulling large images.
Presently, the image pull process in the guest hinges on `CreateContainerRequest`, which defaults to a 60-second timeout.
However, this duration may prove insufficient for pulling larger images, such as those containing AI models.
Consequently, we must devise a method to extend the timeout period for large image pull.
Fixes: #8141
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR updates the journal log name for nerdctl artifacts to make
sure that we have different names in case we add a parallel GHA job.
Fixes#9357
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
the service might not listen on the default port, use the full service
address to ensure we are talking to the right resource.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
this can be used on kcli or other systems where cluster nodes are
accessible from all places where the tests are running.
Fixes: #9272
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The automated release workflow starts with the creation of the release in
GitHub. This is followed by the build and upload of the various artifacts,
which can be very long (like hours). During this period, the release appears
to be fully available in https://github.com/kata-containers/kata-containers/
even though it lacks all the artifacts. This might be confusing for users
or automation consuming the release.
Create the release as draft and clear the draft flag when all jobs are
done. This ensure that the release will only be tagged and made public
when it is fully usable.
If some job fails because of network timeout or any other transient
error, the correct action is to restart the failed jobs until they
eventually all succeed. This is by far the quicker path to complete
the release process.
If the workflow is *canceled* for some reason, the draft release is left
behind. A new run of the workflow will create a brand new draft release
with the same name (not an issue with GitHub). The draft release from
the previous run should be manually deleted. This step won't be automated
as it looks safer to leave the decision to a human.
[1] https://github.com/kata-containers/kata-containers/releasesFixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
The remaining code in network.rs was mostly moved to utils.rs which seems
better home for these utility functions anyway (and a closely related
function open_named_tuntap() has already lived there).
ToString implementation for Address was removed after some consideration.
Address should probably ideally implement Display (as per RFC 565) which
would also supply a ToString implementation, however it implements Debug
instead, probably to enable automatic implementation of Debug for anything
that Address is a member of, if for no other reason. Rather than having
two identical functions this commit simply switches to using the Debug
implementation for printing Address on qemu command line.
Fixes#9352
Signed-off-by: Pavel Mores <pmores@redhat.com>
Make file descriptors to be passed to qemu owned by QemuCmdLine. See
commit 52958f17cd for more explanation.
Signed-off-by: Pavel Mores <pmores@redhat.com>
add_network_device() doesn't need to be passed NetworkInfo since it
already has access to the full HypervisorConfig.
Also, one of the goals of QemuCmdLine interface's design is to avoid
coupling between QemuCmdLine and the hypervisor crate's device module,
if at all possible. That's why add_network_device() shouldn't take
device module's NetworkConfig but just parts that are useful in
add_network_device()'s implementation.
Signed-off-by: Pavel Mores <pmores@redhat.com>
is_running_in_vm() is enough to figure out whether to disable_modern but
it's clumsy and verbose to use. should_disable_modern() streamlines the
usage by encapsulating the verbosity.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This commit replaces the existing NetDevice-based implementation with one
using Netdev and DeviceVirtioNet.
Signed-off-by: Pavel Mores <pmores@redhat.com>
In keeping with architecture of QemuCmdLine implementation we split the
functionality into two objects: Netdev to represent and generate the
-netdev part and DeviceVirtioNet for the -device virtio-net-<transport>
part.
This change is a pure refactor, existing functionality does not change.
However, we do remove some stub generalizations and govmm-isms, notably:
- we remove the NetDev enum since the only network interface types that
kata seems to use with qemu are tuntap and macvtap, both of which are
implemented by the same -netdev tap
- enum DeviceDriver is also left out since it doesn't seem reasonable to
try to represent VFIO NICs (which are completely different from
virtio-net ones) with the same struct as virtio-net
- we also remove VirtioTransport because there's no use for it so far, but
with the expectation that it will be added soon.
We also make struct Netdev the owner of any vhost-net and queue file
descriptors so that their lifetime is tied ultimately to the lifetime of
QemuCmdLine automatically, instead of returning the fds to the caller and
forcing it to achieve the equivalent functionality but manually.
Signed-off-by: Pavel Mores <pmores@redhat.com>
generate_netdev_fds() takes NetworkConfig from which it however only needs
a host-side network device name. This commit makes it take the device name
directly, making the function useful to callers who don't have the whole
NetworkConfig but do have the requisite device name.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The idea of this function is to make sure O_CLOEXEC is not set on file
descriptors that should be inherited by a child (=hypervisor) process.
The approach so far is however rather heavy-handed - clearing *all* flags
is unjustifiably aggresive for a low-level function with no knowledge of
context whatsoever.
This commit refactors the function so that it only does what's expected
and renames it accordingly. It also clarifies some of its call sites.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Since v2.2.6 it can detect TDX guests on Azure, so let's bump it even if
Azure peer-pods are not currently used as part of our CI.
Fixes: #9348
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kata CI has full debug output enabled for the cbl-mariner k8s tests,
and the test AKS node is relatively slow. So debug prints from policy
are expensive during CI.
Fixes: #9296
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The lib.sh script uses the right directory but the wrong path for the
script that installs yq; fix it.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Remove links to tests repo and update with corresponding location in the
current repo.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change scripts and source that uses files in the tests repo to use the
corresponding file in the current repo.
Fixes#9165
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Now that the version is an invariant for the entire workflow, it
isn't required to obtain it with an environment variable. Just
rely on the content of the `VERSION` file like other actions.
Fixes#9064 - part VI
Signed-off-by: Greg Kurz <groug@kaod.org>
We need some docs about how to build a guest kernel to support
both Upcall and Nvidia GPU Passthrough(hotplug) at the same time.
This patch is to do such thing to help users to build a guest
kernel with support both Upcall and Nvidia GPU hotplug/unlplug.
Fixes: #9140
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add a document for kata guest image management design.
Related feature: #8484Fixes: #9225 -- part I
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
CONFIG_CRYPTO_ECDSA is not supported in older kernels such as 5.10.x
which may cause building broken problem if we build such kernel with
NVIDIA GPU in version 5.10.x
So this patch is to add CONFIG_CRYPTO_ECDSA into whitelist.conf to
avoid break building guest kernel with NVIDIA GPU.
Fixes: #9140
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Currently, `.lock().await.clone()` results in `Option<ImageService>` being duplicated in memory with each call to `singleton()`.
Consequently, if kata-agent receives numerous image pulling requests simultaneously,
it will lead to the allocation of multiple `Option<ImageService>` instances in memory, thereby consuming additional memory resources.
In image.rs, we introduce two public functions:
`merge_bundle_oci()` and `init_image_service()`. These functions will encapsulate
the operations on `IMAGE_SERVICE`, ensuring that its internal details remain
hidden from external modules such as `rpc.rs`.
Fixes: #9225 -- part II
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
It is observed that virtiofsd exits immediately on s390x
if there is no attached console devices.
This commit resolves the issue by migrating `appendConsole()`
from runtime and being triggered in `start_vm()`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
For s390x, it requires an additional option `memory-backend` for `-machine`.
Otherwise, virtiofsd exits with HandleRequest(InvalidParam).
This commit is to add a field `memory_backend` to `struct Machine`
and turn it on for s390x.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Like nvdimm for x86_64, a block device for s390x should be
treated differently with `virtio-blk-ccw`.
This is to generate a QEMU command line parameter for a block
device by using `-blockdev` and `-device` if the `vm_rootfs_driver`
is set to `virtio-blk-ccw`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This version brings in a fix for cleaning up k3s/rke2 environments,
which directly impacts the TDX machine that's part of our CI.
Fixes: #9318
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the unbound variables error when trying to run
the setup script locally in order to avoid errors.
Fixes#9328
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add in the full details once cloud-hypervisor/cloud-hypervisor#6103
has been implemented, and the feature is available in a Cloud Hypervisor
release.
Fixes: #8799
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Currently, `*-pci` is used as an argument for the device config.
It is not true for a case where a different type of bus is used.
s390x uses `ccw`.
This commit is to make it flexible to generate the device argument
based on the bus type. A structure `DeviceVhostUserFsPci` and
`VhostVsockPci` is renamed to `DeviceVhostUserFs` and `VhostVsock`
because the structure name is not bound to a certain bus type any more.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR updates the journal log names for kubernetes artifacts
in order to make sure that we have different names when we are
running parallel GHA jobs.
Fixes#9308
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR terminates the nydus namespace to avoid the error of
that the flag needs an argument.
Fixes#9264
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This reverts commit 0fa59ff94b, as now
we'll be able to use the `--generate-notes`, hopefully, without blowing
the allowed limit.
Fixes: #9064 - part VI
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Added an announcement message to the `static-checks.sh` script. It runs
platform / architecture specific code so it would be useful to display
details of the platform the checker is running on to help with
debugging.
Fixes: #9258.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This release is a special case, as we've slacked for 6 months and the
release content is way too long ... long enough to exceed the allowed
limit for the release notes.
With this in mind we'll just remove the `--generate-notes` for now, and
then revert this commit as soon as the release is out, as releases
should be happening every month and, ideally, we won't reach this
situation never ever again.
Fixes: #9064 - part V
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It has been observed that the runtime stops running around
`sysinfo::total_memory()` while adjusting a config on s390x.
This is to update the crate to the latest version which happened
to resolve the issue. (No explicit release note for this)
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
MTRR, or Memory-Type Range Registers are a group of x86 MSRs providing a way to control access
and cache ability of physical memory regions.
During our test in runtime-rs + Dragonball, we found out that this register support is a must
for passthrough GPU running CUDA application, GPU needs that information to properly use GPU memory.
fixes: #9310
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Those two architectures are not TEE capable, thus we can just skip
running those tests there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed as the TDX machine is hosted inside Intel and relies on
proxies in order to connect to the external world. Not having those set
causes issues when pulling the image inside the guest.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure those two proxy settings are respected, as those will be
widely used when pulling the image inside the guest on the Confidential
Containers case.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When the https_proxy/no_proxy settings are configured alongside agent-policy enabled, the process of pulling image in the guest will hang.
This issue could stem from the instantiation of `reqwest`’s HTTP client at the time of agent-policy initialization,
potentially impacting the effectiveness of the proxy settings during image guest pulling.
Given that both functionalities use `reqwest`, it is advisable to set https_proxy/no_proxy prior to the initialization of agent-policy.
Fixes: #9212
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To access the URL, the component to pull image in the guest needs to send a request to the remote.
Therefore, we need to add CA to the rootfs.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Enable to build kata-agent with PULL_TYPE feature.
We build kata-agent with guest-pull feature by default, with PULL_TYPE set to default.
This doesn't affect how kata shares images by virtio-fs. The snapshotter controls the image pulling in the guest.
Only the nydus snapshotter with proxy mode can activate this feature.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To support handle image-guest-pull block volume from different CRIs, including cri-o and containerd.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
By default the pause image and runtime config will provided
by host side, this may have potential security risks when the
host config a malicious pause image, then we will use the pause
image packaged in the rootfs.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Julien Ropé <jrope@redhat.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Add "guest-pull" feature option to determine that the related dependencies
would be compiled if the feature is enabled.
By default, agent would be built with default-pull feature, which would
support all pull types, including sharing images by virtio-fs and
pulling images in the guest.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
support to pass image information to guest by KataVirtualVolumeImageGuestPullType
in KataVirtualVolume, which will be used to pull image on the guest.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
As we do not employ a forked containerd in confidential-containers, we utilize the KataVirtualVolume
which storing the image information as an integral part of `CreateContainer`.
Within this process, we store the image information in rootfs.storage and pass this image url through `CreateContainerRequest`.
This approach distinguishes itself from the use of `PullImageRequest`, as rootfs.storage is already set and initialized at this stage.
To maintain clarity and avoid any need for modification to the `OverlayfsHandler`,we introduce the `ImagePullHandler`.
This dedicated handler is responsible for orchestrating the image-pulling logic within the guest environment.
This logic encompasses tasks such as calling the image-rs to download and unpack the image into `/run/kata-containers/{container_id}/images`,
followed by a bind mount to `/run/kata-containers/{container_id}`.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Containerd can support set a proxy when downloading images with a environment variable.
For CC stack, image download is offload to the kata agent, we need support similar feature.
Current we add https_proxy and no_proxy, http_proxy is not added since it is insecure.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
With image-rs pull_image API, the downloaded container image layers
will store at IMAGE_RS_WORK_DIR, and generated bundle dir with rootfs
and config.json will be saved under CONTAINER_BASE/cid directory.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Arron Wang <arron.wang@intel.com>
Co-authored-by: Jiang Liu <gerry@linux.alibaba.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: wllenyj <wllenyj@linux.alibaba.com>
This commit updates `filter_k8s_test.sh` to handle skipped tests that
include comments. In addition to the existing parameter expansion,
the following expansions have been added:
- Removal of a comment
- Stripping of trailing spaces
Fixes: #9304
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to place a pre-action step for the kata-deploy job in order to
clean up the github workspace directory before checking out the repo.
Fixes: #9301
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Currently we're only running the small instance tests. This adds the
normal instance tests as well.
Fixes: #9298
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Fix the Docker install by removing the second (erroneous) call to
`containerd_installed()` in `handle_docker()`.
Without this fix, installing using Docker (`-D`) will work *iff* you
already have containerd installed. However, if you do not have
containerd installed, the `containerd_installed()` function returns 1,
which exits the script as we're running with `set -e`, leaving a broken
Docker installation.
> **Note:** containerd is installed via Docker's `get-docker.sh` script.
Fixes: #9292.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Now we don't have minor and major releases and
we are now generating a new version
in the release workflow, we can
tidy up the arch specific releases workflows to remove
the extra required inputs
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This drops the documentation of the legacy release scripts and adds
a quick description of the scripts of the new workflow. It also
highlights the bump of the `VERSION` file.
Signed-off-by: Greg Kurz <groug@kaod.org>
All releases are now created in the `main` branch following
the very same workflow. No need to special case pre-releases.
Signed-off-by: Greg Kurz <groug@kaod.org>
This is done in order to avoid having to push a commit to the main
branch, which is against the defined rules on GitHub.
By doing this, we need to educate ourselves to always bump the VERSION
file as soon as a release is cut out.
As a side effect of this change, we can drop the release-major and
release-minor workflows, as those are not needed anymore.
Fixes: #9064 - part IV
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
File descriptors that are passed to QEMU need some special care.
We want them to be closed when the QEMU process is started. But
at the same time, it is required that the associated rust File
structures, either coming from the` std::fs` or the `tokio::fs`
crates, are still in scope when the QEMU process is forked. This
is currently achieved by keeping File structures in variables
at the outer scope of `start_vm()`. This scheme is currently
duplicated, with similar justifications in the corresponding
comments.
Consolidate all this handling in one place with a more generic
explanation.
Fixes#9281
Signed-off-by: Greg Kurz <groug@kaod.org>
The agent now has a number of optional build-time features that can be
enabled.
Add details of these features to the following areas:
- Version output (`kata-agent --version`)
- Announce message (so that the details are always added to the journal
at agent startup).
- The response message returned by the ttRPC `GetGuestDetails()` API.
Fixes: #9285.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Some previous contribution missed to run cargo clippy.
Fix the dependency now so that it doesn't cause noise
in future contributions.
Signed-off-by: Greg Kurz <groug@kaod.org>
Fixes: #9269
From https://github.com/opencontainers/runtime-spec/blob/main/config.md#mounts
type (string, OPTIONAL) The type of the filesystem to be mounted.
bind may be only specified in the oci spec options -> flags update r#type
The agent will ignore bind mounts if they are only specified in the OCI spec options and not in the flags.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
s390x supports a different machine type `s390-ccw-virtio` and it is
not required to configure cpu features by default for the platform.
A hypervisor `dragonball` is not supported on s390x so that `DBCMD`
is not necessary. `vm-rootfs_driver` should be set to `virtio-blk-ccw`.
This commit is to set the architecture-specific flags for Makefile.
Fixes: #9158
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Initialize $GITHUB_ENV to avoid nounset error when running the scripts locally
out of Github Actions.
Fixed commit 9ba5e3d2a8Fixes#9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The guest_hook_path item in configuration.toml allows OCI hook scripts
to be executed within Kata's guest environment. Traditionally, these
guest hook programs are pre-built and included in Kata's guest rootfs
image at a fixed location.
While setting guest_hook_path = "/usr/share/oci/hooks" in configuration.toml
works, it lacks flexibility. Not all guest hooks reside in the path
/usr/share/oci/hooks, and users might have custom locations.
To address this, a more flexible and configurable approach is to be proposed
that allows users to specify their desired path. This could include using a
sandbox bind mount path for hooks specific to that particular container.
However, The current implementation of guest hooks and bind mounts in kata-agent
has a reversed order of execution compared to the desired behavior.
To achieve the intended functionality, we simply need to swap the order of their
implementation.
Fixes: #9274
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
This PR ensures that the self hosted runner is prepared by taking
necesary actions before running the workflow. The script prepare_runner.sh
checks the following:
1. Ensure that containerd/docker is up and running
2. Make sure that the repository workspace is cleaned up and has no conflicts
3. Remove/cleanup any leftover files from the previous runs
Fixes: #9262
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
we are grepping for an expected message but it might contain extra bits
of information fruitful for later debugging. Let's include it in the
output and the full log in case of an error.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
we're hitting this timeout quite often, looks like newer OCP takes
longer to reconfigure. Increase the timeout to 1200.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This script doesn't serve as a complete cleanup, but it can be used as a
best-effort cleaner between deploying different versions of
kata-containers on the same OCP cluster.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
sometimes we want to test a different than the latest image (eg. when
verifying a PR via ghcr images or when bisecting a failure over older
builds). Let's add a KATA_DEPLOY_IMAGE variable for that while keeping
the latest image by default.
Fixes: #9228
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
previously we only replaced the image when the previously defined one
matched the "old_img". This is good to avoid modifying developers custom
changes, but it might lead to hard-to-debug issues when the image stays
different. Let's ensure we always replace the image with the one we
asked for.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
the latest upstream kata-containers requires the skip_mount_home to be
enabled, which is default on OCP 4.14+ but disabled on OCP 4.13-. Let's
use a "WORKAROUND_9206_CRIO" (called by kata-containers GH issue)
variable to allow users to enable this treatement when needed.
Related to: #9206
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Instead of failing the pipeline let's proceed with an error message that
selinux setup failed so, in case of a later failure, we know what might
have caused it while keeping the coverage in case of a false setup
issue.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
previously we only waited for a random pod to finish the selinux
relabel, which could be error-prone. Let's wait for all of the podst to
contain the expected message.
Increase the timeout to 120s as some pods might take a little bit longer
to finish.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
in case we re-apply the selinux workaround or if user had already
existing similar rule the relabel_selinux was failing. Let's allow it to
modify the existing rules as well to avoid such issues.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The open_named_tuntap function is designed as a public function to
open a tuntap device with the specified name. However, in order to
reference existing methods in dbs_utils, we still need to keep the
reference "path = "../../../dragonball/src/dbs_utils" in dependencies
and cannot hide it.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add network helpers and impl ToQemuParams trait to build
netdev params which are put into cmdline for Qemu VM running.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need ensure the add_network_device happens in netns and
move qemu process into netns which keeps the qemu process
running in this net namespace.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Add network device handler in start_vm, which is sepcially
for Qemu VM running with added net params to command line.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need add a new netns field in struct QemuInner, and
initialize it with argument passed down in prepare_vm().
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The enter_netns function is designed as a public method to help
VMMs running as a independent process enter a network namespace,
reducing duplicate code.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
It just move the related code to a public file(utils.rs) and make
it a common method for both vsock and network, or some others.
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Signed-off-by: Pavel Mores <pmores@redhat.com>
In order to better support non-builtin vmm usage of NetnsGuard and
reduce code duplication, we need to move it to a common path that
can be referenced by both hypervisor and resource manager.
In this patch, it just do moving code from network/utils/netns.rs
to kata-sys-utils/src/netns.rs
Fixes: #8865
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
The PID needs to be initialized before calling isClhRunning.
waitVMM() uses isClhRunning and is called by launchClh() just
before returning from function.
Fixes: #9230
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Use containerd's default environment for container images that don't
specify the Env field.
Also, re-enable policy env variable verification, now that these
uncommon images are supported too.
Fixes: #9239
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
We're trying to keep just the bare minimum info, as we really would like
to not have the list of commits, and mainly the list of new
contributors, trucated from the release notes.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to ensure the release type is passed down to workflows,
otherwise we'll fail to get the correct release version for tagging the
daemonset images.
Fixes: #9064 - part III
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Recently confidential-containers/kbs repository was renamed to
confidential-containers/trustee. Github will automatically resolve the
old URL but we better adjust it in code.
The trustee repository will be cloned to $COCO_TRUSTEE_DIR. Adjusted
file paths and pushd/popd's to use $COCO_KBS_DIR
($COCO_TRUSTEE_DIR/kbs).
On versions.yaml changed from `coco-kbs` to `coco-trustee` as in the
future we might need other trustee components, so keeping it generic.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added the kbs_set_resources_policy() function to set the KBS policy. Also the
kbs_set_allow_all_resources() and kbs_set_deny_all_resources to set the
"allow all" and "deny all" policy, respectively.
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added utility functions to manage resources in KBS:
- kbs_set_resource(), where the resource data is passed via argument
- kbs_set_resource_from_file(), where the resource data is found in a
file
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added kbs_install_cli function to build and install the kbs-client
executable if not present into the system.
Removed the stub from gha-run.sh; now the install kbs-client in the
.github/workflows/run-kata-deploy-tests-on-aks.yaml will effectively
install the executable.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Scripts sourcing ci/lib.sh need to set $GOPATH otherwise it will
fail. This ensure that GOPATH is set to ${HOME}/go unless it is
already exported.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Added functions to return the service host, port or full-qualified
HTTP address, respectively, kbs_k8s_svc_host(), kbs_k8s_svc_port(),
and kbs_k8s_svc_http_addr().
Fixes#9056
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This change updates the module import to use 'util' instead of the deprecated 'io/util'
Fixes: #9166
Signed-off-by: Chungeun Choi <ce.choi@okestro.com>
copyBuffer returns and the streams will be closed when error occurs.
If the error contains "blocked by policy" it means the log output is
disabled by policy with "ReadStreamRequest" and "WriteStreamRequest" set
to false. But at this moment, we want the real stream still working (not
be seen) because we might want to enable logging for debugging purpose,
so we repeat copybuffer in this case to avoid streams being closed.
Fixes: #8797
Signed-off-by: Linda Yu <linda.yu@intel.com>
logging/debugging information might probably be disabled in production
due to security consideration, but we'd better provide an approach for
customer to get logging information during runtime, this PR implement
setpolicy function in kata-runtime tools, although it can set whole policy
other than logging.
setpolicy would evokes remote attestation, which means before setting
policy during runtime, user has to reconfigure new policy hash in KBS/AS.
usage: kata-runtime policy set policy.rego --sandbox-id XXXXXXXX
Fixes: #8797
Signed-off-by: Linda Yu <linda.yu@intel.com>
Disable env variable verification to unblock CI, until container
images that don't specify the Env variables will be handled correctly
(see #9239).
Also, mark the image config Env field as optional, thus allowing
policy generation for these container images.
Fixes: #9240
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Fix the `grep(1)` warning caused by the unnecessary escaping of the
hash/sharp symbol.
Fixes: #9235.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Move the "build from source" and "manual installation" details to the
developer guide. This makes the installation landing page clearer for
users.
Fixes: #9234.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The k3s distribution of k8s uses an embedded version of containerd and
configures it to log to a file, not the journal. Hence, although we
collect the journal as a test artifact, we also need to collect the
actual log files for containerd.
Also collect the k3s containerd config files to help with debugging.
Fixes: #9104.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The jobs running on garm will collect journal information. The data gathered
is based on the time the tests started running. The $start_time is
exported on run_tests() and used in collect_artifacts(). It happens that
run_tests() and collect_artifacts() are called on different steps of the
workflow and the environment variables aren't preserved between them,
i.e, $start_time exported on the first step is not available on the
subsequents.
To solve that issue, let's save $start_time in the file pointed out by
$GITHUB_ENV that Github actions uses to export variables. In case $GITHUB_ENV is
empty then probably it is running locally outside of Github, so it won't
save the start time value.
Fixes#9217
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the general tests documentation in main README of the
kata containers repository.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds general README documentation for the tests section
in the kata containers repository.
Fixes#9209
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add details to the README for `kata-manager` showing how to list
available hypervisor configs (packaged and local), and switch between
the configurations. Also, update the hypervisors page to show a lot more
detail about the hypervisor configurations, including the "short name"
used by `kata-manager` for switching hypervisor config.
> **Note:**
>
> These changes only apply to the current default golang runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add new options to allow the configured hypervisor to be changed:
- `-L`: List available _packaged_ hypervisor config short names.
- `-e`: List available _local_ hypervisor config names.
- `-H <hypervisor>`: Install Kata then switch to the specified hypervisor.
- `-S <hypervisor>`: Switch to the specified hypervisor (by config short name [Errors if Kata not installed]).
For example, to install Kata and configure it to use Cloud Hypervisor
with the golang Kata runtime:
```bash
$ kata-manager.sh -H clh
```
To switch back to the default hypervisor:
```bash
$ kata-manager.sh -S default
```
To show details of the available packaged configs:
```bash
$ kata-manager.sh -L
```
To show details of the local configs:
```bash
$ kata-manager.sh -e
```
> **Notes:**
>
> - This change **only** applies to the current default (golang) Kata runtime.
>
> - Although this is mainly for users wishing to switch hypervisor (by
> changing the Kata config file to another of the packaged config files
> provided for specific hypervisors), strictly it allows users to change
> to _any_ config file. For example, if the user has a config file called
> `/etc/kata-containers/configuration-my-custom-config.toml`, they could
> switch to this by running:
>
> ```bash
> $ kata-manager.sh -S my-custom-config
> ```
>
> - The "config short names" are the hypervisor specific part of the configuration file name.
> For example, the config short name for file `configuration-qemu.toml` is
> `qemu` and the config short name for `configuration-clh.toml` is `clh`.
Fixes: #8305.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The `configure_kata()` function modifies the configuration file to
enable debug. But it was doing this by calling `sed -i` which, by
default, creates a new _file_ from the `configuration.toml` symbolic
link. This defeated the point of the symbolic link which is supposed to
resolve to the local copy of the pristine config file, so we now use
the GNU sed(1) specific `---follow-symlinks` option to retain the
sym-link.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add an info message just before the archive file is checked. This keeps
the user informed about what is happening as it can take a few seconds
to perform the checks on slower systems.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
a) There is some unknown syscalls triggered in new github virt machine
that would break the make test process with SIGSYS after applying
SeccompFilter. In order to fix this, we change the allowlist in this
unit test for seccompfileter into a blocklist to avoid meeting the unknown syscalls.
b) lazy static METRICS is not fully initialize in the unit test and may lead to
unstable result for this UT.
fixes: #9207
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
the mmap region start guest addr hard-code a value and later there
would be check whether the mentioned addr is larger than or equal
to mem_end (default to host_phy_mem >> 1) in order to satisfy the
requirement for DaxMemory. Since github virt machine phy_mem is larger
than previous CI machine we use, the hard-code value could no longer be
worked. To fix this, we change the address to mem_end in unit test to
avoid the influence of host machine change.
fixes: #9207
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
- enable CONFIG_MTRR,CONFIG_X86_PAT on x86_64 for nvidia gpu
- optimize -f of build-kernel.sh, clean old kernel path and config before setup
- add kernel 5.16.x
Fixes: #9143
Signed-off-by: Jimmy-Xu <xjimmyshcn@gmail.com>
PR #8760 tentatively tried to have the shim to run in its own mount
namespace for the sake of improving isolation between the sandbox and
the host. Thus crio storage drivers shouldn't create a PRIVATE
bind mount on their home directory. Otherwise, the container's rootfs
mount wouldn't be propagated to kata runtime's mount namespace, and
kata runtime couldn't access the container's rootfs files.
So, when kata cooperated with crio, crio should set
skip_mount_home=true for its storage overlay.
Fixes: #9028
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
_print_cluster_name() create a string based information like the
pull request number and commit SHA. However, when you are developing the
scripts you might want to use an arbitrary name, so it was introduced
the $AKS_NAME variable that once exported it will overwrite the
generated name.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Until this point the deployed KBS service is only reachable from within
the cluster. This introduces a generic mechanism to apply an Ingress
configuration to expose the service externally.
The first implemened ingress is for AKS. In case the HTTP application
routing isn't enabled in the cluster (this is required for ingress), an
add-on is applied.
It was added the get_cluster_specific_dns_zone() and
enable_cluster_http_application_routing() helper functions
to gha-run-k8s-common.sh.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Introduce the tests/integration/kubernetes/confidential_kbs.sh library
that contains functions to manage the KBS on CI. Initially implemented
the kbs_k8s_deploy() and kbs_k8s_delete() functions to, respectively,
deploy and delete KBS on Kubernetes. Also hooked those functions in the
tests/integration/kubernetes/gha-run.sh script to follow the convention
of running commands from Github Workflows:
$ .tests/integration/kubernetes/gha-run.sh deploy-coco-kbs
$ .tests/integration/kubernetes/gha-run.sh delete-coco-kbs
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Kustomize has been used on some of our internal components (e.g.
kata-deploy) to manage k8s deployments. On CI it has been used
the `sed` tool to edit kustomization.yaml files, but `kustomize` is
more suitable for that purpose. So in order to use that tool on CI
scripts in the future, this commit introduces the `install_kustomize()`
function that is going to download and install the binary in
/usr/local/bin in case it's found on $PATH.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This PR adds the collect artifacts function in gha-run script for
the kubernetes tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR helps to store the artifacts for different logs for k8s tests
on garm.
Fixes#9103
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This backports the:
9060e930caf2d20f413df07778d3ab497493161c
ci.ocp: Add debug output on HTTP service failure
these logs are vital to analyze a setup failure.
a10a1e2c9cbc21afc1e80f22b0fb8634d27cbd8d
ci.ocp: Improve the service-up detection
waiting for the first response is not sufficient as OCP returns html
page without error even when the route is not yet established describing
the issue (why it doesn't reply with 500?). Waiting for the correct
output should do better.
commits from the kata-containers/tests repo.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
This fixes a panic on tracing on container exit.
The root cause is that global var needs to be set by "=" instead of
":=".
Fixes: #9102
Signed-off-by: Liu Bo <liub.liubo@gmail.com>
Currently, the checking for kata-deploy is running assume that the
daemonset scheduled at least one pod, however it might not had and the
kubectl wait command fails due to "error: no matching resources found".
On CI I've observed that fail intermittently. I suspect the service
account kata-deploy-sa take a while to show up then no kata-deploy is
scheduled in meanwhile.
Changed the checker logic to use waitForProcess() to keep testing if it is
already running, or hit the timeout (still 10m).
Fixes#9183
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The step to deploy KBS on run-k8s-tests-on-aks workflow should be
increased so that there is enough time for checking the service is
healthy and exposed. Likewise the step that builds the kbs-client
which requires enough time to build the executable.
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Kubernetes v1.29 introduced a new `PodReadyToStartContainers` condition
that gets inserted at index 0 in the conditions array. This means that
the expected `PodCompleted` reason can now be either at index 0 with
kubernetes v1.28 and older or at index 1 starting with kubernetes v1.29.
This is fragile at best since the `kubectl wait` doesn't allow to combine
multiple checks. Also, checking the reason is dubious as it doesn't really
tell if the pods have actually completed or not.
Check the pod phase to be `Succeeded` instead, this guarantees that :
> All containers in the Pod have terminated in success, and will not
> be restarted.
Fixes#9178
Signed-off-by: Greg Kurz <groug@kaod.org>
This was introduced by a45988766c, but
didn't follow the correct format for the env declaration.
Fixes: #9064 - part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want all payloads to be built and published, regardless if there's a
new PR merged.
This will help people to easily trace / debug issues.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We were never passing the arguments to add the PAUSE_IMAGE to the
rootfs, leading to it never being present in the confidential image /
initrd.
Fixes: #9032 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR implements general fixes to the gha-run script for the
cri-containerd tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR skips the cri-containerd in gha-run script for cloud hypervisor
runtime-rs.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the cri-containerd tests.
Fixes#9181
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR updates the code pr advice document to make the proper
references now that we have move the test repository to the kata containers
repository.
Fixes#9171
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is not needed anymore as we can run the tests from any branch, and
we can patch this locally before doing a test.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GitHub actions is fun and always willing to play tricks with us. This
nice little kid decided that `echo "FOO=\"bar zaz\"" >> $GITHUB_ENV` is
not valid, and it simply breaks things in a way that is a pain to debug.
But hey, we take this path, and after doing so I realised that the
correct way to export that is `echo "FOO=bar zaz" >> $GITHUB_ENV`.
I know, this looks incorrect, but this fellow never stops surprising us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I'm getting here the most relevant parts of what we had as part of the
release-notes.sh script. As the script will not be used anymore, it's
been removed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For some reason we need to force its installation in the GOPATH,
otherwise yq is not found.
Ideally we should switch to a packaged version of yq, but that's a topic
for another series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to adjust the tags as when this workflow ends up being called
from the release side, we'll receive "refs/tags/main" as the
GITHUB_REF, and in that case we must use the release version.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is utterly counter intuitive, but if we change a file during the
GitHub Action, the checkout done for the next workflow won't have that
file updated, but rather the branch on its original state when the
workflow was created.
This makes us safe to always "calculate" the next release version from
the VERSION file at the time the workflow was triggered.
This requires us to have the release type exported for the whole
workflow.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Without those we'll end up running steps in parallel that should
actually wait for a previous step to be completed.
While here, let's also correct some of the "needs" that were waiting fro
the wrong workflow to be finished.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We missed doing this as part of
50011e89a0, but we also need to check for:
* RELEASE_VERSION
* GH_TOKEN
* ARCHITECTURE
* KATA_STATIC_TARBALL
While here, let's fix a ARCHITECURE -> ARCHITECTURE typo.
Fixes: #9064 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Changed the "run k8s tests on AKS" workflows to get the CoCo KBS
installed so that we can run attestation tests.
The plan is to run attestation tests only on a subset of non-TEE jobs
initially, so this commit restricts to install KBS only on kata-qemu
configuration. Actually at this point it is added only stubs commands
to tests/integration/kubernetes/gha-run.sh that should be implemented
in a future commit.
Fixes#9058
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Add all agent configuration options to README so that users can more easily understand
what these options do and how to configure them at runtime.
Fixes: #9109
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This commit should be merged as it's now, then we trigger a test
release, fix whatever has to be fixed, and drop it as soon as we know
our workflows are working as expected.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those are not needed anymore as we're automating our release process
around GitHub actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're not doing backports anymore, as we're getting rid of the stable
branches in favour of having a better release cadence from the main
branch.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're not maintaining a stable branch anymore, let's get rid of the
kata-deploy stable pieces.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those will allow us to cut a release just by a single click, instead of
the current process we have.
Fixes: #9064 -- part I
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The release workflow is now updated to be a `workflow_call`, and it
includes the steps that had to be manually done in the past, such as
updating the needed files and creating the release itself.
While on this, the kata-deploy multiarch manifest tags have been updated
to match the new release scheme.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function returns the version of the next release (the one about to
be cut), and it'll be used as part of our new workflow that will take
care of the release.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a function that will be responsible for bumping the project's
version in the VERSION file, and push it to the branch as part of the
release process that will be introduced.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a helper function that will be used to create a new release as
part of our release process workflow (which will still be modified).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the name of the function says, it's responsible for uploading the
libseccomp source tarballs as par of our release process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As hinted by the name of the function, this is used to generate and
upload the vendored code we have as its own tarball.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the name says, this function will be used to upload the versions.yaml
file during a given release process of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function, as it names says, will be used to upload the
kata-static.tar.xz tarballs generated during the release process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function, as it names says, will be used to publish multiarch
manifests for the Kata Containers CI and Kata Containers releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For now this script does nothing, but we're introducing it in order to
redduce the diffs for the next commits in this series.
My intention is to have as much as possible related to the release as
part of this helper script, and it'll be populated function by function
while replacing content that's "hard coded" (and duplicated) on
different GitHub actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This refactor the teardown() of tests/integration/nydus/nydus_tests.sh:
* Moved boilerplate code that kill process to a loop;
* Doesn't leave teardown() if a process failed to get killed, so that
other clean up routines are ran;
* Check if the pid exist then attempt to kill the process, so avoid this
misleading message:
```
Usage:
kill [options] <pid> [...]
Options:
<pid> [...] send signal to every <pid> listed
-<signal>, -s, --signal <signal>
specify the <signal> to be sent
-q, --queue <value> integer value to be sent with the signal
-l, --list=[<signal>] list all signal names, or convert one to a name
-L, --table list all signal names in a nice table
-h, --help display this help and exit
-V, --version output version information and exit
For more details see kill(1).
```
Fixes#8948
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
It's recommended to avoid images from docker.io to avoid errors related
with hitting the pull limits that happens mostly on bare-metal machines.
So this replaced the docker.io's busybox with
quay.io/prometheus/busybox.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The "run ps command" test has failed once in a while because it doesn't
wait the sh command to start within the container, consequently `ps`
won't report the amount of lines expected.
Fixes#8975
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Migrated runk tests from pure shell script to bats to be consistent with
other test suites.
The install_dependencies() will install the bats tool locally.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Relax the timeout for calling CLH's CreateVM + BootVM APIs. When
hitting the older 1s timeout, killing a half-booted Guest and
retrying the same boot sequence could have been wasteful and resulting
in unstable CI testing on slower Hosts.
Fixes: #9152
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add a link in the release notes to the Kata Container survey, to
advertise it, and hopefully encourage users to take the survey.
Fixes: #9074.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add a "twistie" / arrow (`▶`) that the user can click on to see the full
list of commits _if they want to_.
This way, the release notes become easier to read and we can display
information below the shortlog which would (probably) normally not be
seen due to the huge long list of commits.
Fixes: #9075.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
For consistency with the go runtime.
As the shim itself is not using the network (all its communication with
other processes is done with local unix sockets), there is no reason to
keep gathering and reporting shim-specific network metrics.
Actual network usage of the kata containers can be found from the existing
agent network metrics (kata_guest_netdev_stat).
Signed-off-by: Julien Ropé <jrope@redhat.com>
As part of the shim network metrics, the shim is reporting network interfaces
from the host with no namespace isolation - this gives insight in interfaces
not tied to the kata containers, and causes an increase in resource usage for
kata metrics.
As the shim itself is not using the network (all its communication with
other processes is done with local unix sockets), there is no reason to
keep gathering and reporting shim-specific network metrics.
Actual network usage of the kata containers can be found from the existing
hypervisor network metrics (kata_hypervisor_netdev) and from the agent
network metrics (kata_guest_netdev_stat).
Fixes: #5738
Signed-off-by: Julien Ropé <jrope@redhat.com>
CI failed to deploy nydus snapshotter because it was not cleaned up last time.
So we can try to cleanup nydus snapshotter before deploying it.
Fixes: #9121
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need initailize the pci_hotplug_enabled with true before we do GPU
passthrough with runtime-rs/dragonball. Otherwise it fails with error
`InvalidOperation`.
Fixes: #9129
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
teardown() gets executed after each test case, so there is no need to
clean-up before teardown.
Fixes: #9072
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add set_namespace_to_policy_settings() for changing the pod namespace
in genpolicy settings.
Fixes: #9072
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
When cgroup v2 is in use, a container should only see its part of the
unified hierarchy in `/sys/fs/cgroup`, not the full hierarchy created
at the OS level. Similarly, `/proc/self/cgroup` inside the container
should display `0::/`, rather than a full path such as :
0::/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde291f58_8f20_4d44_aa89_c9e538613d85.slice/crio-9e1823d09627f3c2d42f30d76f0d2933abdbc033a630aab732339c90334fbc5f.scope
What is needed here is isolation from the OS. Do that by running the
container in its own cgroup namespace. This matches what runc and
other non VM based runtimes do.
Fixes#9124
Signed-off-by: Greg Kurz <groug@kaod.org>
Remove specific kernel/initrd/image leftovers in Makefile of
local-build, which is the part of #9026.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR adds the error script to display the error message with
much more information to help debugging.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds an enhanced die function in order to dump more information
in a yaml format that will help with the debugging.
Fixes#9105
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In case an error is encountered while removing a network endpoint during
network cleanup, we cuurently return immediately with the error.
With this change, in case of error we simply log the error and proceed
towards removing the next endpoint. With this, we can cleanup the
network changes made by the shim as much as possible.
This is especially important when multiple interfaces are passed to the
network namespace using a network plugin like multus.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Move the defer for cleaning up network before the call to add network.
This way if any change made by add network is reverted by in case of
failure. This is particulary important for physical network interfaces
as with this step we make sure that driver for the physical interface is
reverted back to the original host driver. Without this the physical
network iterface will remain bound to vfio.
Fixes: #8646
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This has been introduced by 53bc4a432b,
where the condition was changed.
The correct condition is:
* If the list of supported tees does not contain the kata hypervisor
and the list of supported non tees does not contain the kata
hypervisor.
The error is that we were checking whether kata-hypervisor would contain
the list of supported tees, and that would almost always be false
(unless in the case where the list had an one and only one element).
Fixes: #9055 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is required to allow creation of SNP coco on host kernel
(e.g. https://github.com/AMDESE/linux ,branch:snp-host-latest)
supporting guest private memory for SNP using gmem.
Note: This qemu does not work if the host kernel does not support
gmem/UPM.
Fixes: #9092
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
A test `vcpu allocation k8s test` exhibits different behavior on s390x
For more details, please refer to issue #9093.
This commit is to make the test skipped until the issue is resolved on
the platform.
Fixes: #9093
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Remove some unnecessary whitespace from a couple of `kata-deploy` files.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
whitespace
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Now we're using a "confidential" image that has support for all of
those.
Fixes: #9010 -- part II
#8982 -- part II
#8978 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR pulls the docker image needed for the test as part of the dependencies
in order to avoid failures of timeouts mainly because the image was not
properly download it and it is unable to find it.
Fixes#9089
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the ability to run k8s confidential tests in a
non-TEE environment.
Fixes#9055
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will ensure no leftovers are in the node, which has been cause the
TDX CI to fail every now and then.
Fixes: #9081
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the same way that doesn't make sense to ship the pause-image, it also
doesn't make sense to ship the coco-guest-components itself as part an
release artefact.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It doesn't make sense to ship the pause-image itself as an release
artefact.
The reason we build it and cache it is in order to use it inside the
rootfs, and that's it, there's not need to ship it as part of the
release, at all.
Fixes: #9032 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the CI link into the README as currently we are
using GHA workflows and they are now part of the kata containers
repository.
Fixes#9078
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Delete the debugger pod created during the test, rather than already
existing debugger pods.
Also, send the output of "kubectl delete" to stderr, just in case it's
useful for debugging.
Fixes: #9069
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Last but not least, all placeholders for argument replacement
should be configured to generate a configuration file when `QEMUCMD`
is defined. This enriches those variables.
Additionally, this involves creating a symbolic link to `configuration-qemu.toml`
if QEMU is defined as the default hypervisor.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Avoid error when building for amd64 using:
USE_CACHE=no AGENT_POLICY=yes DEBUG=1 \
tools/packaging/kata-deploy/local-build/kata-deploy-binaries.sh \
--build=rootfs-image
Fixes: #9067
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
There are some variables newly introduced to runtime-rs, such as:
- runtime.name
- runtime.hypervisor_name
- runtime.agent_name
- vm_rootfs_driver
Additionally some of the placeholders for argument replacement are
made hypervisor-specific based on the changes made for dragonball.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We can use daemonset to deploy nydus snapshotter, which will decrease
one manual step both for Kata Containers and Confidential Containers CI.
Fixes: #8584
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR replaces the add_kernel_initrd_annotations_to_yaml function
more generic so later can be used for other components.
Fixes#9054
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
1. add_requests_to_policy_settings allows one or more ttrpc requests
from the Host to the Guest. Example:
add_requests_to_policy_settings "${policy_settings_dir}" \
"ReadStreamRequest" "WriteStreamRequest"
2. add_copy_from_host_to_policy_settings allows executing on the Guest
the commands initiated behind the scenes by "kubectl cp" from the
Host to the Guest. Example:
add_copy_from_host_to_policy_settings "${policy_settings_dir}"
3. add_copy_from_guest_to_policy_settings allows executing on the Guest
the commands initiated behind the scenes by "kubectl cp" from the
Guest to the Host. Example:
add_copy_from_guest_to_policy_settings "${policy_settings_dir}" \
"/tmp/file.txt"
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
For Confidential containers stack, the pause image is managed by host side,
then it may configure a malicious pause image, we need package
a pause image inside the rootfs and don't the pause image from host.
But the installation of skopeo is not included in 20.04 release, so we
can not directly install skopeo in rootfs and pull pause image.
So I plan to let the task as a static build stuff, which would not be influenced
by the system version in rootfs. And the pause image will be part of the Kata Containers rootfs
that's used by the Confidential Containers usecase. This commit enables the component to be built
both locally and in our CI environment with the command: make pause-image-tarball.
Fixes: #9032
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
For example, Kata CI's k8s-copy-file.bats transfers files between the
Host and the Guest using "kubectl exec", and that results in
CloseStdinRequest being called from the Host.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Remove PullImageRequest because that is not used in the main
branch. It was used in the CCv0 branch.
2. Add default false values for the remaining Kata Agent ttrpc
requests.
These changes don't change the functionality of the auto generated
Policy, but they help with easier understanding the Policy text and
the logging from the Rego rules.
Fixes: #9049
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Rename install_kata_common to install_kata_core.
2. Add TODO for better way to install the Kata tools.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Since cri-o doesn't seem to use address for event publishing as mentioned
in the previous commit it will not send it. However, the exact way of
not sending it is unfortunately different from what is assumed by
runtime-rs. Due to an implementation detail of cri-o which uses containerd
libraries for some low-level tasks, TTRPC_ADDRESS will not be missing from
environment as assumed, instead it will be present with an empty value.
This commit contains a small adjustment to account for that and use
LogForwarder even if TTRPC_ADDRESS is present, but with an empty value.
Fixes#8985
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR removes the jenkins reference which is not longer being used
in the kernel documentation.
Fixes#9046
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Bump nydus snapshotter version to v0.13.7.
The new release name of nydus snapshotter is `nydus-snapshotter-v0.13.7-linux-amd64.tar.gz`,
which differs from the version used by kata (`nydus-snapshotter-v0.12.0-x86_64.tgz`).
Therefore, we need to update the script to obtain the correct nydus snapshotter name.
Fixes: #9044
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Automatically generate the test policy for k8s-attach-handlers.bats,
if AUTO_GENERATE_POLICY is enabled.
Steps:
- Create a temporary directory for the current test and copy the
common genpolicy settings into this new directory.
- Change genpolicy settings in the temp directory to allow the
"kubectl exec" command that this test needs. (For CoCo, exec is
blocked by the default policy settings)
- Auto-generate the policy for the test YAML file.
- Test as usual, using the YAML file.
- Clean-up the temporary settings described above.
Fixes: #8921
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Enable AUTO_GENERATE_POLICY for one of the Kata CI K8s test platforms.
Additional platforms will be enabled after testing them.
When AUTO_GENERATE_POLICY is enabled, create genpolicy settings that
are common for all tests. Some of the tests will make temporary copies
of these common settings and customize them as needed.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
1. Avoid repeating "kata-containers-k8s-tests".
2. Allow users to specify a different test namespace.
3. Introduce the TEST_CLUSTER_NAMESPACE variable, that will also be
useful when auto-generating the Agent Policy for these tests.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is needed to fix the bug which is not allowing to create SEV container
on SNP enabled host anymore. This is a regression that was introduced as
part of the following commit:
de39fb7d38Fixes: #9036
Signed-off-by: Niteesh Dubey <niteesh@us.ibm.com>
It makes sense to reuse a configuration template for runtime-golang
as a base. This is simply to copy it into the config directory.
Fixes: #8441
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR skips the k8s tests that are not working with cloud hypervisor
runtime-rs with its proper issue.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It appears that under the shim v2 protocol, a shim has no use of its own
for the -address value, it just passes it back to container runtime's
(mostly containerd or cri-o) event-publishing binary. Since the -address
value only flows through the shim, being passed to the shim by a container
runtime and then essentially passed back by shim to the container runtime,
it seems inappropriate for a shim to validate the value that is fully
owned and only used by the container runtime.
This commit removes such validation from runtime-rs. Doing so, it solves
(part of) an interoperability problem between runtime-rs and cri-o. cri-o
seems to intentionally choose not to implement the event-publishing part
of the shim v2 protocol and thus it has no value it could pass to
runtime-rs for -address. As a result, it sends an empty string which has
been failing the excessive validation performed by runtime-rs so far.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This is required as the tdx-attest-rs crate, which is used as part of
the guest components, has a runtime dependency on libattest-tdx.
Fixes: #9021 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The emergent Kata CI tests for Policy use confidential_guest = false
in genpolicy-settings.json. That value is inconsistent with the
following mount settings:
"emptyDir": {
"mount_type": "local",
"mount_source": "^$(cpath)/$(sandbox-id)/local/",
"mount_point": "^$(cpath)/$(sandbox-id)/local/",
"driver": "local",
"source": "local",
"fstype": "local",
"options": [
"mode=0777"
]
},
We need to keep those settings for confidential_guest = true, and
change confidential_guest = false to use:
"emptyDir": {
"mount_type": "local",
"mount_source": "^$(cpath)/$(sandbox-id)/rootfs/local/",
"mount_point": "^$(cpath)/$(sandbox-id)/local/",
"driver": "local",
"source": "local",
"fstype": "local",
"options": [
"mode=0777"
]
},
The value of the mount_source field is different.
This change unblocks testing using Kata CI's pod-empty-dir.yaml:
genpolicy -u -y pod-empty-dir.yaml
kubectl apply -f pod-empty-dir.yaml
k get pod sharevol-kata
NAME READY STATUS RESTARTS AGE
sharevol-kata 1/1 Running 0 53s
Fixes: #8887
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Due to the restrictions on instance provisioning for self-hosted runners, performing
static checks (36 jobs at the time of writing) on them each time a PR is updated could
significantly burden them, consequently slowing down the entire CI system. To address
this, the decision is to trigger these checks only when an 'ok-to-test' label is added.
Meanwhile, the checks for x86_64, which are supported by GitHub-hosted runners, will
remain unchanged.
Fixes: #8998
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The loop that generate test cases for virtio-mem enabled/disabled
doesn't return the integers '1' and '0' as expected. Instead it returns
the strings '{1,' and '0}'.
Fixes#9024
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Let's install the coco-guest-components into the confidential rootfs
image and initrd.
Fixes: #9021
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make our lives easier to figure out whether the components are
being installed or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to pass this to the container where the rootfs is built, so it
can actually be unpacked inside the rootfs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When moving the webhook we skipped the common.bash as (close-enough)
version is already in `/tests` but we forgot to update the source path,
fixing it here.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
As DESTDIR was not being passed, we've been installing the final
binaries in a container path that was not exposed to the host, leading
to creating an empty tarball with the guest components.
Now, theoretically, guest-components should respect a PREFIX passed, but
that's not the case and we're manually adding "/usr/local/bin" to the
passed DESTDIR.
Here's the result of the tarball:
```bash
⋊> kata-containers ≡ tar tf build/kata-static-coco-guest-components.tar.xz
./
./usr/
./usr/local/
./usr/local/bin/
./usr/local/bin/confidential-data-hub
./usr/local/bin/attestation-agent
./usr/local/bin/api-server-rest
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR is a split of #8585.
make the changes on the Github workflows, and the skeleton to deploy_snapshotter()
and cleanup_snapshotter() in tests/integration/kubernetes/gha-run.sh in this commit.
After initially merging this patch to trigger CI jobs for CoCo, which will begin executing
the dummy functions deploy_snapshotter() and cleanup_snapshotter(), the implementation details for these functions
remain in #8585. Our subsequent step involves transferring this logic to the PR #8484, enabling the PR to undergo CI testing prior to its merge.
Fixes: #8997
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Implementing Persist API for cloud-hypervisor was done partially with
initial support for cloud-hypervisor. Store and retrieve additional
fields to/from the hypervisor state.
Fixes: #6202
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
The filtering of testing cases installs/uses yq and expects GOPATH to be present. Hence, add it to the workflow.
Fixes: #9018
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The original handling method does not reach user expectations.
When the ClientSocketAddress method stats the corresponding
path of runtime-rs and has not found it yet, we should return
an error message here that includes the reason for the failure
(which should be an error display indicating that both runtime-go
and runtime-rs were not found). Instead of simply displaying the
corresponding path of runtime-rs as the final error message to
users.
It is also necessary to return the error promptly to the caller
for further error handling.
Fixes: #8999
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Now that we have a confidential image / initrd being built, instead of a
specific one for each TEE, let's use it everywhere possible.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's use a single rootfs image / initrd for confidential workloads,
instead of having those split for different TEEs.
We can easily do this now as the soon-to-be-added guest-components can
be built in a generic way.
Fixes: #8982
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we're using the kernel-confidential, let the rootfs depending
on it, instead of depending on the TEE specific ones.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to do this in order to ensure that the measure boot will be
taking the latest kernel bits, as needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is already done for the TDX kernel, and should have been done also
for the confidential one.
This action requires us to bump the kernel version as the resulting
kernel will be different from the cached one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
delete_cluster() has tried to delete the az resources group regardless
if it exists. In some cases the result of that operation is ignored,
i.e., fail to resource group not found, but the log messages get a
little dirty. Let's delete the RG only if it exists then.
Fixes#8989
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
As we're building a single confidential kernel, we should rely on it
rather than keep using the specific ones for TDX / SEV / SNP.
However, for debugability-sake, let's do this change TEE by TEE.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this we can properly generate and the the `-confidential` kernel,
which supports SEV / SNP / TDX as part of our configuration files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
I made this a required argument during the series and ended up
forgetting to add that while calling the function.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This issues was introduced due to a typo not caught during reviews on
e5bca90274.
Fixes: #6415 -- part II
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Until now, runtime-rs couldn't be compiled on s390x.
We need to lift those restrictions in Makefile first.
Fixes: #8446
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests different with devmapper.
Fixes#8995
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This addresses an internal AKS issue that intermittently prevents
clusters from getting created. The fix has been rolled out to eastus but
not yet eastus2, so we unblock the CI by switching. No downsides in
general.
This supersedes #8990.
Fixes: #8989
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
It fails to compile virt_container because Dragonball is only
used in the implementation of the trait method Persist::restore().
As the hypervisor is not compiled on s390x and QEMU implements
the trait method, this commit is to let the method use QEMUi's.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Dragonball and cloud-hypervisor are not supported on s390x.
We need to exclude the plugins for these hypervisors from compilation.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Till now we didn't have a logic to consume the kernel modules cached
tarball. Let's make sure those are consumed as it'll save us a
reasonable amount of build time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will save us a lot of time, as right now the CI is rebuilding the
kernel for absolutely no reason.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's start doing this for the confidential kernels (and also for SEV,
till it gets removed).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This allows us to add a map, in the format of:
`"tarball1_name:tarball1_path tarball2_name:tarball2_path ..."`
With this we have a base to start doing a better job when caching extra
artefacts, like kernel modules.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is just being added but not used yet. The idea is to use
this to both cache and later on untar the kernel modules needed for some
of the kernel targets we have (specifically looking at the confidential
one).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch can reduce load on systemd process, and
increase the k8s deployment density when using go runtime.
Fixes: #8758
Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
As already done for docker, let's also add support for installing
nerdctl + kata containers.
For now, at least for now, we are explicitly not allowing the
combination of installing both docker and nerdctl in the same
installation in order to reduce the script complexity.
Also, nerdctl installation, for now, is limited to x86_64 and aarch64 as
those are the only architectures that nerdctl releases a "full" package
for.
Fixes: #8358
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds workflow for running kubernetes test suite on ppc64le.
It uses scripts to create and delete the cluster using kubeadm as none of the current cluster creation tools are supported on Power.
Fixes: #7950
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Kata CI's pod-sandbox-vcpus-allocation.yaml ends with "---", so the
empty YAML document following that line should be ignored.
To test this fix:
genpolicy -u -y pod-sandbox-vcpus-allocation.yaml
Fixes: #8895
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Allow users to specify in genpolicy-settings.json a default cluster
namespace other than "default". For example, Kata CI uses as default
namespace: "kata-containers-k8s-tests".
Fixes: #8976
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
There is a race condition in agent HVSOCK_STREAMS hashmap, where a
stream may be taken before it is inserted into the hashmap. This patch
add simple retry logic to the stream consumer to alleviate this issue.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Linux forbids opening an existing socket through /proc/<pid>/fd/<fd>,
making some images relying on the special file /dev/stdout(stderr),
/proc/self/fd/1(2) fail to boot in passfd io mode, where the
stdout/stderr of a container process is a vsock socket.
For back compatibility, a pipe is introduced between the process
and the socket, and its read end is set as stdout/stderr of the
container process instead of the socket. The agent will do the
forwarding between the pipe and the socket.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
In passfd io mode, when not using a terminal, the stdout/stderr vsock
streams are directly used as the stdout/stderr of the child process.
These streams are non-blocking by default.
The stdout/stderr of the process should be blocking, otherwise
the process may encounter EAGAIN error when writing to stdout/stderr.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
We want the io connection keep connected when the containerd closed
the io pipe, thus it can be attached on the io stream.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Support the hybrid fd passthrough mode with passing pipe fd,
which can specify this connection kept even when the pipe
peer closed, and this connection can be reget wich re-opening
the pipe.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In linux, when a FIFO is opened and there are no writers, the reader
will continuously receive the HUP event. This can be problematic
when creating containers in detached mode, as the stdin FIFO writer
is closed after the container is created, resulting in this situation.
In passfd io mode, open stdin fifo with O_RDWR|O_NONBLOCK to avoid the
HUP event.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
When container exits, the agent should clean up the term master fd,
otherwise the fd will be leaked.
Fixes: kata-containers#6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
When one end of the connection close, the epoll event will be triggered
forever. We should close the connection and kill the connection.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Currently in the kata container, every io read/write operation requires
an RPC request from the runtime to the agent. This process involves
data copying into/from an RPC request/response, which are high overhead.
To solve this issue, this commit utilize the vsock fd passthrough, a
newly introduced feature in the Dragonball hypervisor. This feature
allows other host programs to pass a file descriptor to the Dragonball
process, directly as the backend of an ordinary hybrid vsock connection.
The runtime-rs now utilizes this feature for container process io. It
open the stdin/stdout/stderr fifo from containerd, and pass them to
Dragonball, then don't bother with process io any more, eliminating
the need for an RPC for each io read/write operation.
In passfd io mode, the agent uses the vsock connections as the child
process's stdin/stdout/stderr, eliminating the need for a pipe
to bump data (in non-tty mode).
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Two toml options, `use_passfd_io` and `passfd_listener_port` are introduced
to enable and configure dragonball's vsock fd passthrough io feature.
This commit is a preparation for vsock fd passthrough io feature.
Fixes: #6714
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Using custom input paths with -i is counter-intuitive. Simplify path handling with explicit flags for rules.rego and genpolicy-settings.json.
Fixes: #8568
Signed-Off-By: Malte Poll <1780588+malt3@users.noreply.github.com>
The necessary files for the agent builder image can be found in
`tools/packaging/static-build/agent`,
`ci/install_libseccomp.sh` and
`tools/packaging/kata-deploy/local-build/kata-deploy-copy-libseccomp-installer.sh`.
Identifying the correct files addresses the previously misreferenced path
used to name the builder image.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Removed the setting of default values for runtime fields. Added explicit checks for missing or empty fields, reporting errors with clear messages.
Fixes: #8838
Signed-off-by: yaoyinnan <35447132+yaoyinnan@users.noreply.github.com>
By running `kustomize edit fix` on those files they have changed
deprecated instructions ('bases' and 'patchesStrategicMerge') as well as
'apiVersion' and 'kind' were added.
Fixes#8268
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
correct the lib file locations after the move from
tests->kata-containers repo and add a minimized version of the
".ci/lib.sh" library into the "ci/openshift-ci" as we don't really
utilize all of the features.
Fixes: #8653
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Move the f15be37d9bef58a0128bcba006f8abb3ea13e8da version of scripts
required for openshift-ci from "kata-containers/tests/.ci/openshift-ci"
into "kata-containers/kata-containers/ci/openshift-ci" and required
webhook+libs into "kata-containers/kata-containers/tools/testing" as is
to simplify verification, the different location handling will be added
in following commit.
Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
The `noop-method-call` is a rustc lint that has existed since v1.52.0.
This lint has been moved to the warn by default lint level since v1.73.0.
Therefore build is failing with this version and above.
This commit removes the unnecessary call to `<&T as Deref>::deref` on `T: !Deref`.
Fixes: #8586
Signed-off-by: Kvlil <kalil.pelissier@gmail.com>
This PR re-arranged the nerdctl tests to avoid random failures.
In this PR first will run the tests with RunC and then with the kata hypervisor.
This PR tries to avoid the random failures that is happening with cloud-hypervisor
and clh.
Fixes#8963
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The auto-generated Policy already allows these volumes to be mounted,
regardless if they are:
- Present, or
- Missing and optional
Fixes: #8893
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This involves using Ubuntu 20.04 as a build environment for an agent to match with a runtime environment.
Fixes: #8955
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Qemu stderr monitoring runs in its own asynchronous green thread.
For that, `stderr` is taken out of the Child representing the qemu child
process to avoid partial move and make it possible for the main thread
still to call functions on QemuInner::qemu_process (e.g. kill(), id()).
Fixes#8937
Signed-off-by: Pavel Mores <pmores@redhat.com>
We'll want to capture qemu's stderr in parallel with normal runtime-rs
execution. Tokio's primitives make this much easier than std's. This
also makes child process management more consistent across runtime-rs
(i.e. virtiofsd child process is already launched and managed using tokio).
Some changes were necessary due to tokio functions being slightly different
from their std counterparts. Child::kill() is now async and Child::id()
now returns an Option.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Allow Kata CI's pod-nested-configmap-secret.yaml to work with
genpolicy and current cbl-mariner images:
1. Ignore the optional type field of Secret input YAML files.
It's possible that CoCo will need a more sophisticated Policy
for Secrets, but this change at least unblocks CI testing for
already-existing genpolicy features.
2. Adapt the value of the settings field below to fit current CI
images for testing on cbl-mariner Hosts:
"kata_config": {
"confidential_guest": false
},
Switching this value from true to false instructs genpolicy to
expect ConfigMap volume mounts similar to:
"configMap": {
"mount_type": "bind",
"mount_source": "$(sfprefix)",
"mount_point": "^$(cpath)/watchable/$(bundle-id)-[a-z0-9]{16}-",
"driver": "watchable-bind",
"fstype": "bind",
"options": [
"rbind",
"rprivate",
"ro"
]
},
instead of:
"confidential_configMap": {
"mount_type": "bind",
"mount_source": "$(sfprefix)",
"mount_point": "$(sfprefix)",
"driver": "local",
"fstype": "bind",
"options": [
"rbind",
"rprivate",
"ro"
]
}
},
This settings change unblocks CI testing for ConfigMaps.
Simple sanity testing for these changes:
genpolicy -u -y pod-nested-configmap-secret.yaml
kubectl apply -f pod-nested-configmap-secret.yaml
kubectl get pods | grep config
nested-configmap-secret-pod 1/1 Running 0 26s
Fixes: #8892
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR updates the required packages for the TensorFlow ResNet50
Int8 Dockerfile.
Fixes#8950
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Due to the changes done in the CI, we need to set the correct
subscription to be used with the account from now on, otherwise we'd end
up using CoCo subscription.
Fixes: #8946
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This fixes a flaw pointed out in review of PR #8185. Creation of the
directory semantically fits better into VM preparation than VM launch.
Signed-off-by: Pavel Mores <pmores@redhat.com>
We need to actually check that the env var is not empty. :-)
This was introduced by 8307718842.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The Confidential Containers guest-components will, in the very short
future, be part of the Kata Containers rootfs that's used by the
Confidential Containers usecase.
This commit introduces the ability to, standalone, build the component
locally and as part of our CI, and this can be done by calling:
`make coco-guest-components-tarball`
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Co-authored-by: Jakob Naucke <jakob.naucke@ibm.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Co-authored-by: zhouliang121 <liang.a.zhou@linux.alibaba.com>
Co-authored-by: Alex Carter <alex.carter@ibm.com>
Co-authored-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
Co-authored-by: Xynnn007 <xynnn@linux.alibaba.com>
Validating the node name is currently outside the scope of the CoCo
policy.
This change unblocks testing using Kata CI's test-pod-file-volume.yaml
and pv-pod.yaml.
Fixes: #8888
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Those are not yet being cached for no reason, and they better be as
it'll allow us to save a considerable amount of time building the
rootfs.
Fixes: #8917
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The existing confidential basic test titled `Test unencrypted
confidential container launch success and verify that we are
running in a secure enclave` has been updated to incorporate
IBM Secure Execution (`qemu-se`).
Previously, a secure image was absent from kata-deploy, hindering
the inclusion of IBM SE in the test.
Thanks to the #6755 update, it is now possible to test the TEE.
This modification extends the existing test by introducing
`qemu-se`. The specific changes are outlined below:
- Add an additional test `cc-se-e2e-tests` to s390x nightly
- Expansion of `REMOTE_COMMAND_PER_HYPERVISOR` for `qemu-se`
- Temporary exclusion of two test cases currently incompatible with IBM SE
(`cpu-ns` is a common issue across all TEEs, while `inotify`
will be addressed in a subsequent pull request).
Fixes: #8913
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Add metadata containing the Policy annotation if the user didn't
provide any metadata in the input yaml file.
For a simple sanity test using a Kata CI YAML file:
genpolicy -u -y job.yaml
kubectl apply -f job.yaml
kubectl get pods | grep job
job-pi-test-64dxs 0/1 Completed 0 14s
Fixes: #8891
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's start relying on the already cached agent to be deployed inside
the rootfs. By doing this we save a lot of time in our CI, and we have
a better way, for developers, to play with changes in the agent.
Fixes: #8915
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we'll be untarring the agent tarball (and any other component that
may be part of the rootfs) into the rootfs, we have to have xz
installed.
For debian and ubuntu the package is called xz-utils; for centos,
alpine and cbl-mariner the package is called xz.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This env var will serve us to pass the agent tarball to the rootfs
builder, which will then just unpack the content into the rootfs instead
of building the agent again.
AGENT_TARBALL and AGENT_SOURCE_BIN should never be used together.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving away from alpine and using ubuntu in order to be able to
build the agent for all the architectures we need.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This has been missed during reviews and is already a problem as we're
trying to build the agent outside of the rootfs for other architectures
than x86_64.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This has been missed during reviews and will become a problem when the
tools start to be built in different architectures.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Several use-cases need a vTPM lets enable it for QEMU, a follow up patch will introduce the runtime config.
Fixes: #8902
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This PR uses a variable that is already defined in the build base
qemu script to have uniformity across the script as this variable
is already used in the script.
Fixes#8903
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Lets remove the GHA target actions first so the the follow-up PR #8874 tests are succeeding.
Fixes: #8900
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Add --show-output-of-passing-tests to the k8s integration tests. The
output of a passing test can be helpful when investigating a failure
of the same test.
Fixes: #8885
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR is to add a document for how to run kata containers under IBM
Secure Execution environment.
Fixes: #7025
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR updates the libseccompt version to v2.5.5 which includes
the following changes:
- Update the syscall table for Linux
- Fix minor issues with binary tree testing and with empty binary trees
Fixes#8883
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The versions.yaml has a default for the amd64 binary, but there is no
code to actually build the arm64 binary, which seems an overlook.
Let's simplify the OPA logic by removing the direct link to the binary,
and construct that link as part of the checks we do to decide whether we
need to build OPA or not.
Fixes: #8373
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the packages necessary to build the ResNet50 fp32
Dockerfile to run properly the benchmark.
Fixes#8875
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is a follow up to the work of minimizing targets, unifying TDX,SNP builds for NVIDIA GPUs
Fixes: #8828
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
SharedVersion fiel add a versiontable property that isn't supported by upstream QEMU.
This is dead code since virtcontainers isn't setting SharedVersions to true.
Fixes: #7720
Signed-off-by: Kvlil <kalil.pelissier@gmail.com>
The changes to install and test genpolicy must come later, after CI
picks up these gha changes.
Fixes: #8856
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Ignore pod DNS settings because policing the network traffic is
currently outside the scope of the Agent Policy.
Example from Kata CI: pod-custom-dns.yaml
Fixes: #8832
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deploy the framework added by the previous commit to generate qemu
command line and launch the VM.
We now properly store the child process object which allows us to
implement remaining Hypervisor functions necessary for a simple but
successful VM lifecycle, get_vmm_master_tid() and stop_vm().
Fixes#8184
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR updates the firecracker version to v1.6.0 which includes
the following features
- Added support for per net device metrics. In addition to aggregate metrics net, each individual net device will emit metrics under the label "net_{iface_id}". E.g. the associated metrics for the endpoint "/network-interfaces/eth0" will be available under "net_eth0" in the metrics json object.
- Added support for per block device metrics. In addition to aggregate metrics block, each individual block device will emit metrics under the label "block_{drive_id}". E.g. the associated metrics for the endpoint "/drives/{drive_id}" will be available under "block_drive_id" in the metrics json object.
- Added a new vm-state subcommand to info-vmstate command in the snapshot-editor tool to print MicrovmState of vmstate snapshot file in a readable format. Also made the vcpu-states subcommand available on x86_64.
- Added source-level instrumentation based tracing. See tracing for more details.
- Added developer preview only (NOT for production use) support for vhost-user block devices. Firecracker implements a vhost-user frontend. Users are free to choose from existing open source backend solutions or their own implementation. Known limitation: snapshotting is not currently supported for microVMs containing vhost-user block devices. See the related doc page for details. The device emits metrics under the label "vhost_user_{device}_{drive_id}".
Fixes#8854
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
- test_volume_capacity_stats: verify the file block size against the fetched size via statfs()
- test_reseed_rng: Correct the request codes for RNDADDTOENTCNT and RNDRESEEDCRNG when platform is ppc64le
- test list_routes: Add the route only if destination is not empty
- test_new_fs_manager: skip the test if cgroups v2 is used by default
- skip test cases rpc::tests::test_do_write_stream, sandbox::tests::test_find_process, sandbox::t
ests::test_find_container_process and sandbox::tests::add_and_get_container on ppc64le as they are fl
aky
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
kata-ctl currently fails to build on ppc64le. Skip it for running static checks and the issues will be fixed and tracked in a seperate issue.
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
A few CPU related test cases were failing as the version was being verified against Power8 while the CI machine is Power9.
Fixes: #5531
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
It was observed that a tmporary file `/tmp/kata_hybrid_vsock02.hvsock`
for test_setup_hvsock_failed() is not removed from time to time.
This leads to a test failure for the same test next time due to the
file permission on a self-hosted runner.
This commit is to explicitely delete the file before the check starts.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
At the moment, a project `dragonball` and `runtime-rs` does not support
for s390x. During the enablement, some errors due to the misconfiguration
of Makefile for `make check` and `make vendor` were identified.
This is to skip the build for the affected target of the projects.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Some linting errors were identified during the enablement of `make check`.
These have not been found by the Jenkins CI job because `make test` was
only triggered.
The errors for the `agent` occurs under the s390x specific tests while
the other ones for the `kata-ctl` are the architecture-specific code.
This commit is to fix those errors.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
If `yq_path` is set to `/usr/local/bin/yq`, there could be a situation
where the `yq` cannot be installed without `sudo`.
This commit handles the situation by putting `sudo` in front of `curl`
and `chmod`, respectively.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
As part of the CI migration from Jenkins to GitHub Action, a CI job named
`kata-containers-2.0-ubuntu-s390x-unit-PR` is covered by the static check.
This commit is to enable the check for s390x by incorporating a runner
`s390x` with the corresponding workflow.
Fixes: #8482
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The idea of most of these is just to prevent running into todo!()s where
we can at the moment, while implementing the fundamental functionality of
VM launch.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Install the default genpolicy OPA rules and settings JSON files, in
addition to the genpolicy binary.
Fixes: #8844
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
DirectVolume/Rawblock doesn't work well when device's block driver
is virtio-blk-pci and the storage handler is DRIVER_BLK_PCI_TYPE.
Fixes: #8707
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Temporarily disable the allow_storages() rules, because they are based
on the tarfs snapshotter + container image integrity information that
are not available yet in the main branch - see #8833.
Fixes: #8834
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Adjust genpolicy-settings.json to match the container root path from
the main branch + cbl-mariner Guest VMs.
This configuration might have to be adjusted again when other types of
Guest VMs will be tested during CI using genpolicy, in the future.
Also, improve logging from allow_root_path(), to easier debug these
issues in the future.
Fixes: #8835
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR removes the references to virtiofs from memory average
calculation when the container uses a shared file system other than
virtiofs.
Fixes: #8807
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR removes the iperf3 server protocol as this server definition is
also used for the UDP iperf3 benchmarks to avoid duplication of the
same yaml files.
Fixes#8829
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses a specific python version to run tensorflow benchmark
as it needs python 3.8 to run correctly and avoid failures.
Fixes#8791
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Replace the `todo!()` calls with a minimal NOP implementation to return
the CH driver to working order since the `todo!()`'s forcibly crash the
driver at runtime. Full implementations for these APIs will be added on
issues #8800, #8801, and #8802.
Fixes: #8784.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the `todo!()` macro which would cause a runtime crash and replace
with a implementation that returns an error as a stop-gap until #8800 is
implemented.
Fixes: #8785.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Containerd runtime options with wrong setting cause it failed.
Correct it as below:
...
[plugins.cri.containerd.runtimes.${runtime}.options]
ConfigPath= "${KATA_CONFIG_PATH}"
...
Fixes: #8746
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
It is a little bit heavy for the runtime-rs to forwards events via
containerd CLI, contrast to the ttrpc way. Plus, for runtimes that haven't
this mechanism, e.g. CRI-O, we can't get those events anywhere.
This patch introduces two types of forwarders:
- `ContainerdForwarder`: Acquire ttrpc address from environment variables
and forward events via ttrpc connection.
- `LogForwarder`: Write event info into logs.
Fixes: #7881
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR updates the runc version to 1.1.11 which includes the
following improvements
- Fix several issues with userns path handling.
- Support memory.peak and memory.swap.peak in cgroups v2. Add
swapOnlyUsage in MemoryStats. This field reports swap-only usage.
For cgroupv1, Usage and Failcnt are set by subtracting memory usage
from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
are set.
- build(deps): bump github.com/cyphar/filepath-securejoin.
Fixes#8795
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We're using a Kernel based on v6.7, which should include all te
patches needed for SEV / SNP / TDX.
By doing this, later on, we'll be able to stop building the specific
kernel for each one of the targets we have for the TEEs.
Let's note that we've introduced the "confidential" target for the
kernel builder script, while the TEE specific builds are being kept as
they're -- at least for now.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As all the supported architectures are disabling the virtiofsd build,
there's no need to keep the switch statement there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've not been building QEMU experimental for a very long time, and the
entry there has only been serving the purpose to clutter the
versions.yaml (in the best case scenario) or even confuse new
contributors to the project.
Mind that the machinery to build the QEMU experimental is not touched,
and that's used to build the TEEs capabale artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR removes the check images function from stressng test as now
it will part of the install dependencies function from gha-run script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To avoid random failures while trying to build and install the stressng image,
this PR moves that step as part of the install dependencies in order to move
the stability tests and avoid timeouts.
Fixes#8787
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is not necessarily meant to work, just to stub out unimplemented
functionality while focusing on more fundamental things.
Signed-off-by: Pavel Mores <pmores@redhat.com>
This PR fixes the indentation of the build static stratovirt script
for kata containers.
Fixes#8777
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Log the list of the current pods between tests because these pods
might be related to cluster nodes occasionally running out of memory.
Fixes: #8769
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The agent registers an event fd in `memory.oom_control`. An OOM event is
forwarded to containerd when the event is emitted, regardless of the
content in that file.
I observed content indicating that events should not be forwarded, as shown
below. When `oom_kill` is set to 0, it means no OOM has occurred. Therefore,
it is important to check the content to avoid mistakenly forwarding OOM
events.
```
oom_kill_disable 0
under_oom 0
oom_kill 0
```
Fixes: #8715
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR adds the qemu-experimental hypervisor in the function to
kill kata components.
Fixes#8775
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Don't release the lock between is_allowed and set_policy calls,
because the policy might change in between these calls.
Also, move more policy code into policy.rs.
Fixes: #8734
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
- `ttrpc` from `0.7.1` to `0.8`.
- `containerd-shim-protos` from `0.3.0` to `0.6.0`.
Fixes: #8756
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
As the StratoVirt VMM has been added, we can update the docs
and make some intoduction to StratoVirt, thus users can know more
about the hypervisor choices.
Fixes: #8645
Signed-off-by: Zhigang Wang <wangzhigang17@huawei.com>
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
In order to avoid rust-vmm upstream change breaks Dragonball
compilation, we introduce Cargo.lock to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
As reported in #8767, we have found that the root cause is that rust-vmm's vmm-sys-utils
introduce a new release 0.12.1 and dbs-pci rely on rust-vmm's vfio-ioctls which uses >=
to declare vmm-sys-utils so it automatically upgrade vmm-sys-utils to 0.12.1.
That's how two different versions of vmm-sys-utils is introduced and this breaks the compilation.
In order to fix this and also avoid future problems, we introduce Cargo.lock file to dbs crates.
fixes: #8770
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This PR improves the iperf3 cleanup to ensure all the components are
being deleted properly to avoid the random failures of leaving
the iperf3 clients on the kata metrics CI.
Fixes#8765
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Previously, Dragonball did not support PCI device hot-plugging or
VFIO device passthrough. Therefore, the runtime-rs support for
Dragonball was incomplete. it is time to complete it so that users
can use Dragonball's PCI hot-plugging and VFIO passthrough capabilities.
Fixes: #8748
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
vfio commits introduce quite a lot change in runtime-rs, this commit is
for all the changes related to ci, including compilation errors and so on.
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
add pci add and del guest kernel patch as the extension
in the upcall device manager server side.
also, dump config version to 120 since we need to add config
for dragonball pci in upcall
fixes: #8741
Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Introduce two new vmm action to implement pci hotplug
and pci hot-unplug: PrepareRemoveHostDevice and RemoveHostDevice.
PrepareRemoveHostDevice is to call upcall to unregister the pci device
in the guest kernel.
RemoveHostDevice should be called after PrepareRemoveHostDevice, it is used
to clean the PCI resource in the Dragonball side.
fixes: #8741
Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Add a pcie_topology field to DeviceManager and initialize
pcie_topology when ResourceManager calls DeviceManager's new()
with TopologyConfigInfo.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Before calling the device driver to attach a device, register
the device to PCIe topology and allocate a PciPath for it.
However, for some hypervisor such as CLH, the allocation is invalid
when plugging devices to VM, they have the ability to return
DeviceInfo containing PciPath. It'll update the PciPath with the
returned pci path in the PCIe topology for them to prevent the
inferred pcipath from being different from the actual value returned.
But the update will not be executed if the pcipath value doesn't change.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce helper macros to simplify PCIe device register/unregister
and update, which provides a convenient way to handle devices in
topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Add one more argument with type &mut Option<&mut PCIeTopology>
in attach and detach to inroduce methods within PCIe Topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Implement Trait PCIeDevice register/unregister for pcie/pci
device, such as vfio device which needs set/get device's pci
path for kata agent's device handler.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce Trait PCIeDevice with register/unregister, which are
used to register or unregister pcie device within the PCIe topology.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Due to different ways that different VMMs handle PCI devices,
we expect to provide a general PCIe topology processing framework
that is as compatible as possible with VMMs such as dragonball,
qemu, clh(Though it has its own management method, no conflict).
Currently,it's mainly developed for kinds of PCIe/PCI devices in
dragonball/clh which are attached on the pci/pcie root bus directly.
More will be added when Qemu is ready in runtime-rs.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
A TopologyConfigInfo added to store device config info for PCIe/PCI
devices in the VM from Hypervisor DeviceInfo.
And TopologyConfigInfo::new will be the entry to initialize PCIe
Topology for each VM.
Fixes: #7218
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Bridge the gap between user requirements for direct block device access
and the DirectVolume capabilities provided by Kata runtimes
(kata-runtime/runtime-rs), and facilitate seamless integration with CSI
to improve user experience.
It aims to integrate DirectVolume CSI support into Kata, enabling users
to benefit from its performance and flexibility advantages.
Fixes: #8602
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Otherwise `jq` will complain about:
```sh
jq: error: nydus/0 is not defined at <top-level>, line 1:
.plugins."io.containerd.grpc.v1.cri".containerd.runtimes."kata-clh".snapshotter=nydus
jq: 1 compile error
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch introduces a feature of supporting vhost-user-blk device.
This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --virblks option:
--virblks '{
"drive_id": "8623",
"device_type": "Spdk",
"path_on_host": "spdk:///var/tmp/vhost.sock",
"is_root_device": false,
"is_read_only": false,
"is_direct": false,
"no_drop": false,
"num_queues": 1,
"queue_size": 256
}'
Fixes: #8631
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: fupan <fupan.lfp@antgroup.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Qinqi Qu <quqinqi@linux.alibaba.com>
In the way the script is without this patch, we're trying to set
```toml
[`$shim`]
snapshotter = $snapshotter
```
However, what we actually want to set is the full runtime table instead
of shim.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We want to check whether the shim is part of the "plain text" shims
passed to the daemonset (meaning, checking against `$SHIMS`). Before
this fix we were checking against `$shims`, which is an array of shims
instead of a string, resulting on a broken check.
Fixes: #8732
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
move `SKIP_GO_VERSION_CHECK` after `make` command to skip
checking golang version.
And also upgrade golang to 1.19.
Fixes: #8728
Signed-off-by: Bin Liu <bin@hyper.sh>
The compiler will give a warning if a developer forget to add an arm for
a new variants defined.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
DAN reads vhost-user-net device from JSON config. It only supports VMM
running as server right now.
Fixes: #8625
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The changes involve:
- Expose VhostUserConfig struct to runtime-rs.
- Set a default value while num_queues or queue_size are 0.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This commit introduces VhostUserEndpoint and supports relative to
vhost-user-net devices for device manager. For now, Dragonball is able to
attach vhost-user-net devices.
Fixes: #8625
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Config space of network device is shared and accord with virtio 1.1 spec.
It is a good way to abstract the common part into one function.
`set_config_space()` implements this.
Plus, this patch removes `vq_pairs` from vhost-net devices, since there is
a possibility of data inconsistency. For example, some places read that
from `self.vq_pairs`, others read from `queue_sizes.len() / 2`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Add application that infers K8s user's intentions based on user's
K8s YAML file, and generates a Rego/OPA based policy for that YAML.
Just Pod YAML files are supported as input using this initial source
code. Support for other types of YAML files will come with upcoming
commits.
Fixes: #7673
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Vhost-user-net has a dependency on address space from `MmioV2DeviceState`.
The addition of the address space is introduced in this patch. Plus, it
makes sure all unit tests have the according parameter as well.
Fixes: #8502
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The default network backend of runtime-rs with Dragonball is vhost-net
after #8609 merged. The tests might be failed if vhost modules are not
loaded.
Fixes: #8717
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
It's important to ensure that these tasks which setup vfio
devices are completed before add_device.
So Moving vfio device setup code to a dedicated method at device
building time which does not affect the behavior of other code.
And this change makes it easier to understand the difference
between create and attach, and also makes the boundaries
clearer.
Fixes: #8665
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Make VhostUserConfig pci_path's type more specific, change it
from Option<String> to Option<PciPath>.
Fixes: #8665
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Since containerd 1.7.0 we can easily set a specific snapshotter to be
used with a runtime handler, and we should take advantage of this,
mostly as it'll help setting up any runtime using devmapper or nydus
snapshotters.
This implementation here has a few caveats:
* The format expected for the SNAPSHOTTER_HANDLER_MAPPING is:
`shim:snapshotter,shim:snapshotter,...`
* It only works with containerd 1.7 or newer
* We **never** change the default containerd snapshotter
* We don't do any check on our side to verify whether the snapshotter
required is properly deployed
* Users will have to add an annotation to their pods, in order to use
the snapshotter set up per runtime handler
* Example:
```
metadata:
...
annotations:
io.containerd.cri.runtime-handler: kata-fc
```
Fixes: #8615
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Switch to use function from Kata repo in common.bash to reduce
dependency on the tests repo.
Fixes#8713
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add implementations for the following `Hypervisor` trait methods which
simply return the same details as the `get_vmm_master_tid()` method:
- `get_thread_ids()`
- `get_pids()`
Fixes: #6438.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This reverts commit ee5fa08a27.
This is perfectly fine to do as we narrwoed down the issue to be on the
version of `jq` provided by alpine, and we've already updated it in the
previous commit (in this very same series).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`jq` coming from alpine is in its 1.6 version, and that has a bug that
hits us quite hard, as it changes a float to an int whenever the number
is in the `x.0` format.
One example is:
```bash
/ # jq --version
jq-1.6
/ # echo '{"foo": 1.0}' | jq .foo
1
```
With this in mind, let's switch, at least for now, to using the `jq`
released directly on github, as it does address the issue we've been
hitting.
```bash
⋊> Downloads ./jq-linux-amd64 --version
jq-1.7
⋊> Downloads echo '{"foo": 1.0}' | jq .foo
1.0
```
Fixes: #8678
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
After kata-deploy has installed, check that the worker nodes
are still in Ready state and don't have a containerd://Unknown
container runtime versions, identicating that container isn't working
to ensure that we didn't corrupt the containerd config during kata-deploy's edits
Fixes: #8678
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
To become more resilient against these kinds of errors:
deployment.apps/confidential-unencrypted created
pod/confidential-unencrypted-c5fdd6964-rrb6q condition met
ssh: connect to host 10.42.0.109 port 22: Connection refused
Fixes: #8687
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is to reintroduce a configuration rule for IBM Z Secure Execution,
where no initrd path should be configured. For the TEE of interest,
only a kernel image should be specified with `confidential_guest=true`.
Fixes: #8692
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
`make SUPPORT_VIRTUALIZATION=1 test` iterates through all subcrates and
does test.
Plus, this patch fixes some issues about unit tests:
- Feed too much parameters to `I8042Device::new()`.
- Virtqueue checks have been introduced since `virtio-queue v0.7.0`.
- GHA might have no access to `/var/tmp` dir on runner.
Fixes: #8690
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR fixes the indentation of the confidential common
script for kubernetes tests.
Fixes#8698
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
As a follow-up for #8516, guest_cid and vhost_fd are not necessarily initialised
via new(). Instead, the fields should be initialised later when they are really
used to construct hypervisor's parameters.
This commit is to separate init_config() from new() to initialise guest_cid
and vhost_fd and leave only the assignment of id for the existing function.
Fixes: #8671
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We now use the static checks script from the main kata containers repo
and not the tests repo; update documentation to reflect this.
Fixes#8681
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change the two functions in lib.sh to use the static checks script from
the kata containers repo instead of tests. Remove cloning the repo from
these functions since we don't need it anymore. Leave these two
functions because the document checking one may be used locally and the
static checks one is called from the virtcontainers Makefile.
Fixes#8681
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
If a PR contains files from the ignore-paths, these actions do not run
as intended. However, the actions are make as required. And there does
not seem to be a way to mark these as non-required in that case.
As a result a PR containing the files from the ignore-paths remains
stalled.
Hence remove the ignore-paths until github provides a way to mark
actions that are skipped due to ignore-paths as non-required/passed.
Fixes: #8663
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Update the Speed & Density metric tests baseline for StratoVirt
and re-enable them, and skip other metric tests temporarily.
Fixes: #8656
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
fix error when running checks and tests:
error: failed to run custom build command for `devicemapper-sys v0.1.5`
fatal error: 'libdevmapper.h' file not found
thread 'main' panicked at 'Could not generate dm.h bindings:
ClangDiagnostic("dm.h:2:10: fatal error: 'libdevmapper.h' file not found\n")',
/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/devicemapper-sys-0.1.5/build.rs:24:10
stack backtrace:
0: rust_begin_unwind
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
1: core::panicking::panic_fmt
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
2: core::result::unwrap_failed
at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/result.rs:1651:5
3: core::result::Result<T,E>::expect
4: build_script_build::main
5: core::ops::function::FnOnce::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make: *** [../../utils.mk:177: standard_rust_check] Error 101
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Fix error when making checks:
```
error: failed to run custom build command for `image-rs v0.1.0
(https://github.com/confidential-containers/guest-components?tag=v0.8.0#e849dc89)`
Caused by:
process didn't exit successfully: `/home/runner/work/kata-containers/kata-containers/src/
agent/target/release/build/image-rs-fd932206d09362b7/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=./protos/getresource.proto
cargo:rerun-if-changed=./protos
--- stderr
thread 'main' panicked at 'Could not find `protoc` installation and this build crate cannot proceed without
this knowledge. If `protoc` is installed and this crate had trouble finding
it, you can set the `PROTOC` environment variable with the specific path to your
installed `protoc` binary.If you're on debian, try `apt-get install protobuf-compiler`
or download it from https://github.com/protocolbuffers/protobuf/releases
```
Fixes#8673
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
We use a matching direct-volume path to determine whether an OCI mount
is a DirectVolume. However, we should handle the case where no match is
found appropriately.
This error will be defined as a non-DirectVolume type when judging the
OCI mount but not failed.
Fixes: #8619
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
DirectVolume structure in runtime-rs is different from it in kata-runtime,
which causes they has no unified handling method for DirectVolumeMountInfo
and MountInfo.
We should align the two by simply adding the attribute #[serde(rename="x")
to each field in DirectVolumeMountInfo
Fixes: #8619
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Dragonball sets a default queue config in the case of `None`. The
queue_size and num_queues of vhost-net are set to `Some(0)` by default.
Therefore, we might get an invalid queue config. This patch fixes this
issue.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This patch set vhost-net as default backend of networking. It allows users
to set `disable_vhost_net` to `true` to reenable virtio-net backend.
Plus, which backend to use is a matter of hypervisor, runtime-rs will no
longer need to know that.
Fixes: #8608
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR improves the latency network cleanup by removing the pods
even if the test fails.
Fixes#8658
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This is to introduce a pre-action to all the workflows for building artifacts.
The action could take care of tasks such as cleaning up files and reinstalling
packages, which prevents a workflow from getting affected by the environment.
This also includes the removal of the step `Adjust a permission for repo`,
because it could be incorporated into the action.
Fixes: #8648
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We never have to add a configuration for the "default" case, as we're
already creating the runtime class pointing to what should be the
"default" handler.
This helps to simplify the logic by quite a lot.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The comment adds absolutely nothing to the runtime handler added, and
it'd make our life slightly harder to properly say which VMM is being
used when setting the default `kata` handler.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This save us a lot of trouble on properly sed'ing content that may or
may not be in the containerd configuration file.
Fixes: #8638
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to have an easier time playing with the containerd
configuration, instead of having to sed the **** out of it, which is
super error prone.
`tomlq` is a tool that comes from https://github.com/kislyuk/yq, and
that depends on `jq` to do the toml parsing / editing.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Direct the Makefile to use the static checks script in the tests
directory of the main Kata Containers repo so it is run in GHA.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Some copyright dates were not updated with the most recent changes to
code; update them.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Change directory for running make due to local errors when building with
make -C.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Make the CH (Cloud Hypervisor) `stop_vm()` method check the VM state before
attempting to stop the VM, and update the state once the VM has stopped.
This avoids the method failing if called multiple times which will
happen if the workload exits before the container manager requests that
the container stop.
This change ensures the CH driver finishes cleanly.
Fixes: #8629.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR updates the TensorFlow ResNet50 Int8 Dockerfile to use the
proper python version for kata metrics.
Fixes#8643
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add a `--show-default-config-paths` command line option for parity with
`kata-runtime`.
Note that this requires the `KataCtlCli.command` to be optional so that
the user can run simply:
```bash
$ kata-ctl --show-default-config-paths
```
... without also specifying a (sub-)command.
Fixes: #8640.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Fix paths for yqdir (where the install_yq.sh script currently is) so
that static checks can run without error.
Fixes#8595
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
The layout of packed virtqueue isn't supported by `Endpoint::negotiate()`.
Communication between device and driver will be failed due to the failure
of parsing virtqueue if we don't disable the packed feature. This patch
fixes this issue.
Fixes: #8633
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Right now, cargo fmt check in Dragonball only test with the default
features but not all features. This will cause some code being untested
by the fmt tool.
This PR adds --all option for the Dragonball CI and also fix some code
that forgets to do cargo fmt --all.
fixes: #8598
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This is to fix a broken usage for `k3s kubectl version` by switching
an option `--short` to `--client=true`.
Fixes: #8621
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Make the URL checker cycle through a list of user agent values until we
hit one the remote server is happy with.
This is required since, unfortunately, we really, really want to check
these URLs, but some sites block clients based on their `User-Agent`
(UA) request header value. And of course, each site is different and can
change its behaviour at any time.
Our strategy therefore is to try various UA's until we find one the
server accepts:
- No explicit UA (use `curl`'s default)
- Explicitly no UA.
- A blank UA.
- Partial UA values for various CLI tools.
- Partial UA values for various console web browsers.
- Partial UA for Emacs's built-in browser.
- The existing UA which is used as a "last ditch" attempt where the UA implies multiple platforms and browser.
> **Notes:**
>
> - The "partial UA" values specify specify the UA "product" but not the
> UA "product version": we specify `foo` and not `foo/1.2.3`). We do
> this since most sites tested appear to not care about the version.
> This is as expected given that the version is strictly optional (see `[*]`).
>
> - We now log all errors and display an error summary if none of the UAs
> worked, in addition to the simple list of the URLs we believe to be
> invalid. This should make future debugging simpler.
`[*]` - https://www.rfc-editor.org/rfc/rfc9110#section-10.1.5Fixes: #8553.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed the Azure Portal URL (https://portal.azure.com) since this
causes problems with our static checks script: that URL returns HTTP 403
("Forbidden") when queried using command-line tools like `curl(1)`,
which is used by the static check script.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Split the call to `curl` in the URL checker out into a new
`run_url_check_cmd()` function to make `check_url()` slightly clearer.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
If a wrong configuration.toml file is used by accidentally, runtime-rs
binary could run into panic because of unwrap().
This fixes the panic by returning errors instead of unwrap().
fixes: #8565
Signed-off-by: Liu Bo <liub.liubo@gmail.com>
This is to make GHA secrets inherited for the workflow titled
`build-kata-static-tarball-s390x` to configure an environment
variable `CI_HKD_PATH` for a `build-asset-boot-image-se` step.
Fixes: #8611
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Log-parser-rs was always intended to become a sub-functionality of
kata-ctl, but it was useful to develop it and initaly merge it as a
standalone program, and migrate it to a subcommand later.
Fixes#6797
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
A prerequisite for measuring kata network bandwidth is
run Iperf3 tool at a the transport layer provided by a
k8s service for exposing a network where the clients
inside the cluster can use to contact Pods in the service.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR fixes small issues like:
1. Cleaning up the k8s environment by removing the iperf test
implementation even when the test fails.
2. Checks if the workload returned a result before generating
an empty results json file as it was bein done.
3. Removes the redundancy of calls to functions that process
subtests and should compose the results json file only when
all results are ready and not before.
4. The tcp service manifest was added to the server deployment
which targets TCP port 5201.
Fixes: #8534
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Since cloud-hypervisor is no longer built as an optional feature,
lets mention cloud-hypervisor in the list of hypervisors supported by
runtime-rs.
Fixes: #8587
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
It is to remove the build redundancy of `kernel` and `rootfs-initrd` by making `boot-image-se` built based on them at the second build stage.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make `build_se_image.sh` incorporate the key verification originally supported by `genprotimg`.
It can be achieved by specifying two environment variables called `SIGNING_KEY_CERT_PATH` and `INTERMEDIATE_CA_CERT_PATH`.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make kernel parameters configurable during the secure image build by adding an environment variable SE_KERNEL_PARAMS.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This is to make a base builder image build genprotimg without a package
manager under the cross-compilation environment.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the Cloud Hypervisor driver, integrated with the runtime-rs,
as part of the kubernetes tests.
Fixes#8559
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR enables but do not run the nerdctl tests for cloud hypervisor
runtime-rs until we find out how stable they are.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It is to remove a GITHUB_WORKSPACE directory for self-hosted runners
when a workflow fails due to the merge conflict. This will prevent
the subsequent workflows from getting stuck in the same situation.
Fixes: #8600
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Implement try_from trait function to convert runtime-rs BlockConfig
to cloud-hypervisor DiskConfig. This can allow for code reuse in the
future.
Fixes: #8581
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR updates the python version for the TensorFlow ResNet FP32
dockerfile so the benchmark can run without issues.
Fixes#8593
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Fixes: #8508
Create a stable overlay for kata-deploy.yaml so we do not have to maintain two files, only one.
Single source for both. This is also preparation for the helm-overlay
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This patch implements the virtio-fs device used for filesystem sharing
and heavily based on the vhost-user protocol.
This vhost-user-fs device defines 5 parameters:
- path: vhost-user socket path
- tag: mount tag used from the guest to mount the filesystem
- req_num_queues: number of request virtqueues
- queue_size: depth of each virtqueue
- cache_size: cache window size for dax
This device needs to be defined before the VM instance is started,
which can be done through the dbs-cli tool with --fs option:
--fs '{
"sock_path":"/path/to/virtiofs.socket",
"tag":"myfs",
"num_queues":1,
"queue_size":1024,
"cache_size":0,
"thread_pool_size":1,
"cache_policy":"auto",
"writeback_cache":true,
"no_open":true,
"xattr":true,
"drop_sys_resource":false,
"mode":"vhostuser",
"fuse_killpriv_v2":true,
"no_readdir":false,
}'
Fixes: #8428
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
This is required for clh to work with nerdtcl and docker.
This fixes the issues seen with nerdctl while starting a container.
Hoewever, container exit with docker is still broken due to an unrelated
issue.
Fixes: #8579
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR updates the config containerd url link in the containerd
kata documentation.
Fixes#8577
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Fixes: #8569
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make the life easier for dragonball developers to properly
enable the tests once the tests are ready.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #8483 changed the location of the rust runtime config files to
`/etc/kata-containers/runtime-rs/`. However, if you haven't updated your
system to create that directory, attempting to create a container using
the rust runtime was giving the following cryptic message
(formatted for easier reading):
```
failed to handler message try init runtime instance
Caused by:
0: load config
1: load toml config
2: entity not found
```
Now, the message is as follows (again, reformatted for easier reading):
```
failed to handle message try init runtime instance
Caused by:
0: load config
1: load TOML config failed (tried [
\"/etc/kata-containers/runtime-rs/configuration.toml\",
\"/usr/share/defaults/kata-containers/runtime-rs/configuration.toml\",
\"/opt/kata/share/defaults/kata-containers/runtime-rs/configuration.toml\"
])
```
Fixes: #8557.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This will ensure no job will be stuck forever, as we've noticed with a
few jobs already.
Fixes: #8572
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the cri installation url link for the containerd
documentation.
Fixes#8539
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Cloud Hypervisor driver, integraedwith the runtime-rs,
as part of the stability tests.
Fixes#8462
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The driver enables the userspace interface to communicate with the TDX
module to request the TDX guest details, like the attestation report.
Fixes: #8555
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Bring support for legacy vsock and add Vsock to the ResourceConfig
enum type, and add the processing flow of the Vsock device to the
prepare_before_start_vm function.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Instroduce prepare_vm_socket_config to VirtSandbox for vm
socket config, including Vsock and Hybrid Vsock.
Use the capabilities() trait of the hypervisor to get the
vm socket supported in VMM.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Add Cap of HybridVsockSupport for hypervisors CLH and Dragonball
which use hybrid-vsock, default for Qemu, which uses legacy vsock.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce HybridVsock Cap to judge which kind of vm socket will
be supported by the Hypervisor.
Use `is_hybrid_vsock_supported` to tell if an hypervisor supports
hybrid-vsock, if not, it supports legacy vsock.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Remove the redundant `kata-ctl` `root` check when running the `env`
command. This check duplicated the `GuestProtection` check, and that
check is now no longer necessary anyway.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
It is no longer necessary to be `root` to query the guest protection
(TDX) on `x86_64` systems, so drop the requirement.
> **Note:**
>
> This change drops the `nix` `Uid` import required for the `root` check.
> But at the same time it adds it for PPC64le since that implementation of
> `available_guest_protection()` needs it and it was previously missing.
Fixes: #8548.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
PR #8311 inadvertently broke the logging since no log messages below the
`Info` level are logged now, regardless of the requested log level.
Resolve the issue by storing the requested log level in the
`RuntimeComponentLevelFilter` and using that level in the `log()`
function, rather than hard-coding `Info` as the default where no entry
is found in the `FILTER_RULE` hashmap.
Fixes: #8546.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Device mapper is the only supported block device driver so far,
which seems limiting. Kata Containers can work well with other
block devices. It is necessary to enhance supporting of multiple
kinds of host block device.
Fixes#4714
Signed-off-by: yuchen.cc <yuchen.cc@alibaba-inc.com>
This reverts commit b0157ad73a.
```
commit b0157ad73a
Refs: 3.3.0-alpha0-124-gb0157ad73
Author: Fabiano Fidêncio <fabiano.fidencio@intel.com>
AuthorDate: Fri Aug 11 14:55:11 2023 +0200
Commit: Fabiano Fidêncio <fabiano.fidencio@intel.com>
CommitDate: Fri Nov 10 12:58:20 2023 +0100
runtime: confidential: Do not set the max_vcpu to cpu
We don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
```
This commit was removing a requirement that was made previously, but due
to the SMP issue we're facing with the QEMU used for TDX (see commit
d1b54ede290e95762099fff4e0bcdad10f816126*), QEMU will fail to start due
to:
```
Invalid CPU topology: product of the hierarchy must match maxcpus:
sockets (1) * dies (1) * cores (1) * threads (1) != maxcpus (240)"
```
This has no affect on the SEV / SNP workflow and hopefully we'll be able
to re-revet this soon enough, when this gets solved on te QEMU side.
Last but not least, this is not a "clean" revert as we're using
conf.NumVCPUs() instead of conf.NumVCPUs, to ensure we're dealing with
uint32.
Fixes: #8532
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is a commit that's a pre-req for #6826, as that PR will merge
log-parser-rs into kata-ctl, but that will result in a CI breakage.
So, let's deal with the CI changes here, thanks to GHA and our favourite
`pull_request_target` event, unblocking that PR to be merged.
Fixes: #6797 (not really, but related).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Print more information useful for debugging. Also, use a separate YAML
file for this test, instead of reusing someone else's file.
Fixes: #8270
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
- Update the remote hypervisor code to match the re-genned code for
the ttrpc Hypervisor Service
Fixes: #8519
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
As the configuration for the runtime-rs based drivers are now placed in
a different location than the golang ones, we should adapt this script
accordingly.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`kata-ctl` is the tool for runtime-rs, and it should be used instead of
`kata-runtime`.
`kata-ctl` requires sudo, and that's the reason it's also been added as
part of the calls.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`kata-runtime env` is an alias for `kata-runtime kata-env, and calling
it with the `env` paramenter allows us to easily extend the scripts to
use `kata-ctl` instead of `kata-runtime` when dealing with runtime-rs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've done some changes in the runtime-rs based drivers to install
their configuration into a different location, this should also be
reflected as part of this test.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Updates to scripts for static-checks.sh functionality, including common
functions location, the move of several common functions to the existing
common.bash, adding hadolint and xurls to the versions file, and changes
to static checks for running in the main kata containers repo.
The changes to the vendor check include searching for existing go.mod
files but no other changes to expand the test.
Fixes#8187
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
This reverts commit 44899d4cdf, as we've
decided to keep both golang and rust runtime installable and usable at
the same time.
The decision of having both runtimes installable and usable will help
users to test and easily catch any possible differences between those
runtimes, helping us to get on par with both implementations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This change for now doesn't do much, apart from making it easier to
expand which runtimes should be linked to the runtime-rs containerd shim
binary.
Also, this matches the logic used for the config files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update `kata-deploy` to modify the rust runtime configuration files in
their new `runtime-rs/` directory.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR updates the iperf3 network documentation to include
the parallel bandwidth.
Fixes#8523
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Implements the following test case:
Scenario: Check incorrect hash fails
**Given** I have a version of kata installed that has a kernel with the
initramfs built and config with rootfs_verity.scheme=dm-verity
rootfs_verity.hash=<incorrect hash of rootfs> set in the kernel_params
**When** I try and create a container a basic pod
**Then** The pod is doesn't run
**And** Ideally we'd get a helpful message to indicate why
Currently on CI only qemu-tdx is built with measured
rootfs support in the kernel, so the test is restriced to that
runtimeclass.
Fixes#7415
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Bring the setup_common() from CCv0 branch test's
integration/kubernetes/confidential/tests_common.sh. It should be used
to reduce boilerplates on the setup() of the tests.
Unlike the original code, this won't export the `test_start_time` variable
as it wouldn't be accurate to grab logs from the worker nodes due
date/time mismatch between the running tests machine and the worker
node. The function export the `node` variable which holds the name of
a random node which has kata installed. Apart from that, it exports the
`node_start_time` which capture the date/time when the test started,
relative to the `node`.
Tests that should inspect the logs can schedule pods/resources to the `node`
and use `node_start_time` as the value reference to grep the logs.
Fixes#7590
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Bring the assert_logs_contain() from CCv0 branch tests'
integration/kubernetes/confidential/lib.sh.
Introduced the print_node_journal() which uses `kubectl debug` to print
the systemd's journal of a k8s's node.
Fixes#7590
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This new function allow to the annotations to metadata section in a yaml
configuration file.
Co-authored-by: Ryan Savino <ryan.savino@amd.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The new clean-generated-files make target allows for removing the
generated files (including the configuration.toml files).
The tools/packaging/static-build/shim-v2/build.sh script now uses that
target to always force the re-generation of those files.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Copied the new_pod_config() and pod-config.yaml.in from CCv0 branch
tests' integration/kubernetes/confidential/tests_common.sh and fixtures.
Unlike the original version, new_pod_config() now gets the runtimeclass
by parameter as the RUNTIMECLASS environment variable seems not broadly
used on main branch's CI.
The pod-config.yaml.in was changed as the diff shows below. In
particular the imagePullSecrets was removed to avoid it throwing a
warning on the pod's log.
```
--- a/tests/integration/kubernetes/runtimeclass_workloads/pod-config.yaml.in
+++ b/tests/integration/kubernetes/runtimeclass_workloads/pod-config.yaml.in
@@ -5,12 +5,10 @@
apiVersion: v1
kind: Pod
metadata:
- name: busybox-cc
+ name: test-e2e
spec:
runtimeClassName: $RUNTIMECLASS
containers:
- - name: nginx
+ - name: test_container
image: $IMAGE
- imagePullPolicy: Always
- imagePullSecrets:
- - name: cococred
\ No newline at end of file
+ imagePullPolicy: Always
\ No newline at end of file
```
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The following functions were copied from CCv0's branch test's
integration/kubernetes/confidential/lib.sh. I did just smalls
refactorings (shortened their names and delinted shellcheck warnings):
- k8s_delete_all_pods_if_any_exists()
- k8s_wait_pod_be_ready()
- k8s_create_pod()
- assert_pod_fail()
Co-authored-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Co-authored-by: Georgina Kinge <georgina.kinge@ibm.com>
Co-authored-by: Jordan Jackson <jordan.jackson@ibm.com>
Co-authored-by: Megan Wright <Megan.Wright@ibm.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Co-authored-by: Wang, Arron <arron.wang@intel.com>
Re-wrote the logic of init.sh to follow the rules:
* the root device MUST exist always because it will be either mounted
or verified (then mounted)
* if rootfs verifier is enabled then the hash device MUST exist. Avoid
the case where dm-verity is set but the hash device does not exist and
so the verification is silently skipped
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
When measured toofs is enabled then the shim-v2 build should find the
guest rootfs hash file, otherwise might (silently) generate configuration
files with empty hash.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The KATA_BUILD_CC variable plus the existence (or not) of the initramfs
were used to determine whether to build the kernel for measured rootfs
or not. Currently the variable MEASURED_ROOTFS has been used
to trigger the feature build and when it is activated it should expect
the initramfs exist. In other words, this changed the kernel build
so that if `MEASURED_ROOTFS=yes` then the initramf file must exist and
be found.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Moved the measure rootfs logic from kata-deploy-binaries.sh to the
shim-v2's builder script so that the former get less bloated with
components's specific code.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Moved the measure rootfs logic from kata-deploy-binaries.sh to the
kernel's builder script so that the former get less bloated with
components's specific code.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
By convention the caller of tools/packaging/kernel/build-kernel.sh changes
the script behavior by passing arguments, whereas, for measured rootfs
it has used an environment variable (MEASURED_ROOTFS). This refactor
the script so that the caller now must pass the "-m" argument to enable
the build of the kernel with measured rootfs support.
Fixes#6674
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
As the configuration files are different, we can safely remove those as
any new installation of the binary should also bring in the new
configurations.
This makes things less error-prone in the future, as we're ensuring that
the rust runtime will only be reading the rust configuration files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update the `DEFAULT_RUNTIME_CONFIGURATIONS` list to include a number of
rust runtime specific paths to try to load before checking the
"traditional" (golang) runtime configuration paths.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Install the rust runtime configuration files to a `runtime-rs/`
directory to distinguish them from the golang config files (which may
have a different syntax).
The default values mean that the rust config files are now installed to
`/opt/kata/share/defaults/kata-containers/runtime-rs/` rather than
`/opt/kata/share/defaults/kata-containers/`.
See: https://github.com/kata-containers/kata-containers/issues/6020Fixes: #8444.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
(1) Add enum DirectVolumeType for direct volumes.
(2) Reimplement spdk volume into direct_volume and
do alignment of rawblock volume.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
The current infra(K8S, CSI, CRI, Containerd) for Kata containers is
unable to properly handle direct volumes, resulting in the need for
workarounds like searching/comparision and then patch up volume type.
In this commit, reimplement of handling method is added to support
raw block volume which backends may be rawdisk or other format file.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As vsock device will be used in Qemu or other VMMs, the Vsoock
is reintroduced to DeviceType enum.
Fixes: #8474
Signed-off-by: Pavel Mores <pmores@redhat.com>
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Currently encounters difficulty in utilizing the clone operation
on VsockConfig due to the implicit management of the vhost fd
within the runtime-rs. This responsibility should be delegated to
the VMM(especially QEMU) child process, as it's not runtime-rs core
responsibilities. We'll remove the member vhost_fd from VsockConfig
and make the VsockConfig/VsockDevice Cloneable.
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Introduce a new function generate_vhost_vsock_cid to generate
a guest CID and set guest CID for vsock fd.
Also this commit wouldn't introduce functional change and it's
just splited from the previous VsockDevice::new().
Fixes: #8474
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
(1) rawblock volume is directvol mount type.
(2) block volume is based on the bind mount type.
Fixes: #8300
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
1) Creating storage for all `io.katacontainers.volume=` messages in rootFs.Options,
and then aggregates all storages into `containerStorages`.
2) Creating storage for other data volumes and push them into `volumeStorages`.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
To enhance the construction and administration of `Katavirtualvolume` storages,
this commit expands the 'sharedFile' structure to manage both
rootfs storages(`containerStorages`) including `Katavirtualvolume` and other data volumes storages(`volumeStorages`).
NOTE: `volumeStorages` is intended for future extensions to support Kubernetes data volumes.
Currently, `KataVirtualVolume` is exclusively employed for container rootfs, hence only `containerStorages` is actively utilized.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
The snapshotter will place `KataVirtualVolume` information
into 'rootfs.options' and commence with the prefix 'io.katacontainers.volume='.
The purpose of this commit is to transform the encapsulated KataVirtualVolume data into device information.
Fixes: #8495
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Co-authored-by: Feng Wang <feng.wang@databricks.com>
Co-authored-by: Samuel Ortiz <sameo@linux.intel.com>
Co-authored-by: Wedson Almeida Filho <walmeida@microsoft.com>
Update cloud hypervisor implementation to allow hybrid vsock device to
be handled.
Fixes#6692
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
In the case of Cloud Hypervisor running on arm64 architecture,
only arm AMBA UART (pl011) is supported as the TTY. Consequently,
when enabling Hypervisor debug mode, it's essential to configure
the console as "ttyAMA0" rather than "ttyS0
Fixes: #8381
Signed-off-by: briwan01 <brian.wang@arm.com>
- statically linked qemu requires slof.bin to run, hence remove it from blacklist
- By default, initrd is used for Power, modify the configuration.toml accordingly
Fixes: #8458
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
The test utils will be used by the upcoming feature tests: vhost-user-net,
vhost-user-blk and vhost-user-fs.
Signed-off-by: Beiyue <beiyue@linux.alibaba.com>
Signed-off-by: Huang Jianan <jnhuang@linux.alibaba.com>
It mainly focus on the two parts:
(1) redesign the ShareFsConfig with ShareFsMountConfig
The device mount operation must depend on the fact that sharefs
device exists, and re-design the structure of SharesFsConfig and
move the ShareFsMountConfig into it with Option type, which is to
describe the relation between ShareFsConfig and ShareFsMountConfig.
(2) move virtiofs into device manager
Currently, virtio-fs is still outside of the device manager.
To do Enhancement of device manager, it will bring virtio-fs
device in device-manager for unified management
Fixes: #7915
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
As a follow-up PR for #8404, this is to set a default value for an environment variable `ALLOWED_HYPERVISOR_ANNOTATIONS`.
This will prevent a pod launching without an explicit configuration for the variable from getting into a `CrashLoop` state.
Fixes: #8477
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
In order to support PCI VFIO functionality in Dragonball, we should
first add PCI bus and PCI device Interrupt information in Dragonball
mptable setup process.
This patch add :
1. pci_legacy_irqs transfered to setup_mptable function.
2. pci bus support in mptable mem
3. pci interrupt support in mptable mem
fixes: #8449
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This will be used for selecting the correct runtimes and runtimeclasses
to be deployed with kata-deploy.
Fixes: #8475
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add the corresponding data structure in the runtime part according to
kata-containers/kata-containers/pull/7698.
Fixes: #8472
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
This PR fixes the result finding for the general throughput for
the tensorflow benchmark.
Fixes#8466
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
It's CCv0 specific for now, and it's needed as the Operator is now
delegating the runtimeclass creation to the kata-deploy daemonset.
Fixes: #7550
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 2df6cb7609)
In order to support different pod VM instance type via
remote hypervisor implementation (cloud-api-adaptor),
we need to pass machine_type, default_vcpus
and default_memory annotations to cloud-api-adaptor.
The cloud-api-adaptor then uses these annotations to spin
up the appropriate cloud instance.
Reference PR for cloud-api-adaptor
https://github.com/confidential-containers/cloud-api-adaptor/pull/1088Fixes: #7140
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
(based on commit 004f07f076)
This patch updates the template configuration file for
the remote hypervisor to set static_sandbox_resource_mgmt
to be true. The remote hypervisor uses the peer pod config
to determine the sandbox size, so requires this to be set to
true by default.
Fixes: #6616
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(based on commit 938447803b)
Add the SELinux setting to ensure it is passed through to the remote
hypervisor
Fixes: #5936
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(based on commit 3ef2fd1784)
This patch adds the support of the remote hypervisor type.
Shim opens a Unix domain socket specified in the config file,
and sends TTPRC requests to a external process to control
sandbox VMs.
Fixes#4482
Co-authored-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(based on commit f9278f22c3)
This patch adds a protobuf definiton of the remote hypervisor type.
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
Co-authored-by: stevenhorsman <steven@uk.ibm.com>
(based on commit 150e8aba6d)
This PR fixes k8's configmap/secrets etc update propagation when filesystem sharing is disabled.
The commit introduces below changes with some limitations:
- creates new timestamped directory in guest
- updates the '..data' symlink
- creates user visible symlinks to newly created secrets.
- Limitation: The older timestamped directory and stale user visible symlinks exist in guest
due to missing DELETE api in agent.
Fixes: #7398
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
As part of the CI migration, this PR is to add workflows for containerd and k8s for s390x.
Fixes: #7930
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Improve the `CODEOWNERS` file by specifying more groups.
Since GitHub automatically checks the `CODEOWNERS` file when a PR is
created and adds all matching groups as reviewers for the PR, this may
help reduce the PR backlog since the right people will be alerted and
requested to review the PR. That should improve the quality of reviews
(and thus the quality of the landed code). It may also have a positive
effect on PR velocity.
> **Note:**
>
> This PR combines the other `CODEOWNERS` files so we have
> a single, visible, top-level file.
See: https://github.com/kata-containers/community/issues/253Fixes: #3804.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This commit enables StratoVirt hypervisor to be tested in kata GHA,
incluing k8s, metrics, cri-containerd, nydus and so on.
Meanwhile, adding some unit tests for StratoVirt to make sure it works.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Allow kata-deploy process to pull StratoVirt from release binaries, and
add them as a part of kata release.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Add configuration-stratovirt.toml.in to generate the StratoVirt configuration,
and parser to deliver config to StratoVirt.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
Initial support of the MicroVM machine type of StratoVirt
hypervisor for the kata go runtime.
Fixes: #7794
Signed-off-by: Liu Wenyuan <liuwenyuan9@huawei.com>
This PR adds the iperf udp information to the network README
for the kata metrics CI.
Fixes#8452
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When multiple containers in a kata pod share one direct volume,
it's important to make sure that the corresponding block device
is only mounted once in the guest. This means that there should
be only one mount entry for the device in the mount information.
Fixes: #8328
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When a direct volume is used by multiple containers in Kata,
Generating many shared paths with cids will cause IO error
as the result of one direct volume mounts more than once.
To correct it, use the device_id instead of cid which
ensures that the guest only mounts the FS once.
Fixes: #8328
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This was added as part of d4d65bed38, but
install_kata has never actually used the passed enable_debug var.
With this in mind, let's just remove it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this change, we give the users the change to try kata-containers
with their own pre-built tarball.
This will become very useful in the CI context, as we won't be
downloading a specific version of kata-containers, but rather installing
whatever was built in previous steps of the CI pipeline.
Fixes: #8438
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The virtio vsock driver has a small window during initialization
where it can silently drop replies to connection requests.
Because no reply is sent, kata waits for 10 seconds and in the
end it generates a connection timeout error in HybridVSockDialer.
Fixes: #8291
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
This patch is to remove vhost-net dependency on virtio-net for
dbs-virtio-devices crate. Then, the feature of vhost-net is able to enable
without enabling virtio-net device, error, etc.
Fixes: #8423
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Introduce the `update_device` trait in Hypervisor to enable
device updates for VMMs.This trait will initially be utilized
for virtiofs Mount operations.
Fixes: #7915
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Two workflows, run-nerdctl-tests-on-garm.yaml and
run-docker-tests-on-garm.yaml, are removed from commit b481d39. However,
they are referenced by CI workflow. It leads to the CI not working
properly. This patch is to remove those files from ci.yaml.
Fixes: #8433
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
There's no need to keep those as separate files, and by having those in
the basic-ci-amd64.yaml file actually helps us to avoid the
undocummented GHA limitation about the number of files imported.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Peng Tao made this move as part of 1280f85343, and here we're
simply adjusting to the move.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #8311 inadvertently broke the runtime-rs / Cloud Hypervisor TDX
handling. It also introduced unrecoverable failure scenarios. Hence,
replace slow, fallible regex matching in logging fast path with single pass
non-failing multi-string log level matching.
Also, added a unit test for `parse_ch_log_level()`.
Fixes: #8418.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
- Remove two panic statements from InsertNetworkDevice test.
- Rename `NUM_QUEUES` to `DEFAULT_NUM_QUEUES`, `QUEUE_SIZE` to
`DEFAULT_QUEUE_SIZE` for vhost-net and virtio-net.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
`test_networkconfig_to_netconfig` from clh depends on `NetworkConfig` which
has some new fields in this PR. Therefore, this commit gives the test
missing fields.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
- Dragonball's vhost-net feature not depends on virtio-net feature.
- Remove `TapError` from dbs-virtio-devices's Error, and add `VirtioNet`
and `VhostNet` two fields.
- Downgrade visiblity of two fields of `VhostNetDeviceMgr` from
`pub(crate)`.
- File an issue to record a todo for network rate limiter.
- Print internal errors with `{0:?}.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
As we've done some changes in the VMM vcpu allocation, let's introduce
basic tests to make sure that we're getting the expected behaviour.
The test consists in checking 3 scenarios:
* default_vcpus = 0 | no limits set
* this should allocate 1 vcpu
* default_vcpus = 0.75 | limits set to 0.25
* this should allocate 1 vcpu
* default_vcpus = 0.75 | limits set to 1.2
* this should allocate 2 vcpus
The tests are very basic, but they do ensure we're rounding things up to
what the new logic is supposed to do.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With the change done in the last commit, instead of calculating milli
cpus, we're actually converting the CPUs to a fraction number, a float.
Let's update the function name (and associated vars) to represent that
change.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
First of all, this is a controversial piece, and I know that.
In this commit we're trying to make a less greedy approach regards the
amount of vCPUs we allocate for the VMM, which will be advantageous
mainly when using the `static_sandbox_resource_mgmt` feature, which is
used by the confidential guests.
The current approach we have basically does:
* Gets the amount of vCPUs set in the config (an integer)
* Gets the amount of vCPUs set as limit (an integer)
* Sum those up
* Starts / Updates the VMM to use that total amount of vCPUs
The fact we're dealing with integers is logical, as we cannot request
500m vCPUs to the VMMs. However, it leads us to, in several cases, be
wasting one vCPU.
Let's take the example that we know the VMM requires 500m vCPUs to be
running, and the workload sets 250m vCPUs as a resource limit.
In that case, we'd do:
* Gets the amount of vCPUs set in the config: 1
* Gets the amount of vCPUs set as limit: ceil(0.25)
* 1 + ceil(0.25) = 1 + 1 = 2 vCPUs
* Starts / Updates the VMM to use 2 vCPUs
With the logic changed here, what we're doing is considering everything
as float till just before we start / update the VMM. So, the flow
describe above would be:
* Gets the amount of vCPUs set in the config: 0.5
* Gets the amount of vCPUs set as limit: 0.25
* ceil(0.5 + 0.25) = 1 vCPUs
* Starts / Updates the VMM to use 1 vCPUs
In the way I've written this patch we introduce zero regressions, as
the default values set are still the same, and those will only be
changed for the TEE use cases (although I can see firecracker, or any
other user of `static_sandbox_resource_mgmt=true` taking advantage of
this).
There's, though, an implicit assumption in this patch that we'd need to
make explicit, and that's that the default_vcpus / default_memory is the
amount of vcpus / memory required by the VMM, and absolutely nothing
else. Also, the amount set there should be reflected in the
podOverhead for the specific runtime class.
One other possible approach, which I am not that much in favour of
taking as I think it's **less clear**, is that we could actually get the
podOverhead amount, subtract it from the default_vcpus (treating the
result as a float), then sum up what the user set as limit (as a float),
and finally ceil the result. It could work, but IMHO this is **less
clear**, and **less explicit** on what we're actually doing, and how the
default_vcpus / default_memory should be used.
Fixes: #6909
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
We don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a command-line option to list the installed and available versions
of Kata and containerd.
Fixes: #8355.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Rather than creating a container called `test-kata`, prefix with the
script name to make it a bit "more unique" and less likely for users to
have an existing container with the test container name. The new test
container name is `kata-manager-sh-test-kata`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Update `github_get_latest_release()` to use `sort -V` rather than
sub-sorting on the major, minor and patch level version number elements.
The new approach is safer and more accurate.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add an extra sanity check to ensure that only a single download URL is
found for the specified release version.
Fixes: #8364.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve some of the information messages a little by adding
more detail and quoting file names.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
As "/etc/containerd/config.toml" is used from more than one place, let's
just make it a global var.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for also installing the Docker CLI, giving users the chance
to try Kata Containers with docker in the same way we provide users the
chance to try Kata Containers with `ctr`.
Fixes: #8357
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The current kata-deploy code has been doing a `sed` to add allowed
hypervisor annotations, so CBL mariner can be tested with their own
kernel and initrd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently the only way one can specify allowed hypervisor annotations is
during build time, which is a big issue for users grabbing kata-deploy
as we provide.
Fixes: #8403
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now we're only testing with `ctr` and there's no change in
behaviour with this commit. However, allowing to pass a tool to run the
tests with gives us an easier time when expanding kata-manager to
support, for instance, docker and nerdctl.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no reason to escape the first + on the +k3s[0-9]\+ regex, as
shown here:
```sh
ubuntu@k3s:~$ /usr/local/bin/k3s kubectl version --short 2>/dev/null | \
grep "Client Version" | \
sed \
-e 's/Client Version: //' \
-e 's/+k3s[0-9]\+//'
v1.27.7
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It seems that with the new k3s release, they've bumped their kubectl
version from x.y.z+k3s1 to x.y.z+k3s2.
Let's ensure our regexp is more generic and future proof for such
changes.
Fixes: #8410
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TestCheckHostIsVMContainerCapable removes sysModuleDir to simulate a
case that the kernel modules are not loaded. However,
checkKernelModules() executes modprobe <module> if a module not
found in that directory. Loading those modules is required to be denied
temporarily.
Fixes: #8390
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
- Disable device cgroup restriction while pod cgroup is not available.
- Remove balcklist-related names and change whitelist-related names to
allowed_all.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
`TestDeviceCgroup` is added to cri-containerd's integration tests. The test
launches two containers. Each container has a block device. It checks the
validity of device cgroup.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
FSManager of systemd cgroup manager is responsible for setting up cgroup
path. The container launching will be failed if the FSManager is in
read-only mode.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The changes include:
- Change to debug logging level for resources after processed.
- Remove a todo for pod cgroup cleanup.
- Add an anyhow context to `get_paths_and_mounts()`.
- Remove code which denys access to VMROOTFS since it won't take effect. If
blackmode is in use, the VMROOTFS will be denyed as default. Otherwise,
device cgroups won't be updated in whitelist mode.
- Add a unit test for `default_allowed_devices()`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The runk is a standard OCI runtime that isnt' aware of concept of sandbox.
Therefore, the `devcg_info` argument of `LinuxContainer::new()` is
unneccessary to be provided.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
The target is to guarantee that containers couldn't escape to access extra
devices, like vm rootfs, etc.
Assume that there is a cgroup, such as `/A/B`. The `B` is container cgroup,
and the `A` is what we called pod cgroup. No matter what permissions are
set for the container (`B`), the `A`'s permission is always `a *:* rwm`. It
leads that containers could acquire permission to access to other devices
in VM that not belongs to themselves.
In order to set devices cgroup properly, the order of setting cgroups is
that the pod cgroup comes first and the container cgroup comes after.
The `Sandbox` has a new field, `devcg_info`, to save cgroup states. To
avoid setting container cgroup too early, an initialization should be done
carefully. `inited`, one of the states, is a boolean to indicate if the pod
cgroup is initialized. If no, the pod cgroup should be created firstly, and
set default permissions. After that, the pause container cgroup is created
and inherits the permissions from the pod cgroup.
If whitelist mode which allows containers to access all devices in VM is
enabled, then device resources from OCI spec are ignored.
This feature not supports systemd cgroup and cgroup v2, since:
- Systemd cgroup implemented on Agent hasn't supported devices subsystem so
far, see: https://github.com/kata-containers/kata-containers/issues/7506.
- Cgroup v2's device controller depends on eBPF programs, which is out of
scope of cgroup.
Fixes: #7507
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Add test to verify kata supports ipvlan networks.
This test can be bit tricky as it requires knowledge about host interfaces
to be used as a master for the ipvlan network.
However, with github actions, we can assume interface called eth0 to be
present on the host and functioning.
Fixes: #8366
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Since moving from network coldplug to hotplug, the only case verified
was veth endpoints. Support for network hotplug for ipvlan and macvlan was
broken/not added. Fix it.
Fixes: #8391
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Remove the redundant `VmConfigError::EmptyVsockSocketPath` error from
the Cloud Hypervisor config crate since this scenario is already handled
by the `VsockConfigError::NoVsockSocketPath` error.
Fixes: #8385.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the redundant `parse_mac()` function: this was never used and we
already have an implementation in `crates/resource/src/network/utils/mod.rs`.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Virtio-net and vhost-net share a common virtio config, and vhost-user-net
uses another config, named `VhostUserConfig`. Thus, the virtio config could
be added into `NetworkConfig` instead of `Backend`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Moving Dragonball structs convertions out of device drivers to keep driver
neutral. The convertions include `NetworkBackend` to
`DragonballNetworkBackend` and `NetworkConfig` to
`DragonballNetworkConfig`.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Changes include:
- Merge `VhostNetDeviceError` import item.
- Replace if with match in `add_vhost_net_device()`
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Network backends determine the virtio dataplane implementations. Common
protocols include virtio-net, vhost-net and vhost-user-net, etc. Network
config has a new field named `backend` to specify which protocol to use.
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
PLEASE NOTE THAT this pull request just implements vhost-net support for
Dragonball, and adaptation for the Runtime-rs. And this pull request
DOESN'T provide an item to config which backend to use. To sum up,
virtio-net as a default backend is only choice for the user so far.
This pull request introduces vhost-net device for the Dragonball. In
addition, this pull request includes changes of Runtime-rs to improve
network configuration abilities.
The Dragonball part implements a vhost-net device and a vhost-net device
manager, named `VhostNetDeviceMgr`, to manage vhost-net device.
`NetworkInterfaceConfig` is introduced as a high-level abstract for network
config. Then, the Dragonball is able to distinguish network backends, e.g.
virtio-net, vhost-net, vhost-user-net(WIP), etc.
The Runtime-rs part adds support of multiple network backends as well.
`NetworkConfig` has a couple of new fields, like `backend`,
`use_shared_irq`, etc. And Dragonball's network config structs are
implmented `From` trait which allow to be converted from the Runtime-rs's
network config conveniently.
Fixes: #7674
Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: wllenyj <wllenyj@linux.alibaba.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Now that most of the test repository got migrated to the main Kata repository,
it is no longer needed to tag the test repository when doing a release.
Update the documentation accordingly by dropping all references to the test
repository and only mention *the* Kata repository.
Fixes#8302
Signed-off-by: Greg Kurz <groug@kaod.org>
The hub tool is deprecated. Releases are now based on the official gh
CLI. A notable improvement : when properly setup (see [1]), gh allows
to directly use HTTPS with one's GitHub credentials, instead of having
to setup proper SSH access for pushes to the repo.
Adjust the documentation accordingly.
Fixes#8302
[1] https://docs.github.com/en/github-cli/github-cli/quickstart#prerequisites
Signed-off-by: Greg Kurz <groug@kaod.org>
TestCheckHostIsVMContainerCapable is failing on AMD machines.
kata-check_amd64_test.go:96 has no AMD modules, also getCPUType is
missing.
Fixes#8384.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This update includes necessary changes due to the version bump of
containerd and its dependencies. It's part of a broader initiative to
phase out gogo protobuf, which has been deprecated, and to align with
the current supported libraries.
Fixes#7420.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The +fieldpath option, specific to gogoprotobuf, enabled dynamic field
access in protobuf messages, allowing nested fields to be accessed via
string paths.
This change is part of a larger effort to transition to the official Go
protobuf library for better maintainability and community support.
Upon review, no instances of dynamic field access were found in the
codebase, confirming that the feature is not in use.
By removing this unused feature, we simplify the build process and make
it easier to complete the transition away from gogoprotobuf.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Those mappings are not used by our .proto files and there is no
difference between .pb.go files generated.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This PR makes the change to using the SIGKILL signal instead
of SIGTERM to force stop each kata component before start
running any metric test.
Fixes: #8336
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Remove earlier functionality that tries to assign PCI path to vfio
devices from the host assuming pci slots to start from 1.
Get this from the hypervisor instead.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
If PCI path for block device is not empty for a block device, use
that as identifier for agent instead of virt path which is valid only
for mmio devices.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Block(virtio-blk) and vfio devices are currently not handled correctly
by the agent as the agent is not provided with correct PCI paths for
these devices.
The PCI paths for these devices can be inferred from the PCI information
provided by the hypervisor when the device is added.
Hence changing the add_device trait function to return a device copy
with PCI info potentially provided by the hypervisor. This can then be
provided to the agent to correctly detect devices within the VM.
This commit includes implementation for PCI info update for
cloud-hupervisor for virtio-blk devices with stubs provided for other
hypervisors.
Removing Vsock from the DeviceType enum as Vsock currently does not
implement the Device Trait, it has no attach and detach trait functions
among others. Part of the reason is because these functions require Vsock
to implement Clone trait as these functions need cloned copies to be
passed down the hypervisor.
The change introduced for returning a device copy from the add_device
hypervisor trait explicitly requires a device to implement
Copy trait. Hence removing Vsock from the DeviceType enum for now, as
its implementation is incomplete and not currently used.
Note, one of the blockers for adding the Clone trait to Vsock is that it
currently includes a file handle which cannot be cloned. For Clone and
Device Traits to be implemented for Vsock, it requires an implementation
change in the future for it to be cloneable.
Fixes: #8283
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This patch re-generates the client code for Cloud Hypervisor v35.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #8378
Signed-off-by: Bo Chen <chen.bo@intel.com>
kata-manager is a very nice tool, but we shouldn't be trying to take
care of "everything" in "all possible scenarios", and we should focus on
installing Kata Containers dependencies that are supported.
With this in mind, let's limit a little bit the scope of which versions
of containerd can be installed, limitting to "active" and "lts", which
will then install the latest version of those "flavours". The default
value will always be "lts" as that's supposed to be the stable one.
NOTE: This is a breaking change, as it changes the behaviour of what the
script takes in its `-c` parameter. I'm assuming here we're safe to do
so as the majority of the users should / would only be using the full
installation by default.
Fixes: #8356
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to avoid waiting till the next time cron would trigger
the action to test
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is causing the following error:
```
Unable to resolve action action/stale, repository not found
```
Fixes: #8347
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As that's an operation that can easily fail, and it's quite simple /
cheap for us to run it, let's just do it and avoid the failure.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Our goal. as discussed in the Architecture Committee meeting held on
October 31st, 2023, is to take a more aggressive action on issues and
PRs that have been opened for a long time.
This commit is the very first step, and it's **only** targetting
**PRs**. What this action will do is:
* Mark all the PRs that have no activity for more than 180 days,
starting from May 1st, 2023, as stale.
* A message will be added, letting the contributor know that they can
simply comment on the PR in order to make it "not stale".
* If there's no activity on the PR for 7 days, the PR will be
automatically closed.
Fixes: #8347
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix: Kata-Manager fails to retrieve the correct Containerd string name
for architectures other than amd64.
Update the 'github_get_release_file_url()' function to make it compatible
with different architecture expressions. eg. aarch64/arm64, or x86_64/amd64,
allowing it to acquire the correct URL addresses
Fixes: #7071
Signed-off-by: briwan.wang <briwan.wang@arm.com>
The Cargo.lock for agent needs to be updated to include
"safe-path" dependency.
Fixes: #8350
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
make check is giving us the following error:
error: this expression creates a reference which is immediately
dereferenced by the compiler.
Fixes#8344
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The hub tool is deprecated. Convert this script to use the
official GitHub CLI gh instead of hub.
A typical gh setup is able to access repos using HTTPS along with
GitHub credentials. It is only needed to patch the remote url when
using SSH.
Signed-off-by: Greg Kurz <groug@kaod.org>
The hub tool is deprecated. Convert this script to use the
official GitHub CLI gh instead of hub.
A couple of adjustments had to be made :
- the notes.md temporary file is moved to ${tmp_dir} in order to silent gh,
otherwise it complains about an untracked file,
- title of a PR no longer goes to the notes.md file since gh requires the
title to be passed with a dedicated --title option.
Fixes#8303
Signed-off-by: Greg Kurz <groug@kaod.org>
We don't want to mess with the official repo when testing a change
in the release scripts. Adapt `update-repository-version.sh` to
be able to use an alternate repo just like `tag_repos.sh` already
does.
This means that the following command :
$ OWNER="$SOME_ORG" ./update-repository-version.sh -p "$NEW_VERSION" "$BRANCH"
will only create a PR in this repo :
http://github.com/$SOME_ORG/kata-containers.git
Signed-off-by: Greg Kurz <groug@kaod.org>
By modifying RuntimeLevelFilter drain to improve logging control,
enabling isolation of change effect of the loggers between components,
tuning clh logs to be logged according to their log levels
given by cloud-hypervisor.
Fixes: #8310
Signed-off-by: Ruoqing He <linuxwatcher@outlook.com>
This PR removes double quotes in jq output to return raw strings
as input of checkmetrics tool.
Fixes: #8331
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR increases the number of attempts to stop kata components
when it is required usually before starting a metrics test.
Fixes: #8307
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR enables the new FIO test based on the containerd client
which is used to track the I/O metrics in the kata-ci environment.
Additionally this PR fixes the parsing of results.
Fixes: #8199
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR updates the values of the fio parameters for iodepth
requests and for the number of jobs, in order to increase the
number of sequential operations.
Additionally, it adds the list of packages needed to parse the
results.
Fixes: #8198
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the tracing capability for dragonball and it depends on the tracing::Subscriber of the upper layer.
Fixes: #7249
Signed-off-by: Songqian Li <mail@lisongqian.cn>
We used the approach of cold-plugging network interface for pre-shimv2
support for docker.Since the hotplug approach was not required,
we never really got to implementing hotplug support for certain network
endpoints, ipvlan and macvlan being among them.
Since moving to shimv2 interface as the default for
runtime, we switched to hotplugging the network interface for supporting
docker and nerdctl. This was done for veth endpoints only.
Implement the hot-attach apis for ipvlan and macvlan as well to support
ipvlan and macvlan networks with docker and nerdctl.
Fixes: #8333
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Legacy device manager adds device metrics to METRICS when a device is created and removes metrics when a device is dropped.
Fixes: #7248
Signed-off-by: Songqian Li <mail@lisongqian.cn>
Contained release files include the version number without a "v" prefix.
However, the tag for the equivalent release does include it so handle
this distinction and also tighten up the Kata check by specifying an
explicit version number in the regex.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In bare-metal machines the git tree might get on unstable state with the
previous rebase left halfway. So let's attempt to abort any rebase before.
Fixes#8318
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The archive names for x86_64 [Kata releases](https://github.com/kata-containers/kata-containers/releases)
used to include the tag `x86_64`, but that has now been changed to
`amd64`, which unfortunately broke `kata-manager.sh`:
```
kata-static-3.1.3-x86_64.tar.xz
~~~~~~
expected
kata-static-3.2.0-alpha3-x86_64.tar.xz
~~~~~~
expected
kata-static-3.2.0-alpha4-amd64.tar.xz
~~~~~
changed
```
Fixes: #8321.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve the code by fixing some lint issues:
- defining variables before using them.
- Using `grep -E` rather than `egrep`.
- Quoting variables.
- Adding a check for invalid CLI arguments.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
In the migration from the tests repo to the kata containers repo we
missed two huspell dictionaries for static checks; add them.
Fixes#8315
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add the hypervisor security details to the output of the `kata-runtime
env` and `kata-ctl env` commands so the user can see, amongst other
things, the value of `confidential_guest`.
Fixes: #8313.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The config template file for clh is in the new format for runtime-rs.
It is a result of merging the new format file and options supportted by
cloud-hypervisor.
Some config options from the golang runtime are missing as they may not
be currently supported by the rust runtime. An example of this is the
selinux options, rate limiting options as these are not currently
supported or verified with the rust runtime.
Fixes: #8249
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Balloon device manager adds balloon device metrics to METRICS when a device is created and remove metrics when a device is dropped.
Fixes: #7248
Signed-off-by: Songqian Li <mail@lisongqian.cn>
This is to skip a flaky test `create_tmpfs()` on s390x until a root cause is identified and fixed.
Fixes: #4248
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
hub is now deprecated, which has been causing issues with our release
process.
Let's move to the GH cli (https://cli.github.com/manual), and unblock
this release.
**NOTE**: This commit is purposefully not touching anywhere else hub is
used, as that would require more time and investigation to do the
switch, and right now we just want to unblock the release.
Fixes: #8286
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-manager currently adds default config which currently is qemu.
Add config for clh as well to containerd configuration.
This should allow new users to get started with clh using kata-manager.
Also add config related to enabling privileged_without_host_devices.
Always good to have this config enabled when users try to run privileged
containers so that devices from host are not inadverdantly passed to the
guest.
Fixes: #8280
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This is needed in order to properly run the CIs in branches that are not
the main one, as the kata-deploy.yaml file on those branches do not have
the `latest` tag, but rather the latest stable release.
Fixes: #8274
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- kata-deploy-stable: Switch to using the ubuntu based payload
- libs: protection: Fix typo in TDX output
- ci: k8s: Fix bogus firecracker check in k8s-credentials-secrets.bat
- tests: Enable agent stability test
- docs: Fix paths to build kernel in SNP VMs documentation
- runtime-rs: ch: Add TDX CH features check
- runtime: Validate hypervisor section name in config file
- tests: query data from the OPA service
- release: tag_repos: Stop tagging the `tests` repo
- metrics: fixes common.sh function to always return true
- Memory footprint test removing trailing commas to make json results file valid
- policy: allow access to ReseedRandomDev
- runtime/kata-ctl: update dependencies
- runtime-rs : fix Nydus support for runtime-rs + Dragonball
- metrics: removal of reference in the documentation to the fio dax subtest.
- runtime-rs: ch: Detect Intel TDX version
- runitme-rs: use the same base64 as kata-runtime/direct-volume does
- tests: Enable scability test for stability CI
- runtime-rs: Add support for adding vfio device for cloud-hypervisor
- tests: Enable soak parallel stability test
- dragonball: vcpu metrics change to be recorded per vcpu
- ci: k8s: adapt gha-run.sh to run locally
- metrics: removes kata components and k8s deployment when test finishes
- GHA: fix up referenced yaml exceeding 20 limit problem
- gha: ci: Revert tracing test PR to unbreak CI
- runtime-rs: ch: Enable feature
- gha: ci: Port runk tests over
- ci: gha: Port tracing tests over
- Enable fio test using containerd client
- gha: Add stability tests workflow for gha
- gha: arm64: Ensure the builder is arm64-builder
- kata-deploy: Build kata-agent as we build all the other components
- versions: migrate out of k8s.gcr.io
- doc: Update crictl pod-config
- gha: Fix k0s deployment
- tests: Add stability test for kata CI
- docs: Update url in kata vra document
- gpu: Adding CDI support for cold and hot-plug of VFIO devices
- kata-deploy: build & ship the rust components from src/tools/
- metrics: Add latency value limits for kata CI
- runtime: fix reading cgroup stats of sandboxes
- Upgrade to Cloud Hypervisor v35.0
- ci: Port kata-monitor tests from Jenkins to GHA
- metrics: Fix latency yamls path
- metrics: Fix metrics README
- metrics: Fix C-Ray documentation
- runtime-rs: ch: Enable Intel TDX
- ci: k8s: crio: Follow up patches to have CRI-O also working as part of our CI
- metrics: Enable latency test in gha run script
- local-build: Fix .docker ownership before build-payload
- runtime-rs: Add network support for cloud-hypervisor
- osbuild: Reduce guest components binary size with strip
- gha: Add pandoc as a dependency for static checks
- ci: rootfs-image build-asset is failing
- feat(runtime-rs): introduce huge page mode to select VM RAM's backend
- clh: Direct IO support for block devices
- gha: Install hunspell for static checks
- ci: Trigger payload-after-push on workflow_dispatch
- ci: Actually enable the CRI-O tests
- protocol: remove gogoprotobuff tests
- ci: k8s: Also run tests with CRI-O
- runtime: support kernel params including spaces
- ci: kata-deploy: Fix runner name
- metrics: Enable parallel bandwidth iperf limit
- ci: kata-deploy: Enable all k8s flavours that we support
- ci: Create clusters in individual resource groups
- versions: Bump virtiofsd to v1.8.0
- clh: arm: Use static_sandbox_resource_mgmt=true
- Bump nydus versions and update nydus tests
- runtime/qemu: Rework QMP/HMP support
- clh:arm64: use arm AMBA UART for hypervisor debug
- ci: Use variable size of VMs depending on the tests running
- ci: Rework static checks
- runtime: incorrect handling of non-empty []Endpoint parameter in Remo…
- ci: cache: Check the sha256sum of the components & fix ovmf-sev cache usage
- ci: cache: Use the artefacts stored in ghcr.io/kata-containers/cached-artefacts/${component}
- ci: Run some of the GARM tests in smaller instances
- ci: Reduce the size of the AKS VMs
- ci: cache: Allow pushing our artefacts to an OCI registry
- metrics: Add iperf value for cpu utilization
- ci: cache: Export env vars needed to use ORAS
- gha: vfio: Import test script
- tests: fix kernel and initrd annotations
- metrics: Add iperf bandwidth value for kata metrics
- metrics: Add Cassandra Metrics documentation
- metrics: Remove warning from metrics documentation
- ci: docker: nerdctl: Switch to tcp port 80 ping
- runtime: Naming conflict of network devices
- Remove gogoproto.nullable extension
- metrics: Ensure docker is running in init_env
- metrics: this PR skips the FIO test temprarily to fix issues
- ci: Add a very basic nerdctl sanity test
- runtime-rs: hypervisor: Remove debug kernel options
- versions: Bump rust version
- ci: Add a very basic docker sanity test
- dragonball: fix for non-deterministic builds
- runtime-rs: bring hybrid vsock devices in manager.
- ci: use github.ref_name instead of $GITHUB_REF_NAME
- ci: Add more target-branch related fixes
- ci: Fix target-branch usage
- agent: optimize the code of systemd cgroup manager
- gha: Manually rebase PR atop of the target branch before testing
- Update kernel to the latest LTS release (v6.1.52) and bring in erofs patches needed for the CC work
- kata-deploy: Fix aarch64 image build
- runtime: Fix more virtiofs args
- kata-deploy: Switch to an alpine image
- metrics: Use TensorFlow optimized image
- metrics: fix FIO test initialization
- ci: k8s: Add clean-up-garm argument for gha-run.sh
- ci: k8s: Second round of fix-ups with the devmapper CI
- metrics: re-enable memory-usage initialization step
- Dragonball: optimize the placement of dbs-upcall features
- ci: k8s: Fix typo in run-k8s-tests-on-garm.yaml
- ci: k8s: Add k8s devmapper tests (part 0)
- kata-deploy: Create kata-static.tar with correct ownership
- runtime: run prestart hooks before starting VM for FC
- metrics: Add write 95 percentile FIO value
- runtime: Allow virtio_fs_extra_args annotation
- packaging: do not install docker-compose-plugin for s390x|ppc64le
- runtime-rs: Fix volumes and rootfs cleanup issues
- metrics: Enable iperf benchmark on gha for kata metrics
- CI: switch static-checks-dragonball CI machines to Azure
- metrics: Add README for kata metrics report
- osbuilder: Remove chcon operation for guest SELinux
- kata-sys-util: protection: Update TDX checks
- Improve the way to clean up storage devices for sandbox
- agent: avoid possible leakage of storage device
- tests: add policy to existing tests
- gha: Rebase PR atop of the target branch before testing
- versions: Update alpine to its 3.18 version
- runtime: Fix data race in ioCopy
- metrics: Add grabdata script for metrics report
- Fixes tests on AMD machines
- metrics: Enable FIO limits for kata metrics
- metrics: Add metrics report script
- metrics: Fix memory inside limits for kata metrics
- metrics: fix parsing issue on memory-usage test
- dragonball: vsock add fifo/pipe stream support for passed fd hybridSt…
- tests: Add confidential test
- tdx: Update the components needed for using the 6.2 kernel stack
- tests: delete k8s deployment at the test's end
- tests: use unique test name
- runtime-rs: check peer close in log_forwarder
- gha: Avoid "fail-fast" in tests that are known to be flaky
- Refine storage device management for kata-agent
- metrics: Remove unused variable in tensorflow nhwc script
- kata-deploy: Don't try to remove /opt/kata
- metrics: Add TensorFlow ResNet50 FP32 benchmark
- gha: vfio: Run on Ubuntu 23.04 runner
- kata-agent: use default filemode for block device when it is set to 0
- kata-types: introduce KataVirtualVolume to support nydus, direct volume and image pull
- libs,tests: fix typo disable_guest_seccomp in configuration-anno-1.toml
- local-build: Remove GID before creating group
- kata-deploy: Avoid failing on content removal
- runtime: fix image and initrd assets handling
- metrics: Add disk link to README
- metrics: Fix FIO path
- gha: capture additional kata-deploy output
- metrics: Use function from metrics common in pytorch script
- metrics: Enable kata runtime in K8s for FIO test.
- metrics: Fix README for pytorch
- metrics: Remove unused variable in tensorflow mobilenet script
- rootfs: agent: Policy support with AGENT_INIT=yes
- gha: k8s: kata-deploy: Move kata-deploy specific tests from integration/kubernetes to functional/kata-deploy
- metrics: Fix check results for tensorflow benchmark
- metrics: Add Tensorflow ResNet50 int8 benchmark
- kata-deploy: Properly create default runtime class
- agent: simplify error handling
- metrics: Fix MobileNet help me description
- gha: ci: Start running kata-deploy tests
- runk: Modify kill command's error message for containerd tests
- runtime-rs: add driver option
- gha: cri-containerd: Enable tests
- metrics: Rename tensorflow scripts
- gha: tests: Add kata-deploy functional tests -- Part 1
- agent: runtime: add Agent Policy feature
- runk: Support without pid ns
- metrics: Add Cassandra Kubernetes benchmark for kata metrics
- metrics: Add common functions to the common script
- metrics: fix the loop used to stop kata components
- docs: Remove installation step in virtcontainers doc
- Propogate secrets, config maps etc into guest if sharedFS not available
- kata-deploy: Preliminary k0s support
- gha: static-checks: Move to the Azure instances
- versions: Update firecracker version to 1.4.0
- agent: Allow clippy::redundant_clone in the unit tests
- agent: avoid creating new `Vec` instances when easily avoidable
- metrics: compute tensorflow statistics
- metrics: Add network nginx benchmark
- metrics: install kata once and run multiple checks
- ci: unencrypted-image: Fix build context
- ci: create-confidential-image: Add dependent actions
- Follow up fixes for https://github.com/kata-containers/kata-containers/pull/7596
- tests: Create image that will be used in the unencrypted confidential tests
- kata-deploy: Ensure we cover SHIMS / DEFAULT_SHIM as part of our tests
- tests: upgrade bats version
- Fix mimor bugs and improve coding stype of agent rpc/sandbox/mount
- deps: Bump dependent crate versions
- fix number of queues handling in dragonball share fs device
- runtime-rs: Introduce directly attachable network
- metrics: General improvements to mobilenet tensorflow test
- gha: Add iperf network metrics
- docs: Use control-plane term instead of master
- agent: avoid unnecessary calls to `Arc::clone`
- metrics: Add network latency test
- Image pulling on the host
- Use version 0.10.4 of `fuse-backend-rs`
- kata-deploy: Use host's systemctl
- release: Revert kata-deploy changes after 3.2.0-rc0 release
- metrics: stop kata components before start a metric test.
- runtime-rs: Add block device handling for cloud hypervisor
a93fdb014 kata-deploy-stable: Adapt to what we're using in the stable branch
36109da93 ci: k8s: Fix bogus firecracker check in k8s-credentials-secrets.bat
d01daf749 tests: Adjust timeout for agent stability test
9b14dda14 libs: protection: Fix typo in TDX output
0e0867f15 runtime-rs: ch: Add TDX CH features check
409eadddb runtime-rs: ch: Improve readability of guest protection checks
82a0814fc tests: Enable agent stability test
32be8e3a8 tests: query data from the OPA service
b81c0a669 tests: encode policy file during test
4f9681b41 metrics: fixes common.sh function to always return true
2ef2b2a6d docs: Fix paths to build kernel in SNP VMs documentation
408b59c02 runtime-rs: fix bugs to support Nydus v5
157caea9f Revert "nydus: Temporarily skip tests on dragonball"
678fe3cd3 Dragonball: fix Nydus config serde problem
b6ec62138 policy: allow access to ReseedRandomDev
908519db9 metrics: skips docker restart when it is not installed or is masked.
c2763120a metrics: removing trailing comma characters from json file.
3e8cf6959 runtime: Validate hypervisor section name in config file
ef6388e81 tests: Remove unused function from scability test
fbc8f8f46 scripts: Use install_yq from the `kata-containers` repo
65b1a2d27 release: tag_repos: Stop tagging / updating the `tests` repo
87b760f56 runtime-rs: ch: Detect Intel TDX version
73e81f5e3 runitme-rs: unify base64 encoding for direct-volume
c6463cb5a tests: Fix path for versions yaml for soak parallel test
89c9454fc metrics: removal of reference in the documentation to the dax test.
30ff58904 tests: Enable scability test for stability CI
8d6f7b909 runtime-rs: Add support for handling vfio device for cloud-hypervisor
e786b2b01 gha: Add install dependencies for stability tests
dbfe6512f dragonball: vcpu metrics change to be recorded per vcpu
fa60fbe02 dragonball: METRICS is refactored to RwLock<DragonballMetrics>
500d1c5ce kata-ctl: update rustls-webpki/webpki dependency
d7660d82a runtime: unify gopkg.in/yaml.v3 to v3.0.1
fc9a107e8 runtime: unify swag and testify dependency
79ebb959c runtime: update runc dependency to v1.1.9
7f3e8bd65 runtime: unify golang.org/x/text to v0.7.0
df325ae37 runtime: update golang.org/x/net to v0.7.0
bba34910d metrics: stops kata components and k8s deployment when test finishes
84e3d884e gha: Add general dependencies to stability tests
dec3951ca tests: Add soak parallel stability test
0f04d527d tests: Enable soak parallel test
e669282c2 ci: k8s: set KUBERNETES default value
c30c3ff18 tests: run k8s-volume on a given node
666993da8 tests: run k8s-file-volume on a given node
3a00fc910 tests: exec_host() now gets the node name
61c9c17bf tests: add get_one_kata_node() to tests_common.sh
68f083c4d ci: k8s: set KATA_HYPERVISOR default value
6677a61fe ci: k8s: configurable deploy kata timeout
200e54292 ci: k8s: shellcheck fixes to gha-run.sh
4af78be13 kata-deploy: re-format kata-[deploy|cleanup].yaml
d54e6d9cd ci: k8s: run_tests() for kcli
c2ef1f0fb ci: k8s: add deploy-kata-kcli() to gh-run.sh
d2be8eef1 ci: k8s: add cleanup-kcli() to gha-run.sh
cbb9aa15b ci: k8s: set default image for deploy_kata()
89bef7d03 ci: k8s: create k8s clusters with kcli
954d40cce gha: combine coco jobs into a single yaml
b60e0a9b5 gha: combine basic amd64 jobs into a single yaml
e9bd85211 gha: ci: Revert tracing test PR to unbreak CI
b8a46a4b8 runtime-rs: ch: Enable feature
0f2dc8c67 gha: Add containerd stability tests to ci yaml
da91c9df8 ci: Port runk tests to this repo
7f2377276 ci: Add placeholder for runk tests
9205acc3d ci: Move tracing tests here
85d290a04 gha: Add stability gha run script
54f0c8f88 gha: Add stability tests workflow for gha
3bb2923e5 ci: Add placeholder for tracing tests
2c3bf406d ci: Create a function to install docker
119f03de2 gha: arm64: Ensure the builder is arm64-builder
8c498ef5e metrics: Use jq tool to pretty-print json metrics output
a2159a636 metrics: Enables FIO test for kata containers
70e7ec3e2 gha: Fix k0s deployment
560bbffb5 packaging: tools: Remove `set -x` leftover
18fa483d9 packaging: release: Mention newly added images
ca3b88837 packaging: tools: Fix container image env var name
5ca66795c packaging: Allow passing the TOOLS_CONTAINER_BUILDER
02acef957 gha: Build the kata-agent as part of our workflows
5208386ab packaging: Build the kata-agent
1727487ee agent: Allow specifying DESTDIR and AGENT_POLICY via env vars
45c118883 packaging: Add get_agent_image_name()
0db8fb8f9 versions: migrate out of k8s.gcr.io
a1a054367 doc: Fix spelling
6339605a1 tests: Add general stability fixes
59ae24444 doc: Update crictl pod-config
fd19f4082 tests: Add agent stability test
215577032 tests: Add cassandra stress in stability tests
f2d3ea988 tests: Add stressng dockerfile for stability tests
6493aa309 tests: Add stressor CPU test for stability tests
ef68a3a36 metrics: Add stability test for kata CI
7c934dc7d gpu: Fix cold-plug of VFIO devices
8d66ef518 metrics: Increase qemu jitter value
5600e28b5 metrics: Increase jitter value for clh
a6b1f5e21 ci: Build src/tools components as part of our tests / releases
501a168a8 kata-deploy: Build components from src/tools
6ef42db5e static-build: Add scripts to build content from src/tools
4d08ec29b packaging: Add get_tools_image_name()
98097c96d packaging: Use git abbreviated hash
489caf1ad ci: kata-monitor: Move tests over
a3fb067f1 ci: Add placeholder for kata-monitor tests
57cb4ce20 ci: Make install_kata aware of container engines
de1eeee33 ci: Create a generic install_crio function
64a200085 ci: Add install_cni_plugins helper
8132fe15c ci: Modify containerd default config
8cb7df1be metrics: Add checkmetrics for latency test
e90440ae2 metrics: Add qemu latency value limit
a74a8f8a9 metrics: Add latency value limits for kata CI
d7def8317 metrics: Fix general check static warnings
928553d1b docs: Update url in kata vra document
b0a3293d5 runtime-rs: ch: Enable Intel TDX
523399c32 runtime-rs: ch: Add more consts
dea806581 runtime-rs: ch: Remove unused function
995f2c015 runtime-rs: ch: Only handle particular pending device types
b1b96a5c4 runtime-rs: ch: Remove erroneous "virtio-blk-mmio" check
9ac29b8d3 metrics: Add init_env function to latency test
dfd0c9fa9 runtime: clh: Re-generate the client code
8f9f087e3 versions: Upgrade to Cloud Hypervisor v35.0
81c8babca metrics: Fix latency yamls path
481573682 metrics: Fix C-Ray documentation
ef63d67c4 ci: crio: Trail '\r' from exec_host() output
74c12b292 ci: crio: Enable default capabilities
358dc2f56 kata-deploy: Fix CRI-O detection
ebaa4fa4c ci: crio: Pass `-y` to apt
97e73b223 metrics: Fix spelling warnings
36c8cd6f1 metrics: Fix metrics README
15425a2b8 local-build: Fix .docker ownership before build-payload
13ca7d9f9 gha: Add pandoc as a dependency for static checks
08bc8e4db metrics: Add latency benchmark for gha
6776b55d7 metrics: Enable latency test in gha run script
94e2ccc2d runtime: fix reading cgroup stats of sandboxes
d507d189b fc: Add support for noflush cache option
2ca781518 clh: Direct IO support for block devices
0c95697cc ci: Trigger payload-after-push on workflow_dispatch
28cbc3b51 ci: rootfs-image build-asset is failing Fixes: #802787a861648 gha: Install hunspell for static checks
8c3c50ca8 ci: Actually enable the CRI-O tests
3a6510ad6 osbuild: Reduce guest components binary size with strip
07a6e63a6 ci: k8s: rke2: Use sudo to call systemd
03b82e848 ci: k8s: Add a CRI-O test
d7105cf7a ci: k8s: Add a method to install CRI-O
54c0a471b ci: k8s: k0s: Allow passing parameters to the k0s installer
730ef5169 deps: updating dependencies
3a2c83d69 ci: kata-deploy: Fix runner name
82ff2db46 runtime: support kernel params including spaces
604a9dd67 protocol: remove gogoprotobuff tests
f7fa7f602 ci: Enable kata-deploy tests for all the supported k8s flavours
2c908b598 ci: kata-deploy: Add the ability to deploy rke2
eaf616491 ci: kata-deploy: Add the ability to deploy k0s
001525763 ci: kata-deploy: Add deploy-k8s argument to gha-run.sh
bf2cb0228 ci: kata-deploy: Expland tests to run on k0s / rke2
b12b9e188 ci: kata-deploy: Add placeholder for tests on GARM
9e1fb8a96 ci: kata-deploy: Export KUBERNETES env var
09cc0ed43 ci: Move deploy_k8s() to gha-run-k8s-common.sh
486fe14c9 ci: Properly set K8S_TEST_UNION
d9ef1352a ci: Add first letter of the K8S_TEST_HOST_TYPE to resource group name
68267a399 ci: Create clusters in individual resource groups
9aa8d1c91 metrics: Add parallel bandwidth limit for qemu
44c7c082d versions: Bump virtiofsd to v1.8.0
af59d4bf4 metrics: Enable parallel bandwidth iperf limit
aba36ab18 nydus: Temporarily skip tests on dragonball
b8a8dfcd1 nydus: Use `kata-${KATA_HYPERVISOR}` instead of `kata`
f6df3d6ef static-build: Fix arch error on nydus build
2f9c9e2e6 tests: nydus: Update nydus tests
c9a4e7e46 versions: Bump nydus and nydus-snapshotter to its latest release
b73bde320 gha: nydus: Populate run()
b3904a1a3 gha: nydus: Populate install_dependencies()
d2b3b67f5 gha: nydus: Actually install kata when `install-kata` is called
0ec00ad42 gha: nydus: Get rid of nydus{,-snapshotter} install from nydus_test.sh
568439c77 tests: nydus: Add timeout to the crictl calls
5ac3b76eb tests: nydus: Add uid / namespace to the nydus container / sandbox
376574a16 tests: nydus: Decorate some calls with `sudo`
4290fd4b6 tests: nydus: Adapt "source ..." to GHA
a84efa3e8 tests: nydus: Adapt check to "clh" instead "cloud-hypervisor"
56a14b395 tests: common: Add install_nydus_snapshotter()
b6563783e tests: common: Add install_nydus()
72599f191 clh: arm: Use static_sandbox_resource_mgmt=true
1f16b6627 runtime/qemu: Rework QMP/HMP support
8b1e9b0c7 ci: static-checks: Clean up static-checks job
2c5ca2eaf ci: static-checks: Run tests depending on KVM
509c309ab ci: static-checks: Move "sudo make test" to the new test matrix
4e963cedf ci: static-checks: Move "make test" to the new test matrix
08f2e5ae0 runtime-rs: Ensure static-checks-build is a dep of `make test`
2bc3a616a kata-ctl: Use `loop` instead of `kvm` module in tests
46daddc50 kata-ctl: Ensure GENERATED_CODE is a dep of `make test`
ec826f328 agent: Ensure GENERATED_CODE is a dep of `make test`
1d32410a8 ci: install_libseccomp: Do not depend on the tests repo
bf888b9a5 ci: static-checks: Move "make check" to the new test matrix
473ec8780 kata-ctl: Add `kata-types` to the Cargo.lock file
ea19549a9 kata-ctl: Ensure GENERATED_CODE is a dep of `make check`
e12577586 tests: install_rust: Also install clippy
e2c61a152 ci: static-checks: Move vendor check to its own job
6794d4c84 tests: Move install_rust.sh from the tests repo
e64508c30 tests: install_go: Remove tests repo dependency
11dff731b tests: Move functions from kata_arch script here
75c974c80 ci: static-checks: Move kernel config check to its own job
9c233bb9e test: Add test to verify try_from for clh Netconfig
c69a1e33b ci: Use variable size of VMs depending on the tests running
9049d311d runtime-rs: Add network support for cloud-hypervisor
eecd5bf2a ci: cache: Fix ovmf-sev cache
86c41074b ci: cache: Check the sha256sum of the component
460988c5f ci: cache: Remove the script used to cache artefacts on Jenkins
4533a7a41 ci: cache: Also store the ${component} sha256sum
eccc76df6 ci: cache: Use the cached artefacts from ORAS
7f5e77bcb kernel: enable Arm pl011 support
241c355e0 clh:arm64: use arm AMBA uart for hypervisor debug
094b6b2cf ci: k8s: Temporarily disable tests that require a bigger VM instance
d0c257b3a ci: cache: Push cached artefacts to ghcr.io
108f1b60d kata-deploy: Generate latest_{artefact,image_builder} files
be2eb7b37 ci: cache: Install ORAS in the kata-deploy binaries builder container
fb24fb0dc ci: k8s: devmapper: Use a smaller / cheaper VM instance
1daf02f5d ci: nydus: Use a smaller / cheaper VM instance
e60d81f55 ci: nerdctl: Use a smaller / cheaper VM instance
4db416997 ci: docker: Use a smaller / cheaper VM instance
32841827b ci: cri-containerd: Use a smaller / cheaper VM instance
92fff129f ci: k8s: Don't set cpu limit request for k8s-inotofy test
faf98c062 ci: Reduce the size of the AKS VMs
adc18ecdb ci: cache: For consistency, read all used env vars
c7a851efd ci: cache: Pass the exposed env vars to the kata-deploy binaries in docker
6bd15a85d ci: cache: Export env vars needed to use ORAS
cd4fd1292 metrics: Add iperf cpu utilization limit for qemu
df5cd10ea metrics: Add iperf value for cpu utilization
a96050a7a tests: Apply timeout to 'ctr t kill'
9d9303678 tests/vfio: Bump VM image to Fedora 38
faee59b52 tests/vfio: Accept single device in vfio group for CLH
df3dc1105 tests/vfio: Get rid of sync's
7211c3dcc gha: vfio: Set test timeout to 15m
1b02f89e4 packaging: kernel: Enable VIRTIO_IOMMU on x86_64
3a1db7a86 runtime: clh: Support enabling iommu
9f1a42c6c tests/vfio: Give commands 30s to execute
b46b0ecf8 tests/vfio: Configure a value for 'hot_plug_vfio' for both vmms
bfc93927f runtime: Remove redundant check in checkPCIeConfig
7c4e73b60 runtime: Add test cases for checkPCIeConfig
fc51e4b9e runtime: Check config for supported CLH (cold|hot)_plug_vfio values
509771e6f runtime: clh: Add hot_plug_vfio entry to config
5f6475a28 tests/vfio: Gather debug info and disable tdp_mmu
8fffdc81c tests/vfio: Capture journal from vm
df815087e tests/vfio: Change to get the test working in GHA
a92ddeea1 tests/vfio: Move dependency installation to gha-run.sh
5a551a85b gha: vfio: Import jobs scripts from tests repo
49e2fa189 metrics: Increase jitter value for qemu
49234433a metrics: Increase value limit for jitter in clh
813bfdec0 ci: docker: nerdtl: Use io.containerd.kata-${KATA_HYPERVISOR}.io
46bc0b1c0 ci: nerdctl: Create the containerd config
13968aa7f ci: nerdctl: Switch to tcp port 80 ping
e0c811678 ci: docker: Switch to tcp port 80 ping
1636abbe1 runtime: issue with non-empty []Endpoint in RemoveEndpoints
0aa073967 metrics: Add iperf bandwidth value for qemu
c0ad91476 tests: fix kernel and initrd annotations
615c1cbf1 metrics: Add iperf bandwidth value for kata metrics
d53eb73ee metrics: Ensure docker is running in init_env
ad08321b8 metrics: Add Cassandra Metrics documentation
a58ea6659 metrics: this PR skips the FIO test temprarily to fix issues
f536ef5ce ci: docker: Also run the smoke test with runc
c83f167c5 ci: docker: Run the tests after the kata-static is created
12d833d07 ci: Add a very basic nerdctl sanity test
348b8644d ci: Add a very basic docker sanity test
a75fd5eb8 runk: Fix rust unecessary mut error
a31c14517 kata-ctl: useless-vec warning
c8419fc3b kata-ctl: Resolve non-minimal-cfg warning
3eaf68d95 agent-ctl: Allow clippy lint
1d8b78959 runtime-rs: Fix useless-vec warning
99f3d69e9 runtime-rs: Remove mut
16fbc27b0 dragonball: Allow ambiguous-glob-reexports
bbf191951 dragonball: Resolve non-minimal-cfg warning
75cfdd5d5 agent: config: Allow clippy lint
f3a0fd590 agent: config: Fix useles-vec warning
9e423bd3d libs: Fix clippy unnecesary hashes error
444395050 versions: Bump rust version
a16b0962b chore(cargo): update cargo lock
ca4b6b051 runtime: Naming conflict of network devices
202049f35 feat(runtime-rs): introduce huge page type to select VM RAM's backend
f811b064c ci: use github.ref_name instead of $GITHUB_REF_NAME
6d795c089 ci: Add more target-branch related fixes
8509c3187 ci: Fix target-branch usage
060499dca metrics: Remove warning from metrics documentation
c0f697fcc runtime: Allow kernel_params annotation
b03e49794 dragonball: fix for non-deterministic builds
976d10150 runtime-rs: hypervisor: Remove debug kernel options
fde34610c kernel: Add erofs patches needed for CC related work
dc6a4588a versions: Bump kernel to the latest LTS release (6.1.52)
52f6449b7 kata-manager: Remove initcall_debug kernel option
8b4a0b368 kata-deploy: Remove curl after it's used
139c7f03a kata-deploy: Fix aarch64 image build
470d06541 agent: optimize the code of systemd cgroup manager
bd24afcf7 gha: Manually rebase PR atop of the target branch before testing
72c510d05 runtime/virtiofsd: Drop all references to "--cache=none"
ead724bec protocol: removing gogo.nullable feature
d8e4bb985 protocol: remove unused PROTO_FILE env
5e1106a77 protocol: remove unused import_path
87accaaec protocol: use workdir during build
711a7ed96 protocol: remove mapping definitions
8db84c1bd protocol: force GOPATH to be set
68156d77a protocol: breaking lines to improve readability
670a8e9c7 kata-deploy: Switch to an alpine image
9d74b7ccc k8s: ci: Skip "Pod quota" test with firecracker
f6cd3930c ci: k8s: Remove useless skip statement from tests
3cc20b47a ci: k8s: Also check for "fc" (for firecracker)
b5bad3cb0 ci: k8s: Add clean-up-garm argument for gha-run.sh
aaec5a09f ci: k8s: devmapper tests should be using ubuntu 20.04
27fa7d828 ci: k8s: Add a kata-deploy-garm target
fa62a4c01 ci: k8s: Export KUBERNETES env var
8c9380a79 ci: k8s: Install bats on GARM runners
3de23034f ci: k8s: Wait some time after restarting k3s
adfea55b8 metrics: fix FIO test initialization
2df183fd9 ci: k8s: Append, instead of overwrite, the devmapper config
369a8af8f ci: k8s: Decrease k3s sleep from 4 to 2 minutes
ada65b988 ci: k8s: Use vanilla kubectl with k3s
ad45ab5d3 ci: k8s: Ensure k3s is deploy with --write-kubeconfig-mode=644
028a97e0d ci: k8s: Use the proper command for sleep
3a427795e metrics: Use TensorFlow optimized image
8d99972a8 ci: k8s: Fix typo in run-k8s-tests-on-garm.yaml
deed1b927 Dragonball: optimize the placement of dbs-upcall features
0e8bd50cb ci: k8s: Add k8s devmapper tests (part 0)
b28b54df0 ci: k8s: Add a function to configure devmapper for containerd
54f711721 ci: k8s: Add a function to deploy k3s
81536f21a runtime/qemu: Pass "--xattr" to virtiofsd instead of "-o xattr"
b1dd09a4d runtime: Allow virtio_fs_extra_args annotation
2efda20c7 packaging: do not install docker-compose-plugin for s390x|ppc64le
438fbf966 metrics: Add write 95 percentile for FIO for qemu
024b4d2ff metrics: Add write 95 percentile FIO value
e98e5cdea metrics: Add checkmetrics to gha run script
c1edfe551 metrics: Add checkmetrics value for qemu for iperf
6a79ecedf metrics: Add jitter value for clh
f609a9a75 metrics: Add test selector to iperf metrics
5b8db3042 metrics: Enable iperf benchmark on gha for kata metrics
60f733d30 CI: switch static-checks-dragonball CI machines to Azure
7870b33a2 runtime-rs: bring hybridVsock devices in manager.
18c94ebbe kata-deploy: Create kata-static.tar with correct ownership
57e7bf14a agent: refine StorageDeviceGeneric::cleanup()
53edb1937 agent: implement StorageDeviceGeneric::cleanup()
0c63453e2 types: make StorageDevice::cleanup() return possible error code
3a3d77b3b agent: move StorageDeviceGeneric from kata-types into agent
b151cfd14 metrics: re-enable memory-usage initialization step
f3e1a6a94 osbuilder: alpine: Change mirror
ac612aef5 osbuilder: alpine: Match the version on versions.yaml
9cd706d1c agent: avoid possible leakage of storage device
bf21411e9 tests: add policy to k8s tests
d0e061067 runtime: config: use the SEV initrd for SNP
67fed26f1 runtime: Use TDX image with in the qemu-tdx config
ac939c458 gha: Rebase atop of the target branch
82cd14ba3 versions: Update alpine to its 3.18 version
666882575 metrics: Add grabdata script for metrics report
c290eaed8 kata-sys-util: protection: Update TDX checks
d7a996c68 gha: Update to checkout@v3 action
c2ba29c15 runtime: Fix data race in ioCopy
211de08d9 osbuilder: Remove chcon operation for guest SELinux
9f21fa9b3 metrics: Add report generator link to general documentation
c0ed5ea0a metrics: Add README for kata metrics report
a7b59a5bf metrics: Add limit for 90 percentile for qemu value
99db6568e metrics: Add limit for write 90 percentile value for clh
6e06392c5 metrics: Enable FIO limits for kata metrics
2e4c87472 runtime/vc: runPrestartHooks should ignore GetHypervisorPid failure
21204caf2 runtime: fail early when starting docker container with FC
32fd01371 runtime: run prestart hooks before starting VM for FC
00e7ffd98 tests: check vmx only on Intel machines
c8dd3c073 metrics: Fix memory footprint qemu limit
8877ec62f metrics: Fix memory inside limits for kata metrics
80146f207 tests: Fixes cpuType check on AMD machines
7e364716d metrics: Add test setup details to metrics report
17dc1b976 metrics: Add boot lifecycle times to metrics report
3b0d6538f metrics: Add memory inside container to metrics report
79fbb9d24 metrics: Add scaling system footprint in metrics report
8e6d4e6f3 metrics: Add metrics reportgen
139ffd4f7 metrics: Add report file titles
878d1a2e7 metrics: Generate PNGs alongside the PDF report
fce248797 metrics: Add metrics report R files
08812074d metrics: Add report dockerfile
69781fc02 metrics: Add metrics report script
e286e842c tests: Expand confidential test to support TDX
e31f099be tests: Expand confidential test to support SNP
c3b9d4945 tests: Add confidential test for SEV
538c965c2 metrics: fix parsing issue on memory-usage test
3818bf331 local-build: Remove $HOME/.docker/buildx/activity/default
d1b54ede2 qemu: tdx: Workaround SMP issue with TDX 1.5
1e34220c4 qemu: tdx: Adapt to the TDX 1.5 stack
8115a0522 versions: tdx: Update Kernel to 6.2 + TDX
ec18180f3 versions: tdx: Update TDVF to the "edk2-stable202302"
9803b2428 versions: tdx: Update QEMU to v7.2 + TDX v1.10
dffc16e5b runtime-rs: check peer close in log_forwarder
aaa5ab126 agent: simplify storage device by removing StorageDeviceObject
fb49d5d7c gha: Avoid "fail-fast" in tests that are known to be flaky
183f51d6f tests: use unique test name
6a974679f tests: delete k8s deployment at the test's end
32a778b6d metrics: Remove unused variable in tensorflow nhwc script
d8f3ce649 kata-deploy: Don't try to remove /opt/kata
936e8091a gha: vfio: Run on Ubuntu 23.04 runner
0e7248264 agent: move storage device related code into dedicated files
268e84655 runtime-rs: Fix volumes and rootfs cleanup issues
8f49ee33b agent: refine storage related code a bit
60ca12ccb agent: switch to new storage subsystem
fcbda0b41 kata-types: introduce StorageDevice and StorageHandlerManager
b03b1f613 agent: simplify the way to manage storage object
8392c71bf sys-util: support more mount flags in parse_mount_options()
c00d8f3d4 agent: use create_mount_destination() from kata-sys-util
5e867f053 types: add more mount related constants
880e6c9a7 agent: use function from kata-sys-utils to reduce code
3b881fbc0 local-build: Remove GID before creating group
959ca4944 metrics: Add TensorFlow ResNet50 fp32 Dockerfile
4b7d72c4a metrics: Add TensorFlow ResNet50 FP32 benchmark
5cba38c17 kata-deploy: Avoid failing on content removal
18d42da21 runtime/fc: fix image/initrd annotation handling
9fda7059a runtime/clh: fix image/initrd annotation handling
1a0092d63 runtime/qemu: fix image/initrd annotation handling
22d8f335d libs,tests: fix typo disable_guest_seccomp in configuration-anno-1.toml
8afd158ce metrics: Add disk link to README
40914b25d kata-agent: use default filemode for block device when it is set to 0
eee2ee6ee metrics: Fix FIO path
39bc3488f metrics: Use function from metrics common in pytorch script
400eb8874 gha: capture additional kata-deploy output
4aee3eade kata-types: implement serde methods for KataVirtualVolume
b875e3932 kata-types: validate KataVirtualVolume object
fa2fdc105 kata-types: implement two conversion helpers for KataVirtualVolume
6326af20e kata-types: introduce KataVirtualVolume
c8b43f8b3 metrics: Fix README for pytorch
fb571f8be metrics: Enable kata runtime in K8s for FIO test.
cb056f8cb rootfs: agent: Policy support with AGENT_INIT=yes
85c02828e metrics: Update tensorflow name in gha run script
e8a511934 metrics: Fix check results for tensorflow benchmark
2d896ad12 gha: kata-deploy: Do the runtime class cleanup as part of the cleanup
4ffc2c86f gha: kata-deploy: Add the first kata-deploy test
8616c050a metrics: Remove unused variable in tensorflow mobilenet script
285e616b5 tests: common: Ensure test_type is used as part of the cluster's name
790bd3548 tests: commob: Don't fail if yq is not part of the cache
ce6adecd0 gha: kata-deploy: Add run-kata-deploy-tests.sh
cfc29c11a gha: k8s: Stop running kata-deploy tests as part of the k8s suite
f4dd15286 tests: k8s: Call ensure_yq() in setup.sh
339569b69 kata-deploy: Properly create default runtime class
2a491e9b1 metrics: Fix MobileNet help me description
d19a75e80 gha: ci: Start running kata-deploy tests
d90f7ac68 runtime-rs: add unit test for block driver
e44919f0d runtime-rs: add load_test_config for unit test
7f48a6937 runtime-rs: add driver option
bade6a5c3 docs: Fix TensorFlow word across the document
1a1b20776 docs: Add Tensorflow Resnet50 documentation
24baededc metrics: Add Dockerfile for ResNet50 int8
6d971ba8d metrics: Add Tensorflow ResNet50 int8 benchmark
25d151bd1 runk: Modify kill command's error message for containerd tests
b3592ab25 gha: cri-containerd: Enable tests
84dd02e0f gha: cri-containerd: Add timeout to the crictl calls on testContainerStop
b29782984 gha: cri-containerd: Show pod before deleting it
ae0930824 gha: cri-containerd: Print kata logs in case of error
6c8b2ffa6 gha: cri-containerd: Group containerd logs
9e898701f gha: cri-containerd: Ensure RUNTIME takes KATA_HYPERVISOR into account
76dac8f22 agent: simplify error handling
18a7fd8e4 metrics: Rename tensorflow scripts
e55fa93db tests: kata-deploy: Add placeholder for kata-deploy-tests-on-tdx
d9ee17aae tests: kata-deploy: Add placeholder for kata-deploy-tests-on-aks
ab829d103 agent: runtime: add the Agent Policy feature
831e73ff9 tests: kata-deploy: Add functional/kata-deploy/gha-run.sh placeholder
af1b46bbf tests: Add gha-run-k8s-common.sh
416445e7e docs: Remove installation step in virtcontainers doc
72cbcf040 kata-deploy: Add k0s support
767434d50 metrics: fix the loop used to stop kata components #76295d0f0d43c metrics: Add cassandra statefulset yaml
c1dcc1396 metrics: Add cassandra service yaml
2297a0d1c metrics: Add block loop pvc yaml for cassandra
e3d511946 metrics: Add block loop pv yaml for cassandra test
989027159 metrics: Add block loop pvc for cassandra test
349b89969 metrics: Add Cassandra Kubernetes benchmark for kata metrics
c52d09052 gha: static-checks: Move to the Azure instances
8815ed066 runtime: Remove config warnings
afe1a6ac5 agent: support copying of directories and symlinks
ab13ef87e runtime: propagate configmap/secrets etc changes for remote-hyp
c074ec4df runtime: Copy shared files recursively
fdcd52ff7 metrics: Add check containers are running in tensorflow mobilenet
36337ee14 metrics: Add check containers are up in tensorflow script
f700f9b0b metrics: Remove unused variable in tensorflow script
833cf7a68 metrics: Add check containers are running function
918c78308 metrics: Add check containers are up in tensorflow mobilenet script
9d57a1fab metrics: Use check containers are up in tensorflow script
1c84680d8 metrics: Add check containers are up in common script
d3e57cf45 metrics: Use collect_results function in tensorflow mobilenet test
286de046a metrics: Remove collect results function definition
9879709aa metrics: Add common functions to the common script
4746fa3da docs: Specify supported Firecracker version using `versions.yaml`
cc922be5e versions: Update firecracker version to 1.4.0
39e67b06e dragonball: vsock add fifo/pipe stream support for passed fd hybridStream
473b0d3a3 metrics: compute tensorflow statistics
03d1fa67b ci: unencrypted-image: Fix build context
eb463b38e ci: unencrypted-image: Don't fail to build on s390x
a2d731ad2 ci: create-confidential-image: Add dependent actions
d1a629622 metrics: Add nginx documentation to network README
498f7c054 metrics: Add nginx kubernetes yaml
f8a5255cf metrics: Add network nginx benchmark
43fe5d1b9 ci: k8s: tees: Ensure PR_NUMBER is exported
54f6a7850 ci: {{ pr-number }} should be {{ inputs.pr-number }}
034d7aab8 tests: k8s: Ensure the runtime classes are properly created
fac8ccf5c ci: Add build-and-publish-tee-confidential-unencrypted-image
ab5f603ff ci: k8s: Add the image used for unencrypted confidential tests
1e8fe131b k8s: tests: Take advantage of `SHIMS` and `DEFAULT_SHIM` env vars
729b2dd61 agent: avoid creating new `Vec` instances when easily avoidable
aeaec9dae tests: upgrade bats version
e66496986 metrics: install kata once and run multiple checks
baabfa9f1 agent: refine implementation of mount related code
98ba211a3 agent: fix a bug in update_ephemeral_mounts()
5333618d7 agent: make add_storage() take &[Storage] instead of Vec<Storage>
37f34781d agent: simplify function online_cpu_memory()
d3c542237 agent: refine style of code related to sandbox
71a9f6778 agent: avoid unwrap() in function do_remove_container()
84badd89d agent: avoid clone objects when possible
b23c5ed15 deps: Bump dependent crate versions
863283716 metrics: General improvements to mobilenet tensorflow test
3c319d8d4 metrics: Add iperf to gha run script
5b5caf890 gha: Add iperf network metrics
66db5b535 metrics: Add latency test to network README
c36572418 agent: avoid unnecessary calls to `Arc::clone`
4fbe0a3a5 runtime: bind-mount mounted block device into container
7e1b1949d runtime: add support for kata overlays
6c867d9e8 agent: add io.katacontainers.fs-opt.overlay-rw option
6163c3565 agent: skip mount options that start with "io.katacontainers."
b2ff97aa0 dragonball: use version 0.10.4 of `fuse-backend-rs`
845eeb4d7 agent: Allow clippy::redundant_clone in the unit tests
1163fc9de release: Revert kata-deploy changes after 3.2.0-rc0 release
3958a39d0 runtime-rs: Introduce directly attachable network
1e15369e5 metrics: Improve naming testing containers in launch times test
5dbe88330 metrics: Clean kata components before start a metric test.
3b45060b6 metrics: Add latency server yaml
9bb8451df metrics: Add latency client yaml
64fdb9870 metrics: Add network latency test
a81ad3b58 runtime-rs: Add block device handling in cloud hypervisor
3230dec95 kata-deploy: Use host's systemctl
1b21a4624 docs: Use control-plane term instead of master
28e5e9c86 runtime-rs: fix number of queues handling in dragonball share fs device
f1d8de9be runk: Allow runk to launch a container without pid namespace
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is basically to make sure that folks trying to use the kata-deploy
script from the main branch, to deploy **stable** kata-deploy images, do
not have a hard time.
Fixes: #7194
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove the ability to block access to kata agent endpoints by using
agent-config.toml. That functionality is now implemented using the
Agent Policy feature (#7573).
The CCv0 branch relied on blocking endpoints using agent-config.toml
but will set-up an equivalent default policy file instead (#8219).
Fixes: #8228
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Add the missing closing bracket to the output of the TDX details,
so rather than:
```bash
$ sudo kata-ctl env 2>/dev/null | grep available_guest_protection
available_guest_protection = "tdx (major_version: 1, minor_version: 0"
: ^
: Missing ')' !
```
... we now have:
```bash
$ sudo kata-ctl env 2>/dev/null | grep available_guest_protection
available_guest_protection = "tdx (major_version: 1, minor_version: 0)"
: ^
: Aha!
```
Added a unit test for this scenario.
Fixes: #8257.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
If you attempt to create a container (a TD) on a TDX system using a
custom build of Cloud Hypervisor (CH) that was not built with the `tdx`
CH feature, Kata will report the following, somewhat cryptic, CH error:
```
ApiError(VmBoot(InvalidPayload))
```
Newer versions of CH now report their build-time features in the ping
API response message so we now use that, if available, to detect this
scenario and generate a user-friendly error message instead.
This changes improves the readability of `handle_guest_protection()` and
adds a couple of additional tests for that method.
Fixes: #8152.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Improve the way `handle_guest_protection()` is structured by inverting
the logic and checking the value of the `confidential_guest` setting
before checking the guest protection. This makes the code easier to
understand.
> **Notes:**
>
> - This change also unconditionally saves the available guest protection
> (where previously it was only saved when `confidential_guest=true`).
> This explains the minor unit test fix.
>
> - This changes also errors if the CH driver finds an unexpected
> protection (since only Intel TDX is currently tested).
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This PR adds the iperf udp benchmark for bandwdith measurement
for network metrics.
Fixes#8246
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
ACPI PCI device hotplug on qemu virt is not supported. The only way to
hotplug pci device is pcie native way. Thus we need create pcie root
port as default.
Pcie root port number depends on following:
1. reserved one for network device as default;
2. virtio-mem dev;
3. add enough port for vhost user blk dev;
Fixes: #7646
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Add GetEndpointsNum API for Network Interface to get the number of
network endpoints. This is used for caculate the number of pcie root
port for QemuVirt.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Encode policy file during test - easier to understand than hard-coding
the encoded file contents.
Fixes: #8214
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR corrects the init env() helper function, to make that
systemctl always returns true when enumerating masked services,
and preventing the test from failing
Fixes: #8242
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR fixes the correct path to setup, build and install properly
the kernel for snp.
Fixes#8156
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
1. enable virtio-fs-pro in Dragonball to have the ability to process nydus backend registry
2. change passthrough for rw layer's readonly config to false to have the accurate read write ability.
Fixes:#8013
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Since Nydus snapshotter has been updated in previous commits, there is a
problem that the config passthrough to Dragonball during mount_rafs is
RafsConfig instead of ConfigV2, but Dragonball could only serde ConfigV2
so it will panic.
We need to add the support for RafsConfig
Fixes:#8013
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Allow access to the ReseedRandomDev endpoint by default. Using false
for ReseedRandomDevRequest was unintended.
Fixes: #8225
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
To avoid errors when initializing the test environment, the
kill_processes_before_start() helper function needs to verify that
docker is installed before attempting to stop it.
Fixes: #8218
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR removes trailing commas so that the json results
file is valid.
This PR also changes the way data results are collected by
terating through the array of memory values to calculate
their average.
Fixes: #8204
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Previously, if you accidentally modified the name of the hypervisor
section in the config file, the default golang runtime gives a cryptic
error message ("`VM memory cannot be zero`"). This can be demonstrated
using the `kata-runtime` utility program which uses the same golang
config package as the actual runtime (`containerd-shim-kata-v2`):
```bash
$ kata-runtime env >/dev/null; echo $?
0
$ sudo sed -i 's!^\[hypervisor\.qemu\]!\[hypervisor\.foo\]!g' /etc/kata-containers/configuration.toml
$ kata-runtime env >/dev/null; echo $?
VM memory cannot be zero
1
```
The hypervisor name is now validated so that the behaviour becomes:
```bash
$ kata-runtime env >/dev/null; echo $?
0
$ sudo sed -i 's!^\[hypervisor\.qemu\]!\[hypervisor\.foo\]!g' /etc/kata-containers/configuration.toml
$ ./kata-runtime env >/dev/null; echo $?
/etc/kata-containers/configuration.toml: configuration file contains invalid hypervisor section: "foo"
1
```
Fixes: #8212.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
As the file is already part of the kata-containers repo, and the tests
repo is about to become read-only, we're good to drop the tests
references from here and use everything coming from the
`kata-containers` repo instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've moved all the tests to the `kata-containers` repo, the `tests`
repo will become a read-only repo.
Fixes: #8200
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Improve the `GuestProtection` handling to detect the version of
Intel TDX available.
The TDX version is now logged by the Cloud Hypervisor driver.
Fixes: #8147.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Direct-volume needs to use the same base64 character set as
kata-runtime/direct-volume does.
Fixes: #8175
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR removes the reference in the documentation to the DAX
subtest of the FIO benchmark, because this metric is currently
WIP.
Fixes: #8159
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This change adds support for adding and removing vfio devices for
cloud-hypervisor.
Fixes: #6691
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
To pick up fix for the following issue:
A maliciously crafted HTTP/2 stream could cause excessive CPU
consumption in the HPACK decoder, sufficient to cause a denial of
service from a small number of small requests.
Fixes: #8190
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
This PR adds a trap whenever the scrip exits, it deletes the iperf
k8s deployment and k8s services, and deletes the kata components.
This way, when the script finishes, it verifies that there are
indeed no kata components still running.
Fixes: #8126
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The KUBERNETES variable is mostly used by kata-deploy whether to apply
k3s specific deployments or not. It is used to select the type of
kubernetes to be installed (k3s, k0s, rancher...etc) and it is always
set on CI. Running the script locally we want to set a value by default
to avoid `KUBERNETES: unbound variable` errors.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
This test can give false-positive on a multi-node cluster. Changed it to
use the new get_one_kata_node() and the modified exec_host() to run the
setup commands on a given node (that has kata installed) and ensure the
test pod is scheduled at that same node.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The exec_host() simply fails on cluster with multi-nodes because
`kubectl get node -o name" will return a list o names. Moreover, it will
return control nodes names which usually don't have kata installed.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The introduced get_one_kata_node() returns the first node that
has the kata-runtime=true label, i.e., supposedly a node with
kata installed.
This is useful for tests that should run on a determined worker
node on a multi-nodes cluster.
Fixes#7619
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Let KATA_HYPERVISOR be qemu by default in gh-run.sh as this variable
is required to tweak some configurations of kata-deploy.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The deploy-kata() of gha-run.sh will wait for 10 minutes for the kata
deploy installation finish. This allow users of the script to overwrite
that value by exporting the KATA_DEPLOY_WAIT_TIMEOUT environment
variable.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Fixed a couple of warns shellcheck emitted and disabled others:
* SC2154 (var is referenced but not assigned)
* SC2086 (Double quote to prevent globbing and word splitting)
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The .tests/integration/kubernetes/gh-run.sh script run `yq write` a
couple of times to edit the kata-[deploy|cleanup].yaml, resulting
on the file being formatted again. This is annoying because leaves
the git tree dirty.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The only difference to the other platforms is that it needs to
export KUBECONFIG.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The cleanup-kcli() behaves like other deploy kata for
bare-metal (e.g. sev, tdx...etc) except that KUBECONFIG
should be exported.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The cleanup-kcli() behaves like other clean up for bare-metal (e.g. sev,
tdx...etc) except that KUBECONFIG should be exported.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
On CI workflows the variables DOCKER_REGISTRY, DOCKER_REPO and
DOCKER_TAG are exported to match the built image. However, when running
the script outside of CI context, a developer might just use the latest
image which in this case will be
`quay.io/kata-containers/kata-deploy-ci:kata-containers-latest`.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Adapted the gha-run.sh script to create a Kubernetes cluster locally
using the kcli tool.
Use `./gha-run.sh create-cluster-kcli` to create it, and
`./gha-run.sh delete-cluster-kcli` to delete.
Fixes#7620
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
GHA has an undocumented limitation that there can be at most 20
referenced yamls in a single yaml file. We workaround it by combining
multiple jobs into a single yaml file.
Fixes: #8161
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Enable the Cloud Hypervisor driver (the `cloud-hypervisor` build feature) for the rust runtime.
Fixes: #6264.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
I'm basically moving the runk tests from the tests repo to this one, and
I'm adding the "Signed-off-by:" of every single contributor the tests.
Fixes: #8116
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Chen Yiyang <cyyzero@qq.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The runk test has been executed as part of the former "ubuntu" jenkins
CI.
We're porting it to GHA and running it against LTS containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The tracing tests are currently running as part of the Jenkins CI with
the following setups:
* Container Engines: containerd
* VMMs: QEMU | Cloud Hypervisor
* Snapshotters: overlayfs | devmapper
We'll be restricting those tests to be running on LTS version of
containerd, without devmapper.
As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next interations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll use any arm64 machine that's added as a runner, and
whenever new machines are added those may end up being only used for
running some specific set of the tests.
Fixes: #8109
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR enables the use of jq pretty-print feature to
improve the formatting of metric results json files.
Fixes: #8081
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
FIO benchmark is enabled to measure IO in Kata
at different latencies using containerd client,
in order to complement the CI metrics testing set.
This PR asl deprecated the previous Fio bench
based on k8s.
Fixes: #8080
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The tests are failing when setting up k0s, and that happens because we
download a kubectl binary matching the kubernetes version k0s is using,
and we do that by:
```
sudo k0s kubectl version --short 2>/dev/null | ...
```
With kubectl 1.28, which is now the default on k0s, `kubectl version
--short` has been removed, leading us to an empty stringm causing then
the error in the CI.
Fixes: #8105
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've added two new containerd builder images recently, one for the
components under `src/tools` and another one for the Kata Containers
agent.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This follows what we've been doing for all the components we're
building, but was missed as part of #8077.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The kata-agent binary won't be released, just built so it can be used,
later on, as part of our tests and as part of the rootfs build.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed functions to start building the kata-agent, with or
without the OPA support.
For now this build is not used as part of the rootfs build, but later on
this will (not as part of this series, though).
Fixes: #8099
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help to build the agent binary as part of the kata-deploy
localbuild, as we need to pass the DESTDIR to where the agent will be
installed, and also whether we're building the agent with policy support
enabled or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The k8s.gcr.io is deprecated for a while now and has been redirected to
registry.k8s.io. However on some bare-metal machines in our testing
pools that redirection is not working, so let's just replace the
registries.
Fixes#8098
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
(cherry picked from commit b2c3bca558c38deff2117d5909d9071c23c05590)
Spell check failed with:
```
[kata-spell-check.sh:275] WARNING: Word 'overcommitment':
did you mean one of the following?: over commitment, over-commitment,
commitment
```
So update this to pass the static checks
Fixes: #
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Ensure that our documented crictl pod config file contents have
uid and namespace fields for compatibility with crictl 1.24+
This avoids a user potentially hitting the error:
```
getting sandbox status of pod "d3af2db414ce8": metadata.Name,
metadata.Namespace or metadata.Uid is not in metadata
"&PodSandboxMetadata{Name:nydus-sandbox,Uid:,Namespace:default,Attempt:1,}"
getting sandbox status of pod "-A": rpc error: code = NotFound desc = an
error occurred when try to find sandbox: not found
```
Fixes: #8092
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
(cherry picked from commit 8f8c2215)
We need to do proper sandbox sizing when we're doing cold-plug introduce CDI,
the de-facto standard for enabling devices in containers. containerd
will pass-through annotations for accumulated CPU,Memory and now CDI
devices. With that information sandbox sizing can be derived correctly.
Fixes: #7331
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Let's add targets and actually enable users and oursevles to build those
components in the same way we build the rest of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we'd like to ship the content from src/tools, we need to build them
in the very same way we build the other components, and the first step
is providing scripts that can build those inside a container.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The kata-monitor tests is currently running as part of the Jenkins CI
with the following setups:
* Container Engines: CRI-O | containerd
* VMMs: QEMU
When using containerd, we're testing it with:
* Snapshotter: overlayfs | devmapper
We will stop running those tests on devmapper / overlayfs as that hardly
would get us a functionality issue.
Also, we're restricting this to run with the LTS version of containerd,
when containerd is used.
As it's known due to our GHA limitation, this is just a placeholder and
the tests will actually be added in the next iterations.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will serve us quite will in the upcoming tests addition, which will
also have to be executed using CRi-O.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will become handy when doing tests with CRI-O, as CRI-O doesn't
install the CNI plugins for us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we have runc running with `SystemdCgroups = false`,
otherwise we'll face failures when running tests depending on runc on
Ubuntu 22.04, woth LTS containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Pause and resume task do not currently update the status of the
container to paused or running, so fix this. This is specifically for
pausing the task and not the VM.
Fixes#6434
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Allow Cloud Hypervisor to create a confidential guest (a TD or
"Trust Domain") rather than a VM (Virtual Machine) on Intel systems
that provide TDX functionality.
> **Notes:**
>
> - At least currently, when built with the `tdx` feature, Cloud Hypervisor
> cannot create a standard VM on a TDX capable system: it can only create
> a TD. This implies that on TDX capable systems, the Kata Configuration
> option `confidential_guest=` must be set to `true`. If it is not, Kata
> will detect this and display the following error:
>
> ```
> TDX guest protection available and must be used with Cloud Hypervisor (set 'confidential_guest=true')
> ```
>
> - This change expands the scope of the protection code, changing
> Intel TDX specific booleans to more generic "available guest protection"
> code that could be "none" or "TDX", or some other form of guest
> protection.
Fixes: #6448.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Introduce a few new constants (for PCI segment count and FS queues) and
move the disk queue constants to `convert.rs` to allow them to be used
there too.
> **Note:**
>
> This change gives the `ShareFs` code it's own set of values rather
> than relying on the disk queue constants.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Modify the Cloud Hypervisor `add_device()` method to add `ShareFs` and
`Network` devices to the list of pending devices since only these two
device types need to be cached before VM startup. Full details in the
comments.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Remove the `VIRTIO_BLK_MMIO` check which appears to have been added
erroneously in the first place.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
This patch re-generates the client code for Cloud Hypervisor v35.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #8057
Signed-off-by: Bo Chen <chen.bo@intel.com>
We've faced this as part of the CI, only happening with the CRI-O tests:
```
not ok 1 Test readonly volume for pods
# (from function `exec_host' in file tests_common.sh, line 51,
# in test file k8s-file-volume.bats, line 25)
# `exec_host "echo "$file_body" > $tmp_file"' failed with status 127
# [bats-exec-test:38] INFO: k8s configured to use runtimeclass
# bash: line 1: $'\r': command not found
#
# Error from server (NotFound): pods "test-file-volume" not found
```
I must say I didn't dig into figuring out why this is happening, but we
may be safe enough to just trail the '\r', as long as all the tests keep
passing on containerd.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need the default capabilities to be enabled, especially `SYS_CHROOT`,
in order to have tests accessing the host to pass.
A huge thanks to Greg Kurz for spotting this and suggesting the fix.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
Some of the "k8s distros" allow using CRI-O in a non-official way, and
if that's done we cannot simply assume they're on containerd, otherwise
kata-deploy will simply not work.
In order to avoid such issue, let's check for `cri-o` as the container
engine as the first place and only proceed with the checks for the "k8s
distros" after we rule out that CRI-O is not being used.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the network metrics section at the README by leaving
the current tests that we have in our kata metrics.
Fixes#8017
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The permissions on .docker/buildx/activity/default are regularly broken by us
passing docker.sock + $HOME/.docker to a container running as root and then
using buildx inside. Fixup ownership before executing docker commands.
Fixes: #8027
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
To avoid the failure of not finding pandoc command this PR adds that
package as a dependency for static checks.
Fixes#8041
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The cgroup stats come from resourcecontrol package in the form of pointers
to structs. The sandbox Stat() method incorrectly was expecting structs.
This caused the cpu and memory stats to always be 0, which in turn caused
incorrect pod overhead metrics.
Fixes#8035
Signed-off-by: Peteris Rudzusiks <rye@stripe.com>
Firecracker supports noflush semantic via Unsafe cache type.
There is no support for direct i/o, remove it from config file
Fixes: #7823
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Clh suports direct i/o for disks. It doesn't
offer any support for noflush, removed passing
of option to cloud-hypervisor internal config
Fixes: #7798
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Seems like the static checks are failing due the missing of the hunspell
package this PR fixes that.
Fixes#8019
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The test has been added to the repo, but we have to also add it to the
list of jobs to be executed.
Fixes: #8005
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll face the following error:
```
Failed to enable unit: Interactive authentication required.
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we'll also be testing k8s using CRI-O.
For now, we'll only be running the CRI-O test with QEMU. Once it
becomes stable we can expand this to other Hypervisors as well.
Fixes: #8005
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Support quoted kernel command line parameters that include space
characters. Example:
dm-mod.create="dm-verity,,,ro,0 736328 verity 1
/dev/vda1 /dev/vda2 4096 4096 92041 0 sha256
f211b9f1921ef726d57a72bf82be23a510076639fa8549ade10f85e214e0ddb4
065c13dfb5b4e0af034685aa5442bddda47b17c182ee44ba55a373835d18a038"
Fixes: #8003
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This is part of a bigger effort to drop gogoprotobuff from our code
base. IIUC, those options are basically used by *pb_test.go, and since
we are dropping gogoprotobuff and those are auto generated tests, let's
just remove it.
Fixes#7978.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This will be very useful in the near future, when we start testing
kata-deploy with rke2 as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will be very useful in the near future, when we start testing
kata-deploy with k0s as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll be using exactly the same code used for the k8s tests, which are
already deploying k3s on GARM.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We just need to make sure the correct overlay is applied, following what
we already have been doing for k3s.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll be testing kata-deploy with different kubernetes flavours as part
of our GARM tests, and this is a place-holder for this.
Once enabled, we'll do nothing, just `return 0`, so we can then properly
add the tests after this commit gets merged.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So we have a better control on which flavour of kubernetes kata-deploy
is expected to be targetting.
This was also done as part of fa62a4c01b,
for the k8s tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Ideally we'd add the instance_type or the full K8S_TEST_HOST_TYPE but
that exceeds the maximum amount of characteres allowed for the cluster
name. With this in mind, let's use the first letter of
K8S_TEST_HOST_TYPE instead.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This makes it so that each AKS cluster is created in its own individual
resource group, rather than using the "kataCI" resource group for all
test clusters.
This is to accommodate a tool that we recently introduced in our Azure
subscription which automatically deletes resource groups after a set
amount of time, in order to keep spending under control.
The tool will automatically delete any resource group, unless it has a
tag SkipAutoDeleteTill = YYYY-MM-DD. When this tag is present, the
resource group will be retained until the specified date.
Note that I tagged all current resource groups in our subscription with
SkipAutoDeleteTill = 2043-01-01 so that we don't lose any existing
resources.
Fixes: #7982
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We're hitting a specific issue after updating, which will require some
work on dragonball before it can be re-added here.
The issue:
```
...
3: failed to do rafs mount\\n
4: fail to attach rafs \\\"/var/lib/containerd-nydus/snapshots/2/fs/image/image.boot\\\"\\n
5: add share fs mount\\n
6: Mount rafs at
/rafs/197ef3db03c86b91bf3045ff59183ce8b5750941ad1d3484f4a8301a70f5109f/rootfs_lower
error: Failed to Mount backend
...
Caused by:
vmm action error: FsDevice(AttachBackendFailed(\\\"attach/detach a
backend filesystem failed:: missing field `version` at line 1 column
489\\\"))\"): unknown"
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will ensure we're testing with the correct runtime, instead of
using the `default` one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix the arch error when downloading the nydus tarball.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Steven Horsman <steven@uk.ibm.com>
To support the v0.12.0 nydus-snapshotter, we need to update the config
files and the commandline to start nydus-snapshotter.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
And with this we finally enable the nydus tests to run as part of our
GHA CI.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been simply doing nothing whenever `install-kata` was called, and
that was the intent when we added the placeholder calls.
Now, let's install kata, as expected. :-)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we've added install_nydus() and install_nydus_snapshotter(), which do
conform with the pattern we're following on GHA, let's rely on them
rather than relying on the bits coming from nydus_test.sh.
Later on we'll have install_nydus() and install_nydus_snapshotter() as
part of the dependencies install in our `gha-run.sh`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to what's been done for the cri-containerd tests, as part of
84dd02e0f9, we need to add the timeout
here for the crictl calls.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we may face errors like:
```
getting sandbox status of pod "d3af2db414ce8": metadata.Name,
metadata.Namespace or metadata.Uid is not in metadata
"&PodSandboxMetadata{Name:nydus-sandbox,Uid:,Namespace:default,Attempt:1,}"
getting sandbox status of pod "-A": rpc error: code = NotFound desc = an
error occurred when try to find sandbox: not found
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we canoot properly start the nydus snapshotter, nor properly
kill it after it's been started.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The "source ..." we've been doing was not changed since those tests were
part of the Jenkins tests, and we need to adapt them, either setting the
correct path or entirely removing the ones that are not relevant to us
anymore.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will be used to download and install the
nydus-snapshotter, and it follows the same pattern we already have
introduced for downloading and installing another dependencies from
GitHub.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will be used to download and install nydus, and it follows
the same pattern we already have introduced for downloading and
installing another dependencies from GitHub.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Users have noticed that this is needed, as CLH does not yet implement a
way to hotplug resources on aarh64.
With this patch, when building for x86_64, I can see the this is the
resulting config:
```
$ ARCH=amd64 make
...
$ cat config/configuration-clh.toml | grep static_sandbox_resource_mgmt
static_sandbox_resource_mgmt=false
```
And when building for aarch64:
```
$ ARCH=arm64 make
...
$ cat config/configuration-clh.toml | grep static_sandbox_resource_mgmt
static_sandbox_resource_mgmt=true
```
Fixes: #7941
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
PR #6146 added the possibility to control QEMU with an extra HMP socket
as an aid for debugging. This is great for development or bug chasing
but this raises some concerns in production.
The HMP monitor allows to temper with the VM state in a variety of ways.
This could be intentionally or mistakenly used to inject subtle bugs in
the VM that would be extremely hard if not even impossible to debug. We
definitely don't want that to be enabled by default.
The feature is currently wired to the `enable_debug` setting in the
`[hypervisor.qemu]` section of the configuration file. This setting has
historically been used to control "debug output" and it is used as such
by some downstream users (e.g. Openshift). Forcing people to have the
extra HMP backdoor at the same time is abusive and dangerous.
A new `extra_monitor_socket` is added to `[hypervisor.qemu]` to give
fine control on whether the HMP socket is wanted or not. This setting
is still gated by `enable_debug = true` to make it clear it is for
debug only. The default is to not have the HMP socket though. This
isn't backward compatible with #6416 but it is for the sake of "better
safe than sorry".
An extra monitor socket makes the QEMU instance untrusted. A warning is
thus logged to the journal when one is requested.
While here, also allow the user to choose between HMP and QMP for the
extra monitor socket. Motivation is that QMP offers way more options to
control or introspect the VM than HMP does. Users can also ask for
pretty json formatting well suited for human reading. This will improve
the debugging experience.
This feature is only made visible in the base and GPU configurations
of QEMU for now.
Fixes#7952
Signed-off-by: Greg Kurz <groug@kaod.org>
Now that the static-checks job only takes care of running the
static-checks, let's clean it up, remove all the unneeded steps, make
sure that we're using the actions in their latest version, and have it
running in a cost free runner.
At some point I'd like to see those tests done in parallel, in the same
way that I've organised the build-checks, but that's something for
someone else, at some other time.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With this we're removing the dragonball static-checks CI, as the test is
running here now. :-)
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` and `make check` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` and `make check` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will simply fail with:
```
error[E0583]: file not found for module `config`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This makes it pssible to run the tests in the cost free runners, which
are not KVM capable.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will simply fail with:
```
error[E0583]: file not found for module `version`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise `make test` will fail with:
```
error[E0583]: file not found for module `version`
```
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're moving it out of the previous "static-checks" confusing matrix,
and adding it to the matrix that was currently being used for the `make
vendor` checks.
This will allow us to have one job per component, and with that we can
easily run those in parallel and on the zero cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
clippy is used as part our tests, so it's useful to have it installed
while we're already installing rust.
In case of developers, they also better be using it. :-)
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Similarly to the static-check jobs, those jobs can be run on the zero
cost runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We'll use it as part of the refactoring we're doing in the static check
tests.
I can see a lot of other uses of this, but changing all of them to this
one is out of the scope for this PR.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We can use this a lot as part of our CI, but right now I'm just moving
those here with the intent to use later on in this series.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It doesn't make sense to run this for all the bits of the matrix,
neither it's demanding enough to require running this in one of our
Azure sponsored runners.
Fixes: #7974 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let me start with a fair warning that this commit is hard to split into
different parts that could be easily tested (or not tested, just
ignored) without breaking pieces.
Now, about the commit itself, as we're on the run to reduce costs
related to our sponsorship on Azure, we can split the k8s tests we run
in 2 simple groups:
* Tests that can be run in the smaller Azure instance (D2s_v5)
* Tests that required the normal Azure instance (D4s_v5)
With this in mind, we're now passing to the tests which type of host
we're using, which allows us to select to run either one of the two
types of tests, or even both in case of running the tests on a baremetal
system.
Fixes: #7972
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds support for adding a network device before starting the
cloud-hypervisor VM.
Support for adding and removing network devices is not really added to
the resource manager, so supporting this for cloud-hypervisor is not
scoped in this PR.
This also changes "pending_devices" for clh implementation from an
Option of vector to simply a vector. This simplifies the structure a bit
as we can simple iterate over the pending devices instead of having to
check for a "Some" value as this is not really required.
Fixes: #6333
Signed-off-by: Shuaiyi Zhang <zhang_syi@qq.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
We've removed this in the part 2 of this effort, as we were not caching
the sha256sum of the component. Now that this part has been merged,
let's get back to checking it.
Fixes: #7834 -- part 3
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
That's not needed anymore, as we've switched to using ORAS and an OCI
registry to cache the artefacts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is something that was done by our Jenkins jobs, but that I ended up
missing when writing d0c257b3a7.
Now, let's also add the sha256sum to the cached artefact, and in a
coming up PR (after this one is merged) we will also start checking for
that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the previous series related to the artefacts we build, we've
switching from storing the artefacts on Jenkins, to storing those in the
ghcr.io/kata-containers/cached-artefacts/${artefact_name}.
Now, let's take advantage of that and actually use the artefacts coming
from that "package" (as GitHub calls it).
NOTE: One thing that I've noticed that we're missing, is storing and
checking the sha256sum of the artefact. The storing part will be done
in a different commit, and the checking the sha256sum will be done in a
different PR, as we need to ensure those were pushed to the registry
before actually taking the bullet to check for them.
Fixes: #7834 -- part 2
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
cloud hypervisor on arm64 only support arm AMBA UART(pl011) as
tty. So, the console should be set to "ttyAMA0" instead of "ttyS0"
when enable hypervisor debug mode.
Fixes: #5080
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The list of tests which require a bigger VM instance is:
* k8s-number-cpus.bats -- failing on all CIs
* k8s-parallel.bats -- only failing on the cbl-mariner CI
* k8s-scale-nginx.bats -- only failing on the cbl-mariner CI
We'll keep those disabled while we re-work the logic to **only run
those** in a bigger (and more expensive) VM instance.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's push the artefacts to ghcr.io and stop relying on jenkins for
that.
Fixes: #7834 -- part 1
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is not used, but it'll be used when we start caching the
artefacts using ORAS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
ORAS is the tool which will help us to deal with our artefacts being
pushed to and pulled from a container registry.
As both the push to and the pull from will be done inside the
kata-deploy binaries builder container, we need it installed there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need to run on a D4s_v5. as those tests are not CPU / memory
intense. With this is mind, let's use a smaller version of the
instance, the D2s_v5 one.
Fixes: #7958
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Without setting the cpu limit / request to 1, we can make this test run
in a smaller VM instance without any issue.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the environment variables are now being passed down from the GitHub
Actions, let's make sure they're exposed to the container used to build
the kata-deploy binaries, and during the build process we'll be able to
use those to log in and push the artefacts to the OCI registry, using
ORAS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We do the build of our artefacts inside a container image, and we need
to expose some env vars to the container so ORAS can be used there to
push the artefacts we want to cache to ghcr.io.
The env vars we're exposing are:
* ARTEFACT_REGISTRY: The registry where we're going to save the
artefacts.
* ARTEFACT_REGISTRY_USERNAME: The username to log in to the registry, as
ORAS does not use the same json file used by docker.
* ARTEFACT_REGISTRY_PASSWORD: The pasword to log in to the the registry,
as the ORAS does not use the same json file used by docker.
* TARGET_BRANCH: The target branch, which will be part of the tag of the
artefact, as we may end up caching the artefacts for both main and
stable branches.
Fixes: #7834 -- part 0
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need a very recent L2 guest kernel to fix all the bugs that occur in nested
virtualization.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
cloud hypervisor does not emulate pcie switches or pci bridges, so we need to
accept a lonely device.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
It is fine to start a VM with the disk image without syncing it as we now run
the test in an ephemeral Azure instance.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Sometimes the test gets stuck running commands in the container - need to
investigate why later.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Cloud Hypervisor exposes a VIRTIO_IOMMU device to the VM when IOMMU support is
enabled. We need to add it to the whitelist because dragonball uses kernel
v5.10 which restricted VIRTIO_IOMMU to ARM64 only.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
by enabling IOMMU on the default PCI segment. For hotplug to work we need a
virtualized iommu and clh exposes one if there is some device or PCI segment
that requests it. I would have preferred to add a separate PCI segment for
hotplugging vfio devices but unfortunately kata assumes there is only one
segment all over the place. See create_pci_root_bus_path(),
split_vfio_pci_option() and grep for '0000'.
Enabling the IOMMU on the default PCI segment requires passing enabling IOMMU on
every device that is attached to it, which is why it is sprinkled all over the
place.
CLH does not support IOMMU for VirtioFs, so I've added a non IOMMU segment for
that device.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There is no way for this branch to be hit, as port is only set when it is
different than config.NoPort.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
These test cases shows which options are valid for CLH/Qemu, and test that we
correctly catch unsupported combinations.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The only supported options are hot_plug_vfio=root-port or no-port.
cold_plug_vfio not supported yet.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
hot_plug_vfio needs to be set to root-port, otherwise attaching vfio devices to
CLH VMs fails. Either cold_plug_vfio or hot_plug_vfio is required, and we have
not implemented support for cold_plug_vfio in CLH yet.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
tdp_mmu had some issues up until around Linux v6.3 that make it work
particularly bad when running nested on Hyper-V. Reload the module at the start
of the test and disable the tdp_mmu param.
Gather debug info at the end of the test to make it easier to figure out what
went wrong. This uses github actions group syntax so that each section can be
collapsed.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
- reduce memory and cpu usage to fit in a D4s_v5
- source correct lib
- mount workspace from 9p
- disable cpu mitigations for speed
- drop unused commands and variables
- install containerd
- install kata from built artifacts
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This imports the vfio test scripts github.com/kata-containers/tests. The test
case doesn't work yet but doing the changes in a separate commit will make it
easier to track the changes. The only change in this commit is renaming
vfio_jenkins_job_build.sh -> vfio_fedora_vm_wrapper.sh
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Otherwise we'll fail to configure kata-containers in the `install-kata`
step.
This is mostly needed because the nerdctl-full tarball doesn't provide a
contaienrd configuration, just the binary, as contaienrd does not
actually require a configuration file to run with the default config.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TIL that the Azure VMs we use are created without an explicit outbund
connectivity defined.
This leads us to issues using `ping ...` as part of our tests, and when
consulting Jeremi Piotrowski about the issue he pointed me out to two
interesting links:
* https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/default-outbound-access
* https://learn.microsoft.com/en-us/archive/blogs/mast/use-port-pings-instead-of-icmp-to-test-azure-vm-connectivity
For your own sanity, do not read the comments, after all this is
internet. :-)
Anyways, the suggestion is to use nping instead, which is provided by
the nmap package, so we can explicitly switch to using the tcp port 80
for the ping. With this in mind, I'm switching the image we use for the
test and using one that provided nping as a possible entry point, and
from now on (this part of) the tests should work.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TIL that the Azure VMs we use are created without an explicit outbund
connectivity defined.
This leads us to issues using `ping ...` as part of our tests, and when
consulting Jeremi Piotrowski about the issue he pointed me out to two
interesting links:
* https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/default-outbound-access
* https://learn.microsoft.com/en-us/archive/blogs/mast/use-port-pings-instead-of-icmp-to-test-azure-vm-connectivity
For your own sanity, do not read the comments, after all this is
internet. :-)
Anyways, the suggestion is to use nping instead, which is provided by
the nmap package, so we can explicitly switch to using the tcp port 80
for the ping. With this in mind, I'm switching the image we use for the
test and using one that provided nping as a possible entry point, and
from now on (this part of) the tests should work.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the RemoveEndpoints(), when the endpoints paramete isn't empty,
using idx may result in wrong endpoint removals. To improve,
directly passing the endpoint parameter helps
locate the correct elements within n.eps.
Fixes: #7732
Signed-off-by: shixuanqing <1356292400@qq.com>
Fixes: #7732
Signed-off-by: shixuanqing <1356292400@qq.com>
Update src/runtime/virtcontainers/network_linux.go
Co-authored-by: Xuewei Niu <justxuewei@apache.org>
Fix kernel and initrd annotations in the k8s tests on Mariner. These
annotations must be applied to the spec.template for Deployment, Job
and ReplicationController resources.
Fixes: #7764
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR ensures that docker is running as part of the init_env function
in kata metrics to avoid failures like docker is not running and making
the kata metrics CI to fail.
Fixes#7898
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
FIO test is showing ongoing issues when running in k8s.
Working on running FIO on the ctr client which has been
shown to be stable.
Fixes: #7920
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
There's no reason to wait till the payload is created to run the tests,
as we rely on the tarball, not on the kata-deploy payload.
That was a mistake on my side, and that's already fixed for the nerdctl
tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a very basic sanity test to check that we can spawn a
containers using nerdctl + Kata Containers.
This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.
In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.
Fixes: #7911
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a very basic sanity test to check that we can spawn a
containers using docker + Kata Containers.
This will ensure that, at least, we don't regress to the point where
this feature doesn't work at all.
For now we're running this test against Cloud Hypervisor and QEMU only,
due to an already reported issue with dragonball:
https://github.com/kata-containers/kata-containers/issues/7912
In the future, we should also test all the VMMs with devmapper, but
that's for a follow-up PR after this test is working as expected.
Fixes: #7910
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the any combinator.
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Allow `clippy::redundant-closure-call`
which has issues with the guard function passed into
the `run_if_auto_values` macro
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
The bindgen generated code is triggering lots of
ambiguous-glob-reexports warnings in rust 1.70+
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- In rust 1.72, clippy warned clippy::non-minimal-cfg
as the cfg has only one condition, so doesn't
need to be wrapped in the all combinators.
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
- Allow `clippy::redundant-closure-call` in `from_cmdline`
which has issues with the guard function passed into
the `parse_cmdline_param` macro
Fixes: #7902
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
When creating a new endpoint, we check existing endpoint names and automatically adjust the naming of the new endpoint to ensure uniqueness.
Fixes: #7876
Signed-off-by: shixuanqing <1356292400@qq.com>
This commit allows us to specify the huge page backend when enabling huge
page. Currently, we support two backends: thp and hugetlbfs, the default
is hugetlbfs.
To ensure backward compatibility, we introduce another configuration item
"hugepage_type" to select the memory backend, which is available only when
"enable_hugepages" is true. Besides, we add an annotation
"io.katacontainers.config.hypervisor.hugepage_type" to configure huge page
type per pod.
Fixes: #6703
Signed-off-by: Guixiong Wei <weiguixiong@bytedance.com>
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
As, regardless of what's mentioned in the documentation, it seems that
$GITHUB_REF_NAME is passed down as a literal string.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The ones for the payload-after-push.yamland ci-nightly.yaml are not that
much important right now, but they're needed for when we start running
those on stable branches as well.
The other ones were missed during
bd24afcf73.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the metrics migration from the tests to kata containers has been completed, this PR removes the warning from the main metrics documentation.
Fixes#7894
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
To support the removal of the `initcall_debug` and `earlyprintk=`
options from the default guest kernel cmdline, add `kernel_params` to the list
of enabled annotations to allow those kernel options (or others) to be
set using `kata-deploy` for either runtime.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Removed the following kernel command line options:
- `earlyprintk=ttyS0`
- `initcall_debug`
Both these options are only useful when debugging a guest kernel failure
which is not a common occurrence.
Further, the `earlyprintk=` option can have a large negative performance
impact (it can increase the VM boot time significantly).
If the user wishes to use either of these options, they can add them to the
`kernel_params=` setting in the Kata configuration file's hypervisor
stanza.
Fixes: #7886.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
All the patches have already been merged upstream and they've just been
cherry-picked to this branch.
Fixes: #7885
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're bumping here in order to make our lives easier backporting EROFS
patches needed for the CC related work.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removed the addition of the `initcall_debug` kernel option when agent
debugging enabled. This option has nothing to do with the agent.
If the user wishes to use this option, they can add it to the
`kernel_params=` setting in the Kata configuration file's hypervisor
stanza.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Similarly to what's been done for x86_64 -> amd64, we need to do a
aarch64 -> arm64 change in order to be able to download the kubectl
binary.
Fixes: #7861
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
1. Directly support CgroupManager::freeze through systemd API.
2. Avoid always passing unit_name by storing it into DBusClient.
3. Realize CgroupManager::destroy more accurately by killing systemd unit rather than stop it.
4. Ignore no such unit error when destroying systemd unit.
5. Update zbus version and corresponding interface file.
Acknowledgement: error handling for no such systemd unit error refers to
Fixes: #7080, #7142, #7143, #7166
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
We're changing what's been done as part of ac939c458c, as we've
notcied issues using `github.event.pull_request.merge_commit_sha`.
Basically, whenever a force-push would happen, the reference of
merge_commit_sha wouldn't be updated, leading us to test PRs with the
old code. :-/
In order to get the rebase properly working, we need to ensure we pull
the hash of the commit as part of checkout action, and ensure
fetch-depth is set to 0.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This syntax belongs to the legacy C virtiofsd implementation that
we don't support anymore since kata-containers 3.1.3 because
of other API breaking changes.
People have been warned to switch from "none" to "never" since
kata-containers 2.5.2. Let's officially do that.
The compat code that would convert "none" to "never" isn't
needed anymore. Just drop it.
Fixes#7864
Signed-off-by: Greg Kurz <groug@kaod.org>
gogo.nullable is the main gogo.protobuf' feature used here. Since we are
trying to remove gogo.protobuf, the first reasonable step seems to be
remove this feature. This is a core update, and it will change how the
structs are defined. I could spot only a few places using those structs,
based on make check/build.
Fixes#7723.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
There is no reference to PROTO_FILE and this is not working. Also we are
not inside a Makefile, so makes sense to adapt the usage to reflect the
script instead of a make command.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
import_path is used as the default package when no input files specify
go_package. However, all the files we are currently building already
have a go_package definition, making this behavior both redundant and
error-prone.
Additionally, one of our files (types.pb.go) resides outside the grpc
directory, indicating that it's indeed ignored but also inconsistent.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
Currently, the script searches for .proto files within $GOPATH/.
Consequently, modifications to a definition file in the current working
directory won't influence the output .pb.go if the directory is outside
of $GOPATH. For developers, it's more intuitive to alter the local
codebase than the version stored in $GOPATH.
With this modification, the generated .pb.go files will be relative to
the current working directory, removing the need to clone this project
under $GOPATH/src/github.com/kata-containers.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
The definitions are already specified in the .proto files using the
go_package option. Centralizing them in one location reduces the
potential for errors and simplifies the script.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
There's absolutely no need to have the skip check as part of the test
itself when it's already done as part of the setup function.
We're only touching the files here that were touched in the previous
commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's keep both checks for now, but in the future we'll be able to
remove the check for "firecracker", as the hypervisor name used as part
of the GitHub Actions has to match what's used as part of the
kata-deploy stuff, which is `fc` (as in `kata-fc for the runtime class)
instead of `firecracker`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
That's what we've been using as part of Jenkins, so let's ensure things
will work as they did before, and only after that consider upgrading the
base OS used for the tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been using the `kata-deploy-tdx` target as that also uses k3s as
base, but it's better to just have a specific garm target.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
So we have a better control on which flavour of kubernetes kata-deploy
is expected to be targetting.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GARM runners do not come with the whole set of tools we need, or are
used to when it comes to the GHA runners, so we need to manually install
bats on those.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR changes the order in which the FIO test first
cleans the environment and then checks if the environment
is indeed clean.
Fixes: #7869
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
As we were using `tee` without the `-a` (or `--apend`) aptton, the
containerd config would be overwritten, leading to a NotReady state of
the Node.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's download the vanilla kubectl binary into `/usr/bin/`, as we need
to avoid hitting issues like:
```sh
error: open /etc/rancher/k3s/k3s.yaml.lock: permission denied
```
The issue basically happens because k3s links `/usr/local/bin/kubectl`
to `/usr/local/bin/k3s`, and that does extra stuff that vanilla
`kubectl` doesn't do.
Also, in order to properly use the k3s.yaml config with the vanilla
kubectl, we're copying it to ~/.kube/config.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise the /etc/rancher/k3s/k3s.yaml is not readable by other users
than root.
As --write-config-mode is being passed, and that's an option that has to
be passed to the `server`, -s is also added to the command line.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`wait` waits for a job to complete, not a number of seconds. Not sure
how I got that wrong in the first place, but it's what it's.
Fixes: #6542
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR replaces the ubuntu image for one which has TensorFlow optimized
for kata metrics.
Fixes#7866
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently, the dbs-upcall features have 2 problems that are needed to be
fixed :
There are redundant dbs-upcall features that are needed to be removed.
Some place should be controlled by dbs-upcall but not being implemented.
This commit will fix those two problems.
fixes: #6878
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Let's enable the devmapper kubernetes tests to match exactly what's been
tested as part of the Jenkins CI.
Fixes: #6542
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function right now is completely based on what's part of the tests
repo[0], and that's the reason I'm keeping the `Signed-off-by` of all
the contributors to that file.
This is not perfect, though, as it changes the default snapshotter to
devmapper, instead of only doing so for the Kata Containers specific
runtime handlers. OTOH, this is exactly what we've always been doing as
part of the tests.
We'll improve it, soon enough, when we get to also add a way for
kata-deploy to set up different snapshotters for different handlers.
But, for now, this is as good (or as bad) as it's always been.
It's important to note that the devmapper setup doesn't take into
consideration a BM machine, and this is not suitable for that. We're
really only targetting GHA runners which will be thrown away after the
run is over.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
One can use different kubernetes flavours for getting a kubernetes
cluster up and running.
As part of our CI, though, I really would like to avoid contributors
spending time maintaining and updating kubernetes dependencies, as done
with the tests repo, and which has been proven to be really good on
getting things rotten.
With this in mind, I'm taking the bullet and using "k3s" as the way to
deploy kubernetes for the devmapper related tests, and that's the reason
I'm adding a function to do so, and this will be used later on as part
of this series.
It's important to note that the k3s setup doesn't take into
consideration a BM machine, and this is not suitable for that. We're
really only targetting GHA runners which will be thrown away after the
run is over.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Some use cases may just require passing extra arguments to virtiofsd,
and having this disabled by default makes it impossible to set when
using kata-deploy, as changes in the configuration file would be
overwritten by the daemon-set.
With this in mind, let's allow users to pass whatever thet need (and
here I'm specifically looking at `--xattr`) as a virtio_fs_extra_arg.
Fixes: #7853
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR is to skip installing docker-compose-plugin while buiding a `build-kata-deploy` image for s390x|ppc64le.
It is a temporary solution to fix current CI failures for s390x regarding `hash sum mismatch`.
Fixes: #7848
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
This PR adds the write 95 percentile for FIO for qemu for
checkmetrics for kata metrics.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the write 95 percentile FIO value for checkmetrics
for kata metrics.
Fixes#7842
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Previously, static-checks-dragonball is using machines from Alibaba
Cloud to run all the CI jobs.
Currently, we are going through an internal process to apply for the new
machines for Dragonball CI. Before the internal process is over, we will
temporarily use Azure VM to run static-checks-dragonball jobs.
fixes: #7838
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Currently, virtio_vsock are still outside of the device
manager. This causes some management issues,such as the
inability to unify PCI address management.
Just do some work for hybrid vsock.
Fixes: #7655
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Pass --owner and --group to the tar invokation to prevent gihtub runner user
from leaking into release artifacts.
Fixes: #7832
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
When a storage device is used by more than one container, the second
and forth instances will cause storage device reference count leakage,
thus cause storage device leakage. The reason is:
add_storages() will increase reference count of existing storage device,
but forget to add the device to the `mount_list` array, thus leak the
reference count.
Fixes: #7820
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Use AGENT_POLICY=yes when building the Guest images, and add a
permissive test policy to the k8s tests for:
- CBL-Mariner
- SEV
- SNP
- TDX
Also, add an example of policy rejecting ExecProcessRequest.
Fixes: #7667
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Let's make sure we use the TDX image as part of the QEMU TDX
configuration, which will help us to have the policies tested here.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We have two scenarios we care about this, `pull_request` and
`pull_request_target` events triggered a job.
`pull_request` event:
When using the checkout action, it'll already provide a "rebased atop of
main" repo for us, nothing else is needed, and that's basically what we
already have as part of the jobs in our CI.
`pull_request_target` event:
This one is a little bit tricky, as the checkout action, unless passing
a spsecific repo, give us the PR checked out rebased atop of the HEAD of
the PR branch. Jeremi Piotrowski nicely pointed out that we could use
github.event.pull_request.merge_commit_sha instead, which is the result
of the PR's branch with the official repo target branch.
Now, the only cases where the contributor's rebase would still be needed
is when the action itself has been changed.
Fixes: #7414
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds the grabdata script so it can be used for the metrics report
for kata metrics.
Fixes#7812
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Update the protection checking code to detect newer versions of Intel
TDX (whose userland interface has now stabilised).
> **Note:** that we don't need to retain the existing behaviour since:
>
> - We haven't yet landed the TDX feature (#6448).
> - Systems wishing to use TDX will need to use the latest available
> system components (such as firmware and host kernel).
Also added an explicit TDX unit test.
Fixes: #7384.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
IoCopy is a tricky function (I don't claim to fully understand its contract),
but here is what I see: The goroutine that runs it spawns 3 goroutines - one
for each stream to handle (stdin/stdout/stderr). The goroutine then waits for
the stream goroutines to exit. The idea is that when the process exits and is
closed, the stdout goroutine will be unblocked and close stdin - this should
unblock the stdin goroutine. The stderr goroutine will exit at the same time as
the stdout goroutine. The iocopy routine then closes all tty.io streams.
The problem is that the stdout goroutine decrements the WaitGroup before
closing the stdin stream, which causes the iocopy goroutine to race to close
the streams. Move the wg.Done() of the stdout routine past the close so that
*this* race becomes impossible. I can't guarantee that this doesn't affect some
unspecified behavior.
Fixes: #5031
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
If we are running FC hypervisor, it is not started when prestart hooks
are executed. So we should just ignore such error and just go ahead and
run the hooks.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
FC does not support network device hotplug. Let's add a check to fail
early when starting containers created by docker.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Add a new hypervisor capability to tell if it supports device hotplug.
If not, we should run prestart hooks before starting new VMs as nerdctl
is using the prestart hooks to set up netns. To make nerdctl + FC
to work, we need to run the prestart hooks before starting new VMs.
Fixes: #6384
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
When running on amd machines, those tests will fail because there is no
vmx flag. Following other tests that checks for cpuType, let's adapt
them to restrict vmx only on Intel machines.
Fixes#7788.
Related #5066
Signed-off-by: Beraldo Leal <bleal@redhat.com>
This PR fixes the memory inside limit for clh for kata metrics due
to the recent changes that we had in the script which impacted
in the performance measurement.
Fixes#7786
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's expand the confidential test to also support TDX.
The main difference on the test, though, is that we're not grepping for
a string in the `dmesg` output, but rather relying on `cpuid` to detect
a TDX guest.
Fixes: #7184
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add a test case for the launch of unencrypted confidential
container, verifying that we are running inside a TEE.
Right now the test only works with SEV, but it'll be expanded in the
coming commits, as part of this very same series.
Fixes: #7184
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes an issues in the parsing results stage,
by collecting just the n-results from the n-running
containers, discarding irrelevant data.
Fixes: #7774
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The file can be removed between builds without causing any issue, and
leaving it around has been causing us some headache due to:
```
ERROR: open /home/runner/.docker/buildx/activity/default: permission denied
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The log_forwarder task does not check if the peer has closed, causing a
meaningless loop during the period of “kata vm exit”, when the peer
closed, and “ShutdownContainer RPC received” that aborts the log forwarder.
This patch fixes the problem.
Fixes: #7741
Signed-off-by: Zixuan Tan <tanzixuan.me@gmail.com>
Otherwise we'll have to re-run all the tests due to a flaky behaviour in
one of the parts.
Fixes: #7757
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
k8s-pid-ns.bats was already using the test name from
k8s-kill-all-process-in-container.bats - probably a copy/paste bug.
Fixes: #7753
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
The directory is a host path mount and cannot be removed from within the
container. What we actually want to remove is whatever is inside that
directory.
This may raise errors like:
```
rm: cannot remove '/opt/kata/': Device or resource busy
```
Fixes: #7746
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The vfio test requires nested-nested virtualization:
L0 Azure host
-> L1 Ubuntu VM
-> L2 Fedora VM
-> L3 Kata
This hits a kernel bug on v5.15 but works quite nicely on the v6.2 kernel
included in Ubuntu 23.04. We can switch back to Ubuntu 22.04 when they roll out
v6.2.
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There are several processes for container exit:
- Non-detach mode: `Wait` request is sent by containerd, then
`wait_process()` will be called eventually.
- Detach mode: `Wait` request is not sent, the `wait_process()` won’t be
called.
- Killed by ctr: For example, a container runs `tail -f /dev/null`, and
is killed by `sudo ctr t kill -a -s SIGTERM <CID>`. Kill request is
sent, then `kill_process()` will be called. User executes `sudo ctr c
rm <CID>`, `Delete` request is sent, then `delete_process()` will be
called.
- Exited on its own: For example, a container runs `sleep 1s`. The
container’s state goes to `Stopped` after 1 second. User executes
the delete command as below.
Where do we do container cleanup things?
- `wait_process()`: No, because it won’t be called in detach mode.
- `delete_process()`: No, because it depends on when the user executes the
delete command.
- `run_io_wait()`: Yes. A container is considered exited once its IO ended.
And this always be called once a container is launched.
Fixes: #7713
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
Refine storage related code by:
- remove the STORAGE_HANDLER_LIST
- define type alias
- move code near to its caller
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Introduce StorageDevice and StorageHandlerManager, which will be used
to refine storage device management for kata-agent.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Simplify the way to manage storage objects, and introduce
StorageStateCommon structures for coming extensions.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
docker install now creates a group with gid 999 which happens to match what we
need to get docker-in-docker to work. Remove the group first as we don't need
it.
Fixes: #7726
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
We can simply use `rm -f` all over the place and avoid the container
returning any error.
Fixes: #7733
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now if we configure an image annotation and have a config file
setting initrd, the initrd config would override the image annotation.
Make sure annotations are preferred over config options in image and initrd
path handling.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
We should make sure annotations are preferred over
config options in image and initrd path handling.
Fixes: #7705
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Right now if we configure an image annotation and have a config file
setting initrd, the initrd config would override the image annotation.
Add a helper function ImageOrInitrdAssetPath to make sure annotations
are preferred over config options in image and initrd path handling.
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
When the FileMode field for the device is unset (0), use a default value instead
to allow the use of the device from the container.
This behaviour is seen from cri-o typically.
Note: this is what runc is doing, which is why regular containers don't have an
issue. This change makes sure kata behaves the same as runc.
Fixes: #7717
Signed-off-by: Julien Ropé <jrope@redhat.com>
Introduce structure KataVirtualVolume to to encapsulate information
for extra mount options and direct volumes, so we could build a common
infrastructure to handle these cases.
Fixes: #7699
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This PR configures the corresponding kata runtime in K8s
based on the tested hypervisor.
This PR also enables FIO metrics test in the kata metrics-ci.
Fixes: #7665
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
When building with AGENT_POLICY=yes and AGENT_INIT=yes:
1. Include OPA and the Policy settings in rootfs.
2. Start OPA from the kata agent.
Before these changes, building with both AGENT_POLICY=yes and
AGENT_INIT=yes was unsupported.
Starting OPA from systemd (when AGENT_INIT=no) was already supported.
Fixes: #7615
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR fixes the check results for tensorflow benchmark now
that we change the name of the test.
Fixes#7684
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Instead of doing this as part of the test itself, let's ensure it's done
before running the tests and during the tests cleanup.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This test, at least for now, only checks whether the runtimeclasses
have been properly created.
This is just a migration from a test we had as part of the k8s suite.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
By doing this we can make sure there won't be any clash on the cluster
name created for either the k8s or the kata-deploy tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will have the same function as run-k8s-tests.sh has, but for
kata-deploy.
Right now it doesn't have any tests, and the command to actually run the
tests is commented out, but right now this is just a placeholder that
will be populated sooner than later.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In a follow-up series, we'll add a whole suite for the kata-deploy
tests. With this in mind, let's already get rid of this one and avoid
more kata-deploy tests to land here.
Fixes: #7642
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The default `kata` runtime class would get created with the `kata`
handler instead of `kata-$KATA_HYPERVISOR`. This made Kata use the wrong
hypervisor and broke CI.
Fixes: #7663
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Let's add the tests as part of the ci.yaml, so they an be triggered as
part of each PR.
For this PR those tests won't be triggered, courtesy to the
`pull_request_target` event we rely on.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the TensorFlow word across the document to have uniformity
across all the document.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The error message when the kill command is executed with the container's
state == Stopped should be "container not running" because the containerd
tests expect that OCI runtimes return the error message and compare it.
If the error message is different from the expected one, the tests fail.
Fixes: #7650
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
As the cri-containerd tests have been fully migrated to GHA, let's make
sure we get them running.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As part of the runners, we're hitting a timeout that I cannot reproduce,
at all, when allocating the same instance and running the tests
manually.
The default timeout to connect to the server is 2s when using `crictl`.
Let's increase this to 20s.
It's fairly important to mention that in the first tests I used a
timeout of 10s, and that helped but we still hit issues every now and
then.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We extend the `Result` and `Option` types with associated types that
allows converting a `Result<T, E>` and `Option<T>` into
`ttrpc::Result<T>`.
This allows the elimination of many `match` statements in favor of
calling the map function plus the `?` operator. This transformation
simplifies the code.
Fixes: #7624
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This PR renames the tensorflow scripts to include the data format
that is being used as we will have multiple tests with different
data and model formats for tensorflow so this will help us to
distinguish them.
Fixes#7645
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.
Fixes: #7642
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will not be tested as part of the PR, thanks to the
`pull_request_target` event, but we want it to be added so we can build
atop of that in a coming up series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #7573
To enable this feature, build your rootfs using AGENT_POLICY=yes. The
default is AGENT_POLICY=no.
Building rootfs using AGENT_POLICY=yes has the following effects:
1. The kata-opa service gets included in the Guest image.
2. The agent gets built using AGENT_POLICY=yes.
After this patch, the shim calls SetPolicy if and only if a Policy
annotation is attached to the sandbox/pod. When creating a sandbox/pod
that doesn't have an attached Policy annotation:
1. If the agent was built using AGENT_POLICY=yes, the new sandbox uses
the default agent settings, that might include a default Policy too.
2. If the agent was built using AGENT_POLICY=no, the new sandbox is
executed the same way as before this patch.
Any SetPolicy calls from the shim to the agent fail if the agent was
built using AGENT_POLICY=no.
If the agent was built using AGENT_POLICY=yes:
1. The agent reads the contents of a default policy file during sandbox
start-up.
2. The agent then connects to the OPA service on localhost and sends
the default policy to OPA.
3. If the shim calls SetPolicy:
a. The agent checks if SetPolicy is allowed by the current
policy (the current policy is typically the default policy
mentioned above).
b. If SetPolicy is allowed, the agent deletes the current policy
from OPA and replaces it with the new policy it received from
the shim.
A typical new policy from the shim doesn't allow any future SetPolicy
calls.
4. For every agent rpc API call, the agent asks OPA if that call
should be allowed. OPA allows or not a call based on the current
policy, the name of the agent API, and the API call's inputs. The
agent rejects any calls that are rejected by OPA.
When building using AGENT_POLICY_DEBUG=yes, additional Policy logging
gets enabled in the agent. In particular, information about the inputs
for agent rpc API calls is logged in /tmp/policy.txt, on the Guest VM.
These inputs can be useful for investigating API calls that might have
been rejected by the Policy. Examples:
1. Load a failing policy file test1.rego on a different machine:
opa run --server --addr 127.0.0.1:8181 test1.rego
2. Collect the API inputs from Guest's /tmp/policy.txt and test on the
machine where the failing policy has been loaded:
curl -X POST http://localhost:8181/v1/data/agent_policy/CreateContainerRequest \
--data-binary @test1-inputs.json
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Right now this file does nothing, as it's not even called by any GHA.
However, it'll be populated later on as part of a different series,
where we'll have kata-deploy specific tests running here.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's split a good portion of `tests/integration/kuberentes/gha-run.sh`
out, and put them in a place where they can be used to the soon-to-come
kata-deploy specific tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove the installation step in the virtcontainers doc
because the virtcontainers install/uninstall targets have
been removed by 86723b51ae
and they are not used anymore.
Fixes: #7637
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Add k0s support to kata-deploy, in the very same way kata-containers
already supports k3s, and rke2.
k0s support requires v1.27.1, which is noted as part of the kata-deploy
documentation, as it's the way to use dynamic configuration on
containerd CRI runtimes.
This support will only be part of the `main` branch, as it's not a bug
fix that can be backported to the `stable-3.2` branch, and this is also
noted as part of the documentation.
Fixes: #7548
Signed-off-by: Steve Fan <29133953+stevefan1999-personal@users.noreply.github.com>
This PR fixed the loop that stops the kata-shim and the
hypervisors used in metrics checks.
Fixes: #7628
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The GHA runners are not exactly powerful, which makes the static-checks
take way too long (almost an hour).
Let's give a try and move those to the same size of Azure instances used
as part of our CI, and probably have this time reduced.
Fixes: #7446
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove configuration file shared_fs = none warnings
now that there is a solution to updating configMaps, secrets etc
Fixes: #7210
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This patch allows copying of directories and symlinks when
static file copying is used between host and guest. This change is
necessary to support recursive file copying between shim and agent.
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
(cherry picked from commit de232b8030)
For remote hypervisor, the configmap, secrets, downward-api or project-volumes are
copied from host to guest. This patch watches for changes to the host files
and copies the changes to the guest.
Note that configmap updates takes significantly longer than updates via downward-api.
This is similar across runc and Kata runtimes.
Fixes: #7210
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
Signed-off-by: Julien Ropé <jrope@redhat.com>
(cherry picked from commit 3081cd5f8e)
(cherry picked from commit 68ec673bc4d9cd853eee51b21a0e91fcec149aad)
This PR adds check containers are running in tensorflow mobilenet
that is being defined in common script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are up function from common
in tensorflow script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are running function the common metrics
script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the check containers are up in the common script
in the tensorflow mobilenet script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses the check containers are up from the common script
in the tensorflow script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR uses the collect results function defined in common for
the tensorflow mobilenet test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR removes the collect results function from tensorflow script
as it is going to be referenced in the common metrics script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Specify the supported version of Firecracker using our `versions.yaml`
to improve the maintainability of the documentation.
Fixes: #7610
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
This patch upgrades Firecracker version from v1.1.0 to v1.4.0.
* Generate swagger models for v1.4.0 (from `firecracker.yaml`)
- The version of go-swagger used is v0.30.0
* The firecracker v1.4.0 includes the following changes.
- Added
* Added support for custom CPU templates allowing users to adjust vCPU features
exposed to the guest via CPUID, MSRs and ARM registers.
* Introduced V1N1 static CPU template for ARM to represent Neoverse V1 CPU
as Neoverse N1.
* Added support for the virtio-rng entropy device. The device is optional. A
single device can be enabled per VM using the /entropy endpoint.
* Added a cpu-template-helper tool for assisting with creating and managing
custom CPU templates.
- Changed
* Set FDP_EXCPTN_ONLY bit (CPUID.7h.0:EBX[6]) and ZERO_FCS_FDS bit
(CPUID.7h.0:EBX[13]) in Intel's CPUID normalization process.
- Fixed
* Fixed feature flags in T2S CPU template on Intel Ice Lake.
* Fixed CPUID leaf 0xb to be exposed to guests running on AMD host.
* Fixed a performance regression in the jailer logic for closing open file
descriptors.
* A race condition that has been identified between the API thread and the VMM
thread due to a misconfiguration of the api_event_fd.
* Fixed CPUID leaf 0x1 to disable perfmon and debug feature on x86 host.
* Fixed passing through cache information from host in CPUID leaf 0x80000006.
* Fixed the T2S CPU template to set the RRSBA bit of the IA32_ARCH_CAPABILITIES
MSR to 1 in accordance with an Intel microcode update.
* Fixed the T2CL CPU template to pass through the RSBA and RRSBA bits of the
IA32_ARCH_CAPABILITIES MSR from the host in accordance with an Intel microcode
update.
* Fixed passing through cache information from host in CPUID leaf 0x80000005.
* Fixed the T2A CPU template to disable SVM (nested virtualization).
* Fixed the T2A CPU template to set EferLmsleUnsupported bit
(CPUID.80000008h:EBX[20]), which indicates that EFER[LMSLE] is not supported.
Fixes: #7610
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Since the passed fd through unix socket would be any
stream fd such as pipe/fifo fd or any other socket
fd, thus we should deal with it as a normal hybrid
stream instead of a unix stream.
Fixes:#7584
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR computes average results for TF bench.
Additionally, it improves the data parsing from
all running containers.
Fixes: #7603
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The build context should be the folder where the Dockerfile is present,
otherwise the files copied into the image won't be found.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Following the example on https://github.com/docker/build-push-action,
it's clear that the actions to "Set up QEMU" and "Set up Docker Buildx"
are missing.
Let's add them, and also take the advantage to bump the
build-push-action to its v4, which, by the way, had a typo on its name
(build-and-push-action does **NOT** exist, build-push-action does).
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now this is not being used, but it'll as the image generated for
the confidential tests have that as part of their tag.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
One of the joys to rely on the `pull_request_target` is to only be able
to catch those after those are merged.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With these 2 simple checks we can ensure that we do not regress on the
behaviour of allowing the runtime classes / default runtime class to be
created by the kata-deploy payload.
Fixes: #7491
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add here the image we'll be using for unencrypted confidential
tests. Later on, we'll make sure to build and use this image as part of
our CI.
The image can easily be built as a multi-arch image, and has `cpuid`
installed in case of `x86_64` build, so it can be used to detect whether
we're running on a TEE guest without having to rely on `dmesg | grep
...`.
Fixes: #7595
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't have to do any sed to replace the runtimeclass being used by
the moment we start taking advantage of the `DEFAULT_SHIM` environment
variable exposed merged in the previous commits.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There are many places where the code currently creates new `Vec`
instances when it's not really needed. The result is a perf hit because
it allocates memory, copies all elements, then frees the memory; in some
cases, copying elements also involves extra allocations (e.g., when
elements are strings, or structs containing strings).
This patch addresses a number of these cases.
Fixes: #7203
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Instead of using package manager to install bats, building
this from source. This gives us the updated version of bats
which supports functions such as setup_file and
teardown_file.
We can use these functions into our current tests.
Fixes: #7597
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
This PR changes the metrics workflow in order to just install
kata once, and run the checks for multiple hypervisor variations.
In this way we save time avoiding installing kata for each
hypervisor to be tested.
Fixes: #7578
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
Refine implementation of mount by:
- log message with `path.display()` instead of `{:?}`
- add prefix "_" to unused variables
- pass by reference instead of by value to avoid creating redundant
array
- exactly matching prefix "fsgid=" instead of "fsgid"
- avoid redundant clone() operations
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
There's a bug in function update_ephemeral_mounts() which only handles
the first storage object and ignores all other storage objects.
Fixes: #7551
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Simplify function online_cpu_memory() by on calling update_cpuset_path()
for containers with cpuset configured.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Refine style of code related to sandbox by:
- remove unnecessary comments for caller to take lock, we have already taken
`&mut self`.
- change "*count < 1 " to "*count == 0", `count` is type of u32.
- make remove_sandbox_storage() to take `&mut self` instead of `&self`.
- group related function to each others
- avoid search the map twice in function find_process()
- avoid unwrap() in function run_oom_event_monitor()
- avoid unwrap() in online_resources()
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Avoid unwrap() in function do_remove_container(), and also make
implmementation symmetric for both timeout and non-timeout cases.
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Optimize agent rpc implementation by:
- avoid clone objects when possible
- avoid unwrap() when possible
- explictly drop object to ensure order
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This pull request is mainly for updating vm-memory and vmm-sys-util.
The affacted crates include:
- vm-memory: from 0.9.0 to 0.10.0
- vmm-sys-util: from 0.10.0 to 0.11.0
- virtio-queue: from 0.6.0 to 0.7.0
- fuse-backend-rs: from 0.10.4 to 0.10.5
- linux-loader: from 0.6.0 to 0.8.0
- nydus-api: from 0.3.0 to 0.3.1
- nydus-rafs: from 0.3.1 to 0.3.2
- nydus-storage: from 0.6.3 to 0.6.4
Fixes: #0000
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR renames the mobilenet tensorflow test to have a more specific
tensorflow name mainly because tensorflow has different configurations
and we will add more tensorflow tests so we want to distinguish each
tensorflow test.
Fixes#7571
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the iperf network metrics to the github actions
for kata metrics.
Fixes#7535
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
These calls cause two extra atomic instructions each time they're used,
one to increment and another one to decrement the refcount.
Since we don't need them because the referred value is guaranteed to
outlive the function, remove the calls.
Fixes: #7190
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
When the mounted block device isn't a layer, we want to mount it into
containers, but since it's already mounted with the correct fs (e.g.,
tar, ext4, etc.) in the pod, we just bind-mount it into the container.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
When at least one `io.katacontainers.fs-opt.layer` option is added to
the rootfs, it gets inserted into the VM as a layer, and the file system
is mounted as an overlay of all layers using the overlayfs driver.
Additionally, if the `io.katacontainers.fs-opt.block_device=file` option
is present in a layer, it is mounted as a block device backed by a file
on the host.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This causes the overlay-fs driver to add the `upperdir` and `workdir`
options to an overlay-fs mount so that the mount becomes writable using
a discardable directory under the container id.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This is so that file systems don't fail when we pass kata-specific
options from the snapshotter to kata.
Fixes: #7536
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Version 0.10.5, which was just released, breaks `nydus-storage`.
This is a workaround to fix the CI which is blocking other PRs.
Fixes: #7541
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Allow `clippy::redundant_clone` in the agent's unit tests
because rustc>=1.70 shows the errors as false-negatives.
These `clone()` are required because the following codes
refer to the variable, but the clippy analyzes them by mistake,
using the conservative and limited approach.
Ref. https://rust-lang.github.io/rust-clippy/master/index.html#/redundant_cloneFixes: #7534
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
As 3.2.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kata containers as VM-based containers are allowed to run in the host
netns. That is, the network is able to isolate in the L2. The network
performance will benefit from this architecture, which eliminates as many
hops as possible. We called it a Directly Attachable Network (DAN for
short).
The network devices are placed at the host netns by the CNI plugins. The
configs are saved at {dan_conf}/{sandbox_id}.json in the format of JSON,
including device name, type, and network info. At the very beginning stage,
the DAN only supports host tap devices. More devices, like the DPDK, will
be supported in later versions.
The format of file looks like as below:
```json
{
"netns": "/path/to/netns",
"devices": [{
"name": "eth0",
"guest_mac": "xx:xx:xx:xx:xx",
"device": {
"type": "vhost-user",
"path": "/tmp/test",
"queue_num": 1,
"queue_size": 1
},
"network_info": {
"interface": {
"ip_addresses": ["192.168.0.1/24"],
"mtu": 1500,
"ntype": "tuntap",
"flags": 0
},
"routes": [{
"dest": "172.18.0.0/16",
"source": "172.18.0.1",
"gateway": "172.18.31.1",
"scope": 0,
"flags": 0
}],
"neighbors": [{
"ip_address": "192.168.0.3/16",
"device": "",
"state": 0,
"flags": 0,
"hardware_addr": "xx:xx:xx:xx:xx"
}]
}
}]
}
```
Fixes: #1922
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This commit provides a new way to name the containers used
in the launch-times-test in this form:
'kata_launch_times_RANDOM_NUMBER', where RANDOM_NUMBER is
in the 0-1000 range.
Fixes: #7529
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
- ci-on-push: Make the CI also run for the stable-* branches
- ci: k8s: Do not fail when gathering info on AKS nodes
- kata-deploy: enable cross build for non-x86
- runtime-rs: add support for gather metrics in runtime-rs
- kata-ctl: add monitor subcommand for runtime-rs
- release: release-note.sh: Fix typos and reference to images
- metrics: Add sysbench performance test
- Simplify implementation of runtime-rs/service
6ad16d497 release: Adapt kata-deploy for 3.2.0-rc0
025596b28 ci-on-push: Make the CI also run for the stable-* branches
7ffc0c122 static-build: enable cross build for qemu
35d6d86ab static-build: enable cross-build for image build
2205fb9d0 static-build: enable cross build for virtiofsd
11631c681 static-build: enable cross build for shim-v2
7923de899 static-build: cross build kernel
e2c31fce2 kata-deploy: enable cross build for kata deploy script
2fc5f0e2e kata-depoly: prepare env for cross build in lib.sh
f5e9985af release: release-note.sh: Fix typos and reference to images
f910c66d6 ci: k8s: Do not fail when gathering info on AKS nodes
632818176 metrics: Add k8s sysbench documentation
b3901c46d runtime-rs: ignore errors during clean up sandbox resources
5a1b5d367 metrics: Add sysbench pod yaml
ad413d164 metrics: Add sysbench dockerfile
151256011 metrics: Add sysbench performance test
62e328ca5 runtime-rs: refine implementation of TaskService
458e1bc71 runtime-rs: make send_message() as an method of ServiceManager
1cc1c81c9 runtime-rs: fix possibe bug in ServiceManager::run()
1a5f90dc3 runtime-rs: simplify implementation of service crate
731e7c763 kata-ctl: add monitor subcommand for runtime-rs The previous kata-monitor in golang could not communicate with runtime-rs to gather metrics due to different sandbox addresses. This PR adds the subcommand monitor in kata-ctl to gather metrics from runtime-rs and monitor itself.
d74639d8c kata-ctl: provide the global TIMEOUT for creating MgmtClient
02cc4fe9d runtime-rs: add support for gather metrics in runtime-rs
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-deploy files must be adapted to a new release. The cases where it
happens are when the release goes from -> to:
* main -> stable:
* kata-deploy-stable / kata-cleanup-stable: are removed
* stable -> stable:
* kata-deploy / kata-cleanup: bump the release to the new one.
There are no changes when doing an alpha release, as the files on the
"main" branch always point to the "latest" and "stable" tags.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
when interacting with systemd. We have occasionally faced issues with
compatibility between the systemctl version used inside the kata-deploy
container and the systemd version on the host. Instead of using a containerized
systemctl with bind mounted sockets, nsenter the host and run systemctl from
there. This provides less coupling between the kata-deploy container and the
host.
Fixes: #7511
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
As we only support one stable branch, it'll be used as part of the
stable-3.2 and onwards.
Fixes: #7518
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Depends on mutiarch feature of ubuntu, we can set up cross build
environment easily and achive as good build performance as native
build.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
It's too long a time to cross build agent based on docker buildx, thus
we cross build rootfs based on a container with cross compile toolchain
of gcc and rust with musl libc. Then we get fast build just like native
build.
rootfs initrd cross build is disabled as no cross compile tolchain for
rust with musl lib if found for alpine and based on docker buildx takes
too long a time.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Based on messense/rust-musl-cross which offer cross build musl lib
environment to cross compile virtiofsd.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
shim-v2 has go and rust code. For rust code, we use messense/rust-musl-cross
to build for speed up as it doesn't depends on qemu emulation. Build go
code based on docker buildx as it doesn't support cross build now.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
kata-deploy-binaries-in-docker.sh is the entry to build kata components.
set some environment to facilitate the following cross build work.
Fixes: #6557
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
We leverage three env, TARGET_ARCH means the buid target tuple;
ARCH nearly the same meaning with TARGET_ARCH but has been widely
used in kata; CROSS_BUILD means if you want to do cross compile.
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
diferent -> different
And also let's make sure we escape the backticks around the kata-deploy
environment variables, otherwise bash will try to interpret those.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise the VM deletion may not delete, leaving us with several
machines behind.
Fixes: #7509
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This requires the GITHUB_UPLOAD_TOKEN. While we're here, let's also fix
the name of the action and remove the "-tarball" suffix, as it's not
really a tarball.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
`stage` has been added, but only hooked up to the amd64 logic, leaving
arm64 and s390x behind.
Let's fix this right now, and make sure no error occurs when passing
this down to the yaml files.
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 7c857d38c1.
I've misunderstood the error given by github action, let's fix this in
the next commit.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll face the following error as part of our GHA:
```
The workflow is not valid.
kata-containers/kata-containers/.github/workflows/release-$foo.yaml
(Line: 13, Col: 14): Invalid input, stage is not defined in the
referenced workflow.
```
Fixes: #7497
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we can rely on the tests passing down whether they want
to be tested using Node Feataure Discovery or not.
Right now, only the TDX job has this option set to "true", all the other
jobs have this option set to "false".
We can and have to merge this one before merging the NFD related patches
as:
1) It causes no harm in exporting this environment variable, but not
having it used
2) It will allow us to test the NFD after this one is merged, as changes
in the yaml file, in the case of the pull_request_target event, are
not taken into consideration before they're merged
Fixes: #7495
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If modeVFIO is enabled we need 1st to attach the VFIO control group
device /dev/vfio/vfio an 2nd the actuall device(s) afterwards.Sort the
devices starting with device #1 being the VFIO control group device and
the next the actuall device(s)
/dev/vfio/<group>
Fixes: #7493
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
- tests: Add `k8s-volume` and `k8s-file-volume` tests to GHA CI
- metrics: Update boot time for kata metrics
- metrics: Add FIO report files for kata metrics
- kata-deploy: Allow runtimeclasses to be created by the daemonset
- runtime-rs: change block index to 0
- agent: fix typo in constant
- metrics: Add FIO benchmark for metrics tests
- gha: dragonball: Run only on the dragonball labeled machine
- tests: Fix `k8s-job` test
- agent,libs: Remove unused 'mut' keywords
- runtime-rs: remove unneeded 'mut' keywords
- tests: QoL improvements for running tests locally
- agent: exclude symlinks from recursive ownership change
- cache: kernel: Fix kernel caching
- runk: Add Docker guide to README
- metrics: General improvements to json.bash script
- kata-deploy: Allow shim creation based on what's passed to the daemonset
- gha: ci: Add skeleton of vfio job
- s390x: Fixing device.Bus assignment
- release: Mention the container images used to build the project
- kata-deploy-binaries: kernel_cache: Take module_dir into account
- ci: nydus: Fix typo in "source"
- gha: ci: Add no-op nydus tests to our CI
- Dragonball: migrate dragonball-sandbox crates to Kata
- ci: gha: Add cri-containerd tests (but still do not enable them)
- packaging/tools: Add kata-debug and use it as part of our CI
- cache: kernel: Consider changes in tools/packaging/kernel
- kata-deploy: Properly get the path of the versions.yaml file
- kata-deploy: Add VERSION and versions.yaml to the final tarball
- metrics: Add C-Ray performance test
- metrics: enable TensorFlow benchmark to be run on gha
- metrics: Add function to memory inside container script
- Revert "metrics: Replace backslashes used to escape double quoted key in jq expr"
- versions: Bump virtiofsd to v1.7.0
- metrics: stop hypervirsor and shim at init_env stage
- ci: k8s: Adapt "source ..." to the new location of gha-run.sh
- ci: Move `tests/integration/gha-run.sh` to `tests/integration/kuberentes/` ... and also remove KUBECONFIG from the tdx envs
- versions: Update kernel to version v6.1.x
- agent: Fix exec hang issues with a backgroud process
- agent: Ignore already mounted dev/fs/pseudo-fs
- ci: k8s: Bring TDX tests back
- metrics: Update machine learning documentation
- gha: ci: cri-containerd: Fix KATA_HYPERVSIOR typo
- tests: Add MobileNet Tensorflow performance benchmark
- metrics: replace backslashes used to escape double quoted jq key expr.
- runtime-rs: enhancement of Device Manager for network endpoints.
- feat(Tracing): tracing in Rust runtime
- runtime-rs: ignore unconfigured network interfaces
- metrics: Stop running kata-env before kata is properly installed.
- metrics: use rm -f to remove the oldest continerd config file.
- kernel: Update kernel config name
- kata-deploy: Add a debug option to kata-deploy (and also use it as part of our CI)
- runtime-rs: add parameter for propagation of (u)mount events
- kata-ctl: Move GuestProtection code to kata-sys-util
- tests: Add function before function name in common.bash for metrics
- tests: Add metrics storage documentation
- metrics: Fix metrics ts generator to treat numbers as decimals
- gha: ci: Add cri-containerd tests skeleton -- follow up 1
- dragonball/agent: Add some optimization for Makefile and bugfixes of unit tests on aarch64
- metrics: Enable blogbench test
- tests: Add machine learning performance tests
- tests: gha: ci: Add cri-containerd tests skeleton
- metrics: Enable memory inside container metrics
- tools: Use a consistent target name when building mariner initrd
- gha: ci: Gather info about the node / pods
- runtime-rs: Do not scan network if network model is "none"
- gha: k8s: tdx: Temporarily disable TDX tests
- metrics: Update memory usage script
- gha: Cancel previous jobs if a PR is updated
- gha: nightly: Fix long name of AKS clusters issue and make the CI easier to test
- README: Add badge for our Nightly CI
- gha: Do not run all the tests if only docs are updated
- bugfix: plus default_memory when calculating mem size
- gha: ci: Use github.sha to get the last commit reference
- dragonball: Don't fail if a request asks for more CPUs than allowed
- gha: ci: Fix refernce passed to checkout@v3
- gha: ci: Avoid using env also in the ci-nightly and payload-after-push
- gha: k8s: Ensure cluster doesn't exist before creating it
- gha: ci: More follow up fixes after adding a nightly CI
- tests: Enable running k8s tests on Mariner
- gha: ci: Avoid using env unless it's really needed
- gha: ci: Follow up fixes for the nightly jobs
- tests: Enable memory usage metrics tests
- gha: Add nightly jobs
- metrics: storing metrics workflow artifacts
- gha: k8s: Ensure tests are running on a specific namespace
- metrics: Adds blogbench and webtool metrics tests
- gha: dragonball: Correctly propagate PATH update
- versions: Upgrade to Cloud Hypervisor v33.0
- Convert `is_allowed`, `ttrpc_error` and `sl` to functions
- gha: release: Use a specific release of hub
- metrics: Add checkmetrics to gha-run.sh for metrics CI
- packaging: Fix indentation of build.sh script at ovmf
- doc: Add documentation for the virtualization reference architecture
- gpu: Update kernel building to the latest changes
- runtime: fix PCIe topology for GPUDirect use-case
- metrics: Add memory footprint tests
- runtime: Add "none" as a shared_fs option
- metrics: Uniformity across function names in gha-run.sh
- runtime-rs: support physical endpoint using device manager
- runtime-rs: bugfix for direct volume path's validation.
- metrics: Fix retrieving hypervisor version on metrics
- runtime-rs: fix build error on AArch64
- checkmetrics: Add checkmetrics makefile and documentation
- docs: Add boot time metrics documentation
- runtime-rs: add support spdk/vhost-user based volume.
- static-build: Remove kata-version parameter
- dragonball: avoid obtaining lock twice in create_stdio_console
- metrics: Add checkmetrics for kata metrics CI
- metrics: enable launch-times test on gha-run metrics script
- docs: Add general metrics documentation
- add support vfio device manager
- gha: Don't automatically trigger CI
- kata-ctl: Check for vm capability
- docs: fix spelling of "crate"
- packaging: Fix indentation in init.sh script
- gha: Fix gha actions
- metrics: install kata and launch-times test
- tests: Move tests helper script to this repo
- tests: Add json script for metrics tests
- Cherry pick initramfs caching updates from CCv0
- gha: Fix format for run launchtimes metrics yaml
- tests: Add tests lib common script
- Fix deprecated virtiofsd args (go shim only)
- gha: Add base branch on SHA on pull requst
- gha: ci-on-push: Run metrics tests
- docs: Update Developer Guide
- runtime-rs: Enhance flexibility of virtio-fs config
- versions: Update firecracker version to 1.3.3
- tools: Fix no-op builds
- runtime-rs: update Cargo.lock
- gha: Fix `stage` definition in matrix
- feat(runtime): vcpu resize capability
- packaging: Remove snap package
- gha: Add new build targets for Mariner
- Dragonball: support resize memory
- Port Measured rootfs feature from CCv0 branch to main
- add support direct volume and refactor device manager
- gha: Fix gha-run.sh and unbreak CI
- kata-ctl: Switch to slog logging; add --log-level and --json-logging arguments
- log-parser: Update log parser link at README
- gha: aks: Extract `run` commands to a script
- runtime-rs: handle copy files when share_fs is not available
- agent-ctl: fix the compile error
- agent: fix the issue of exec hang with a backgroud process
- runtime-rs: bugfix: update Cargo.lock
- gha: aks: Use short SHA in cluster name
- README: Display badge for the "Publish Artefacts" job and update the Kata Containers logo
- kata-deploy: Change how we get the Ubuntu k8s key
- gha: aks: Ensure host_os is used everywhere needed
- kubernetes: add agnhost command in pod yaml
- main | release: Standardize kata static file name
- packaging: make BUILDER_REGISTRY configurable
- gha: aks: Add the host_os as part of the aks cluster's name
- kernel: Modify build-kernel.sh to accomodate for changes in version.yaml
- gha: Fix Mariner cluster creation
- gha: Unbreak CI and fix cluster creation step
- Dragonball: support vcpu hotplug on aarch64
- runtime-rs/sandbox_bindmounts: add support for sandbox bindmounts
- runtime-rs/kata-ctl: Enhancement of DirectVolumeMount.
- gha: Create Mariner host as part of k8s tests
- netlink: Fix the issue of update_interface
- gha: Increase timeout for AKS jobs and give more time to start running the tests
- runtime: sending SIGKILL to qemu
- dragonball: convert BlockDeviceMgr and VirtioNetDeviceMgr functions to methods
- dragonball: Remove virtio-net and vsock devices gracefully
- kata-deploy: Improve shim backup / restore
- doc: Update git commands
- kata-deploy: Fix indentation on kata deploy merge script
8353aae41 ci: k8s: Rework get_nodes_and_pods_info()
6ad5d7112 ci: k8s: Do not gather node info before running the tests
5261e3a60 ci: k8s: Group messages to improve readability
9cc6b5f46 ci: k8s: Get logs from kata-deploy
9d285c622 ci: k8s: Let kata-deploy take care of the runtimeclasses
87568ed98 gha: Test split out runtimeclasses are in sync with all-in-one file
39192c608 kata-deploy: Print variables passed to the script
0e157be6f kata-deploy: Allow runtimeclasses to be created by the daemonset
a27433324 kata-deploy: Change default values of DEBUG
69535b808 kata-deploy: runtimeclass: Split out entries
9e1710674 kata-runtimeClasses: Alphabetically sort the enrties
6222bd910 tests: Add k8s-file-volume test
187a72d38 tests: Add k8s-volume test
0c8427035 metrics: Add boot time value for qemu
6520dfee3 metrics: Update boot time for kata metrics
ff2279061 metrics: Update runtime and configuration paths
a5d4e3388 metrics: Add compare virtiofsd dax script
5e937fa62 metrics: Update general FIO tests
b0bea47c5 metrics: Add makefile to report generator
73c57b9a1 metrics: Add FIO report files for kata metrics
c8fcd29d9 runtime-rs: use device manager to handle virtio-pmem
901c19225 runtime-rs: support configure vm_rootfs_driver
5d6199f9b runtime-rs: use device manager to handle vm rootfs
20f1f62a2 runtime-rs: change block index to 0
662f87539 metrics: Add general FIO makefile
c5a87eed2 tests: gha: Add timeout to cluster creation
6daeb08e6 tests: k8s: Clean up node debuggers after running
3aa6c77a0 gha: dragonball: Run only on the dragonball labeled machine
37641a543 metrics: Add example config for fio jobs
314aec73d agent: fix typo in constant
4703434b1 tests: k8s: Allow using custom resource group
350f3f70b tests: Import `common.bash` in `run_kubernetes_tests.sh`
d7f04a64a tests: k8s: Leave `runtimeclass_workloads/` alone
bdde6aa94 tests: k8s: Split deployment and testing commands
91a0b3b40 tests: aks: Simply delete cluster when cleaning up
3c1044d9d metrics: Update FIO paths for k8s runner
6177a0db3 metrics: Add env files for FIO
a45900324 metrics: Add fio exec
ea198fddc metrics: Add FIO runner k8s
8f7ef41c1 metrics: Add FIO vendor code
6293c17bd metrics: Add FIO benchmark for metrics tests
ff4cfcd8a runk: Add Docker guide to README
c8ac56569 cache: kernel: Harmonize commit with fetching side
81775ab1b cache: kernel: Fix SEV kernel caching
717f775f3 gha: ci: Add skeleton of vfio job
b9f100b39 agent,libs: Remove unused 'mut' keywords
a56f96bb2 kata-deploy: Allow shim creation based on what's passed to the daemonset
4a5ab38f1 metrics: General improvements to json.bash script
d4eba3698 kata-deploy-binaries: kernel_cache: Take module_dir into account
b7c9867d6 release: Mention the container images used to build the project
7c4b59781 ci: nydus: Fix typo in "source"
6a680e241 gha: ci: Add placeholder for the nydus tests as part of the CI
fb4f7a002 gha: nydus: Add a no-op GHA for nydus
4a207a16f gha: nydus: Bring tests as they are from the tests repo
2c8f83424 runtime-rs: remove unneeded 'mut' keywords
1fc715bc6 s390x: Add AP Attach/Detach test
e91f5edba ci: cri-containerd: Fix default typo for testContainerStart()
8b8aef09a ci: cri-containerd: Temporarily disable TestContainerSwap
56767001c ci: cri-containerd: Add namespace / uid to the pods
a84773652 ci: cri-containerd: Always use sudo to call crictl
99ba86a1b ci: cri-containerd: Add /usr/local/go/bin to the PATH
7f3b30999 ci: cri-containerd: Add `function` before each function
fde22d6bc ci: cri-containerd: Assume podman is always used
9465a0496 ci: cri-containerd: Adapt "source ..." to this repo
df8d14411 ci: cri-containerd: Remove CI variable
f90570aef ci: cri-containerd: Remove unused runc_runtime_bin
c3637039f ci: cri-containerd: Remove KILL_VMM_TEST env var
bc4919f9b ci: cri-containerd: Always run shim-v2 tests
f9e332c6d ci: cri-containerd: Stop cloning containerd
cfd662fee ci: cri-containerd: Remove ununsed SNAP_CI var
d36c3395c ci: cri-containerd: Update copyright
b5be8a4a8 ci: cri-containerd: Move integration-tests.sh as it was
f2e00c95c ci: cri-containerd: Populate install_dependencies()
897955252 versions: Add "latest" field for cri-tools
1bbcbafa6 ci: Add clone_cri_container()
f66c68a2b ci: Add install_cri_tools()
4dd828414 ci: Add install_cri_containerd()
ad47d1b9f ci: Add download_github_project_tarball()
788c562a9 ci: Add get_latest_patch_release_from_a_github_project()
6742f3a89 ci: Use `function` before each install_go.sh function
5eacecffc ci: Adjust paths for install_go.sh
8ed1595f9 ci: Update copyright for install_go.sh
6123d0db2 ci: Move install_go.sh as it was
8653be71b ci: Do not take cross-build into consideration for kata-arch.sh
6a76bf92c ci: Fix style / identation if kata-arch.sh
72743851c ci: Add `function` before each kata-arch.sh function
9f6d4892c ci: Update copyright for kata-arch.sh
6f73a7283 ci: Move kata-arch.sh as it was
3615d7343 ci: Add get_from_kata_deps()
34779491e gha: kubernetes: Avoid declaring repo_root_dir
f3738beac tests: Use $HOME/go as fallback for $GOPATH
b87ed2741 tests: Move `ensure_yq` to common.bash
124e39033 tests: common: Fix quoting when globbing
db77c9a43 tests: Make install_kata take care of the links
13715db1f tests: Do not call `install_check_metrics` when installing kata
630634c5d ci: k8s: Group logs to make them easier to read
228b30f31 ci: k8s: Gather node info during the cleanup
81f99543e ci: k8s: Cleanup cluster before deleting it
38a7b5325 packaging/tools: Add kata-debug
ae6e8d2b3 kata-deploy: Properly get the path of the versions.yaml file
309e23255 cache: kernel: Consider changes in tools/packaging/kernel
59fdd69b8 kata-deploy: Add VERSION and versions.yaml to the final tarball
5dddd7c5d release: Upload versions.yaml as part of the release
bad3ac84b metrics: Rename C-Ray to cpu performance tests
87d99a71e versions: Remove "kernel-experimental"
545de5042 vfio: Fix tests
62aa6750e vfio: Added better handling of VFIO Control Devices
dd422ccb6 vfio: Remove obsolete HotplugVFIOonRootBus
114542e2b s390x: Fixing device.Bus assignment
371a118ad agent: exclude symlinks from recursive ownership change
e64edf41e metrics: Add tensorflow function in gha-run script
67a6fff4f metrics: Enable tensorflow benchmark on gha
01450deb6 Revert "metrics: Replace backslashes used to escape double quoted key in jq expr."
843006805 metrics: Add function to memory inside container script
bbd3c1b6a Dragonball: migrate dragonball-sandbox crates to Kata
fad801d0f ci: k8s: Adapt "source ..." to the new location of gha-run.sh
55e2f0955 metrics: stop hypervirsor and shim at init_env stage
556e663fc metrics: Add disk link to general metrics README
98c121709 metrics: Add C-Ray README
8e7d9926e metrics: Add C-Ray Dockerfile
e2ee76978 metrics: Add C-Ray performance test
2ee2cd307 ci: k8s: Move gha-run.sh to the kubernetes dir
88eaff533 ci: tdx: Adjust KUBECONFIG
c09e268a1 versions: Downgrade SEV(-SNP) kernel back to v5.19.x
6a7a32365 versions: Bump virtiofsd to v1.7.0
ac5f5353b ci: k8s: Bring TDX tests back
950b89ffa versions: Update kernel to version v6.1.38
8ccc1e5c9 metrics: Update machine learning documentation
f50d2b066 gha: ci: cri-containerd: Fix KATA_HYPERVSIOR typo
620b94597 metrics: Add Tensorflow Mobilenet documentation
6c91af0a2 agent: Fix exec hang issues with a backgroud process
59f4731bb metrics: Stop running kata-env before kata is properly installed.
468f017e2 metrics: Replace backslashes used to escape double quoted key in jq expr.
64f013f3b ci: k8s: Enable debug when running the tests
8f4b1df9c kata-deploy: Give users the ability to run it on DEBUG mode
2c8dfde16 kernel: Update kernel config name
150e54d02 runtime-rs: ignore unconfigured network interfaces
3ae02f920 metrics: use rm -f to remove older continerd config file.
a864d0e34 tests: Add tensorflow mobilenet dockerfile
788d2a254 tests: Add tensorflow mobilenet performance test
3fed61e7a tests: Add storage link to general metrics documentation
b34dda4ca tests: Add storage blogbench metrics documentation
6787c6390 runtime-rs: add parameter for propagation of (u)mount events
6e5679bc4 tests: Add function before function name in common.bash for metrics
62080f83c kata-sys-util: Fix compilation errors
02d99caf6 static-checks: Make cargo clippy pass.
982420682 agent: Make the static checks pass for agent
61e4032b0 kata-ctl: Remove all utility functions to get platform protection
a24dbdc78 kata-sys-util: Move utilities to get platform protection
dacdf7c28 kata-ctl: Remove cpu related functions from kata-ctl
f5d195717 kata-sys-util: Move additional functionality to cpu.rs
304b9d914 kata-sys-util: Move CPU info functions
7319cff77 ci: cri-containerd: Add LTS / Active versions for containerd
2a957d41c ci: cri-containerd: Export GOPATH
75a294b74 ci: cri-containerd: Ensure deps are installed
6924d14df metrics: Fix metrics ts generator to treat numbers as decimals
9e048c8ee checkmetrics: Add blogbench read value for qemu
2935aeb7d checkmetrics: Add blogbench write value for qemu
02031e29a checkmetrics: Add blogbench read value for clh
107fae033 checkmetrics: Add blogbench write value for clh
8c75c2f4b metrics: Update blogbench Dockerfile
49723a9ec metrics: Add double quotes to variables
dc67d902e metrics: Enable blogbench test
438fe3b82 gha: ci: Add cri-containerd tests skeleton
bd08d745f tests: metrics: Move metrics specific function to metrics gha-run.sh
3ffd48bc1 tests: common: Move a few utility functions to common.bash
7f961461b tests: Add machine learning README
bb2ef4ca3 tests: Add `function` before each function
063f7aa7c tests: Add Pytorch Dockerfile
1af03b9b3 tests: Add Pytorch performance test
4cecd6237 tests: Add tensorflow Dockerfile
c4094f62c tests: Add metrics machine learning performance tests
89b622dcb gha: k8s: tdx: Temporarily disable TDX tests
8c9d08e87 gha: ci: Gather info about the node / pods
283f809dd runtime-rs: Enhancing Device Manager for network endpoints.
a65291ad7 agent: rustjail: update test_mknod_dev
46b81dd7d agent: clippy: fix cargo clippy warnings
c4771d9e8 agent: Makefile: enable set SECCOMP dynamically
a88212e2c utils.mk: update BUILD_TYPE argument
883b4db38 dragonball: fix cargo test on aarch64
6822029c8 runtime-rs: Do not scan network if network model is "none"
ce54e43eb metrics: Update memory usage script
fbc2a91ab gha: Cancel previous jobs if a PR is updated
307cfc8f7 tools: Use a consistent target name when building mariner initrd
d780cc08f gha: nightly: Also use `workflow_dispatch` to trigger it
b99ff3026 gha: nightly: Fix name size limit for AKS
aedc586e1 dragonball: Makefile: add coverage target
310e069f7 checkmetrics: Enable checkmetrics for memory inside test
1363fbbf1 README: Add badge for our Nightly CI
1776b18fa gha: Do not run all the tests if only docs are updated
28c29b248 bugfix: plus default_memory when calculating mem size
0c1cbd01d gha: ci: after-push: Use github.sha to get the last commit reference
37a955678 gha: ci: nightly: Use github.sha to get the last commit reference
ed23b47c7 tracing: Add tracing to runtime-rs
96e9374d4 dragonball: Don't fail if a request asks for more CPUs than allowed
38f0aaa51 Revert "gha: k8s: dragonball: Skip k8s-number-cpus"
828a72183 gha: k8s: dragonball: Skip k8s-oom
a79505b66 gha: k8s: dragonball: Skip k8s-number-cpus
275c84e7b Revert "agent: fix the issue of exec hang with a backgroud process"
2be342023 checkmetrics: Add memory usage inside container value for qemu
6ca34f949 checkmetrics: Add memory inside container value for clh
6c6892423 metrics: Enable memory inside container metrics
0ad298895 gha: ci: Fix refernce passed to checkout@v3
86904909a gha: ci: Avoid using env also in the ci-nightly and payload-after-push
f72cb2fc1 agent: Remove shadowed function, add slog-term
1d05b9cc7 gha: ci: Pass down secrets to ci-on-push / ci-nightly
c5b4164cb gha: ci: Fix tarball-suffix passed to the metrics tests
07810bf71 agent: Ignore already mounted dev/fs/pseudo-fs
11e3ccfa4 gha: ci: Avoid using env unless it's really needed
c45f646b9 gha: k8s: Ensure cluster doesn't exist before creating it
1a7bbcd39 gha: ci: Fix typo pull_requesst -> pull_request
ddf4afb96 gha: ci: Fix set-fake-pr-number job
8a0a66655 gha: ci: schedule expects a list, not a map
5c0269dc5 gha: ci: Add pr-number input to the correct job
de83cd9de gha: ci: Use $VAR instead of ${{ env.VAR }}
6acce83e1 metrics: Fix the call to check_metrics function
e067d1833 gha: Add a nightly CI job
7c0de8703 gha: k8s: Ensure tests are running on a specific namespace
106e30571 gha: Create a re-usable `ci.yaml` file
cc3993d86 gha: Pass event specific info from the caller workflow
4e396e728 metrics: Add function keyword to to helper metrics functions
1ca17c2f7 metrics: storing metrics workflow artifacts
5a61065ab checkmetrics: Add checkmetrics value for memory usage in qemu
78086ed1f checkmetrics: Add memory usage value for clh
1c3dbafbf metrics: Fix function of how to retrieve multiple values
18968f428 metrics: Add function to have uniformity
35d096b60 metrics: Adds blogbench and webtool metrics tests
d8f90e89d metrics: Rename function at memory usage script
b9d66e0d5 metrics: Fix double quotes variables in memory usage script
476a11194 tests: Enable memory usage metrics tests
b568c7f7d tests/integration: Provide default value for KATA_HOST_OS
d6e96ea06 tests/integration: Use AzureLinux instead of Mariner
40c46c75e tests/integration: Perform yq install in run_tests()
d8b8f7e94 metrics: Enable launch tests time metrics
72fd562bd gha: release: Use a specific release of hub
0502354b4 checkmetrics: Add checkmetrics json for qemu
b481ef188 makefile: Add -buildvcs=false flag to go build
e94aaed3c ci_worker: Add checkmetrics ci worker for cloud hypervisor
917576e6f metrics: Add double quotes in all variables
cc8f0a24e metrics: Add checkmetrics to gha-run.sh for metrics CI
477856c1e gha: dragonball: Correctly propagate PATH update
1c211cd73 gha: Swap asset/release in build matrix
0152c9aba tools: Introduce `USE_CACHE` environment variable
2b5975689 tests: Build CLH with glibc for Mariner
80c78eadc tests: Use baked-in kernel with Mariner
532755ce3 tests: Build Mariner rootfs initrd
6a21e20c6 runtime: Add "none" as a shared_fs option
5681caad5 versions: Upgrade to Cloud Hypervisor v33.0
b2ce8b4d6 metrics: Add memory footprint tests to the CI
d035955ef doc: Add documentation for the virtualization reference architecture
0f454d0c0 gpu: Fixing typos for PCIe topology changes
6bb2ea819 packaging: Fix indentation of build.sh script at ovmf
0504bd725 agent: convert the `sl` macros to functions
0860fbd41 agent: convert the `ttrpc_error` macro to a function
0e5d6ce6d agent: convert the `is_allowed` macro to a function
f680fc52b agent: change `AGENT_CONFIG`'s lazy type to just `AgentConfig`
beb706368 metrics: Uniformity across function names
1f3e837e4 runtime-rs: fix build error on AArch64
6fd25968c runtime-rs: bugfix for direct volume path's validation.
415578cf3 docs: Add general README
bff4672f7 runtime-rs: support physical endpoint using device manager
32cba7e44 metrics: Fix retrieving hypervisor version on metrics
aa7946de4 checkmetrics: Add general checkmetrics documentation
2fac2b72f checkmetrics: Add checkmetrics makefile
e45899ae0 docs: Add time tests documentation reference
28130d3ce docs: Add boot time metrics documentation
0df2fc270 runtime-rs: add support spdk/vhost-user based volume.
17198089e vendor: Add vendor checkmetrics dependencies
f1dfea6e8 docs: Add metrics documentation reference
8330fb8ee gpu: Update unit tests
859359424 metrics: enable launch-times test on gha-run metrics script
c4ee601bf metrics: Add checkmetrics for kata metrics CI
e0d6475b4 gha: Don't automatically trigger CI
b535c7cbd tests: Enable running k8s tests on Mariner
71071bdb6 docs: Add general metrics documentation
610f7986e check: Relax the unrestricted_guest check when running in a VM
1b406b9d0 kata-ctl:Implement functionality to check host is capable of running VM
adf88eaa8 static-build: Remove kata-version parameter
09720babc docs: fix spelling of "crate"
7185afc50 gha: Fix gha actions
21294b868 packaging: Fix indentation in init.sh script
fad3ac9f5 metrics: install kata and launch-times test
4bbfcfaf1 tests: Move tests helper script to this repo
f152f0e8c metrics: Add launch-times to metrics tests
59510cfee runtime-rs: add support vfio device based volume
1e3b372bb runtime-rs: add support vfio device manager
6b0848930 gha: Fix format for run launchtimes metrics yaml
3cefa43e7 tests: Add json script for metrics tests
6a3710055 initramfs: Build dependencies as part of the Dockerfile
aa2380fdd packaging: Add infra to push the initramfs builder image
1c7fcc6cb packaging: Use existing image to build the initramfs
a43ea24df virtiofsd: Convert legacy `-o` sub-options to their `--` replacement
8e00dc694 virtiofsd: Drop `-o no_posix_lock`
2a15ad978 virtiofsd: Stop using deprecated `-f` option
c3043a6c6 tests: Add tests lib common script
b16e0de73 gha: Add base branch on SHA on pull requst
72f2cb84e gpu: Reset cold or hot plug after overriding
fbacc0964 gpu: PCIe topology, consider vhost-user-block in Virt
bc152b114 gha: ci-on-push: Run metrics tests
dad731d5c docs: Update Developer Guide
b11246c3a gpu: Various fixes for virt machine type
40101ea7d vfio: Added annotation for hot(cold) plug
8f0d4e261 vfio: Cleanup of Cold and Hot Plug
b5c4677e0 vfio: Rearrange the bus assignemnt
b1aa8c8a2 gpu: Moved the PCIe configs to drivers
55a66eb7f gpu: Add config to TOML
da42801c3 gpu: Add config settings tests for hot-plug
de39fb7d3 runtime: Add support for GPUDirect and GPUDirect RDMA PCIe topology
9318e022a gpu: Add CC relates configs
b7932be4b gpu: Add Arm64 Kernel Settings
211b0ab26 gpu: Update Kernel Config
5f103003d gpu: Update kernel building to the latest changes
35e4938e8 tools: Fix no-op builds
347385b4e runtime-rs: Enhance flexibility of virtio-fs config
21d227853 versions: Update firecracker version to 1.3.3
0e2379909 gha: Fix `stage` definition in matrix
ae2cfa826 doc: add vcpu handlint doc for runtime-rs
7b1e67819 fix(clippy): fix clippy error
67972ec48 feat(runtime-rs): calculate initial size
aaa96c749 feat(runtime-rs): modify onlineCpuMemRequest
d66f7572d feat(runtime-rs): clear cpuset in runtime side
a0385e138 feat(runtime-rs): update linux resource when stop_process
a39e1e6cd feat(runtime-rs): merge the update_cgroups in update_linux_resources
fa6dff9f7 feat(runtime-rs): support vcpu resizing on runtime side
8cb4238b4 packaging: Remove snap package
213773998 runtime-rs: update Cargo.lock
56d2ea9b7 kata-ctl: Refactor kernel module check
9f7a45996 gha: Add `rootfs-initrd-mariner` build target
f28a62164 gha: Add `cloud-hypervisor-glibc` build target
8fb7ab751 dragonball: introduce virtio-balloon device
7ed949497 dragonball: introduce virtio-mem device
776a15e09 runtime-rs: add support direct volume.
a8e0f51c5 dragonball: extend DeviceOpContext
abae11404 runtime-rs: refactor device manager implementation
210a15794 dragonball: avoid obtaining lock twice in create_stdio_console
69668ce87 tests: gha-run: Use correct env variable for repo
f487199ed gha: aks: Fix argument in call to gha-run.sh
f6afae9c7 packaging: Add rootfs-image-tdx-tarball target
f62b2670c config: Add root hash value and measure config to kernel params
008058807 kernel: Integrate initramfs into Guest kernel
28b264562 initramfs: Add build script to generate initramfs
5cb02a806 image-build: generate root hash as an separate partition for rootfs
31c0ad207 packaging: Add cryptsetup support in Guest kernel and rootfs
980d084f4 log-parser: Update log parser link at README
410bc1814 agent-ctl: fix the compile error
77519fd12 kata-ctl: Switch to slog logging; add --log-level, --json-logging args
aab603096 gha: aks: Extract `run` commands to a script
e4eb664d2 runtime-rs: update rust to 1.69.0
ed37715e0 runtime-rs: handle copy files when share_fs is not available
5f6fc3ed7 runtime-rs: bugfix: update Cargo.lock
1c6d22c80 gha: aks: Use short SHA in cluster name
3c1f6d36d readme: Update Kata Containers logo
388684113 readme: Add status badge for the "Publish Artefacts" job
26f752038 kata-deploy: Change how we get the Ubuntu k8s key
aebd3b47d gha: aks: Ensure host_os is used everywhere needed
0c8282c22 gha: aks: Add the host_os as part of the aks cluster's name
4b89a6bda release: Standardize kata static file name
9228815ad kernel: Modify build-kernel.sh to accomodate for changes in version.yaml
03027a739 gha: Fix Mariner cluster creation
43e73bdef packaging: make BUILDER_REGISTRY configurable
ffe3157a4 dragonball: add arm64 patches for upcall
560442e6e dragonball: add vcpu_boot_onlined vector
e31772cfe dragonball: add support resize_vcpu on aarch64
64c764c14 dragonball: update dbs-boot to v0.4.0
fd9b41464 dragonball: update comment for init_microvm
af16d3fca gha: Unbreak CI and fix cluster creation step
5ddc4f94c runtime-rs/kata-ctl: Enhancement of DirectVolumeMount.
25d2fb0fd agent: fix the issue of exec hang with a backgroud process
4af4ced1a gha: Create Mariner host as part of k8s tests
eee7aae71 runtime-rs/sandbox_bindmounts: add support for sandbox bindmounts
557b84081 gha: aks: Wait longer to start running the tests
c04c872c4 gha: aks: Increase the timeout time
428041624 kata-deploy: Improve shim backup / restore
14c3f1e9f kata-deploy: Fix indentation on kata deploy merge script
0e47cfc4c runtime: sending SIGKILL to qemu
6a0035e41 doc: Update git commands
433b5add4 kubernetes: add agnhost command in pod yaml
c477ac551 dragonball: Convert VirtioNetDeviceMgr function to method
4659facb7 dragonball: Convert BlockDeviceMgr function to method
ee6deef09 dragonball: Remove virtio-net and vsock devices gracefully
2bda92fac netlink: Fix the issue of update_interface
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The links for pods and kubelets no longer work so update to new links
with relevant info.
Fixes#7487
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Multiple instances of task service may get registered by
ServiceManager::run(), fix it by making operation symmetric.
Fixes: #7479
Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
This PR rounds the axelnet and resnet results in order to extract
properly the result.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR will avoid to have the strconv.atoi parsing error when we
are retrieving the results from the json.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR moves the checkmetrics to gha-run script to gathered
tensorflow information.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The previous kata-monitor in golang could not communicate with runtime-rs
to gather metrics due to different sandbox addresses.
This PR adds the subcommand monitor in kata-ctl to gather metrics from
runtime-rs and monitor itself.
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
Several functions in kata-ctl need to establish a connection with runtime-rs through MgmtClient.
This PR provides a global TIMEOUT to avoid multiple definitions.
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
1. Implemented metrics collection for runtime-rs shim and dragonball hypervisor.
2. Described the current supported metrics in runtime-rs.(docs/design/kata-metrics-in-runtime-rs.md)
Fixes: #5017
Signed-off-by: Yuan-Zhuo <yuanzhuo0118@outlook.com>
The amount of info we've added seemed unnecessary, and ends up making
our lives even harder when trying to find errors.
Let's just rely on the kata-debug container to collect the needed info
for us.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's been proven to not be useful, and ends up making things more
confusing due to the amount of logs printed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we can debug kata-deploy in case something goes wrong
during its execution.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is needed in order to not lose track of what's been created and
what's been added here and there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help folks to debug / understand what's been passed to the
kata-deploy.sh script.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's allow the daemonset to create the runtimeclasses, which will
decrease one manual step a user of kata-deploy should take, and also
help us in the Confidential Containers land as the Operator can just
delegate it to this script.
Fixes: #7409
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This can be easily done as there was no official release with the
previous values.
The reason we're doing so is because when using `yq` to replace the
value, even when forcing `--tag '!!str' "yes"`, the content is placed
without quotes, causing errors in our CI.
While here, we're also removing the fallback value for DEBUG, as it is
**always** set in the kata-deploy.yaml file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will make things simpler to only create the handlers defined by the
kata-deploy user.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will become handy in the near future, as we want to have separate
enrties for each file, while still keeping this one.
Having the entries sorted will make our lives easier to test those are
always in sync.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This imports the k8s-file-volume test from the tests repo and modifies
it slightly to set up the host volume on the AKS host.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This imports the k8s-volume test from the tests repo and modifies it
slightly to set up the host volume on the AKS host.
Fixes: #6566
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This deletes node debugger pods after execution since their presence may
affect tests that assume only test workloads pods are present.
For example, in `k8s-job` we wait for *any* pod to be in the `Succeeded`
state before proceeding, which causes failures.
Fixes: #7452
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Static checks for dragonball are landing on any of the self-hosted
runners, and the reason for that is because "self-hosted" was the label
selector used.
Let's use "dragonball" instead, as the machine has that label as well.
Fixes: #7464
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This simply allows setting a custom resource group when debugging
locally, so as to prevent name collisions and not pollute the namespace.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Makes it so that `setup.sh` doesn't make changes in
`runtimeclass_workloads/` directly. Instead we treat that as a template
directory and we use the new directory `runtimeclass_workloads_work/` as
a work dir.
This has two advantages:
* Allows rerunning tests without the assumption that `setup.sh` must be
idempotent. E.g. the `set_runtime_class()` step would break.
* Doesn't pollute your git environment with a bunch of changes when
developing.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This splits deploying Kata and running the tests into separate commands
to make it possible to rerun tests locally without having to redeploy
Kata each time.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR adds the FIO benchmark scripts and resources for the metrics
tests section.
Fixes#7441
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
kata-deploy-binaries.sh uses the last commit in
tools/packaging/static-build/kernel for its version check, while the cache
generation uses tools/packaging/kernel. Use tools/packaging/static-build/kernel
as $kata_config_version is already part of the version string and covers any
changes to tools/packaging/kernel.
Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The SEV kernel cache calls create_cache_asset() twice, once for the kernel and
once for modules. Both calls need to use the same version string, otherwise the
second call overwrites the "latest" file of the first one and the cache is not
used.
Fixes: #7403
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This job will run on a nested virt capable Azure VM (improving test
concurrency). This is just a placeholder while we adapt the test to GHA.
Fixes: #6555
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Remove unused `mut` because the agent compilation fails
when the rust compiler is >= 1.71. This is related to #7425Fixes: #7438
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Instead of hardcoding shims as part of the script, let's ensure we can
allow them to be created based on environment variables passed to the
daemonset.
This change brings no functionality change as the default values in the
daemonset are exactly what has been used as part of the scripts.
Fixes: #7407
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds general improvements like putting function before function
name and consistency in how we declare variables and so on to have
uniformity across the metrics scripts.
Fixes#7429
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
`module_dir` has been passed to the function but was never assigned to a
var, leading to errors when trying to use it.
Fixes: #7416
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We should source from `nydus_dir`, instead of `cri_containerd_dir`, and
that was a leftover from fb4f7a002c.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will triger the nydus tests, but as they currently are they'll just
return "okay" without actually executing.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This newly added GHA does nothing, is not even triggered, and it's just
a placeholder that we'll grow in the next commits / PRs, so we can
actually start running the nydus tests as part of our CI.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we have propper AP device support add a
unit test for testing the correct Attach/Detach of AP devices.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Otherwise crictl will fail to remove them with:
```
getting sandbox status of pod "$pod": metadata.Name, metadata.Namespace
or metadata.Uid is not in metadata "..."
```
A huge shout out to Steven Horsman for helping to debug this one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For this set of tests, we'll always be using podman in order to avoid
having containerd pulled in by docker.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We don't need the env var, we just need to restrict the test according
to the KATA_HYPERVISOR used, as right now it's very specifict to QEMU.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We only have shim-v2 as the runtime type, so we always need to run tests
using it. :-)
We had to adjust the script in order to properly run the tests with the
current logic.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move the `integration/containerd/cri/integration-tests.sh` file
from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's install all the dependencies needed for running the
`cri-containerd` tests.
The list of dependencies we have are:
* From the system
- build-essential
- jq
- podman-docker
* From our own repo
- yq
- go
* From GitHub projects
- containerd
- cri-tools
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we don't want to disrupt what we have on the `tests` repo, let's
create a "latest" entry and use that for the GitHub actions tests.
Once we deprecate the `tests` repo we can decide whether we want to
stick to using "latest" or switch back to "version".
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will simply clone containerd repo, specifically on a tag
we want to use to test.
This can be expanded for different projects, and it will be the case as
soon as we grow the tests. But, for now, let's keep it simple.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will install cri-tools in the host, and soon enough (as
part of this PR) we'll be using it to install cri-tools as part of the
cri-containerd tests.
I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-tools to be installed
as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will install cri-containerd in the host, and soon enough
(as part of this PR) we'll be using it to install cri-containerd as part
of the cri-containerd tests.
I've decided to have this as part of the `common.bash` as other tests
that will be added in the future will require cri-containerd to be
installed as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will hel us to get the tarball, from a github project,
that we're going to use as part of our tests.
Right now this is not used anywhere, but it'll soon enough (as part of
this series) be used to download the cri-containerd / cri-tools / cni
tarballs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will help us to get the latest patch release from a
GitHub project.
The idea behind this function is that we don't have to keep updating
versions.yaml that frequently (or worse, have it outdated as it
currently is), and always test against the latest patch release of a
given project's version that we care about.
Although right now this is not used anywhere, this will be used with the
coming cri-containerd tests, which will be part of this series.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's adjust paths for what we source and the scripts we call, after
moving from the tests repo to this one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move `.ci/install_go.sh` file from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Right now we'd need to import lib.sh just in order to get cross-build
information for rust, and it seems a little bit premature to do so at
this stage and only for rust.
Let's skip it and keep this transition simple.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's move `.ci/kata-arch.sh` file from the tests repo to this one.
The file has been moved as it is, it's not used, and in the following
commits we'll clean it up before actually using it.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
First of all, I'm 100% aware that I'm duplicating this function here as
I've copied it from the packaging stuff, and I'm not exactly proud of
that.
However, right now it seems a little bit premature to combine that set
of scripts with this set of scripts in a single one and make them used
by both pieces of our project.
Anyways, this functions helps to get information from the
`versions.yaml` file, and it'll be used as part of the cri-containerd
tests and a few others in the future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is already declared as part of the `common.bash` file, so let's
just make sure we use it from there.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Considering that someone may want to run the tests locally, we shouldn't
rely on having GITHUB_WORKSPACE exported, and fallback to $HOME/go if
needed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When the glob star is inside quotes, there is only one iteration of the loop
and b holds all matches at once. Move the glob out of the quotes so that we
actually iterate over matched paths.
Fixes: #6543
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The `install_kata` function was moved from the metrics' `gha-run.sh`
file to the `common.bash` in the commit 3ffd48bc16, but I didn't notice
that it brought with it a call to `install_check_metrics`, which is
totally unrelated to installing Kata Containers.
Let's remove the call so the function is a little bit less specific, and
move the call to install_check_metrics to the metrics `gha-run.sh` file.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help us to in two fronts:
* catching possible issues related to kata-deploy cleanup
* do more (like, in the future, collect logs) after the tests run
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
kata-debug is a tool that is used as part of the Kata Containers CI to gather
information from the node, in order to help debugging issues with Kata
Containers.
As one can imagine, this can be expanded and used outside of the CI context,
and any contribution back to the script is very much welcome.
The resulting container is stored at the [Kata Containers quay.io
space](https://quay.io/repository/kata-containers/kata-debug) and can
be used as shown below:
```sh
kubectl debug $NODE_NAME -it --image=quay.io/kata-containers/kata-debug:latest
```
Fixes: #7397
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to correctly get the full path of the versions.yaml file as part
of the merge-builds.sh script, as we do a `pushd` there and that leads
to a fail merging the artefacts as the `versions.yaml` file does not
exists in that path.
Fixes: #7405
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Any change in the script used to build the kernel should invalidate the
cache.
Fixes: #7403
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make things simpler to figure out which version of Kata
Containers has been deployed, and also which artefacts come with it.
This will help us immensely in the future, for the TEEs use case, so we
can easily know whether we can deploy a specific guest kernel for a
specific host kernel.
Fixes: #7394
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although this file is far away from being a SBOM, it'll help folks to
easily visualise which components are part of a release, and even have
SBOMs generated from that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've not been using nor shipping this kernel for a very long time.
Regardless, we're leaving behind the logic in the kernel scripts to
build it, in case it becomes necessary in the future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removing HotplugVFIOonRootBus which is obsolete with the latest PCI
topology changes, users can set cold_plug_vfio or hot_plug_vfio either
in the configuration.toml or via annotations.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The device.Bus was reset if a specific combination of
configuration parameters were not met. With the new
PCIe topology this should not happen anymore
Fixes: #7381
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
currently when fsGroup is used with direct-assign, kata agent
recursively changes ownership and permission for each file including
symlinks. However the problem with symlinks is, the permission of
the symlink itself may not be same as the underlying file. So while
doing recursive ownership and permission changes we should skip
symlinks.
Fixes: #7364
Signed-off-by: Alakesh Haloi <a_haloi@apple.com>
This PR adds the tensorflow function in gha-run script in order to
be triggered in the gha.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds function before function of the variables at the memory
inside container script in order to have uniformity across the script.
Fixes#7386
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
In order to make it easier for developers to contribute to Dragonball,
we decide to migrate all dragonball-sandbox crates to Kata.
fixes: #7262
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This PR kills the hypervisor and the kata shim in the
init_env stage prior to launch any metric test.
Additionally this PR adds info messages in the main blocks
of the blogbench test to help in debugging.
Fixes: #7366
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds C-Ray performance test in order to be part of the kata
metrics CI.
Fixes#7375
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We don't need to export KUBECONFIG there. Let's just make sure we have
the server correctly setup and avoid doing that.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
CC-GPU seems to have issues with v6.1, so downgrade the kernels used for
SEV-SNP to a known-working version. It is worth mentioning that TDX is also
still on 5.19.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Now that we have a new TDX machine plugged into our CI, let's re-enable
the TDX tests.
Fixes: #7368
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Kernel v6.1.38 is the current latest LTS version, switch to it. No
patches should be necessary. Some CONFIG options have been removed:
- CONFIG_MEMCG_SWAP is covered by CONFIG_SWAP and CONFIG_MEMCG
- CONFIG_ARCH_RANDOM is unconditionally compiled in
- CONFIG_ARM64_CRYPTO is covered by CONFIG_CRYPTO and ARCH=arm64
Fixes: #6086
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This PR updates the machine learning documentation related with
Tensorflow and Pytorch benchmarks.
Fixes#7359
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the Tensorflow mobilinet documentation for the machine
learning README.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Allow runk to launch a container even though users don't specify the
pid namespace in `config.json` because general container runtimes
such as runc also can launch a container without the namespace.
On the other hand, Kata Containers doesn't allow it due to security issue
so this feature should be enabled in only runk.
Fixes: #7168
Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
Issue #4747 and pull request #4748 fix exec hang issues where the exec
command hangs when a process's stdout is not closed. However, the PR might
cause the exec command not to work as expected, leading to CI failure. The
PR was reverted in #7042. This PR resolves the exec hang issues and has
undergone 1000 rounds of testing to verify that it would not cause any CI
failures.
Fixes: #4747
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
This PR makes kata-env is called only after some metrics have
completed his workload. This fixes a bug that occurs when
kata-env was being called before kata is already installed on the
testing platform.
Fixes: #7348
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR uses squared brackets in a jq expression to access
key values corresponding to metric results in json format.
The values are the data inputs into the checkmetrics tool.
Fixes: #7319
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This will help us to gather more information about Kata Containers in
case of failure.
Fixes: #7343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The DEBUG env var introduced to the kata-deploy / kata-cleanup yaml file
will be responsible for:
* Setting up the CRI Engine to run with the debug log level set to debug
* The default is usually info
* Setting up Kata Containers to enable:
* debug logs
* debug console
* agent logs
This will help a lot folks trying to debug Kata Containers while using
kata-deploy, and also help us to always run with DEBUG=yes as part of
our CI.
Fixes: #7342
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fixes: #7294
When installing the kernel config adjust the name like
the vmlinuz and vmlinux files so that any added suffixes
are also reflected in the kernel config name.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In order to run kata metrics we need to check that the containerd
config file is properly set. When this is not the case, we
need to remove that file, and generate a valid one.
This PR runs rm -f in order to ignore errors in case the
file to delete does not exist.
Fixes: #7336
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the storage metrics documentation for blogbench for kata
metrics.
Fixes#7329
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Add an extra parameter in `bind_mount_unchecked` to specify
the propagation type: "shared" or "slave".
Fixes: #7017
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
This PR adds function before the function name in common.bash script
in order to have uniformity across all the script.
Fixes#7327
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Since these have been added to kata-sys-util, remove these from
kata-ctl. Change all invocations to get platform protection to make use
of kata-sys-util.
Fixes: #7144
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Remove cpu related functions which have been moved to kata-sys-util.
Change invocations in kata-ctl to make use of functions now moved to
kata-sys-util.
Signed-off-by: Nathan Whyte <nathanwhyte35@gmail.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Make certain imports architecture specific as these are not used on all
architectures.
Move additional constants and functionality to cpu.rs.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Move get_single_cpu_info and get_cpu_flags into kata-sys-util.
Add new functions that get a list of flags and check if a flag
exists in that list.
Fixes#6383
Signed-off-by: Nathan Whyte <nathanwhyte35@gmail.com>
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
As we'll be testing against the LTS and the Active versions of
containers, let's add those entries to the versions.yaml file and make
sure we export what we want to use for the tests as an env var.
The approach taken should not break the current way of getting the
containerd version.
LTS and Active versions of containerd can be found at:
https://containerd.io/releases/#support-horizonFixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure this is exported, as it'll be needed in order to install
`yq`, which will be used to get the versions of the dependencies to be
installed.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we install the needed dependencies for running the
`cri-containerd` tests.
Right now this commit is basically adding a placeholder, and later on,
when we'll actually be able to test the job, we'll add the logic of
installing the needed dependencies.
The obvious dependencies we've spotted so far are:
* From the OS
* jq
* curl (already present)
* From our repo
* yq (using the install_yq script)
* From GitHub
* cri-containerd
* cri-tools
* cni plugins
We may need a few more packages, but we will only figure this out as
part of the actual work.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use bc tool to perform math operations even when variables contain
values with leading zero.
Fixes: #7317
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds double quotes to variables in the blogbench script to
have uniformity across all the tests.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR builds the foundation for us to start migrating the
cri-containerd tests from Jenkins to GitHub Actions.
Right now the test does nothing and should always finish successfully.
The coming PRs will actually introduce logic to the `gha-run.sh` script
where we'll be able to run the tests and make sure those pass before
having them actually merged.
Fixes: #6543
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those functions were originally introduced as part of the
`metrics/gha-run.sh` file, but those will be very hand at the time we
start adding more tests.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDX tests need to be temporarily disabled as the current machine
allocated for this will be off for some time, and a new machine only
will be added next week.
Fixes: #7307
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently, network endpoints are separate from the device manager
and need to be included for proper management. In order to do so,
we need to refactor the implementation of the network endpoints.
The first step is to restructure the NetworkConfig and NetworkDevice
structures.
Next, we will implement the virtio-net driver and add the Network
device to the Device Manager.
Finally, we'll unify entries with do_handle_device for each endpoint.
Fixes: #7215
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When running cargo test in container, test_mknod_dev may fail sometimes
because of "Operation not permitted". Change the device path to
"/dev/fifo-test" to avoid this case.
Fixes: #7284
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
1. Update memory end assert because address space layout differs between
x86 and arm.
2. Set guest_addr for aarch64 in test_handler_insert_region case.
Fixes: #7284
TODO: #7290
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This PR updates memory usage script by applying the clean_env_ctr at the main
in order to avoid failures of leaving certain processes not removed.
Fixes#7302
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Currently a mixture of cbl-mariner and mariner is used when creating the
mariner initrd. The kata-static tarball has mariner in the name, but the
jenkins url uses cbl-mariner. This breaks cache usage.
Use mariner as the target name throughout the build, so that caching works.
Fixes: #7292
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This is a very nice suggestion from Steve Horsman, as with that we can
manually trigger the workflow anytime we need to test it, instead of
waiting for a full day for it to be retriggered via the `schedule`
event.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Passing the commit hash as the "pr-number" has shown problematic as it
would make the AKS cluster name longer than what's accepted by AKS.
One easy way to solve this is just passing "nightly" as the PR number,
as that's only used to create the cluster.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will help folks to monitor the history of the failing tests, as
we've done in Jenkins with the "Green Effort CI".
Fixes: #7279
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We should not go through the trouble of running all our tests on AKS /
Azure / baremetal machines in case a PR only changes our documentation.
Fixes: #7258
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've noticed this caused regressions with the k8s-oom tests, and then
decided to take a step back and do this in the same way it was done
before 67972ec48a.
Moreover, this step back is also more reasonable in terms of the
controlling logic.
And by doing this we can re-enable the k8s-oom.bats tests, which is done
as part of this PR.
Fixes: #7271
Depends-on: github.com/kata-containers/tests#5705
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
As we need to pass down the commit sha to the jobs that will be
triggered from the `push` event, we must be careful on what exactly
we're using there.
At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash. In order to actually get
the commit hash, Let's use ${{ github.sha }} instead.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we need to pass down the commit sha to the jobs that will be
triggered from the `schedule` event, we must be careful on what exactly
we're using there.
At first we were using ${{ github.ref }}, but this turns out to be the
**branch name**, rather than the commit hash. In order to actually get
the commit hash, Let's use ${{ github.sha }} instead, as described by
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's take the same approach of the go runtime, instead, and allocate
the maximum allowed number of vcpus instead.
Fixes: #7270
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 25d2fb0fde.
The reason we're reverting the commit is because it to check whether
it's the cause for the regression on devmapper tests.
Fixes: #7253
Depends-on: github.com/kata-containers/tests#5705
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On cc3993d860 we introduced a regression,
where we started passing inputs.commit-hash, instead of
github.event.pull_request.head.sha. However, we have been setting
commit-hash to github.event.pull_request.sha, meaning that we're mssing
a `.head.` there.
github.event.pull_request.sha is empty for the pull_request_target
event, leading the CI to pull the content from `main` instead of the
content from the PR.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The latter workflow is breaking as it doesn't recognise ${GITHUB_REF},
the former would most likely break as well, but it didn't get triggered
yet.
The error we're facing is:
```
Determining the checkout info
/usr/bin/git branch --list --remote origin/${GITHUB_REF}
/usr/bin/git tag --list ${GITHUB_REF}
Error: A branch or tag with the name '${GITHUB_REF}' could not be found
```
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Remove shadowed get_mounts(), added slog-term as a new crate,
slog can directly log to stdout and we can capture output
in the test-cases that are created in the function to be tested.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We have to do this, otherwise we cannot log into azure.
This is a regression introduced by
106e305717.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Instead of passing "-${{ inputs.tag }}-amd64", we must only pass
"-${{ inputs.tag }}".
This is a regression introduced by
106e305717.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Using an initrd and setting KATA_INIT=yes meaning we're using the kata-agent
as the init process we need to make sure that the agent is not segfaulting
if mounts are already happened. Some workloads need to configure several
things in the initrd before the kata-agent starts which involves having
/proc or /sys already mounted.
Fixes: #6992
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
de83cd9de7 tried to solve an issue, but it
clearly seems that I'm using env wrongly, as what ended up being passed
as input was "$VAR", instead of the content of the VAR variable.
As we can simply avoid using those here, let's do it and save us a
headache.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It has to have steps declared, and we need to make it a dependency for
the nightly kata-containers-ci-on-push job.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Otherwise we'll get the following error from the workflow:
```
The workflow is not valid. .github/workflows/ci-on-push.yaml (Line: 24,
Col: 20): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.COMMIT_HASH .github/workflows/ci-on-push.yaml (Line: 25,
Col: 18): Unrecognized named-value: 'env'. Located at position 1 within
expression: env.PR_NUMBER
```
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the call to check_metrics function as KATA_HYPERVISOR
is not needed to be passed.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The idea is to mimic what's been done with Jenkins and the "Green CI"
effort, but now using our GHA and the GHA infrastructure.
Fixes: #7247
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we run our tests in a specific namespace, as in case of
any kind of issue, we will just get rid of the namespace itself, which
will take care of cleaning up any leftover from failing tests.
One important thing to mention is why we can get rid of the `namespace:
${namespace}` on the tests that are already using it, and let's do it in
parts:
* namespace: default
We can easily get rid of this as that's the default namespace where
pods are created, so it was a no-op so far.
* namespace: test-quota-ns
My understanding is that we'd need this in order to get a clean
namespace where we'd be setting a quota for. Doing this in the
namespace that's only used for tests should **not** cause any
side-effect on the tests, as we're running those in serial and there's
no other pods running on the `kata-containers-k8s-tests` namespace
Last but not least, we're not dynamically creating namespaces as the
tests are not running in parallel, **never**, not in the case of having
2 tests being ran at same time, neither in the case of having 2 jobs
being scheduled to the same machine.
Fixes: #6864
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is based on the `ci-on-push.yaml` file, and it's called from ther
The reason to split on a new file is that we can easily introduce a
`ci-nightly.yaml` file and re-use the `ci.yaml` file there as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're not relying, on any of the called workflows, on event
specific information.
Right now, the two information we've been relying on are:
* PR number, coming from github.event.pull_request.number
* Commit hash, coming from github.event.pull_request.head.sha
As we want to, in the future, add nightly jobs, which will be triggered
by a different event (thus, having different fields populated), we
should ensure that those are not used unless it's in the "top action"
that's trigerred by the event.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Use the 'function' keyword to prevent bash aliases from colliding
with other function's name.
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR enables storing metrics workflow artifacts in two
separated flavours: clh and qemu.
Fixes: #7239
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR adds the function name before the function to have uniformity
across all the test.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds blogbench and webtooling metrics checks to this repo.
The function running the test intentionally returns zero, so
the test will be enabled in another PR once the workflow is
green.
Fixes: #7069
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR usses double quotes in all the variables as well as general fixes
to the memory usage script in order to have uniformity.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Non AKS k8s tests (SEV/SNP/TDX) don't currently set KATA_HOST_OS, so provide a
default empty value for the variable so that those tests can run.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
as OSSKU value, to get rid of this warning when creating the AKS cluster:
WARNING: The osSKU "AzureLinux" should be used going forward instead of
"CBLMariner" or "Mariner". The osSKUs "CBLMariner" and "Mariner" will
eventually be deprecated.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
We only need to install in run_tests() so that the yq install is picked up by
kubernets/setup.sh as well. We also need to either use (sudo &&
INSTALL_IN_GOPATH=false) || (INSTALL_IN_GOPATH=true).
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This PR adds the checkmetrics ci worker file for cloud hypervisor in
order to check the boot times limit.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds double quotes in all variables to have uniformity across
all the gha-run.sh script.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds checkmetrics installation for gha-run.sh in order to compare
results limits as part of the metrics CI.
Fixes#7198
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
cargo/rust is installed in one step, we need to write the PATH update to
GITHUBENV so that it becomes visible in the next steps.
Fixes: #7221
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This simply displays the asset name first in GH's UI, so that the
release name (always "test") is truncated rather than the asset name.
Makes things slightly easier to read.
e.g.
build-asset (cloud-hypervisor-glibc, te...
instead of
build-asset (test, cloud-hypervisor-gli...
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This allows setting `USE_CACHE=no` to test building e2e during
developmet without having to comment code blocks and so forth.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This enables building CLH with glibc and the mshv feature as required
for Mariner. At test time, it also configures Kata to use that CLH
flavor when running Mariner.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Mariner ships a bleeding-edge kernel that might be ahead of upstream, so
we use that to guarantee compatibility with the host.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
* Adds a new `rootfs-initrd-mariner` build target.
* Sets the custom initrd path via annotation in `setup.sh` at test
time.
* Adapts versions.yaml to specify a `cbl-mariner` initrd variant.
* Introduces env variable `HOST_OS` at deploy time to enable using a
custom initrd.
* Refactors the image builder so that its caller specifies the desired
guest OS.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
Currently, even when using devmapper, if the VMM supports virtio-fs /
virtio-9p, that's used to share a few files between the host and the
guest.
This *needed*, as we need to share with the guest contents like secrets,
certificates, and configurations, via Kubernetes objects like configMaps
or secrets, and those are rotated and must be updated into the guest
whenever the rotation happens.
However, there are still use-cases users can live with just copying
those files into the guest at the pod creation time, and for those
there's absolutely no need to have a shared filesystem process running
with no extra obvious benefit, consuming memory and even increasing the
attack surface used by Kata Containers.
For the case mentioned above, we should allow users, making it very
clear which limitations it'll bring, to run Kata Containers with
devmapper without actually having to use a shared file system, which is
already the approach taken when using Firecracker as the VMM.
Fixes: #7207
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR adds memory foot print metrics to tests/metrics/density
folder.
Intentionally, each test exits w/ zero in all test cases to ensure
that tests would be green when added, and will be enabled in a
subsequent PR.
A workflow matrix was added to define hypervisor variation on
each job, in order to run them sequentially.
The launch-times test was updated to make use of the matrix
environment variables.
Fixes: #7066
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
There is nothing in them that requires them to be macros. Converting
them to functions allows for better error messages.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
There is nothing in it that requires it to be a macro. Converting it to
a function allows for better error messages.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Having a function allows for better error messages from the type checker
and it makes it clearer to callers what can happen. For example:
is_allowed!(req);
Gives no indication that it may result in an early return, and no simple
way for callers to modify the behaviour. It also makes it look like
ownership of `req` is being transferred.
On the other hand,
is_allowed(&req)?;
Indicates that `req` is being borrowed (immutably) and may fail. The
question mark indicates that the caller wants an early return on
failure.
Fixes: #7201
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
Since it is never modified, it doesn't really need a lock of any kind.
Removing the `RwLock` wrapper allows us to remove all `.read().await`
calls when accessing it.
Additionally, `AGENT_CONFIG` already has a static lifetime, so there is
no need to wrap it in a ref-counted heap allocation.
Fixes: #5409
Signed-off-by: Wedson Almeida Filho <walmeida@microsoft.com>
This PR adds the word function before the function names in order to have
uniformity across the script as some are using this and some are not.
Fixes#7196
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Vfio support introduce build error on AArch64. Remove arch related
annotation can avoid this error.
Fixes: #7187
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
The failure mainly caused by the encoded volume path and
the mount/src. As the src will be validated with stat,but
it's not a full path and encoded, which causes the stat
mount source failed.
Fixes: #7186
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR adds link to the unreference docs in the cmd path to make
them more discoverable.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds checkmetrics makefile which is used to process the
metrics json results files.
Fixes#7172
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds time tests documentation reference in the general README
for kata metrics.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Unlike the previous usage which requires creating
/dev/xxx by mknod on the host, the new approach will
fully utilize the DirectVolume-related usage method,
and pass the spdk controller to vmm.
And a user guide about using the spdk volume when run
a kata-containers. it can be found in docs/how-to.
Fixes: #6526
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR adds the metrics documentation as a general reference in the
main README for kata containers.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the checkmetrics scripts that will be used for the kata metrics CI.
Fixes#7160
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We have GH configured so that manual approval is required for CI runs
triggered by outside contributors. However, because CI is triggered by
the `pull_request_target` event, this setting isn't being honored
(see [1]). This means that an attacker could trivially extracts secrets
by submitting a PR.
This change aims to mititgate this issue by preventing PRs from
triggering CI unless the `ok-to-test` label is set.
Note: For further context, we use the `pull_request_target` event and
manually check out the PR branch because it is the only way to both
access secrets and test incoming code changes.
Fixes: #7163
[1]: https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
When running on a VM, the kernel parameter "unrestricted_guest" for
kernel module "kvm_intel" is not required. So, return success when running
on a VM without checking value of this kernel parameter.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Implement functionality to add to the env output if the host is capable
of running a VM.
Fixes: #6727
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This PR removes an unrecognized value located in one of the yamls for the
gha in order to make it work the CI again.
Fixes#7149
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR replaces single spaces for tabs in order to fix the indentation
in the init.sh script.
Fixes#7147
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR installs kata static tarball on metrics runner
and run launch-times tests.
Fixes: #7049
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
The common.sh script includes helper functions used in
our metrics tests, so we are gradually adding more
metrics used in kata.
Fixes: #7108
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This test measures the duration of a workload that starts, and then
immediately stops the contianer. Also measures the workload period,
the time to quit period, and the time to kernel period.
Fixes: #7049
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
A new choice of using vfio devic based volume for kata-containers.
With the help of kata-ctl direct-volume, users are able to add a
specified device which is BDF or IOMMU group ID.
To help users to use it smoothly, A doc about howto added in
docs/how-to/how-to-run-kata-containers-with-kinds-of-Block-Volumes.
Fixes: #6525
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Limitations:
As no ready rust vmm's vfio manager is ready, it only supports
part of vfio in runtime-rs. And the left part is to call vmm
interfaces related to vfio add/remove.
So when vmm/vfio manager ready, a new PR will be pushed to
narrow the gap.
Fixes: #6525
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This PR fixes the format for the run launchtimes metrics yaml which
is causing to the workflow to fail.
Fixes#7130
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR adds the json script which allow us to save the metrics results
into a json file which will be used in the kata containers metrics.
Fixes#7128
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This will help to not have to build those on every CI run, and rather
take advantage of the cached image.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit c720869eef)
Let's add the needed infra for only building and pushing the initramfs
builder image to the Kata Containers' quay.io registry.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit 111ad87828)
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder for the initramds.
This will save us some CI time.
Fixes: #7084
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
(cherry picked from commit ebf6c83839)
The `-o` option is the legacy way to configure virtiofsd, inherited
from the C implementation. The rust implementation honours it for
compatibility but it logs deprecation warnings.
Let's use the replacement options in the go shim code. Also drop
references to `-o` from the configuration TOML file.
Fixes#7111
Signed-off-by: Greg Kurz <groug@kaod.org>
The C implementation of virtiofsd had some kind of limited support
for remote POSIX locks that was causing some workflows to fail with
kata. Commit 432f9bea6e hard coded `-o no_posix_lock` in order
to enforce guest local POSIX locks and avoid the issues.
We've switched to the rust implementation of virtiofsd since then,
but it emits a warning about `-o` being deprecated.
According to https://gitlab.com/virtio-fs/virtiofsd/-/issues/53 :
The C implementation of the daemon has limited support for
remote POSIX locks, restricted exclusively to non-blocking
operations. We tried to implement the same level of
functionality in #2, but we finally decided against it because,
in practice most applications will fail if non-blocking
operations aren't supported.
Implementing support for non-blocking isn't trivial and will
probably require extending the kernel interface before we can
even start working on the daemon side.
There is thus no justification to pass `-o no_posix_lock` anymore.
Signed-off-by: Greg Kurz <groug@kaod.org>
The rust implementation of virtiofsd always runs foreground and
spits a deprecation warning when `-f` is passed.
Signed-off-by: Greg Kurz <groug@kaod.org>
This PR adds the test lib common script that is going to be used
for kata containers metrics.
Fixes#7113
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The run-launchtimes-metrics workflow needs to get the commit ID
for the last commit to the head branch of the PR.
Fixes: #7116
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
If we override the cold, hot plug with an annotation
we need to reset the other plugging mechanism to NoPort
otherwise both will be enabled.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
In Virt the vhost-user-block is an PCIe device so
we need to make sure to consider it as well. We're keeping
track of vhost-user-block devices and deduce the correct
amount of PCIe root ports.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This gh-workflow prints a simple msg, but is the base for future
PRs that will gradually add the jobs corresponding to the kata
metrics test.
Fixes: #7100
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
This PR updates the developer guide at the connect to the debug console
section.
Fixes#7094
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Now it is possible to configure the PCIe topology via annotations
and addded a simple test, checking for Invalid and RootPort
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Removed the configuration of PCIeRootPort and PCIeSwitchPort, those
values can be deduced in createPCIeTopology
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Refactor the bus assignment so that the call to GetAllVFIODevicesFromIOMMUGroup
can be used by any module without affecting the topology.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The hypervisor_state file was the wrong location for the PCIe Port
settings, moved everything under device umbrella, where it can be
consumed more easily and we do not get into circular deps.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For the GPU CC use case we need to set several crypto algorithms.
The driver relies on them in the CC case.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Use now the sev.conf rather then the snp.conf.
Devices can be prestend in two different way in the
container (1) as vfio devices /dev/vfio/<num>
(2) the device is managed by whataever driver in
the VM kernel claims it.
Fixes: #6844
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This fixes the builds of `cloud-hypervisor-glibc` and
`rootfs-initrd-mariner` to properly create the `build/` directory.
Fixes: #7098
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR updates the firecracker version to 1.3.3 which includes the following
changes
Fixed passing through cache information from host in CPUID leaf 0x80000006.
A race condition that has been identified between the API thread and the VMM
thread due to a misconfiguration of the api_event_fd.
Fixes#7089
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Kubernetes and Containerd will help calculate the Sandbox Size and pass it to
Kata Containers through annotations.
In order to accommodate this favorable change and be compatible with the past,
we have implemented the handling of the number of vCPUs in runtime-rs. This is
This is slightly different from the original runtime-go design.
This doc introduce how we handle vCPU size in runtime-rs.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
In this commit, we refactored the logic of static resource management.
We defined the sandbox size calculated from PodSandbox's annotation and
SingleContainer's spec as initial size, which will always be the sandbox
size when booting the VM.
The configuration static_sandbox_resource_mgmt controls whether we will
modify the sandbox size in the following container operation.
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Some vmms, such as dragonball, will actively help us
perform online cpu operations when doing cpu hotplug.
Under the old onlineCpuMem interface, it is difficult
to adapt to this situation.
So we modify the semantics of nb_cpus in onlineCpuMemRequest.
In the original semantics, nb_cpus represents the number of
newly added CPUs that need to be online. The modified
semantics become that the number of online CPUs in the guest
needs to be guaranteed.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
The declaration of the cpu number in the cpuset is greater
than the actual number of vcpus, which will cause an error when
updating the cgroup in the guest.
This problem is difficult to solve, so we temporarily clean up
the cpuset in the container spec before passing in the agent.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Updating vCPU resources and memory resources of the sandbox and
updating cgroups on the host will always happening together, and
they are all updated based on the linux resources declarations of
all the containers.
So we merge update_cgroups into the update_linux_resources, so we
can better manage the resources allocated to one pod in the host.
Fixes: #5030
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Support vcpu resizing on runtime side:
1. Calculate vcpu numbers in resource_manager using all the containers'
linux_resources in the spec.
2. Call the hypervisor(vmm) to do the vcpu resize.
3. Call the agent to online vcpus.
Fixes: #5030
Signed-off-by: Ji-Xinyou <jerryji0414@outlook.com>
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Nobody has volunteered to maintain the (currently broken) snap build, so
remove it.
Fixes: #6769.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
After we support memory resize in Dragonball, we need to update
Cargo.lock in runtime-rs.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Adding vhost and vhost-net to the kernel modules. These do not require
any kernel module parameters to be checked. Currently, kernel params is
a required field. Make this as optional. Could make this as <Option>,
but making this a slice instead, as a module could have multiple kernel
params. Refactor the function that checks are for kernel modules into
two with one specifically checking if the module is loaded and other
checking for module parameters.
Refactor some of the tests to take into account these changes.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This adds the glibc flavor of CLH to the list of assets as preparation
for #6839. Mariner Kata is only tested with glibc.
Fixes: #7026
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
We introduce virtio-balloon device to support memory resize.
virtio-balloon device could reclaim memory from guest to host.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
We introduce virtio-mem device to support memory resize. virtio-mem
device could hot-plug more memory blocks to guest and could also
hot-unplug them from guest.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
As block/direct volume use similar steps of device adding,
so making full use of block volume code is a better way to
handle direct volume.
the only different point is that direct volume will use
DirectVolume and get_volume_mount_info to parse mountinfo.json
from the direct volume path. That's to say, direct volume needs
the help of `kata-ctl direct-volume ...`.
Details seen at Advanced Topics:
[How to run Kata Containers with kinds of Block Volumes]
docs/how-to/how-to-run-kata-containers-with-kinds-of-Block-Volumes.md
Fixes: #5656
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
In order to support virtio-mem and virtio-balloon devices, we need to
extend DeviceOpContext with VmConfigInfo and InstanceInfo.
Fixes: #6719
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
The key aspects of the DM implementation refactoring as below:
1. reduce duplicated code
Many scenarios have similar steps when adding devices. so to reduce
duplicated code, we should create a common method abstracted and use
it in various scenarios.
do_handle_device:
(1) new_device with DeviceConfig and return device_id;
(2) try_add_device with device_id and do really add device;
(3) return device info of device's info;
2. return full info of Device Trait get_device_info
replace the original type DeviceConfig with full info DeviceType.
3. refactor find_device method.
Fixes: #5656
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
After we have a guest kernel with builtin initramfs which
provide the rootfs measurement capability and Kata rootfs
image with hash device, we need set related root hash value
and measure config to the kernel params in kata configuration file.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Integrate initramfs into guest kernel as one binary,
which will be measured by the firmware together.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
The init.sh in initramfs will parse the verity scheme,
roothash, root device and setup the root device accordingly.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Generate rootfs hash data during creating the kata rootfs,
current kata image only have one partition, we add another
partition as hash device to save hash data of rootfs data blocks.
Fixes: #6674
Signed-off-by: Wang, Arron <arron.wang@intel.com>
Add required kernel config for dm-crypt/dm-integrity/dm-verity
and related crypto config.
Add userspace command line tools for disk encryption support
and ext4 file system utilities.
Fixes: #6674
Signed-off-by: Arron Wang <arron.wang@intel.com>
This PR updates the link to the correspondent Developer Guide at the
enabling full containerd debug that we have for kata 2.0 documentation.
Fixes#7034
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When the version of libc is upgraded to 0.2.145, older getrandom could not adapt
to new API, and this will make agent-ctl fail to compile.
We upgrade the version of `rand`, so the low version of getrandom will no longer
need.
Fixes: #7032
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Fixes: #5401, #6654
- Switch kata-ctl from eprintln!()/println!() to structured logging via
the logging library which uses slog.
- Adds a new create_term_logger() library call which enables printing
log messages to the terminal via a less verbose / more human readable
terminal format with colors.
- Adds --log-level argument to select the minimum log level of printed messages.
- Adds --json-logging argument to switch to logging in JSON format.
Co-authored-by: Byron Marohn <byron.marohn@intel.com>
Co-authored-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Jayant Singh <jayant.singh@intel.com>
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Kelby Madal-Hellmuth <kelby.madal-hellmuth@intel.com>
Signed-off-by: Liz Lawrens <liz.lawrens@intel.com>
Github Actions reads and runs workflow files from the main branch,
rather than from the PR branch. This means that PRs that modify workflow
files aren't being tested with the updated workflows coming from the PR,
but rather with the old workflows from the main branch. AFAIK, this
behavior isn't avoidable for workflow files (but is for other scripts).
This makes it very hard to reliably test workflow changes before they're
actually merged into main and leads to issues that we have to hotifx
(see #6983, #6995).
This PR aims to mitigate that by extracting the commands used in
workflows to a separate script file. The way our CI is set up, those
script files are read from the PR branch and thus changes would be
reflected in the CI checks.
Fixes: #6971
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
In hypervisors that do not support virtiofs we have to copy files in
the VM sandbox to properly setup the network (resolv.conf, hosts, and hostname).
To do that, we construct the volume as before, with the addition of an extra
variable that designates the path where the file will reside in the sandbox.
In this case, we issue a `copy_file` agent request *and* we patch the spec
to account for this change.
Fixes: #6978
Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Signed-off-by: George Pyrros <gpyrros@nubificus.co.uk>
When dragonball update dbs-boot crate in commit
64c764c147, the Cargo.lock in runtime-rs
should also be updated.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Full SHA is 40 characters, while AKS cluster name has a limit of 63. Trim the
SHA to 12 characters, which is widely considered to be unique enough and is
short enough to be used in the cluster name
Fixes: #7010
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Let's start adding the status of our jobs as part of our main page, so
folks monitoring those can easily check whether they're okay, or if
someone has to be pinged about those.
Fixes: #7008
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We added that to create the cluster name, but I forgot to add that to
the part we get the k8s config file, or to the part where we delete the
AKS cluster.
Fixes: #6999
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to do so, otherwise we'll create two clusters for testing Cloud
Hypervisor with exactly the same name, one using Ubuntu, and one using
Mariner.
Fixes: #6999
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The string representing the architecture aarch64 and x86_64 need to be changed to arm64 and amd64 for the release.
Fixes: #6986
Signed-off-by: SinghWang <wangxin_0611@126.com>
There were recent changes for the tdx kernel in the version.yaml that are
not currently accounted for in the build-kernel.sh script.
Attempts to setup a tdx kernel to build local changes seemed to not download
the tdx kernel. Instead the mainline kernel is downloaded which has no
tdx-related changes.
The version.yaml has a new entry for tdx kernel. Use that instead for
setting up and downloading the tdx kernel.
Fixes: #6984
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
While the Mariner Kata host is in preview, we need the `aks-preview`
extension to enable the `--workload-runtime KataMshvVmIsolation` flag.
Fixes: #6994
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This PR is to make an environment variable `BUILDER_REGISTRY` configurable
so that those who want to use their own registry for build can set up
the registry.
Fixes: #6988
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The vcpu hotplug/hotunplug feature is implemented with upcall. This commit
add three patches to support the feature on aarch64. Patches:
> 0005: add support of upcall on aarch64
> 0006: skip activate offline cpus' MSI interrupt
> 0007: set the correct boot cpu number
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This commit implements the vcpu_boot_onlined vector in get_fdt_vm_info.
"boot_enabled" means whether this vcpu should be onlined at first boot.
It will be used by fdt, which write an attribute called boot_enabled,
and will be handled by guest kernel to pass the correct cpu number to
function "bringup_nonboot_cpus".
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
This commit add support of resize_vcpu on aarch64. As kvm will check
whether vgic is initialized when calling KVM_CREATE_VCPU ioctl, all the
vcpu fds should be created before vm is booted.
To support resizing vcpu scenario, we use max_vcpu_count for
create_vcpus and setup_interrupt_controller interfaces. The
SetVmConfiguration API will ensure max_vcpu_count >= boot_vcpu_count.
Fixes: #6010
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
dbs-boot-v0.4.0 refectors the create_fdt interface. It simplifies the
parameters needed to be passed and abstracts them into three structs.
By the way, it also reserves some interfaces for future feature: numa
passthrough and cache passthrough.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Rewrite the comment of Vm::init_microvm method for aarch64.
Fixes cargo test warnings on aarch64.
Fixes: #6969
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
Move the get_volume_mount_info to kata-types/src/mount.rs.
If so, it becomes a common method of DirectVolumeMountInfo
and reduces duplicated code.
Fixes: #6701
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
When run a exec process in backgroud without tty, the
exec will hang and didn't terminated.
For example:
crictl -i <container id> sh -c 'nohup tail -f /dev/null &'
Fixes: #4747
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
The current testing setup only supports running Kata on top of an Ubuntu
host. This adds Mariner to the matrix of testable hosts for k8s
tests, with Cloud Hypervisor as a VMM.
As preparation for the upcoming PR that will change only the actual test
code (rather than workflow YAMLs), this also introduces a new file
`setup.sh` that will be used to set host-specific parameters at test
run-time.
Fixes: #6961
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
sandbox_bind_mounts supports kinds of mount patterns, for example:
(1) "/path/to", default readonly mode.
(2) "/path/to:ro", same as (1).
(3) "/path/to:rw", readwrite mode.
Both support configuration and annotation:
(1)[runtime]
sandbox_bind_mounts=["/path/to", "/path/to:rw", "/mnt/to:ro"]
(2) annotation will alse be supported, restricted as below:
io.katacontainers.config.runtime.sandbox_bind_mounts
= "/path/to /path/to:rw /mnt/to:ro"
Fixes: #6597
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
We're still facing issues related to the time taken to deploy the
kata-deplot daemonset and starting to run the tests.
Ideally, we should solve this with a readiness probe, and that's the
approach we want to take in the future. However, for now, let's just
make sure those tests are not on the way of the community.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've seen tests being aborted close to the end of the run due to the
timeout. Let's increase it, avoiding to hit such cases again..
Fixes: #6964
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're currently backing up and restoring all the possible shim files,
but the default one ("containerd-shim-kata-v2").
Let's ensure this is also backed up and restored.
Fixes: #6957
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR fixes the indentation on the kata deploy merge script
that instead of single spaces uses a tap.
Fixes#6925
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There is a race condition when virtiofsd is killed without finishing all
the clients. Because of that, when a pod is stopped, QEMU detects
virtiofsd is gone, which is legitimate.
Sending a SIGTERM first before killing could introduce some latency
during the shutdown.
Fixes#6757.
Signed-off-by: Beraldo Leal <bleal@redhat.com>
We previously were doing:
* Create a new image on kata-deploy-ci using the commit hash of the
latest tag
* This was used to test on AKS, which is no longer needed as we test
on AKS on every PR
* Create a new image on kata-deploy using the release tag and "latest"
or "stable", by tagging the kata-deploy-ci image accordingly
As part of cfe63527c5, we broke the
workflow described above, as in the first step we would save the PKG_SHA
to be used in the second step, but that part ended up being removed.
Anyways, this back and forth is not needed anymore and we can simplify
the process by doing:
* Create a new image on kata-deploy, using:
- The tag received as ref from the event that triggered this worklow
- "latest" or "stable" tag, depending on whether it's a stable release
or not
Fixes: #6946
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
For some bizarre reason, the login-action will simply fail to
authenticate to docker.io in it's specified as a registry. The way to
proceed, instead, is to *not* specify any registry as it'd be used by
default.
Fixes: #6943
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the nydus version to 2.2.1. This change includes:
nydus-image: fix a underflow issue in get_compressed_size()
backport fix/feature to stable 2.2
[backport] contrib: upgrade runc to v1.1.5
service: add README for nydus-service
nydus: fix a possible panic caused by SubCmdArgs::is_present
Backports two bugfixes from master into stable/v2.2
[backport stable/v2.2] action: upgrade golangci-lint to v1.51.2
[backport] action: fix smoke test for branch pattern
Fixes#6938
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
`docker/login-action@v3` does *not* exist and `docker/login-action@v2`
should be used instead.
Fixes: #6934
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- Fix cache for OVMF and rootfs-initrd (both x86_64)
- Upgrade to Cloud Hypervisor v32.0
- osbuilder: Bump fedora image version
- local-build: Standardise what's set for the local build scripts
- gha: aks: Wait a little bit more before run the tests
- docs: Update container network model url
- gha: release: Fix s390x worklow
- cache: Fix OVMF caching
- gha: payload-after-push: Pass secrets down
- tools: Fix arch bug
22154e0a3 cache: Fix OVMF tarball name for different flavours
b7341cd96 cache: Use "initrd" as `initrd_type` to build rootfs-initrd
b8ffcd1b9 osbuilder: Bump fedora image version
636539bf0 kata-deploy: Use apt-key.gpg from k8s.io
ae24dc73c local-build: Standardise what's set for the local build scripts
35c3d7b4b runtime: clh: Re-generate the client code
cfee99c57 versions: Upgrade to Cloud Hypervisor v32.0
ad324adf1 gha: aks: Wait a little bit more before run the tests
191b6dd9d gha: release: Fix s390x worklow
cfd8f4ff7 gha: payload-after-push: Pass secrets down
75330ab3f cache: Fix OVMF caching
a89b44aab tools: Fix arch bug
11a34a72e docs: Update container network model url
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
75330ab3f9 tried to fix OVMF caching, but
didn't consider that the "vanilla" OVMF tarball name is not
"kata-static-ovmf-x86_64.tar.xz", but rather "kata-static-ovmf.tar.xz".
The fact we missed that, led to the cache builds of OVMF failing, and
the need to build the component on every single PR.
Fixes: #6917 (hopefully for good this time).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been defaulting to "", which would lead to a mismatch with the
latest version from the cache, causing a miss, and finally having to
build the rootfs-initrd as part of the tests, every single time.
Fixes: #6917
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're facing some issues to download / use the public key provided by
google for installing kubernetes as part of the kata-deploy image.
```
The following signatures couldn't be verified because the public key is
not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists... Done
W: GPG error: https://packages.cloud.google.com/apt kubernetes-xenial
InRelease: The following signatures couldn't be verified because the
public key is not available: NO_PUBKEY B53DC80D13EDEF05 E: The
repository 'https://apt.kubernetes.io kubernetes-xenial InRelease' is
not signed.
N: Updating from such a repository can't be done securely, and is
therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user
configuration details.
```
Let's work this around following the suggestion made by @dims, at:
https://github.com/kubernetes/k8s.io/pull/4837#issuecomment-1446426585
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've a discrepancy on what's set along the scripts used to build the
Kata Cotainers artefacts locally.
Some of those were missing a way to easily debug them in case of a
failure happens, but one specific one (build-and-upload-payload.sh)
could actually silently fail.
All of those have been changed as part of this commut.
Fixes: #6908
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This patch re-generates the client code for Cloud Hypervisor v32.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #6632
Signed-off-by: Bo Chen <chen.bo@intel.com>
fa832f4709 increased the timeout, which
helped a lot, mainly in the TEE machines. However, we're still seeing
some failures here and there with the AKS tests.
Let's bump it yet again and, hopefully, those errors to start the tests
will go away.
Fixes: #6905
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
GitHub is warning us that:
"""
The workflow is not valid. In .github/workflows/release.yaml (Line: 21,
Col: 11): Error from called workflow
kata-containers/kata-containers/.github/workflows/release-s390x.yaml@d2e92c9ec993f56537044950a4673e50707369b5
(Line: 14, Col: 12): Job 'kata-deploy' depends on unknown job
'create-kata-tarball'.
"""
This is happening as we need to reference
"build-kata-static-tarball-s390x" instead of "create-kata-tarball".
Fixes: #6903
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The "build-assets-${arch}" jobs need to have access to the secrets in
order to log into the container registry in the cases where
"push-to-registry", which is used to push the builder containers to
quay.io, is set to "yes".
Now that "build-assets-${arch}" pass the secrets down, we need to log
into the container registry in the "build-kata-static-tarball-${arch}"
files, in case "push-to-registry" is set to "yes".
Fixes: #6899
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
OVMF has been cached, but it's not been used from cache as the `version`
set in the cached builds has always been empty.
The reason for that is because we've been trying to look for
`externals.ovmf.ovmf.version`, while we should be actually looking for
`externals.ovmf.x86_64.version`.
Setting `x86_64` as the OVMF_FLAVOUR would cause another bug, as the
expected tarball name would then be `kata-static-x86_64.tar.xz`, instead
of `kata-static-ovmf-x86_64.tar.xz`.
With everything said, let's simplify the OVMF_FLAVOUR usage, by using it
as it's passed, and only adapting the tarball name for the TDVF case,
which is the abnormal one.
Fixes: #6897
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- runtime: Use static_sandbox_resource_mgmt=true for TEEs
- update tokio dependency
- resource-control: fix setting CPU affinities on Linux
- runtime: use enable_vcpus_pinning from toml
- gha: k8s: Make the tests more reliable
- gha: Enable SEV-SNP tests on main
- gha: tdx: Use the k3s overlay for kata-cleanup
- runtime: Port sev package to main
- gpu: Rename the last bits from `gpu` to `nvidia-gpu`
- deploy: fix shell script error
- ppc64le: switch virtiofsd from C to rust version
- osbuilder: Fix indentation in rootfs.sh
- virtcontainers/qemu_test.go: Improve coverage
- agent: Add context to errors that may occur when AgentConfig file is …
- virtcontainers/pkg/compatoci/: Improved coverage for for Kata 2.0
- kata-manager: Fix '-o' syntax and logic error
- kata-ctl: Add the option to install kata-ctl to a user specified directory
- runtime-rs: fix building instructions to use correct required Rust ve…
- Dragonball: use LinuxBootConfigurator::write_bootparams
- kata-deploy: Add http_proxy as part of the docker build
- kata-deploy: Do not ship the kata tarball
- kata-deploy: Build improvements
- deploy: Fix arch in image tag
- Revert "kata-deploy: Use readinessProbe to ensure everything is ready"
- virtcontainers: Improved test coverage for fc.go from 4.6% to 18.5%
- main | release: Fix multi-arch publishing is not supported
- cache: More fixes to nvidia-gpu kernels caching
- runtime: remove overriding ARCH value by default for ppc64le
- gha: Fix Body Line Length action flagging empty body commit messages
- gha: Fix snap creation workflow
- cache: Fix nvidia-gpu version
- cache: Update the KERNEL_FLAVOUR list to include nvidia-gpu
- packaging: Add SEV-SNP artifacts to main
- docs: Mark snap installation method as unmaintained
- packaging: Add sev artifacts to main
- kata-ctl: add generic kvm check & unit test
- Log-parser-rs
- warning_fix: fix warnings when build with cargo-1.68.0
- cross-compile: Include documentation and configuration for cross-compile
- runtime: Fix virtiofs fd leak
- gpu: cold plug VFIO devices
- pkg/signals: Improved test coverage 60% to 100%
- virtcontainers/persist: Improved test coverage 65% to 87.5%
- virtcontainers/clh_test.go: improve unit test coverage
- virtcontainers/factory: Improved test coverage
- gha: Also run k8s tests on qemu-snp
- gha: sev: fix for kata-deploy error
- gha: Also run k8s tests on qemu-sev
- Implement the "kata-ctl env" command
- runtime-rs: support keep_abnormal in toml config
- gpu: Build and Ship an GPU enabled Kernel
- kata-ctl: checks for kvm, kvm_intel modules loaded
- osbuilder: Fix D-Bus enabling in the dracut case
- snap: fix docker start fail issue
- kata-manager: Fix containerd download
- agent: Fix ut issue caused by fd double closed
- Bump ttrpc to 0.7.2 and protobuf to 3.2.0
- gpu: Add GPU enabled confguration and runtime
- gpu: Do not pass-through PCI (Host) Bridges
- cache-components: Fix caching of TDVF and QEMU for TDX
- gha: tdx: Ensure kata-deploy is removed after the tests run
- versions: Upgrade to Cloud Hypervisor v31.0
- osbuilder: Enable dbus in the dracut case
- runtime: Don't create socket file in /run/kata
- nydus_rootfs/prefetch_files: add prefetch_files for RAFS
- runtime-rs/virtio-fs: add support extra handler for cache mode.
- runtime-rs: enable nerdctl to setup cni plugin
- tdx: Add artefacts from the latest TDX tools release into main
- runtime: support non-root for clh
- gha: ci-on-push: Run k8s tests with dragonball
- rustjail: Use CPUWeight with systemd and CgroupsV2
- gha: k8s-on-aks: {create,delete} AKS must be a coded-in step
- docs: update the rust version from version.yaml
- gha: k8s-on-aks: Set {create,delete}_aks as steps
- gha: k8s-on-aks: Fix cluster name
- gha: Also run k8s tests on AKS with dragonball
- gha: Only push images to registry after merging a PR
- gha: aks: Use D4s_v5 instance
- tools: Avoid building the kernel twice
- rustjail: Fix panic when cgroup manager fails
- runtime: add filter metrics with specific names
- gha: Use ghcr.io for the k8s CI
- GHA |Switch "kubernetes tests" from jenkins to GitHub actions
- docs: Update CNM url in networking document
- kata-ctl: add function to get platform protection.
f6e1b1152 agent: update tokio dependency
4cb83dc21 kata-ctl: update tokio dependency
df615ff25 runk: update tokio dependency
ca6892ddb runtime-rs: update tokio dependency
ca1531fe9 runtime: Use static_sandbox_resource_mgmt=true for TEEs
fa832f470 gha: k8s: Make the tests more reliable
cbb9fe8b8 config: Use standard OVMF with SEV
724437efb kata-deploy: add kata-qemu-sev runtimeclass
521dad2a4 Tests: skip CPU constraints test on SEV and SNP
72308ddb0 gha: ci-on-push: Don't skip tests for SEV
da0f92cef gha: ci-on-push: Don't skip tests for SEV-SNP
12f43bea0 gha: tdx: Use the k3s overlay for kata-cleanup
1a3f8fc1a deploy: fix shell script error
87cb98c01 osbuilder: Fix indentation in rootfs.sh
c5a59caca ppc64le: switch virtiofsd from C to rust version
bfdf0144a versions: Bump virtiofsd to 1.6.1
dd7562522 runtime: pkg/sev: Add kbs utility package for SEV pre-attestation
05de7b260 runtime: Add sev package
3a9d3c72a gpu: Rename the last bits from `gpu` to `nvidia-gpu`
4cde844f7 local-build: Fix kernel-nvidia-gpu target name
593840e07 kata-ctl: Allow INSTALL_PATH= to be specified
bdb75fb21 runtime: use enable_vcpus_pinning from toml
20cb87508 virtcontainers/qemu_test.go: Improve test coverage
b9a1db260 kata-deploy: Add http_proxy as part of the docker build
3e85bf5b1 resource-control: fix setting CPU affinities on Linux
5f3f844a1 runtime-rs: fix building instructions with respect to required Rust version
777c3dc8d kata-deploy: Do not ship the kata tarball
50cc9c582 tests: Improve coverage for virtcontainers/pkg/compatoci/ for Kata 2.0
136e2415d static-build: Download firecracker instead of building it
3bf767cfc static-build: Adjust ARCH for nydus
ac88d34e0 static-build: Use relased binary for CLH (aarch64)
73913c8eb kata-manager: Fix '-o' syntax and logic error
2856d3f23 deploy: Fix arch in image tag
e8f81ee93 Revert "kata-deploy: Use readinessProbe to ensure everything is ready"
cfe63527c release: Fix multi-arch publishing is not supported
197c33651 Dragonball: use LinuxBootConfigurator::write_bootparams to writes the boot parameters into guest memory.
4d17ea4a0 cache: Fix nvidia-snp caching version
a133fadbf cache: Fix nvidia-gpu-tdx-experimental cache URL
b9990c201 cache: Fix nvidia-gpu version
c9bf7808b cache: Update the KERNEL_FLAVOUR list to include nvidia-gpu
3665b4204 gpu: Rename `gpu` targets to `nvidia-gpu`
2c90cac75 local-build: fixup alphabetization
4da6eb588 kata-deploy: Add qemu-snp shim
14dd05375 kata-deploy: add kata-qemu-snp runtimeclass
0bb37bff7 config: Add SNP configuration
af7f2519b versions: update SEV kernel description
dbcc3b5cc local-build: fix default values for OVMF build
b8bbe6325 gha: build OVMF for tests and release
cf0ca265f local-build: Add x86_64 OVMF target
db095ddeb cache: add SNP flavor to comments
f4ee00576 gha: Build and ship QEMU for SNP
7a58a91fa docs: update SNP guide
879333bfc versions: update SNP QEMU version
38ce4a32a local-build: add support to build QEMU for SEV-SNP
5f8008b69 kata-ctl: add unit test for kvm check
a085a6d7b kata-ctl: add generic kvm check
772d4db26 gha: Build and ship SEV initrd
45fa36692 gha: Build and ship SEV OVMF
4770d3064 gha: Build and ship SEV kernel.
fb9c1fc36 runtime: Add qemu-sev config
813e4c576 runtimeClasses: add sev runtime class
af18806a8 static-build: Add caching support to sev ovmf
76ae7a3ab packaging: adding caching capability for kernel
12c5ef902 packaging: add support to build OVMF for SEV
b87820ee8 packaging: add support to build initrd for sev
e1f3b871c docs: Mark snap installation method as unmaintained
022a33de9 agent: Add context to errors when AgentConfig file is missing
b0e6a094b packaging: Add sev kernel build capability
a4c0303d8 virtcontainers: Fixed static checks for improved test coverage for fc.go
8495f830b cross-compile: Include documentation and configuration for cross-compile
13d7f39c7 gpu: Check for VFIO port assignments
6594a9329 tools: made log-parser-rs
03a8cd69c virtcontainers: Improved test coverage for fc.go from 4.6% to 18.5%
9e2b7ff17 gha: sev: fix for kata-deploy error
5c9246db1 gha: Also run k8s tests on qemu-snp
c57a44436 gha: Add the ability to test qemu-snp
406419289 env: Utilize arch specific functionality to get cpu details
fb40c71a2 env: Check for root privileges
1016bc17b config: Add api to fetch config from default config path
b908a780a kata-env: Pass cmd option for file path
b1920198b config: Workaround the way agent and hypervisor configs are fetched
f2b2621de kata-env: Implement the kata-env command.
c849bdb0a gha: Also run k8s tests on qemu-sev
6bf1fc605 virtcontainers/factory: Improved test coverage
0d49ceee0 gha: Fix snap creation workflow warnings
138ada049 gpu: Cold Plug VFIO toml setting
defb64334 runtime: remove overriding ARCH value by default for ppc64le
f7ad75cb1 gpu: Cold-plug extend the api.md
0fec2e698 gpu: Add cold-plug test
f2ebdd81c utils: Get rid of spurious print statement left behind.
9a94f1f14 make: Export VERSION and COMMIT
2f81f48da config: Add file under /opt as another location to look for the config
07f7d17db config: Make the pipe_size field optional
68f635773 config: Make function to get the default conf file public
7565b3356 kata-ctl: Implement Display trait for GuestProtection enum
94a00f934 utils: Make certain constants in utils.rs public
572b338b3 gitignore: Ignore .swp and .swo editor backup files
376884b8a cargo: Update version of clap to 4.1.13
17daeb9dd warning_fix: fix warnings when build with cargo-1.68.0
521519d74 gha: Add the ability to test qemu-sev
205909fbe runtime: Fix virtiofs fd leak
5226f15c8 gha: Fix Body Line Length action flagging empty body commit messages
0f45b0faa virtcontainers/clh_test.go: improve unit test coverage
dded731db gpu: Add OVMF setting for MMIO aperture
2a830177c gpu: Add fwcfg helper function
131f056a1 gpu: Extract VFIO Functions to drivers
c8cf7ed3b gpu: Add ColdPlug of VFIO devices with devManager
e2b5e7f73 gpu: Add Rawdevices to hypervisor
6107c32d7 gpu: Assign default value to cold-plug
377ebc2ad gpu: Add configuration option for cold-plug VFIO
c18ceae10 gpu: Add new struct PCIePort
9c38204f1 virtcontainers/persist: Improved test coverage 65% to 87.5%
1c1ee8057 pkg/signals: Improved test coverage 60% to 100%
cc8ea3232 runtime-rs: support keep_abnormal in toml config
96e8470db kata-manager: Fix containerd download
432d40744 kata-ctl: checks for kvm, kvm_intel modules loaded
b1730e4a6 gpu: Add new kernel build option to usage()
3e7b90226 osbuilder: Fix D-Bus enabling in the dracut case
53c749a9d agent: Fix ut issue caused by fd double closed
2e3f19af9 agent: fix clippy warnings caused by protobuf3
4849c56fa agent: Fix unit test issue cuased by protobuf upgrade
0a582f781 trace-forwarder: remove unused crate protobuf
73253850e kata-ctl: remove unused crate ttrpc
76d2e3054 agent-ctl: Bump ttrpc from 0.6.0 to 0.7.1
eb3d20dcc protocols: Add ut for Serde
59568c79d protocols: add support for Serde
a6b4d92c8 runtime-rs: Bump ttrpc from 0.6.0 to 0.7.1
ac7c63bc6 gpu: Add containerd shim for qemu-gpu
a0cc8a75f gpu: Add a kube runtime class
a81fff706 gpu: Adding a GPU enabled configuration
8af6fc77c agent: Bump ttrpc from 0.6.0 to 0.7.1
009b42dbf protocols: Fix unit test
392732e21 protocols: Bump ttrpc from 0.6.0 to 0.7.1
f4f958d53 gpu: Do not pass-through PCI (Host) Bridges
825e76948 gpu: Add GPU support to default kernel without any TEE
e4ee07f7d gpu: Add GPU TDX experimental kernel
a1272bcf1 gha: tdx: Fix typo overlay -> overlays
3fa0890e5 cache-components: Fix TDVF caching
80e3a2d40 cache-components: Fix TDX QEMU caching
87ea43cd4 gpu: Add configuration fragment
aca6ff728 gpu: Build and Ship an GPU enabled Kernel
dc662333d runtime: Increase the dial_timeout
eb1762e81 osbuilder: Enable dbus in the dracut case
f478b9115 clh: tdx: Update timeouts for confidential guest
3b76abb36 kata-deploy: Ensure node is ready after CRI Engine restart
5ec9ae0f0 kata-deploy: Use readinessProbe to ensure everything is ready
ea386700f kata-deploy: Update podOverhead for TDX
e31efc861 gha: tdx: Use the k3s overlay
542bb0f3f gha: tdx: Set KUBECONFIG env at the job level
d7fdf19e9 gha: tdx: Delete kata-deploy after the tests finish
da35241a9 tests: k8s: Skip k8s-cpu-ns when testing TDX
db2cac34d runtime: Don't create socket file in /run/kata
6d315719f snap: fix docker start fail issue
e4b3b0887 gpu: Add proper CONFIG_LOCALVERSION depending on TEE
69ba2098f runtime-rs: remove network entities and netns
b31f103d1 runtime-rs: enable nerdctl cni plugin
69d7a959c gha: ci-on-push: Run tests on TDX
5a0727ecb kata-deploy: Ship kata-qemu-tdx runtimeClass
98682805b config: Add configuration for QEMU TDX
3e1580019 govmm: Directly pass the firmware using -bios with TDX
3c5ffb0c8 govmm: Set "sept-ve-disable=on"
ed145365e runtime/qemu: Drop "kvm-type=tdx"
25b3cdd38 virtcontainers: Drop check for the `tdx` CPU flag
01bdacb4e virtcontainers: Also check /sys/firmwares/tdx for TDX
9feec533c cache: Add ability to cache OVMF
ce8d98251 gha: Build and ship the OVMF for TDX
39c3fab7b local-build: Add support to build OVMF for TDX
054174d3e versions: Bump OVMF for TDX
800fb49da packaging: Add get_ovmf_image_name() helper
fbf03d7ac cache: Document kernel-tdx-experimental
5d79e9696 cache: Add a space to ease the reading of the kernel flavours
6e4726e45 cache: Fix typos
fc22ed0a8 gha: Build and ship the Kernel for TDX
502844ced local-build: Add support to build Kernel for TDX
b2585eecf local-build: Avoid code duplication building the kernel
f33345c31 versions: Update Kernel TDX version
20ab2c242 versions: Move Kernel TDX to its own experimental entry
3d9ce3982 cache: Allow specifying the QEMU_FLAVOUR
33dc6c65a gha: Build and ship QEMU for TDX
eceaae30a local-build: Add support to build QEMU for TDX
f7b7c187e static-build: Improve qemu-experimental build script
3018c9ad5 versions: Update QEMU TDX version
800ee5cd8 versions: Move QEMU TDX to its own experimental entry
1315bb45f local-build: Add dragonball kernel to the `all` target
73e108136 local-build: Rename non vanilla kernel build functions
1d851b4be local-build: Cosmetic changes in build targets
49ce685eb gha: k8s-on-aks: Always delete the AKS cluster
e2a770df5 gha: ci-on-push: Run k8s tests with dragonball
d1f550bd1 docs: update the rust version from versions.yaml
f3595e48b nydus_rootfs/prefetch_files: add prefetch_files for RAFS
3bfaafbf4 fix: oci hook
c1fbaae8d rustjail: Use CPUWeight with systemd and CgroupsV2
375187e04 versions: Upgrade to Cloud Hypervisor v31.0
79f3047f0 gha: k8s-on-aks: {create,delete} AKS must be a coded-in step
2f35b4d4e gha: ci-on-push: Only run on `main` branch
e7bd2545e Revert "gha: ci-on-push: Depend on Commit Message Check"
0d96d4963 Revert "gha: ci-on-push: Adjust to using workflow_run"
c7ee45f7e Revert "gha: ci-on-push: Adapt chained jobs to workflow_run"
5d4d72064 Revert "gha: k8s-on-aks: Fix cluster name"
13d857a56 gha: k8s-on-aks: Set {create,delete}_aks as steps
dc6569dbb runtime-rs/virtio-fs: add support extra handler for cache mode.
85cc5bb53 gha: k8s-on-aks: Fix cluster name
1688e4f3f gha: aks: Use D4s_v5 instance
108d80a86 gha: Add the ability to also test Dragonball
2550d4462 gha: build-kata-static-tarball: Only push to registry after merge
e81b8b8ee local-build: build-and-upload-payload is not quay.io specific
13929fc61 gha: publish-kata-deploy-payload: Improve registry login
41026f003 gha: payload-after-push: Pass registry / repo as inputs
7855b4306 gha: ci-on-push: Adapt chained jobs to workflow_run
3a760a157 gha: ci-on-push: Adjust to using workflow_run
a159ffdba gha: ci-on-push: Depend on Commit Message Check
8086c75f6 gha: Also run k8s tests on AKS with dragonball
fe86c08a6 tools: Avoid building the kernel twice
3215860a4 gha: Set ci-on-push to run on `pull_request_target`
d17dfe4cd gha: Use ghcr.io for the k8s CI
b661e0cf3 rustjail: Add anyhow context for D-Bus connections
60c62c3b6 gha: Remove kata-deploy-test.yaml
43894e945 gha: Remove kata-deploy-push.yaml
cab9ca043 gha: Add a CI pipeline for Kata Containers
53b526b6b gha: k8s: Add snippet to run k8s tests on aks clusters
c444c24bc gha: aks: Add snippets to create / delete aks clusters
11e0099fb tests: Move k8s tests to this repo
73be4bd3f gha: Update actions for release.yaml
d38d7fbf1 gha: Remove code duplication from release.yaml
56331bd7b gha: Split payload-after-push-*.yaml
a552a1953 docs: Update CNM url in networking document
7796e6ccc rustjail: Fix minor grammatical error in function name
41fdda1d8 rustjail: Do not unwrap potential error with cgroup manager
a914283ce kata-ctl: add function to get platform protection.
0f7351556 runtime: add filter metrics with specific names
cbe6ad903 runtime: support non-root for clh
d3bb25418 utils: Add function to check vhost-vsock
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
If a hypervisor debug console is enabled and sandbox_cgroup_only is set,
the hypervisor can fail to open /dev/ptmx, which prevents the sandbox
from launching.
This is caused by the absence of a device cgroup entry to allow access
to /dev/ptmx. When sandbox_cgroup_only is not set, the hypervisor
inherits the default unrestrcited device cgroup, but with it enabled it
runs into allow / deny list restrictions.
Fix by adding an allowlist entry for /dev/ptmx when debug is enabled,
sandbox_cgroup_only is true, and no /dev/ptmx is already in the list of
devices.
Fixes: #6870
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
This PR updates the container network model url that is part of the
virtcontainers documentation.
Fixes#6889
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
When this option is enabled the runtime will attempt to determine the
appropriate sandbox size (memory, CPU) before booting the virtual
machine.
As TEEs do not support memory and CPU hotplug, this approach must be
used.
Fixes: #6818
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We like it or not, every now and then we'll have to deal with flaky
tests, and our tests using GHA are not exempt from that fact.
With this simple commit, we're trying to improve the reliability of the
tests in a few different fronts:
* Giving enough time for the script used by kata-deploy to be executed
* We've hit issues as the kata-deploy pod is considered "Ready" at the
moment it starts running, not when it finishes the needed setup. We
should also be looking on how to solve this on the kata-deploy side
but, for now, let's ensure our tests do not break with the current
kata-deploy behavior.
* Merging the "Deploy kata-deploy" and "Run tests" steps
* We've hit issues re-running tests and seeing even more failures than
the ones we're trying to debug, as a step will simply be taken as
succeeded as part of the re-run, in case it was successful executed
as part of the first run. This causes issues with the kata-deploy
deployment, as the tests would start running before even having the
node set up for running Kata Containers.
Fixes: #6865#6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The AmdSev firmware package should be used with
measured direct boot. If the expected hashes are not
injected into the firmware binary by the VMM, the
guest will not boot. This is required for security.
Currently the main branch does not have the extended
shim support for SEV, which tells the VMM to inject
the expected hashes.
We ship the standard OVMF package to use with SNP,
so let's switch SEV to that for now. This will need
to be changed back when shim support for SEV(-ES)
is added to main.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
In order to populate containerd config file with
support for SEV, we need to add the qemu-sev shim
to the kata-deploy script.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Currently Kata does not support memory / CPU hotplug for SEV or
SEV-SNP so we need to skip tests that rely on it.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that SEV artifacts are built by GHA, remove
conditional that skips tests when using qemu-sev.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that we have SNP artifacts in place and they are built via gha,
remove the condition that skips the tests for SNP.
Fixes: #6809
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
When updating an interface, there's maybe an existed
interface whose name would be the same with the updated
required name, thus it would update failed with interface
name existed error. Thus we should rename the existed interface
with an temporary name and swap it with the previouse interface
name last.
Fixes: #6842
Signed-off-by: fupan <fupan.lfp@antgroup.com>
As the TDX CI runs on k3s, we must ensure the cleanup, as already done
for the deploy, used the k3s overlay.
Fixes: #6857
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR replaces single spaces to tabs in order to fix the
indentation of the rootfs script.
Fixes#6848
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
We have been using the C version of virtiofsd on ppc64le. Now that the issue with
rust virtiofsd have been fixed, let's switch to it.
Fixes: #4259
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
virtiofsd v1.6.1 has been released with the fixes required for running
successfully on ppc64le.
Fixes: #4259
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Supports both online and offline modes of interaction with simple-kbs
for SEV/SEV-ES confidential guests.
Fixes: #6795
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
The sev package provides utilities for launching AMD SEV and SEV-ES
confidential guests.
Fixes: #6795
Signed-off-by: Dov Murik <dovmurik@linux.ibm.com>
Let's specifically name the `gpu` runtime class as `nvidia-gpu`. By
doing this we keep the door open and ease the life of the next vendor
adding GPU support for Kata Containers.
Fixes: #6553
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Update the kata-ctl install rule to allow it to be installed to a given directory
The Makefile was updated to use an INSTALL_PATH variable to track where the
kata-ctl binary should be installed. If the user doesn't specify anything,
then it uses the default path that cargo uses. Otherwise, it will install it
in the directory that the user specified. The README.md file was also updated
to show how to use the new option.
Fixes#5403
Co-authored-by: Cesar Tamayo <cesar.tamayo@intel.com>
Co-authored-by: Kevin Mora Jimenez <kevin.mora.jimenez@intel.com>
Co-authored-by: Narendra Patel <narendra.g.patel@intel.com>
Co-authored-by: Ray Karrenbauer <ray.karrenbauer@intel.com>
Co-authored-by: Srinath Duraisamy <srinath.duraisamy@intel.com>
Signed-off-by: Narendra Patel <narendra.g.patel@intel.com>
Rework TestQemuCreateVM routine to be a table driven test with
various config variations passed to it. After CreateVM a handful
of additional functions are exercised to improve code-coverage.
Also add partial coverage for StartVM routine.
Currently improving from 19.7% to 35.7%
Credit PR to Hackathon Team3
Fixes: #267
Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Add http_proxy and https_proxy as part of the docker build arguments
in order to build properly when we are behind a proxy.
Fixes#6834
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
With this fix the vCPU pinning feature chooses the correct
physical cores to pin the vCPU threads on rather than always using core 0.
Fixes#6831
Signed-off-by: Peteris Rudzusiks <rye@stripe.com>
There's absolutely no reason to ship the kata-static tarball as part of
the payload image, as:
* The tarball is already part of the release process
* The payload image already has uncompressed content of the tarball
* The tarball itself is not used anywhere by the kata-deploy scripts
Fixes: #6828
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no reason for us to build firecracker instead of simply
downloading the official released tarball, as tarballs are provided for
the architectures we want to use them.
Fixes: #6770
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When building from aarch64, just use "arm64" as that's what's used in
the name of the released nydus tarballs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's no need to build Cloud Hypervisor aarch64 as, for a few releases
already, Cloud Hypervisor provides an official release binary for the
architecture.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Fix the syntax and logic error that is only displayed if the user runs
the script with `-o`. This option requests that "only" Kata Containers
is installed and stops containerd from being installed.
Fixes: #6822.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
`uname -m` produces `x86_64`, but container image convention
is to use `amd64`, so update this in the tag
Fixes: #6820
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
This reverts commit 5ec9ae0f04, for two
main reasons:
* The readinessProbe was misintepreted by myself when working on the
original PR
* It's actually causing issues, as the pod ends up marked as not
healthy.
When release is published, kata-deploy payload and kata-static package
can support multi-arch publishing.
Fixes: #6449
Signed-off-by: SinghWang <wangxin_0611@126.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
All the kernel-foo instances, such as "kernel-sev" or "kernel-snp",
should be transformed into "kernel.foo" when looking at the
versions.yaml file.
This was already done for SEV, but missed on the SNP case.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We were passing "kernel-nvidia-gpu-tdx", missing the "-experimental"
part, leading to a non-valid URL.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
c9bf7808b6 introduced the logic to
properly get the version of nvidia-gpu kernels, but one important part
was dropped during the rebase into main, which is actually getting the
correct version of the kernel.
Fixing this now, and using the old issue as reference.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We need to make sure that, when caching a `-nvidia-gpu` kernel, we still
look at the version of the base kernel used to build the nvidia-gpu
drivers, as the ${vendor}-gpu kernels are based on already existing
entries in the versions.yaml file and do not require a new entry to be
added.
Fixes: #6777
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
A few pieces of the local-build tooling are supposed to be
alphabetized. Fixup a couple minor issues that have accumulated.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Now that we have the SNP components in place, make sure that
kata-deploy knows about the qemu-snp shim so that it will be
added to containerd config.
Fixes: #6575
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Since SEV-SNP has limited hotplug support, increase
the pod overhead to account for fixed resource usage.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
SNP requires many specific configurations, so let's make
a new SNP configuration file that we can use with the
kata-qemu-snp runtime class.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
SNP and SEV will share a (guest) kernel. Update the description
in versions.yaml to mention this.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
The x86_64 package of OVMF is required for deployments
that don't use kernel hashes, which includes SEV-SNP
in the short term. We should keep this in the bundle
in the long term in case someone wants to disable
kernel hashes.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Add targets to build the "plain" x86_64 OVMF.
This will be used by anyone who is using SEV or SNP
without kernel hashes. The SNP QEMU does not yet
support kernel hashes so the OvmfPkg will be used
by default.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Since we reshuffled versions.yaml, update the guide so that
we can find the SNP QEMU info.
Once runtime support is merged we should overhaul or remove
this guide, but let's keep it for now.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Refactor SNP QEMU entry in versions.yaml to match
qemu-experimental and qemu-tdx-experimental.
Also, update the version of QEMU to what we are using
in CCv0. This is the non-UPM QEMU and it does not
have kernel hashes support.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Add Make targets and helper functions to build the QEMU
needed for SEV-SNP.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@ibm.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Check that kvm test fails when run as non-root and when device specified
is not /dev/kvm.
Fixes#5338
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
Add kvm check using ioctl macro to create a syscall that checks the kvm
api version and if creation of a vm is successful.
Fixes#5338
Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
We have code that builds initrd for SEV.
thus, adding that to the test and release process.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
SEV requires custom kernel arguments when building.
Thus, adding it to the test and release process.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
Adding config file that can be used with qemu-sev runtime class.
Since SEV has limited hotplug support, increase
the pod overhead to account for fixed resource usage.
Fixes: #6572
Signed-off-by: Unmesh Deodhar <udeodhar@amd.com>
SEV requires special OVMF.
Now that we have ability to build this custom OVMF, let's optimize
it by caching so that we don't have to build it for every run.
Fixes: sev: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
The SEV initrd build requires kernel modules.
So, for SEV case, we need to cache kernel modules tarball in
addition to kernel tarball.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
SEV requires special OVMF to work with kernel hashes.
Thus, adding changes that builds this custom OVMF for SEV.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
We need special initrd for SEV. The work on SEV initrd is based on
Ubuntu. Thus, adding another entry in versions.yaml
This binary will have '-sev' suffix to distinguish it from the generic
binary.
Fixes: #6572
Signed-Off-By: Unmesh Deodhar <udeodhar@amd.com>
The snap package is no longer being maintained so update the docs to
warn readers.
We'll remove the snap installation docs in a few weeks.
See: #6769.
Fixes: #6793.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
When the agent config file is missing, the panic message says "no such file or
directory" but doesn't inform the user about which file was missing. Add
context to the parsing (with filename) and to the from_config_file() calls
(with information where the path is coming from).
Fixes: #6771
Depends-on: github.com/kata-containers/tests#5627
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Expanded tests on fc_test.go to cover more lines of code. Coverage went from 4.6% to 18.5%.
Fixed very simple static check fail on line 202.
Fixes: #266
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
`cross` is an open source tool that provides zero-setup cross compile
for rust binaries. Add documentation on this tool for compiling
kata-ctl tool and Cross.toml file that provides required configuration
for installing dependencies for various targets.
This is pretty useful for a developer to make sure code compiles and
passes checks for various architectures.
Fixes: #6765
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Eventual replacement of kata-log-parser, but for now replicates its
functionaility for the new runtime-rs syntax. Takes in log files,
parses, sorts by timestamp, spits them out in json, csv, xml, toml, and
a few others.
Fixes#5350
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Expanded tests on fc_test.go to cover more lines of code. Coverage went from 4.6% to 18.5%.
Fixes: #266
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
With the changes proposed as part of this PR, a qemu-snp cluster
will be created but no tests will be performed.
GitHub Actions will only run the tests using the workflows that are
part of the **target** branch, instead of the using the ones coming
from the PR. No way to work around this for now.
After this commit is merged, the tests (not the yaml files for the
actions) will be altered in order for the checkout action to help in
this case.
Fixes: #6722
Signed-off-by: Ryan Savino <ryan.savino@amd.com>
Have kata-env call architecture specific function to get cpu details
instead of generic function to get cpu details that works only for
certain architectures. The functionality for cpu details has been fully
implemented for x86_64 and arm architectures, but needs to be
implemented for s390 and powerpc.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add ability to write the environment information to a file
or stdout if file path is absent.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
This is essentially a workaround for the issue:
https://github.com/kata-containers/kata-containers/issues/5954
runtime-rs chnages the Kata config format adding agent_name and
hypervisor_name which are then used as keys to fetch the agent and
hypervisor configs. This will not work for older configs.
So use the first entry in the hashmaps to fetch the configs as a
workaround while the config change issue is resolved.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Expanded tests on factory_test.go to cover more lines of code. Coverage went from 34% to 41.5% in the case of user-mode run tests,
and from 77.7% to 84% in the case of priviledge-mode run tests.
Fixes: #260
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
Fix recurring issues of failing to install dependencies due to stale apt cache.
Uprev actions/checkout to v3 to resolve issue "Node.js 12 actions are deprecated."
Fixes: #5659
Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Currently, ARCH value is being set to powerpc64le by default.
powerpc64le is only right in context of rust and any operation
which might use this variable for a different purpose would fail on ppc64le.
Fixes: #6741
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
These will be consumed by kata-ctl, so export these so that
they can be used to replace variables available to the rust binary.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Most of kata installation tools use this path for installation, so
add this to the paths to look for the configuration.toml file.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add the serde default attribute to the field so that parsing
can continue if this field is not present.
The agent assumes a default value for this, so it is not required
by the user to provide a value here.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
With the changes proposed as part of this PR, a qemu-sev cluster will
be created but no tests will be performed.
GitHub Actions will only run the tests using the workflows that are
part of the **target** branch, instead of the using the ones coming
from the PR. No way to work around this for now.
After this commit is merged, the tests (not the yaml files for the
actions) will be altered in order for the checkout action to help in this
case.
Fixes: #6711
Signed-off-by: Ryan Savino <ryan.savino@amd.com>
The kata runtime invokes removeStaleVirtiofsShareMounts after
a container is stopped to clean up the stale virtiofs file caches.
Fixes: #6455
Signed-off-by: Feng Wang <fwang@confluent.io>
Change the Body Line Length workflow to not trigger when the commit
message contains only a message without a body. Other workflows will
flag the missing body sections, and it was confusing to have an error
message that said 'Body line too long (max 150)' when this was not
actually the case.
Fixes: #5561
Co-authored-by: Jayant Singh <jayant.singh@intel.com>
Co-authored-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Jayant Singh <jayant.singh@intel.com>
Signed-off-by: Luke Phillips <lucas.phillips@intel.com>
Signed-off-by: Kelby Madal-Hellmuth <kelby.madal-hellmuth@intel.com>
Signed-off-by: Liz Lawrens <liz.lawrens@intel.com>
Added driver util function for easier handling of VFIO
devices outside of the VFIO module. At the sandbox level
we may need to set options depending if we have a VFIO/PCIe
device, like the fwCfg for confiential guests.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Some functions may be used in other modules then only in
the VFIO module, extract them and make them available to
other layers like sandbox.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
If we have a VFIO device and cold-plug is enabled
we mark each device as ColdPlug=true and let the VFIO
module do the attaching.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
RawDevics are used to get PCIe device info early before the sandbox
is started to make better PCIe topology decisions
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
For the hypervisor to distinguish between PCIe components, adding
a new enum that can be used for hot-plug and cold-plug of PCIe devices
Fixes: #6687
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Expanded tests on signals_test.go to cover more lines of code. 'go test' won't show 100% coverage (only 66.7%), because one test need to spawn a new
process (since it is testing a function that calls os.Exit(1)).
Fixes: #256
Signed-off-by: Eduardo Berrocal <eduardo.berrocal@intel.com>
This patch adds keep_abnormal in runtime config. If keep_abnormal =
true, it means that 1) if the runtime exits abnormally, the cleanup
process will be skipped, and 2) the runtime will not exit even if the
health check fails.
This option is typically used to retain abnormal information for
debugging and should NOT be enabled by default.
Fixes: #6717
Signed-off-by: mengze <mengze@linux.alibaba.com>
Signed-off-by: quanweiZhou <quanweiZhou@linux.alibaba.com>
Newer containerd releases have an additional static package published.
Because of this, download_url contains two urls causing curl to fail.
To resolve this, pick the first url from the containerd releases to
download containerd.
Fixes: #6695
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Ensure that kvm and kvm_intel modules are loaded.
Renames the get_cpu_info() function to read_file_contents()
Fixes#5332
Signed-off-by: David Esparza <david.esparza.borquez@intel.com>
- D-Bus enabling now occurs only in setup_rootfs (instead of
prepare_overlay and setup_rootfs)
- Adjust permissions of / so dbus-broker will be able to traverse FS
These changes enables kata-agent to successfully communicate with D-Bus.
Fixes#6677
Signed-off-by: Vladimir <amigo.elite@gmail.com>
Never ever try to close the same fd double times, even in a unit test.
A file descriptor is a number which will be reused, so when you close
the same number twice you may close another file descriptor in the second
time and then there will be an error 'Bad file descriptor (os error 9)'
while the wrongly closed fd is being used.
Fixes: #6679
Signed-off-by: Tim Zhang <tim@hyper.sh>
Last but not least add the continerd shim configuration
pointing to the correct configuration-<shim>.toml
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
We need to set hotplug on pci root port and enable at least one
root port. Also set the guest-hooks-dir to the correct path
Fixes: #6675
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
On some systems a GPU is in a IOMMU group with a PCI Bridge and
PCI Host Bridge. Per default no PCI Bridge needs to be passed-through.
When scanning the IOMMU group, ignore devices with a 0x60 class ID prefix.
Fixes: #6663
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
With each release make sure we ship a GPU and TEE enabled kernel
This adds tdx-experimental kernel support
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
The beauty of GHA not allowing us to easily test changes in the yaml
files as part of the PR has hit us again. :-/
The correct path for the k3s deployment is
tools/packaging/kata-deploy/kata-deploy/overlays/k3s instead of
tools/packaging/kata-deploy/kata-deploy/overlay/k3s.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDVF caching is not working as the tarball name is incorrect. The result
expected is kata-static-tdvf.tar.xz, but it's looking for
kata-static-tdx.tar.xz.
This happens as a logic to convert tdx -> tdvf has been added as part of
the building scripts, but I missed doing this as part of the caching
scripts.
Fixes: #6669
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
TDX QEMU caching is not working as expected, as we're checking for its
version looking at "assets.hypervisor.${QEMU_FLAVOUR}.version", which is
correct for standard QEMU. However, for TDX QEMU we should be checking
for "assets.hypervisor.${QEMU_FLAVOUR}.tag"
Fixes: #6668
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When testing on AKS, we've been hitting the dial_timeout every now and
then. Let's increase it to 45 seconds (instead of 30) for all the VMMs,
and to 60 seconfs in case of TEEs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The agent now offloads cgroup configuration to systemd when
possible. This requires to enable D-Bus in order to communicate
with systemd.
Fixes#6657
Signed-off-by: Greg Kurz <groug@kaod.org>
Booting up TDX takes more time than booting up a normal VM. Those
values are being already used as part of the CCv0 branch, and we're just
bringing them to the `main` branch as well.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure the node is ready after the CRI Engine restart, otherwise
we may proceed and scripts may simply fail if they try to deploy a pod
while the CRI Engine is not yet restarted (and, consequently, the node
is not Ready).
Related: #6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
readinessProbe will help us to only have the kata-deploy pod marked as
Ready when it finishes all the needed configurations in the node.
Related: #6649
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As TEEs cannot hotplug memory / CPU, we *must* consider the default
values for those as part of the podOverhead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the TDX machine is using k3s, let's make sure we're deploying
kat-deploy using the k3s overlay.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We must ensure that no kata-deploy is left behind after the tests
finish, otherwise it may interfere with the next run.
Fixes: #6647
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The socket file for shim management is created in /run/kata
and it isn't deleted after the container is stopped. After
running and stopping thousands of containers /run folder
will run out of space.
Fixes#6622
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
Co-authored-by: Greg Kurz <groug@kaod.org>
If conf_guest is set we need to update the CONFIG_LOCALVERSION
to match the suffix created in install_kata
-nvidia-gpu-{snp|tdx}, the linux headers will be named the very
same if build with make deb-pkg for TDX or SNP.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
1. when we use nerdctl to setup network for kata, no netns is created by
nerdctl, kata need to create netns by its own
2. after start VM, nerdctl will call cni plugin via oci hook, we need to
rescan the netns after the interfaces have been created, and hotplug
the network device into the VM
Fixes:#4693
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
Now that we've added a TDX capable external runner, let's make sure we
also run the basic tests using TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we configure containerd for the kata-qemu-tdx handler
and ship the kata-qemu-tdx runtime class for kubernetes.
Fixes: #6537
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the QEMU configuration for TDX differs quite a lot from the normal
QEMU configuration, let's add a new configuration file for the QEMU TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since TDX doesn't support readonly memslot, TDVF cannot be mapped as
pflash device and it actually works as RAM. "-bios" option is chosen to
load TDVF.
OVMF is the opensource firmware that implements the TDVF support. Thus
the command line to specify and load TDVF is ``-bios OVMF.fd``
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we also check /sys/firmwares/tdx for TDX guest
protection, as the location may depend on whether TDX Seam is being used
or not.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the ability to cache OVMF, which right now we're only building
and shipping it for TDX.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's build the OVMF with TDX support as part of our tests, and let's
ship it as part of our releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
OVMF for TDX as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the OVMF for TDX version to what's the latest tested
release of the Intel TDX tools with Kata Containers.
This change requires a newer version of `nasm` than the one provided by
the container used to build the project. This change will also be
needed for SEV-SNP and was originally done by Alex Carter (thanks!).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
As we'll be using this from different places in the near future, let's
create a helper function as part of the libs.sh.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make users aware of the cache_components_main.sh that they can
also cache the kernel-tdx-experimental builds.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's build the kernel with TDX support as part of our tests, and let's
ship it as part of our releases.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
kernel-tdx-experimental as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's create a `install_kernel_helper()` function, as it was already
done for QEMU, and rely on that when calling `install_kernel` and
`install_kernel_dragonball_experimental`.
This helps us to reduce the code duplication by a fair amount.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the Kernel TDX version to what's the latest tested release
of the Intel TDX tools with Kata Containers.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although we've been providing users a way to build kernel with TDX
support, this must be moved to its own experimental entry instead of how
it currently is.
The reason for that is because the patches are not yet merged into
kernel, and this is still an experimental build of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's do what we already did when caching the kernel, and allow passing
a FLAVOUR of the project to build.
By doing this we can re-use the same function used to cache QEMU to also
cache any kind of experimental QEMU that we may happen to have.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed targets and modifications to be able to build
qemu-tdx-experimental as part of the local-build scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure the `qemu_suffix` and `qemu_tarball_name` can be
specified. With this we make it really easy to reuse this script for
any addition flavour of an experimental QEMU that ends up having to be
built (specifically looking at the ones for Confidential Containers
here).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's update the QEMU TDX version to what's the latest tested release of
the Intel TDX tools with Kata Containers.
In order to do such update, we had to relax the checks on the QEMU
version for some of the configuration options, as those were removed
right after the window was open for the 7.1.0 development (thus the
7.0.50 check).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Although we've been providing users a way to build QEMU with TDX
support, this must be moved to its own experimental entry instead of how
it currently is.
The reason for that is because the patches are not yet merged into QEMU,
and this is still an experimental build of the project.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the dragonball kernel is shipped as part of our releases, it must be
added to the `all` target.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In order to make it easier to read, let's just rename the
install_dragonball_experimental_kernel and install_experimental_kernel
to install_kernel_dragonball_experimental and
install_kernel_experimental, respectively.
This allows us to quickly get to those functions when looking for
`install_kernel`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that the infra for running dragonball tests has been enabled, let's
actually make sure to have them running on each PR.
The tests skipped are:
* `k8s-cpu-ns.bats`, as CPU resize doesn't seem to be yet properly
supported on runtime-rs
* https://github.com/kata-containers/kata-containers/issues/6621Fixes: #6605
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
A sandbox annotation used to specify prefetch_files.list
path the container image being used, and runtime will pass
it to Hypervisor to search for corresponding prefetch file:
format looks like:
"io.katacontainers.config.hypervisor.prefetch_files.list"
= /path/to/<uid>/xyz.com/fedora:36/prefetch_file.list
Fixes: #6582
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
1. when do the deserialization for the oci hook, we should use camel
case for createRuntime
2. we should pass the dir of bundle path instead of the path of
config.json
Fixes:#4693
Signed-off-by: Zhongtao Hu <zhongtaohu.tim@linux.alibaba.com>
I should have seen this coming, but currently the "create" and "delete"
AKS workflows cannot be imported and uses as a job's step, resulting on
an error trying to find the correspondent action.yaml file for those.
Fixes: #6630
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're only running this workflow when PRs are opened
against the main branch.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit a159ffdba7.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 3a760a157a.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 7855b43062.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 85cc5bb534.
Unfortunately we have to revert the PRs related to the switch done to
using `workflow_run` instead of `pull_request_target`. The reason for
that being that we can only mark jobs as required if they are targetting
PRs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We've been currently using {create,delete}_aks as jobs. However, it
means that if the tests fail we'll end up deleting the AKS cluster (as
expected), but not having a way to recreate the cluster without
re-running all jobs, which is a waste of resources.
Fixes: #6628
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add support for virtiofsd when virtio_fs_extra_args with
"-o cache auto, ..." users specified.
Fixes: #6615
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
This was missed from the last series, as GHA will use the "target
branch" yaml file to start the workflow.
Basically we changed the name of the cluster created to stop relying on
the PR number, as that's not easily accessible on `workflow_run`.
Fixes: #6611
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
It's been pointed out that D4s_v5 instances are more powerful than the
D4s_v3 ones, and have the very same price. With this in mind, let's
switch to the newer machines.
Fixes: #6606
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With the changes proposed as part of this PR, an AKS cluster will be
created but no tests will be performed.
The reason we have to do this is because GitHub Actions will only run
the tests using the workflows that are part of the **target** branch,
instead of the using the ones coming from the PR, and we didn't find yet
a way to work this around.
Once this commit is in, we'll actually change the tests themselves (not
the yaml files for the actions), as those will be the ones we want as
the checkout action helps us on this case.
Fixes: #6583
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
56331bd7bc oversaw the fact that we
mistakenly tried to push the build containers to the registry for a PR,
rather than doing so only when the code is merged.
As the workflow is now shared between different actions, let's introduce
an input variable to specify which are the cases we actually need to
perform a push to the registry.
Fixes: #6592
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's just print "to the registry" instead of printing "to quay.io", as
the registry used is not tied to quay.io.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We made registry / repo mandatory, but we only adapted that to the amd64
job. Let's fix it now and make sure this is also passed to the arm64
and s390x jobs.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As we're using the `workflow_run` event, the checkout action would
pull the **current target branch** instead of the PR one.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The way previously used to get the PR's commit sha can only be used with
`pull_request*` kind of events.
Let's adapt it to the `workflow_run` now that we're using it.
With this change we ended up dropping the PR number from the tarball
suffix, as that's not straightforward to get and, to be honest, not a
unique differentiator that would justify the effort.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make this workflow dependent of the commit message check, and only
start it if the commit message check one passes.
As a side effect, this allows us to run this specific workflow using
secrets, without having to rely on `pull_request_target`.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As already done for Cloud Hypervisor and QEMU, let's make sure we can
run the AKS tests using dragonball.
Fixes: #6583
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Two different kernel build targets (build,install) have both instructions to
build the kernel, hence it was executed twice. Install should only do
install and build should only do build.
Fixes: #6588
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
This is less secure than running the PR on `pull_request`, and will
require using an additional `ok-to-test` label to make sure someone
deliverately ran the actions coming from a forked repo.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's switch to using the `ghcr.io` registry for the k8s CI, as this
will save us some troubles on running the CI with PRs coming from forked
repos.
Fixes: #6587
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In cases where the D-Bus connection fails, add a little additional context about
the origin of the error.
Fixes: 6561
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Suggested-by: Archana Shinde <archana.m.shinde@intel.com>
Spell-checked-by: Greg Kurz <gkurz@redhat.com>
This workflow becomes redundant as we're already testing kubernetes
using kata-deploy, and also testing it on AKS.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is the very first step to replacing the Jenkins CI, and I've
decided to start with an x86_64 approach only (although easily
expansible for other arches as soon as they're ready to switch), and to
start running our kubernetes tests (now running on AKS).
Fixes: #6541
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This will be shortly used as part of a newly created GitHub action which
will replace our Jenkins CI.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Those will be shortly used as part of a newly added GitHub action for
testing k8s tests on Azure.
They've been created using the secrets we already have exposed as part
of our GitHub, and they follow a similar way to authenticate to Azure /
create an AKS cluster as done in the `/test-kata-deploy` action.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The first part of simplifying things to have all our tests using GitHub
actions is moving the k8s tests to this repo, as those will be the first
vict^W targets to be migrated to GitHub actions.
Those tests have been slightly adapted, mainly related to what they load
/ import, so they are more self-contained and do not require us bringing
a lot of scripts from the tests repo here.
A few scripts were also dropped along the way, as we no longer plan to
deploy kubernetes as part of every single run, but rather assume there
will always be k8s running whenever we land to run those tests.
It's important to mention that a few tests were not added here:
* k8s-block-volume:
* k8s-file-volume:
* k8s-volume:
* k8s-ro-volume:
These tests depend on some sort of volume being created on the
kubernetes node where the test will run, and this won't fly as the
tests will run from a GitHub runner, targetting a different machine
where kubernetes will be running.
* https://github.com/kata-containers/kata-containers/issues/6566
* k8s-hugepages: This test depends a whole lot on the host where it
lands and right now we cannot assume anything about that anymore, as
the tests will run from a GitHub runner, targetting a different
machine where kubernetes will be running.
* https://github.com/kata-containers/kata-containers/issues/6567
* k8s-expose-ip: This is simply hanging when running on AKS and has to
be debugged in order to figure out the root cause of that, and then
adapted to also work on AKS.
* https://github.com/kata-containers/kata-containers/issues/6578
Till those issues are solved, we'll keep running a jenkins job with
hose tests to avoid any possible regression.
Last but not least, I've decided to **not** keep the history when
bringing those tests here, otherwise we'd end up polluting a lot the
history of this repo, without any clear benefit on doing so.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We can easily re-use the newly added build-kata-static-tarball-*.yaml as
part of the release.yaml file.
By doing this we consolidate on how we build the components accross our
actions.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's split those actions into two different ones:
* Build the kata-static tarball
* Publish the kata-deploy payload
We're doing this as, later in this series we'll start taking advantage
of both pieces.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the url for the Container Network Model
in the network document.
Fixes#6563
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
There can be an error while connecting to the cgroups managager, for
example a `ENOENT` if a file is not found. Make sure that this is
reported through the proper channels instead of causing a `panic()`
that does not provide much information.
Fixes: #6561
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Reported-by: Greg Kurz <gkurz@redhat.com>
The kata monitor metrics API returns a huge size response,
if containers or sandboxs are a large number,
focus on what we need will be harder.
Fixes: #6500
Signed-off-by: Miao Xia <xia.miao1@zte.com.cn>
- nydus: upgrad to v2.2.0
- osbuilder: Add support for CBL-Mariner
- kata-deploy: Fix bash semantics error
- make only_kata work without -f
- runtime-rs: ch: Implement confidential guest handling
- qemu/arm64: disable image nvdimm once no firmware offered
- static checks workflow improvements
- A couple of kata-deploy fixes
- agent: Bring in VFIO-AP device handling again
- bugfix: set hostname in CreateSandboxRequest
- packaging / kata-deploy builds: Add the ability to cache and consume cached components
- versions: Update firecracker version
- dependency: update cgroups-rs
- Built-in Sandbox: add more unit tests for dragonball. Part 6
- runtime: add support for Hyper-V
- runtime-rs: update load_config comment
- Add support for ephemeral mounts to occupy entire sandbox's memory
- runtime-rs: fix default kernel location and add more default config paths
- Implement direct-volume commands handler for shim-mgmt
- bugfix: modify tty_win info in runtime when handling ResizePtyRequest
- bugfix: add get_ns_path API for Hypervisor
- runtime-rs: add the missing default trait
- packaging: Simplify get_last_modification()
- utils: Make kata-manager.sh runs checks
- dragonball: support pmu on aarch64
- docs: fix typo in key filename in AWS installation guide
- backport rustjail systemd cgroup fix#6331 to 3.1
- main | kata-deploy: Fix kata deploy arm64 image build error
- workflows: Yet more fixes for publishing the kata-deploy payload after every PR merged
- rustjail: fix cgroup handling in agent-init mode
- runtime/Makefile: Fix install-containerd-shim-v2 dependency
- fix wrong notes for func GetSandboxesStoragePathRust()
- fix(runtime-rs): add exited state to ensure cleanup
- runtime-rs: add oci hook support
- utils: Remove kata-manager.sh cgroups v2 check
- workflows: Fixes for the `payload-after-push` action
- Dragonball: update dependencies
- workflows: Do not install docker
- workflows: Publish kata-deploy payload after a merge
- src: Fixed typo mod.rs
- actions: Use `git-diff` to get changes in kernel dir
- agent: don't set permission of existing directory in copy_file
- runtime: use filepath.Clean() to clean the mount path
- Upgrade to Cloud Hypervisor v30.0
- feat(runtime): make static resource management consistent with 2.0
- osbuilder: Include minimal set of device nodes in ubuntu initrd
- kata-ctl/exec: add new command exec to enter guest VM.
- kernel: Add CONFIG_SEV_GUEST to SEV kernel config
- runtime-rs: Improve Cloud Hypervisor config handling
- virtiofsd: update to a valid path on ppc64le
- runtime-rs: cleanup kata host share path
- osbuilder: fix default build target in makefile
- devguide: Add link to the contribution guidelines
- kata-deploy: Ensure go binaries can run on Ubuntu 20.04
- dragonball: config_manager: preserve device when update
- Revert "workflows: Push the builder image to quay.io"
- Remove all remaining unsafe impl
- kata-deploy: Fix building the kata static firecracker arm64 package occurred an error
- shim-v2: Bump Ubuntu container image to 22.04
- packaging: Cache the container used to build the kata-deploy artefacts
- utils: always check some dependencies.
- versions: Use ubuntu as the default distro for the rootfs-image
- github-action: Replace deprecated command with environment file
- docs: Change the order of release step
- runtime-rs: remove unnecessary Send/Sync trait implement
- runtime-rs: Don't build on Power, don't break on Power.
- runtime-rs: handle sys_dir bind volume
- sandbox: set the dns for the sandbox
- packaging/shim-v2: Only change the config if the file exists
- runtime-rs: Add basic CH implementation
- release: Revert kata-deploy changes after 3.1.0-rc0 release
8b008fc743 kata-deploy: fix bash semantics error
74ec38cf02 osbuilder: Add support for CBL-Mariner
ac58588682 runtime-rs: ch: Generate Cloud Hypervisor config for confidential guests
96555186b3 runtime-rs: ch: Honour debug setting
e3c2d727ba runtime-rs: ch: clippy fix
ece5edc641 qemu/arm64: disable image nvdimm if no firmware offered
dd23f452ab utils: renamed only_kata to skip_containerd
59c81ed2bb utils: informed pre-check about only_kata
4f0887ce42 kata-deploy: fix install failing to chmod runtime-rs/bin/*
09c4828ac3 workflows: add missing artifacts on payload-after-push
fbf891fdff packaging: Adapt `get_last_modification()`
82a04dbce1 local-build: Use cached VirtioFS when possible
3b99004897 local-build: Use cached shim v2 when possible
1b8c5474da local-build: Use cached RootFS when possible
09ce4ab893 local-build: Use cached QEMU when possible
1e1c843b8b local-build: Use cached Nydus when possible
64832ab65b local-build: Use cached Kernel when possible
04fb52f6c9 local-build: Use cached Firecracker when possible
8a40f6f234 local-build: Use cached Cloud Hypervisor when possible
194d5dc8a6 tools: Add support for caching VirtioFS artefacts
a34272cf20 tools: Add support for caching shim v2 artefacts
7898db5f79 tools: Add support for caching RootFS artefacts
e90891059b tools: Add support for caching QEMU artefacts
7aed8f8c80 tools: Add support for caching Nydus artefacts
cb4cbe2958 tools: Add support for caching Kernel artefacts
762f9f4c3e tools: Add support for caching Firecracker artefacts
6b1b424fc7 tools: Add support for caching Cloud Hypervisor artefacts
08fe49f708 versions: Adjust kernel names to match kata-deploy build targets
99505c0f4f versions: Update firecracker version
f4938c0d90 bugfix: set hostname
96baa83895 agent: Bring in VFIO-AP device handling again
f666f8e2df agent: Add VFIO-AP device handling
b546eca26f runtime: Generalize VFIO devices
4c527d00c7 agent: Rename VFIO handling to VFIO PCI handling
db89c88f4f agent: Use cfg-if for s390x CCW
68a586e52c agent: Use a constant for CCW root bus path
a8b55bf874 dependency: update cgroups-rs
97cdba97ea runtime-rs: update load_config comment
974a5c22f0 runtime: add support for Hyper-V
40f4eef535 build: Use the correct kernel name
a6c67a161e runtime: add support for ephemeral mounts to occupy entire sandbox memory
844bf053b2 runtime-rs: add the missing default trait
e7bca62c32 bugfix: modify tty_win info in runtime when handling ResizePtyRequest
30e235f0a1 runtime-rs: impl volume-resize trait for sandbox
e029988bc2 bugfix: add get_ns_path API for Hypervisor
42b8867148 runtime-rs: impl volume-stats trait for sandbox
462d4a1af2 workflows: static-checks: Free disk space before running checks
e68186d9af workflows: static-checks: Set GOPATH only once
439ff9d4c4 tools/osbuilder/tests: Remove TRAVIS variable
43ce3f7588 packaging: Simplify get_last_modification()
33c5c49719 packaging: Move repo_root_dir to lib.sh
16e2c3cc55 agent: implement update_ephemeral_mounts api
3896c7a22b protocol: add updateEphemeralMounts proto
23488312f5 agent: always use cgroupfs when running as init
8546387348 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
736aae47a4 rustjail: print type of cgroup manager
dbae281924 workflows: Properly set the kata-tarball architecture
76b4591e2b tools: Adjust the build-and-upload-payload.sh script
cd2aaeda2a kata-deploy: Switch to using an ubuntu image
2d43e13102 docs: fix typo in AWS installation guide
760f78137d dragonball: support pmu on aarch64
9bc7bef3d6 kata-deploy: Fix path to the Dockerfile
78ba363f8e kata-deploy: Use different images for s390x and aarch64
6267909501 kata-deploy: Allow passing BASE_IMAGE_{NAME,TAG}
3443f558a6 nydus: upgrad nydus to v2.2.0
395645e1ce runtime: hybrid-mode cause error in the latest nydusd
f8e44172f6 utils: Make kata-manager.sh runs checks
f31c79d210 workflows: static-checks: Remove TRAVIS_XXX variables
8030e469b2 fix(runtime-rs): add exited state to ensure cleanup
7d292d7fc3 workflows: Fix the path of imported workflows
e07162e79d workflows: Fix action name
dd2713521e Dragonball: update dependencies
bd1ed26c8d workflows: Publish kata-deploy payload after a merge
fea7e8816f runtime-rs: Fixed typo mod.rs
a9e2fc8678 runtime/Makefile: Fix install-containerd-shim-v2 dependency
b6880c60d3 logging: Correct the code notes
12cfad4858 runtime-rs: modify the transfer to oci::Hooks
828d467222 workflows: Do not install docker
4b8a5a1a3d utils: Remove kata-manager.sh cgroups v2 check
2c4428ee02 runtime-rs: move pre-start hooks to sandbox_start
e80c9f7b74 runtime-rs: add StartContainer hook
977f281c5c runtime-rs: add CreateContainer hook support
875f2db528 runtime-rs: add oci hook support
ecac3a9e10 docs: add design doc for Hooks
3ac6f29e95 runtime: clh: Re-generate the client code
262daaa2ef versions: Upgrade to Cloud Hypervisor v30.0
192df84588 agent: always use cgroupfs when running as init
b0691806f1 agent: determine value of use_systemd_cgroup before LinuxContainer::new()
dc86d6dac3 runtime: use filepath.Clean() to clean the mount path
c4ef5fd325 agent: don't set permission of existing directory
3483272bbd runtime-rs: ch: Enable initrd usage
fbee6c820e runtime-rs: Improve Cloud Hypervisor config handling
1bff1ca30a kernel: Add CONFIG_SEV_GUEST to SEV kernel config Adding kernel config to sev case since it is needed for SNP and SNP will use the SEV kernel. Incrementing kernel config version to reflect changes
ad8968c8d9 rustjail: print type of cgroup manager
b4a1527aa6 kata-deploy: Fix static shim-v2 build on arm64
2c4f8077fd Revert "shim-v2: Bump Ubuntu container image to 22.04"
afaccf924d Revert "workflows: Push the builder image to quay.io"
4c39c4ef9f devguide: Add link to the contribution guidelines
76e926453a osbuilder: Include minimal set of device nodes in ubuntu initrd
697ec8e578 kata-deploy: Fix kata static firecracker arm64 package build error
ced3c99895 dragonball: config_manager: preserve device when update
da8a6417aa runtime-rs: remove all remaining unsafe impl
0301194851 dragonball: use crossbeam_channel in VmmService instead of mpsc::channel
9d78bf9086 shim-v2: Bump Ubuntu container image to 22.04
3cfce5a709 utils: improved unsupported distro message.
919d19f415 feat(runtime): make static resource management consistent with 2.0
b835c40bbd workflows: Push the builder image to quay.io
781ed2986a packaging: Allow passing a container builder to the scripts
45668fae15 packaging: Use existing image to build td-shim
e8c6bfbdeb packaging: Use existing image to build td-shim
3fa24f7acc packaging: Add infra to push the OVMF builder image
f076fa4c77 packaging: Use existing image to build OVMF
c7f515172d packaging: Add infra to push the QEMU builder image
fb7b86b8e0 packaging: Use existing image to build QEMU
d0181bb262 packaging: Add infra to push the virtiofsd builder image
7c93428a18 packaging: Use existing image to build virtiofsd
8c227e2471 virtiofsd: Pass the expected toolchain to the build container
7ee00d8e57 packaging: Add infra to push the shim-v2 builder image
24767d82aa packaging: Use existing image to build the shim-v2
e84af6a620 virtiofsd: update to a valid path on ppc64le
6c3c771a52 packaging: Add infra to push the kernel builder image
b9b23112bf packaging: Use existing image to build the kernel
869827d77f packaging: Add push_to_registry()
e69a6f5749 packaging: Add get_last_modification()
6c05e5c67a packaging: Add and export BUILDER_REGISTRY
1047840cf8 utils: always check some dependencies.
95e3364493 runtime-rs: remove unnecessary Send/Sync trait implement
a96ba99239 actions: Use `git-diff` to get changes in kernel dir
619ef54452 docs: Change the order of release step
a161d11920 versions: Use ubuntu as the default distro for the rootfs-image
be40683bc5 runtime-rs: Add a generic powerpc64le-options.mk
47c058599a packaging/shim-v2: Install the target depending on the arch/libc
b582c0db86 kata-ctl/exec: add new command exec to enter guest VM.
07802a19dc runtime-rs: handle sys_dir bind volume
04e930073c sandbox: set the dns for the sandbox
32ebe1895b agent: fix the issue of creating the dns file
44aaec9020 github-action: Replace deprecated command with environment file
a68c5004f8 packaging/shim-v2: Only change the config if the file exists
ee76b398b3 release: Revert kata-deploy changes after 3.1.0-rc0 release
bbc733d6c8 docs: runtime-rs: Add CH status details
37b594c0d2 runtime-rs: Add basic CH implementation
545151829d kata-types: Add Cloud Hypervisor (CH) definitions
2dd2421ad0 runtime-rs: cleanup kata host share path
0a21ad78b1 osbuilder: fix default build target in makefile
9a01d4e446 dragonball: add more unit test for virtio-blk device.
Signed-off-by: Greg Kurz <groug@kaod.org>
Our CI and release process are currently taking advantage of the
kata-deploy local build scripts to build the artefacts.
Having snap doing the same is the next logical step, and it will also
help to reduce, by a lot, the CI time as we only build the components
that a PR is touching (otherwise we just pull the cached component).
Fixes: #6514
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Add osbuilder support to build a rootfs and image
based on the CBL-Mariner Linux distro
Fixes: #6462
Signed-off-by: Dallas Delaney <dadelan@microsoft.com>
This change provides a preliminary implementation for the Cloud Hypervisor (CH) feature ([currently
disabled](https://github.com/kata-containers/kata-containers/pull/6201))
to allow it to generate the CH configuration for handling confidential guests.
This change also introduces concrete errors using the `thiserror` crate
(see `src/runtime-rs/crates/hypervisor/ch-config/src/errors.rs`) and a
lot of unit tests for the conversion code that generates the CH
configuration from the generic Hypervisor configuration.
Fixes: #6430.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Enable Cloud Hypervisor debug based on the specified configuration
rather than hard-coding debug to be disabled.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
For now, image nvdimm on qemu/arm64 depends on UEFI/ACPI, so if there
is no firmware offered, it should be disabled.
Fixes: #6468
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
passed the only_kata variable through to pre_check, only_kata does not
abort the install when containerd is already installed.
fixes#6385
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
The kata-deploy install method tried to `chmod +x /opt/kata/runtime-rs/bin/*` but it isn't
always true that /opt/kata/runtime-rs/bin/ exists. For example, the
s390x payload does not build the kernel-dragonball-experimental
artifacts. So let's ensure the dir exist before issuing the command.
Fixes#6494
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The kata-deploy-ci payloads for amd64 and arm64 were missing the shim-v2
and kernel-dragonball-experimental artifacts.
Fixes#6493
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
The function is returning "" when called from the script used to cache
the artefacts and one difference noted between this version and the
already working one from the CCv0 is that we make sure to `pushd
${repo_root_dir}` in the CCv0 version.
Let's give it a try here and see if it solves the issue.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add support for caching VirtioFS artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching shim v2 artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching RootFS artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching QEMU artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Nydus artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Kernel artefacts that are generated using
the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Firecracker artefacts that are generated
using the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's add support for caching Cloud Hypervisor artefacts that are
generated using the kata-deploy local-build scripts.
Right now those are not used, but we'll switch to using them very soon
as part of upcoming changes of how we build the components we test in
our CI.
Fixes: #6480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Let's adjust the kernel names in versions.yaml so those can match the
names used as part of the kata-deploy local build scripts.
Right now this doesn't bring any benefit nor drawback, but it'll make
our life easier later on in this same series.
Depends-on: github.com/kata-containers/tests#5534
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This PR updates the firecracker version being used in kata containers
versions.yaml
The changes in version 1.3.1 are
Added
Introduced T2CL (Intel) and T2A (AMD) CPU templates to provide
instruction set feature parity between Intel and AMD CPUs when using
these templates.
Added Graviton3 support (c7g instance type).
Changed
Improved error message when invalid network backend provided.
Improved TCP throughput by between 5% and 15% (depending on CPU) by using
scatter-gather I/O in the net device's TX path.
Upgraded Rust toolchain from 1.64.0 to 1.66.0.
Made seccompiler output bit-reproducible.
Fixed
Fixed feature flags in T2 CPU template on Intel Ice Lake.
Fixes#6482
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
This PR is a continuing work for (kata-containers#3679).
This generalizes the previous VFIO device handling which only
focuses on PCI to include AP (IBM Z specific).
Fixes: kata-containers#3678
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Initial VFIO-AP support (#578) was simple, but somewhat hacky; a
different code path would be chosen for performing the hotplug, and
agent-side device handling was bound to knowing the assigned queue
numbers (APQNs) through some other means; plus the code for awaiting
them was written for the Go agent and never released. This code also
artificially increased the hotplug timeout to wait for the (relatively
expensive, thus limited to 5 seconds at the quickest) AP rescan, which
is impractical for e.g. common k8s timeouts.
Since then, the general handling logic was improved (#1190), but it
assumed PCI in several places.
In the runtime, introduce and parse AP devices. Annotate them as such
when passing to the agent, and include information about the associated
APQNs.
The agent awaits the passed APQNs through uevents and triggers a
rescan directly.
Fixes: #3678
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Generalize VFIO devices to allow for adding AP in the next patch.
The logic for VFIOPciDeviceMediatedType() has been changed and IsAPVFIOMediatedDevice() has been removed.
The rationale for the revomal is:
- VFIODeviceMediatedType is divided into 2 subtypes for AP and PCI
- Logic of checking a subtype of mediated device is included in GetVFIODeviceType()
- VFIOPciDeviceMediatedType() can simply fulfill the device addition based
on a type categorized by GetVFIODeviceType()
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
e.g., split_vfio_option is PCI-specific and should instead be named
split_vfio_pci_option. This mutually affects the runtime, most notably
how the labels are named for the agent.
Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
Since shimv2 create task option is already implemented, we need to update the
corresponding comments.
Also, the ordering is also updated to fit with the code.
fixes: #3961
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
This adds /dev/mshv to the list of sandbox devices so that VMMs can
create Hyper-V VMs.
In our testing, this also doesn't error out in case /dev/mshv isn't
present.
Fixes#6454.
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
When calling `MAKE_KERNEL_NAME` we're considering the default kernel
name will be `vmlinux.container` or `vmlinuz.container`, which is not
the case as the runtime-rs, when used with dragonball, relies on the
`vmlinu[zx]-dragonball-experimental.container` kernel.
Other hypervisors will have to introduce a similar
`MAKE_KERNEL_NAME_${HYPERVISOR}` to adapt this to the kernel they want
to use, similarly to what's already done for the go runtime.
By doing this we also ensure that no changes in the configuration file
will be required to run runtime-rs, with dragonball, as part of our CI
or as part of kata-deploy.
Fixes: #6290
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
On hotplug of memory as containers are started, remount all ephemeral mounts with size option set to the total sandbox memory
Fixes: #6417
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
Some structs in the runtime-rs don't implement Default trait.
This commit adds the missing Default.
Fixes: #5463
Signed-off-by: Li Hongyu <lihongyu1999@bupt.edu.cn>
Currently, we only create the new exec process in runtime, this will cause error
when the following requests needing to be handled:
- Task: exec process
- Task: resize process pty
- ...
The agent do not do_exec_process when we handle ExecProcess, thus we can not find
any process information in the guest when we handle ResizeProcessPty. This will
report an error.
In this commit, the handling process is modified to the:
* Modify process tty_win information in runtime
* If the exec process is not running, we just return. And the truly pty_resize will
happen when start_process
Fixes: #6248
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Implements resize-volume handlers in shim-mgmt,
trait for sandbox and add RPC calls to agent.
Note the actual rpc handler for the resize request is currently not
implemented, refer to issue #3694.
Fixes#5369
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
For external hypervisors(qemu, cloud-hypervisor, ...), the ns they launch vm in
is different from internal hypervisor(dragonball). And when we doing CreateContainer
hook, we will rely on the netns path. So we add a get_ns_path API.
Fixes: #6442
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
Implements get-volume-stats trait for sandbox,
handler for shim-mgmt and add RPC calls to
agent. Also added type conversions in trans.rs
Fixes#5369
Signed-off-by: Tingzhou Yuan <tzyuan15@bu.edu>
We've been seeing the 'sudo make test' job occasionally run out of space in
/tmp, which is part of the root filesystem. Removing dotnet and
`AGENT_TOOLSDIRECTORY` frees around 10GB of space and in my tests the job still
has 13GB of space left after running.
Fixes: #6401
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
{{ runner.workspace }}/kata-containers and {{ github.workspace }} resolve to
the same value, but they're being used multiple times in the workflow. Remove
multiple definitions and define the GOPATH var at job level once.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The last remaining user of the TRAVIS variable in this repo is
tools/osbuilder/tests and it is only used to skip spinning up VMs. Travis
didn't support virtualization and the same is true for github actions hosted
runners. Replace the variable with KVM_MISSING and determine availability of
/dev/kvm at runtime.
TRAVIS is also used by '.ci/setup.sh' in kata-containers/tests to reduce the
set of dependencies that gets installed, but this is also in the process of
being removed.
Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There's no need to pass repo_root_dir to get_last_modification() as the
variable used everywhere is exported from that very same file.
Fixes: #6431
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This is used in several parts of the code, and can have a single
declaration as part of the `lib.sh` file, which is already imported by
all the places where it's used.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
- implement update_ephemeral_mounts rpc
- for each mountpoint passed in, remount it with new options
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
- adds a new rpc call to the agent service named `updateEphemeralMounts`
- this call takes a list of grpc.Storage objects
Signed-off-by: Sidhartha Mani <sidhartha_mani@apple.com>
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)
Caused by:
No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
stack backtrace:
0: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
3: 0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
5: 0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
6: 0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
7: 0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
8: 0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
9: 0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
10: 0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
11: 0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
12: 0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
13: 0x7ff0fe77d3c1 - rust_begin_unwind
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
14: 0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
15: 0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
16: 0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
17: 0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
18: 0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
19: 0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
20: 0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
21: 0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
22: 0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
23: 0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
24: 0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
25: 0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
26: 0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
27: 0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
28: 0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
29: 0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
30: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
31: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
32: 0x7ff0fe7801d5 - std::sys::unix::thread::Thread::new::thread_start::h575491a8a17dbb33
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17
Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Let's make sure the kata-tarball architecture upload / downloaded / used
is exactly the same one that we need as part of the architecture we're
using to generate the image.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Now that we've switched the base container image to using Ubuntu instead
of CentOS, we don't need any kind of extra logic to correctly build the
image for different architectures, as Ubuntu is a multi-arch image that
supports all the architectures we're targetting.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's make sure we use a multi-arch image for building kata-deploy.
A few changes were also added in order to get systemd working inside the
kata-deploy image, due to the switch from CentOS to Ubuntu.
Fixes: #6358
Signed-off-by: SinghWang <wangxin_0611@126.com>
This commit adds support for pmu virtualization on aarch64. The
initialization of pmu is in the following order:
1. Receive pmu parameter(vpmu_feature) from runtime-rs to determine the
VpmuFeatureLevel.
2. Judge whether to initialize pmu devices and add pmu device node into
fdt on aarch64, according to VpmuFeatureLevel.
Fixes: #6168
Signed-off-by: xuejun-xj <jiyunxue@linux.alibaba.com>
As part of bd1ed26c8d, we've pointed to
the Dockerfile that's used in the CC branch, which is wrong.
For what we're doing on main, we should be pointing to the one under the
`kata-deploy` folder, and not the one under the non-existent
`kata-deploy-cc` one.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As the image provided as part of registry.centos.org is not a multi-arch
one, at least not for CentOS 7, we need to expand the script used to
build the image to pass images that are known to work for s390x (ClefOS)
and aarch64 (CentOS, but coming from dockerhub).
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's break the IMAGE build parameter into BASE_IMAGE_NAME and
BASE_IMAGE_TAG, as it makes it easier to replace the default CentOS
image by something else.
Spoiler alert, the default CentOS image is **not** multi-arch, and we do
want to support at least aarch64 and s390x in the near term future.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
When update the nydusd to 2.2, the argument "--hybrid-mode" cause
the following error:
thread 'main' panicked at 'ArgAction::SetTrue / ArgAction::SetFalse is defaulted'
Maybe we should remove it to upgrad nydusd
Fixes: #6407
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Updated the `kata-manager.sh` script to make it run all the checks on
the host system before attempting to create a container. If any checks
fail, they will indicate to the user what the problem is in a clearer
manner than those reported by the container manager.
Fixes: #6281.
Signed-off-by: tg5788re <jfokugas@gmail.com>
These variables are unused since we don't use travis CI. This also allows to
remove two steps:
- 'Setup GOPATH' only printed variables
- 'Setup travis reference' modified some shell local variables that don't have
any influence on the rest of the steps
The TRAVIS var is still used by tools/osbuilder/tests to determine if
virtualization is available.
Fixes: #3544
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Set process status to exited at end of io wait, which indicate process
exited only, but stop process has not been finished. Otherwise, the
cleanup_container will be skipped.
Fixes: #6393
Signed-off-by: Yipeng Yin <yinyipeng@bytedance.com>
In `payload-after-push.yaml` we ended up mentioning cc-*.yaml workflows,
which are non existent in the main branch.
Let's adapt the name to the correct ones.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We have a few actions in the `payload-after-push.*.yaml` that are
referring to Confidential Containers, but they should be referring to
Kata Containers instead.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Since rust-vmm and dragonball-sandbox has introduced several updates
such as vPMU support for aarch64, we also need to update Dragonball
dependencies to include those changes.
Update:
virtio-queue to v0.6.0
kvm-ioctls to v0.12.0
dbs-upcall to v0.2.0
dbs-virtio-devices to v0.2.0
kvm-bindings to v0.6.0
Also, several aarch64 features are updated because of dependencies
changes:
1. update vcpu hotplug API.
2. update vpmu related API.
3. adjust unit test cases for aarch64 Dragonball.
fixes: #6268
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
For the architectures we know that `make kata-tarball` works as
expected, let's start publishing the kata-deploy payload after each
merge.
This will help to:
* Easily test the content of current `main` or `stable-*` branch
* Easily bisect issues
* Start providing some sort of CI/CD content pipeline for those who
need that
This is a forward-port work from the `CCv0` and groups together patches
that I've worked on, with the work that Choi did in order to support
different architectures.
Fixes: #6343
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
$ make install
make: *** No rule to make target 'containerd-shim-kata-v2', needed by 'install-containerd-shim-v2'. Stop.
Spotted when building kata-runtime with a different name for
SHIMV2_OUTPUT. For instance, trying to keep different runtime binaries
installed at the same time, one from master and another from lets say,
the CCv0 branch, with the following small change applied.
diff --git a/src/runtime/Makefile b/src/runtime/Makefile
index 95efaff78..2bab9eb75 100644
--- a/src/runtime/Makefile
+++ b/src/runtime/Makefile
@@ -231,7 +231,7 @@ SED = sed
CLI_DIR = cmd
SHIMV2 = containerd-shim-kata-v2
-SHIMV2_OUTPUT = $(bCURDIR)/$(SHIMV2)
+SHIMV2_OUTPUT = $(CURDIR)/$(SHIMV2)-ccv0
SHIMV2_DIR = $(CLI_DIR)/$(SHIMV2)
MONITOR = kata-monitor
Fixes: #6398
Signed-off-by: Eduardo Lima (Etrunko) <etrunko@redhat.com>
In this commit, we have done:
* modify the tranfer process from grpc::Hooks to oci::Hooks, so the code
can be more clean
* add more tests for create_runtime, create_container, start_container hooks
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
The latest ubuntu runners already have docker installed and trying to
install it manually will cause the following issue:
```
Run curl -fsSL https://test.docker.com/ -o test-docker.sh
Warning: the "docker" command appears to already exist on this system.
If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.
If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.
You may press Ctrl+C now to abort this script.
+ sleep 20
+ sudo -E sh -c apt-get update -qq >/dev/null
E: The repository 'https://packages.microsoft.com/ubuntu/22.04/prod jammy Release' is no longer signed.
```
Fixes: #6390
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Removed the part in the `kata-manager.sh` script that checks if the host system only runs cgroups v2.
Fixes: #6259.
Signed-off-by: Alec Pemberton <pembek1901@gmail.com>
In some cases, network endpoints will be configured through Prestart
Hook. So network endpoints may need to be added(hotpluged) after vm
is started and also Prestart Hook is executed.
We move pre-start hook functions' execution to sandbox_start to allow
hooks running between vm_start and netns_scan easily, so that the
lifecycle API can be cleaner.
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
StartContainer will be execute in guest container namespace in Kata.
The Hook Path of this kind of hook is also in guest container namespace.
StartContainer is executed after start operation is called, and it
should be executed before user-specific command is executed.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
CreateContainer hook is one kind of OCI hook. In kata, it will be
executed after VM is started, before container is created, and after
CreateRuntime is executed.
The hook path of CreateContainer hook is in host runtime namespace, but
it will be executed in host vmm namespace.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
According to the runtime OCI Spec, there can be some hook
operations in the lifecycle of the container. In these hook
operations, the runtime can execute some commands. There are different
points in time in the container lifecycle and different hook types
can be executed.
In this commit, we are now supporting 4 types of hooks(same in
runtime-go): Prestart hook, CreateRuntime hook, Poststart hook and
Poststop hook.
Fixes: #5787
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
This patch re-generates the client code for Cloud Hypervisor v30.0.
Note: The client code of cloud-hypervisor's OpenAPI is automatically
generated by openapi-generator.
Fixes: #6375
Signed-off-by: Bo Chen <chen.bo@intel.com>
The logic to decide which cgroup driver is used is currently based on the
cgroup path that the host provides. This requires host and guest to use the
same cgroup driver. If the guest uses kata-agent as init, then systemd can't be
used as the cgroup driver. If the host requests a systemd cgroup, this
currently results in a rustjail panic:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: I/O error: No such file or directory (os error 2)
Caused by:
No such file or directory (os error 2)', rustjail/src/cgroups/systemd/manager.rs:44:51
stack backtrace:
0: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::libunwind::trace::h8c197fa9a679d134
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x7ff0fe77a793 - std::backtrace_rs::backtrace::trace_unsynchronized::h9ee19d58b6d5934a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7ff0fe77a793 - std::sys_common::backtrace::_print_fmt::h4badc450600fc417
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:65:5
3: 0x7ff0fe77a793 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had334ddb529a2169
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7ff0fdce815e - core::fmt::write::h1aa7694f03e44db2
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/fmt/mod.rs:1209:17
5: 0x7ff0fe74e0c4 - std::io::Write::write_fmt::h61b2bdc565be41b5
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/io/mod.rs:1682:15
6: 0x7ff0fe77cd3f - std::sys_common::backtrace::_print::h4ec69798b72ff254
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:47:5
7: 0x7ff0fe77cd3f - std::sys_common::backtrace::print::h0e6c02048dec3c77
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:34:9
8: 0x7ff0fe77c93f - std::panicking::default_hook::{{closure}}::hcdb7e705dc37ea6e
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:267:22
9: 0x7ff0fe77d9b8 - std::panicking::default_hook::he03a933a0f01790f
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:286:9
10: 0x7ff0fe77d9b8 - std::panicking::rust_panic_with_hook::he26b680bfd953008
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:688:13
11: 0x7ff0fe77d482 - std::panicking::begin_panic_handler::{{closure}}::h559120d2dd1c6180
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:579:13
12: 0x7ff0fe77d3ec - std::sys_common::backtrace::__rust_end_short_backtrace::h36db621fc93b005a
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys_common/backtrace.rs:137:18
13: 0x7ff0fe77d3c1 - rust_begin_unwind
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
14: 0x7ff0fda52ee2 - core::panicking::panic_fmt::he7679b415d25c5f4
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
15: 0x7ff0fda53182 - core::result::unwrap_failed::hb71caff146724b6b
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/result.rs:1791:5
16: 0x7ff0fe5bd738 - <rustjail::cgroups::systemd::manager::Manager as rustjail::cgroups::Manager>::apply::hd46958d9d807d2ca
17: 0x7ff0fe606d80 - <rustjail::container::LinuxContainer as rustjail::container::BaseContainer>::start::{{closure}}::h1de806d91fcb878f
18: 0x7ff0fe604a76 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1749c148adcc235f
19: 0x7ff0fdc0c992 - kata_agent::rpc::AgentService::do_create_container::{{closure}}::{{closure}}::hc1b87a15dfdf2f64
20: 0x7ff0fdb80ae4 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h846a8c9e4fb67707
21: 0x7ff0fe3bb816 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h53de16ff66ed3972
22: 0x7ff0fdb519cb - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1cbece980286c0f4
23: 0x7ff0fdf4019c - <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::hc8e72d155feb8d1f
24: 0x7ff0fdfa5fd8 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::h0a407ffe2559449a
25: 0x7ff0fdf033a1 - tokio::runtime::task::raw::poll::h1045d9f1db9742de
26: 0x7ff0fe7a8ce2 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h4924ae3464af7fbd
27: 0x7ff0fe7afb85 - tokio::runtime::task::raw::poll::h5c843be39646b833
28: 0x7ff0fe7a05ee - std::sys_common::backtrace::__rust_begin_short_backtrace::ha7777c55b98a9bd1
29: 0x7ff0fe7a9bdb - core::ops::function::FnOnce::call_once{{vtable.shim}}::h27ec83c953360cdd
30: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hed812350c5aef7a8
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
31: 0x7ff0fe7801d5 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7df8e435a658960
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/alloc/src/boxed.rs:1987:9
32: 0x7ff0fe7801d5 - std::sys::unix::thread::Thread::new::thread_start::h575491a8a17dbb33
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/sys/unix/thread.rs:108:17
Forward the value of "init_mode" to AgentService, so that we can force cgroupfs
when systemd is unavailable.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Right now LinuxContainer::new() gets passed a CreateOpts struct, but then
modifies the use_systemd_cgroup field inside that struct. Pull the cgroups path
parsing logic into do_create_container, so that CreateOpts can be immutable in
LinuxContainer::new. This is just moving things around, there should be no
functional changes.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Fix path check bypassed issuse introduced by #6082,
use filepath.Clean() to clean path before check
Fixes: #6082
Signed-off-by: XDTG <click1799@163.com>
This patch fixes the issue that do_copy_file changes
the directory permission of the parent directory of
a target file, even when the parent directory already
exists.
Fixes#6367
Signed-off-by: Yohei Ueda <yohei@jp.ibm.com>
This change enables to run cloud-hypervisor VMM using a non-root user
when rootless flag is set true in the configuration
Fixes: #2567
Signed-off-by: Feng Wang <fwang@confluent.io>
Allow an initrd/initramfs image to be used with Cloud Hypervisor, which
is handled differently to the default rootfs image type.
Fixes: #6335.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Replace `cloud_hypervisor_vm_create_cfg()` with a set of `TryFrom` trait
implementations in the new CH specific `convert.rs` to allow the generic
`Hypervisor` configuration to be converted into the CH specific
`VmConfig` type.
Note that device configuration is not currently handled in `convert.rs`
(it's handled in `inner_device.rs`).
This change removes the old hard-coded CH specific configuration.
Fixes: #6203.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Adding kernel config to sev case since it is needed for SNP and SNP will use the SEV kernel.
Incrementing kernel config version to reflect changes
Fixes: #6123
Signed-off-by: Alex Carter <Alex.Carter@ibm.com>
Since the cgroup manager is wrapped in a dyn now, the print in
LinuxContainer::new has been useless and just says "CgroupManager". Extend the
Debug trait for 'dyn Manager' to print the type of the cgroup manager so that
it's easier to debug issues.
Fixes: #5779
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Following Jong Wu suggestion, let's link /usr/bin/musl-gcc to
/usr/bin/aarch64-linux-musl-gcc.
Fixes: #6320
Signed-off-by: SinghWang <wangxin_0611@126.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This reverts commit 9d78bf9086.
Golang binaries are built statically by default, unless linking against
CGO, which we do. In this case we dynamically link against glibc,
causing us troubles when running a binary built with Ubuntu 22.04 on
Ubuntu 20.04 (which will still be supported for the next few years ...)
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
New developers are often confused by some of our requirements, notably porting
labels. While our CONTRIBUTING.md file points to the solution, the developer's
guide does not. Add a link there.
Fixes: #6329
Signed-off-by: Christophe de Dinechin <christophe@dinechin.org>
When starting an initrd the kernel expects to find /dev/console in the initrd,
so that it can connect it as stdin/stdout/stderr to the /init process. If the
device node is missing the kernel will complain that it was unable to open an
initial console. If kata-agent is the initrd init process, it will also result
in log messages not being logged to console and thus not forwarded to host
syslog.
Add a set of standard device nodes for completeness, so that console logging
works. To do that we install the makedev packge which provides a MAKEDEV helper
that knows the major/minor numbers. Unfortunately the debian package tries to
create devnodes from postinst, which can be suppressed if systemd-detect-virt
is present. That's why we create a small dummy script that matches what
systemd-detect-virt would output (anything is enough to suppress mknod).
Fixes: #6261
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
DeviceConfigInfo contains config and device, so when we want to do
update we could simply update config part of the info, and device would
not be changed during update.
Fixes: #6324
Signed-off-by: Helin Guo <helinguo@linux.alibaba.com>
Because crossbeam_channel has more features and better performance than
mpsc::channel and finally rust replace its channel implementation with
crossbeam_channel on version 1.67
Signed-off-by: Tim Zhang <tim@hyper.sh>
Let's bump the base container image to use the 22.04 version of Ubuntu,
as it does bring up-to-date package dependencies that we need to
statically build the runtime-rs on aarch64.
Fixes: #6320
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
previously, if installing on unkown distro, script would tell user that
their distro was unsupported. Changed error message prompting user to
install dependecies manually, then retry.
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Let's push the builder images to a registry, so we can take advantage of
those on each step of our building process.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This, combined with the effort of caching builder images *and* only
performing the build itself inside the builder images, is the very first
step for reproducible builds for the project.
Reproducible builds are quite important when we talk about Confidential
Containers, as users may want to verify the content used / provided by
the CSPs, and this is the first step towards that direction.
Fixes: #5517
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the td-shim.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for building and pushing the OVMF builder
image to the Kata Containers' quay.io registry.
Fixes: #5477
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of buildinf our
own, to be used as a builder image for OVMF.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the QEMU
builder image to the Kata Containers' quay.io registry.
Fixes: #5481
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existsing image, instead of building our
own, to be used as a builder image for QEMU.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the virtiofsd
builder image to the Kata Containers' quay.io registry.
Fixes: #5480
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the virtiofsd.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's ensure we're building virtiofsd with a specific toolchain that's
known to not cause any issues, instead of always using the latest one.
On each bump of the virtiofsd, we'll make sure to adjust this according
to what's been used by the virtiofsd community.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add the needed infra for only building and pushing the shim-v2
builder image to the Kata Containers' quay.io registry.
Fixes: #5478
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's try to pull a pre-existing image, instead of building our own, to
be used as a builder for the shim-v2.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Currently the symbolic link for virtiofsd which is used as
a valid path is not updated on every CI run. Fix it by
using the actual path of installation.
Fixes: #6311
Signed-off-by: Amulyam24 <amulmek1@in.ibm.com>
Let's add the needed infra for only building and pushing the kernel
builder image to the Kata Containers' quay.io registry.
Fixes: #5476
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's first try to pull a pre-existing image, instead of building our
own, to be used as a builder image for the kernel.
This will save us some CI time.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
This function will push a specific tag to a registry, whenever the
PUSH_TO_REGISTRY environment variable is set, otherwise it's a no-op.
This will be used in the future to avoid replicating that logic in every
builder used by the kata-deploy scripts.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Let's add a function to get the hash of the last commit modifying a
specific file.
This will help to avoid writing `git rev-list ...` into every single
build script used by the kata-deploy.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
BUILD_REGISTRY, which points to quay.io/kata-containers/builder, will be
used for storing the builder images used to build the artefacts via the
kata-deploy scripts.
The plan is to tag, whenever it's possible and makes sense, images like:
* ${BUILDER_REGISTRY}:${component}-${unique_identifier}
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Every dependency in check_deps is used inside the script (apart from
git, which may be a historical artifact), and therefore should be
checked even when the -f option is passed to the script. Simply changed
at what point check_deps is called in order to always run it.
Fixes#6302.
Signed-off-by: Gabe Venberg <gabevenberg@gmail.com>
Send and Sync are automatically derived traits,
if a type is composed entirely of Send or Sync types, then it is Send or Sync.
Almost all primitives are Send and Sync,
so we don't need to implement them manually most of the time.
Fixes: #6307
Signed-off-by: Tim Zhang <tim@hyper.sh>
Use `git-diff` instead of legacy `git-whatchanged` to get
differences in the packaging/kernel directory. This also fixes
a bug by grepping for the kernel directory in the output of the
git command.
Fixes: #6210
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
When a new stable branch is created, it is necessary to change the
references in the tests repo from main to the new stable branch.
However this step needs to be performed after the repos have been tagged
as the `tags_repos.sh` script is the one that creates the new branch.
Clarify this in the documentation and move the step to change branch
references in test repo after repos have been tagged.
Fixes: #1824
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Currently ubuntu is already the default distro for all the architectures
but x86_64, which uses clearlinux. However, our CI does *not* test the
clearlinux image we ship.
Taking a look at our CI code [0], we've been using ubuntu as base for
the tests for a few years already, if not forever.
The minimum we can do is to switch to distributing ubuntu, as the tested
rootfs-image, and then decide later on whether we should switch back to
clearlinux (once we switch our CI to using that, and make sure all tests
will be green), or if we move to slimmer distro, such as alpine.
[0]: 0a39dd1a01/.ci/install_kata_image.sh (L44)Fixes: #6303
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
There's a check in the runtime-rs Makefile that basically checks whether
the `arch/$arch-options.mk` exists or not and, if it doesn't, the build
is just aborted.
With this in mind, let's create a generic powerpc64le-options.mk file
and not bail when building for this architecture.
Fixes: #6142
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In the `install_go_rust.sh` file we're adding a
x86_64-unknown-linux-musl target unconditionally. That should be,
instead, based in the ARCH of the host and the appropriate LIBC to be
used with that host.
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
The patchset will help users to easily enter guest VM by debug
console sock.
In order to enter guest VM smoothly, users needs to do some
configuration, options as below:
(1) Set debug_console_enabled = true with default vport 1026.
(2) Or add agent.debug_console agent.debug_console_vport=<PORT>
into kernel_params, and the vport is <PORT> you set.
The detail of usage:
$ kata-ctl exec -h
kata-ctl-exec
Enter into guest VM by debug console
USAGE:
kata-ctl exec [OPTIONS] <SANDBOX_ID>
ARGS:
<SANDBOX_ID> pod sandbox ID
Fixes: #5340
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
For some cases, users will mount system directories as bind volume.
We should not bind mount these kind of directories in the host as it does
not make sense.
Fixes: #6299
Signed-off-by: Yushuo <y-shuo@linux.alibaba.com>
The rust agent had supported to set the guest dns
server in start sandbox request, thus add the dns
in the runtime side.
Fixes:#6286
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
We should make sure the dns's source file's parent
directory exist, otherwise, it would failed to create
the file directly.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In workflow, `set-output` command is deprecated and will be disabled soon.
This commit replaces the deprecated `set-output` command with putting a
value in the environment file `$GITHUB_OUTPUT`.
Fixes#6266
Signed-off-by: jongwooo <jongwooo.han@gmail.com>
Let's not try to sed a file that doesn't exist, which may be the case
depending on the architecture we're building the shim-v2 for.
This is a partial-forward port of
f24c47ea47.
Fixes: #6293
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
As 3.1.0-rc0 has been released, let's switch the kata-deploy / kata-cleanup
tags back to "latest", and re-add the kata-deploy-stable and the
kata-cleanup-stable files.
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Add a few details about the current state of the Cloud Hypervisor (CH)
runtime-rs external hypervisor implementation with pointers to the
appropriate issues.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Add a basic runtime-rs `Hypervisor` trait implementation for Cloud
Hypervisor (CH).
> **Notes:**
>
> - This only supports a default Kata configuration for CH currently.
>
> - Since this feature is still under development, `cargo` features have
> been added to enable the feature optionally. The default is to not enable
> currently since the code is not ready for general use.
>
> To enable the feature for testing and development, enable the
> `cloud-hypervisor` feature in the `virt_container` crate and enable the
> `cloud-hypervisor` feature for its `hypervisor` dependency.
Fixes: #5242.
Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
The .dracut_rootfs.done file is accidentally being picked up as the default
target, regardless of BUILD_METHOD. Move the 'all' target definition up, so
that it's the default (=first) target in the makefile. Additionally make the
.dracut_rootfs.done target conditional on the right BUILD_METHOD being
selected, as building it doesn't make sense with BUILD_METHOD=distro.
Fixes: #6235
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
[](https://github.com/kata-containers/kata-containers/actions/workflows/payload-after-push.yaml) [](https://github.com/kata-containers/kata-containers/actions/workflows/ci-nightly.yaml)
# Kata Containers
@@ -121,7 +123,7 @@ The table below lists the core parts of the project:
| [agent](src/agent) | core | Management process running inside the virtual machine / POD that sets up the container environment. |
| [`dragonball`](src/dragonball) | core | An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads |
| [documentation](docs) | documentation | Documentation common to all components (such as design and install documentation). |
| [tests](https://github.com/kata-containers/tests) | tests | Excludes unit tests which live with the main code. |
| [tests](tests) | tests | Excludes unit tests which live with the main code. |
### Additional components
@@ -132,19 +134,27 @@ The table below lists the remaining parts of the project:
| [packaging](tools/packaging) | infrastructure | Scripts and metadata for producing packaged binaries<br/>(components, hypervisors, kernel and rootfs). |
| [kernel](https://www.kernel.org) | kernel | Linux kernel used by the hypervisor to boot the guest image. Patches are stored [here](tools/packaging/kernel). |
| [osbuilder](tools/osbuilder) | infrastructure | Tool to create "mini O/S" rootfs and initrd images and kernel for the hypervisor. |
| [kata-debug](tools/packaging/kata-debug/README.md) | infrastructure | Utility tool to gather Kata Containers debug information from Kubernetes clusters. |
| [`agent-ctl`](src/tools/agent-ctl) | utility | Tool that provides low-level access for testing the agent. |
| [`kata-ctl`](src/tools/kata-ctl) | utility | Tool that provides advanced commands and debug facilities. |
| [`runk`](src/tools/runk) | utility | Standard OCI container runtime based on the agent. |
| [`ci`](https://github.com/kata-containers/ci) | CI | Continuous Integration configuration files and scripts. |
| [`ci`](.github/workflows) | CI | Continuous Integration configuration files and scripts. |
| [`katacontainers.io`](https://github.com/kata-containers/www.katacontainers.io) | Source for the [`katacontainers.io`](https://www.katacontainers.io) site. |
| [`Webhook`](tools/testing/kata-webhook/README.md) | utility | Example of a simple admission controller webhook to annotate pods with the Kata runtime class |
### Packaging and releases
Kata Containers is now
[available natively for most distributions](docs/install/README.md#packaged-installation-methods).
However, packaging scripts and metadata are still used to generate [snap](snap/local) and GitHub releases. See
the [components](#components) section for further details.
## General tests
See the [tests documentation](tests/README.md).
## Metrics tests
See the [metrics documentation](tests/metrics/README.md).
> While this project's CI has several areas for improvement, it is constantly
> evolving. This document attempts to describe its current state, but due to
> ongoing changes, you may notice some outdated information here. Feel free to
> modify/improve this document as you use the CI and notice anything odd. The
> community appreciates it!
## Introduction
The Kata Containers CI relies on [GitHub Actions][gh-actions], where the actions
themselves can be found in the `.github/workflows` directory, and they may call
helper scripts, which are located under the `tests` directory, to actually
perform the tasks required for each test case.
## The different workflows
There are a few different sets of workflows that are running as part of our CI,
and here we're going to cover the ones that are less likely to get rotten. With
this said, it's fair to advise that if the reader finds something that got
rotten, opening an issue to the project pointing to the problem is a nice way to
help, and providing a fix for the issue is a very encouraging way to help.
### Jobs that run automatically when a PR is raised
These are a bunch of tests that will automatically run as soon as a PR is
opened, they're mostly running on "cost free" runners, and they do some
pre-checks to evaluate that your PR may be okay to start getting reviewed.
Mind, though, that the community expects the contributors to, at least, build
their code before submitting a PR, which the community sees as a very fair
request.
Without getting into the weeds with details on this, those jobs are the ones
responsible for ensuring that:
- The commit message is in the expected format
- There's no missing Developer's Certificate of Origin
- Static checks are passing
### Jobs that require a maintainer's approval to run
These are the required tests, and our so-called "CI". These require a
maintainer's approval to run as parts of those jobs will be running on "paid
runners", which are currently using Azure infrastructure.
Once a maintainer of the project gives "the green light" (currently by adding an
`ok-to-test` label to the PR, soon to be changed to commenting "/test" as part
of a PR review), the following tests will be executed:
- Build all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a tarball with all the components (runs on free cost runners, or bare-metal depending on the architecture)
- Create a kata-deploy payload with the tarball generated in the previous step (runs on free costs runner, or bare-metal depending on the architecture)
- Run the following tests:
- Tests depending on the generated tarball
- Metrics (runs on bare-metal)
-`docker` (runs on Azure small instances)
-`nerdctl` (runs on Azure small instances)
-`kata-monitor` (runs on Azure small instances)
-`cri-containerd` (runs on Azure small instances)
-`nydus` (runs on Azure small instances)
-`vfio` (runs on Azure normal instances)
- Tests depending on the generated kata-deploy payload
- kata-deploy (runs on Azure small instances)
- Tests are performed using different "Kubernetes flavors", such as k0s, k3s, rke2, and Azure Kubernetes Service (AKS).
- Kubernetes (runs in Azure small and medium instances depending on what's required by each test, and on TEE bare-metal machines)
- Tests are performed with different runtime engines, such as CRI-O and containerd.
- Tests are performed with different snapshotters for containerd, namely OverlayFS and devmapper.
- Tests are performed with all the supported hypervisors, which are Cloud Hypervisor, Dragonball, Firecracker, and QEMU.
For all the tests relying on Azure instances, real money is being spent, so the
community asks for the maintainers to be mindful about those, and avoid abusing
them to merely debug issues.
## The different runners
In the previous section we've mentioned using different runners, now in this section we'll go through each type of runner used.
- Cost free runners: Those are the runners provided by GIthub itself, and
those are fairly small machines with no virtualization capabilities enabled -
- Azure small instances: Those are runners which have virtualization
capabilities enabled, 2 CPUs, and 8GB of RAM. These runners have a "-smaller"
suffix to their name.
- Azure normal instances: Those are runners which have virtualization
capabilities enabled, 4 CPUs, and 16GB of RAM. These runners are usually
`garm` ones with no "-smaller" suffix.
- Bare-metal runners: Those are runners provided by community contributors,
and they may vary in architecture, size and virtualization capabilities.
Builder runners don't actually require any virtualization capabilities, while
runners which will be actually performing the tests must have virtualization
capabilities and a reasonable amount for CPU and RAM available (at least
matching the Azure normal instances).
## Adding new tests
Before someone decides to add a new test, we strongly recommend them to go
through [GitHub Actions Documentation][gh-actions],
which will provide you a very sensible background on how to read and understand
current tests we have, and also become familiar with how to write a new test.
On the Kata Containers land, there are basically two sets of tests: "standalone"
and "part of something bigger".
The "standalone" tests, for example the commit message check, won't be covered
here as they're better covered by the GitHub Actions documentation pasted above.
The "part of something bigger" is the more complicated one and not so
straightforward to add, so we'll be focusing our efforts on describing the
addition of those.
> [!NOTE]
> TODO: Currently, this document refers to "tests" when it actually means the
> jobs (or workflows) of GitHub. In an ideal world, except in some specific cases,
> new tests should be added without the need to add new workflows. In the
> not-too-distant future (hopefully), we will improve the workflows to support
> this.
### Adding a new test that's "part of something bigger"
The first important thing here is to align expectations, and we must say that
the community strongly prefers receiving tests that already come with:
- Instructions how to run them
- A proven run where it's passing
There are several ways to achieve those two requirements, and an example of that
can be seen in PR #8115.
With the expectations aligned, adding a test consists in:
- Adding a new yaml file for your test, and ensure it's called from the
"bigger" yaml. See the [Kata Monitor test example][monitor-ex01].
- Adding the helper scripts needed for your test to run. Again, use the [Kata Monitor script as example][monitor-ex02].
Following those examples, the community advice during the review, and even
asking the community directly on Slack are the best ways to get your test
accepted.
## Running tests
### Running the tests as part of the CI
If you're a maintainer of the project, you'll be able to kick in the tests by
yourself. With the current approach, you just need to add the `ok-to-test`
label and the tests will automatically start. We're moving, though, to use a
`/test` command as part of a GitHub review comment, which will simplify this
process.
If you're not a maintainer, please, send a message on Slack or wait till one of
the maintainers reviews your PR. Maintainers will then kick in the tests on
your behalf.
In case a test fails and there's the suspicion it happens due to flakiness in
the test itself, please, create an issue for us, and then re-run (or asks
maintainers to re-run) the tests following these steps:
- Locate which tests is failing
- Click in "details"
- In the top right corner, click in "Re-run jobs"
- And then in "Re-run failed jobs"
- And finally click in the green "Re-run jobs" button
> [!NOTE]
> TODO: We need figures here
### Running the tests locally
In this section, aligning expectations is also something very important, as one
will not be able to run the tests exactly in the same way the tests are running
in the CI, as one most likely won't have access to an Azure subscription.
However, we're trying our best here to provide you with instructions on how to
run the tests in an environment that's "close enough" and will help you to debug
issues you find with the current tests, or even provide a proof-of-concept to
the new test you're trying to add.
The basic steps, which we will cover in details down below are:
1. Create a VM matching the configuration of the target runner
2. Generate the artifacts you'll need for the test, or download them from a
current failed run
3. Follow the steps provided in the action itself to run the tests.
Although the general overview looks easy, we know that some tricks need to be
shared, and we'll go through the general process of debugging one non-Kubernetes
and one Kubernetes specific test for educational purposes.
One important thing to note is that "Create a VM" can be done in innumerable
different ways, using the tools of your choice. For the sake of simplicity on
this guide, we'll be using `kcli`, which we strongly recommend in case you're a
non-experienced user, and happen to be developing on a Linux box.
For both non-Kubernetes and Kubernetes cases, we'll be using PR #8070 as an
example, which at the time this document is being written serves us very well
the purpose, as you can see that we have `nerdctl` and Kubernetes tests failing.
## Debugging tests
### Debugging a non Kubernetes test
As shown above, the `nerdctl` test is failing.
As a developer you can go ahead to the details of the job, and expand the job
that's failing in order to gather more information.
But when that doesn't help, we need to set up our own environment to debug
what's going on.
Taking a look at the `nerdctl` test, which is located here, you can easily see
that it runs-on a `garm-ubuntu-2304-smaller` virtual machine.
The important parts to understand are `ubuntu-2304`, which is the OS where the
test is running on; and "smaller", which means we're running it on a machine
with 2 CPUs and 8GB of RAM.
With this information, we can go ahead and create a similar VM locally using `kcli`.
This document is written **specifically for developers**: it is not intended for end users.
If you want to contribute changes that you have made, please read the [community guidelines](https://github.com/kata-containers/community/blob/main/CONTRIBUTING.md) for information about our processes.
# Assumptions
- You are working on a non-critical test or development system.
@@ -13,11 +15,22 @@ The recommended way to create a development environment is to first
to create a working system.
The installation guide instructions will install all required Kata Containers
components, plus *Docker*, the hypervisor, and the Kata Containers image and
guest kernel.
components, plus a container manager, the hypervisor, and the Kata
On `ppc64le` and `s390x`, `glibc` is used. You will need to install the `libseccomp` library
@@ -435,7 +450,7 @@ You can build and install the guest kernel image as shown [here](../tools/packag
# Install a hypervisor
When setting up Kata using a [packaged installation method](install/README.md#installing-on-a-linux-system), the
`QEMU` VMM is installed automatically. Cloud-Hypervisor and Firecracker VMMs are available from the [release tarballs](https://github.com/kata-containers/kata-containers/releases), as well as through [`kata-deploy`](../tools/packaging/kata-deploy/README.md).
`QEMU` VMM is installed automatically. Cloud-Hypervisor, Firecracker and StratoVirt VMMs are available from the [release tarballs](https://github.com/kata-containers/kata-containers/releases), as well as through [`kata-deploy`](../tools/packaging/kata-deploy/README.md).
You may choose to manually build your VMM/hypervisor.
## Build a custom QEMU
@@ -585,10 +600,15 @@ $ sudo kata-monitor
#### Connect to debug console
Command `kata-runtime exec` is used to connect to the debug console.
You need to start a container for example:
```bash
$ sudo ctr run --runtime io.containerd.kata.v2 -d docker.io/library/ubuntu:latest testdebug
```
Then, you can use the command `kata-runtime exec <sandbox id>` to connect to the debug console.
This document lists the tasks required to create a Kata Release.
This document lists the tasks required to create a Kata Release.
## Requirements
-[hub](https://github.com/github/hub)
* Using an [application token](https://github.com/settings/tokens) is required for hub (set to a GITHUB_TOKEN environment variable).
-GitHub permissions to run workflows.
- GitHub permissions to push tags and create releases in Kata repositories.
## Versioning
- GPG configured to sign git tags. https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key
The Kata Containers project uses [semantic versioning](http://semver.org/) for all releases.
Semantic versions are comprised of three fields in the form:
- You should configure your GitHub to use your ssh keys (to push to branches). See https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/.
* As an alternative, configure hub to push and fork with HTTPS, `git config --global hub.protocol https` (Not tested yet) *
```
MAJOR.MINOR.PATCH
```
When `MINOR` increases, the new release adds **new features** but *without changing the existing behavior*.
When `MAJOR` increases, the new release adds **new features, bug fixes, or
both** and which **changes the behavior from the previous release** (incompatible with previous releases).
A major release will also likely require a change of the container manager version used,
-for example Containerd or CRI-O. Please refer to the release notes for further details.
**Important** : the Kata Containers project doesn't have stable branches (see
[this issue](https://github.com/kata-containers/kata-containers/issues/9064) for details).
Bug fixes are released as part of `MINOR` or `MAJOR` releases only. `PATCH` is always `0`.
## Release Process
### Bump the `VERSION` file
### Bump all Kata repositories
When the `kata-containers/kata-containers` repository is ready for a new release,
first create a PR to set the release in the `VERSION` file and have it merged.
Bump the repositories using a script in the Kata packaging repo, where:
-`BRANCH=<the-branch-you-want-to-bump>`
-`NEW_VERSION=<the-new-kata-version>`
```
$ cd ${GOPATH}/src/github.com/kata-containers/kata-containers/tools/packaging/release
See the commits in [the corresponding PR for stable-2.1](https://github.com/kata-containers/tests/pull/3504) for an example of the changes.
If the workflow fails because of some external environmental causes, e.g. network
timeout, simply re-run the failed jobs until they eventually succeed.
If for some reason you need to cancel the workflow or re-run it entirely, go first
to the [Release page](https://github.com/kata-containers/kata-containers/releases) and
delete the draft release from the previous run.
### Merge all bump version Pull requests
### Improve the release notes
- The above step will create a GitHub pull request in the Kata projects. Trigger the CI using `/test` command on each bump Pull request.
- Trigger the `test-kata-deploy` workflow which is under the `Actions` tab on the repository GitHub page (make sure to select the correct branch and validate it passes).
- Check any failures and fix if needed.
- Work with the Kata approvers to verify that the CI works and the pull requests are merged.
Release notes are auto-generated by the GitHub CLI tool used as part of our
release workflow. However, some manual tweaking may still be necessary in
order to highlight the most important features and bug fixes in a specific
release.
### Tag all Kata repositories
Once all the pull requests to bump versions in all Kata repositories are merged,
tag all the repositories as shown below.
```
$ cd ${GOPATH}/src/github.com/kata-containers/kata-containers/tools/packaging/release
$ git checkout <kata-branch-to-release>
$ git pull
$ ./tag_repos.sh -p -b "$BRANCH" tag
```
### Check Git-hub Actions
We make use of [GitHub actions](https://github.com/features/actions) in this [file](../.github/workflows/release.yaml) in the `kata-containers/kata-containers` repository to build and upload release artifacts. This action is auto triggered with the above step when a new tag is pushed to the `kata-containers/kata-containers` repository.
Check the [actions status page](https://github.com/kata-containers/kata-containers/actions) to verify all steps in the actions workflow have completed successfully. On success, a static tarball containing Kata release artifacts will be uploaded to the [Release page](https://github.com/kata-containers/kata-containers/releases).
### Create release notes
We have a script in place in the packaging repository to create release notes that include a short-log of the commits across Kata components.
Run the script as shown below:
```
$ cd ${GOPATH}/src/github.com/kata-containers/kata-containers/tools/packaging/release
# Note: OLD_VERSION is where the script should start to get changes.
Note, the stable-2.2 branch will still exist with tag 2.2.1, but under current plans it is
not maintained further. The next tag applied to main will be 2.4.0-alpha0. We would then
create a couple of alpha releases gathering features targeted for that particular release (in
this case 2.4.0), followed by a release candidate. The release candidate marks a feature freeze.
A new stable branch is created for the release candidate. Only bug fixes and any security issues
are added to the branch going forward until release 2.4.0 is made.
## Backporting Process
Development that occurs against the main branch and applicable code commits should also be submitted
against the stable branches. Some guidelines for this process follow::
1. Only bug and security fixes which do not introduce inter-component dependencies are
candidates for stable branches. These PRs should be marked with "bug" in GitHub.
2. Once a PR is created against main which meets requirement of (1), a comparable one
should also be submitted against the stable branches. It is the responsibility of the submitter
to apply their pull request against stable, and it is the responsibility of the
reviewers to help identify stable-candidate pull requests.
## Continuous Integration Testing
The test repository is forked to create stable branches from main. Full CI
runs on each stable and main PR using its respective tests repository branch.
### An alternative method for CI testing:
Ideally, the continuous integration infrastructure will run the same test suite on both main
and the stable branches. When tests are modified or new feature tests are introduced, explicit
logic should exist within the testing CI to make sure only applicable tests are executed against
stable and main. While this is not in place currently, it should be considered in the long term.
## Release Management
### Patch releases
Releases are made every four weeks, which include a GitHub release as
well as binary packages. These patch releases are made for both stable branches, and a "release candidate"
for the next `MAJOR` or `MINOR` is created from main. If there are no changes across all the repositories, no
release is created and an announcement is made on the developer mailing list to highlight this.
If a release is being made, each repository is tagged for this release, regardless
of whether changes are introduced. The release schedule can be seen on the
[release rotation wiki page](https://github.com/kata-containers/community/wiki/Release-Team-Rota).
If there is urgent need for a fix, a patch release will be made outside of the planned schedule.
The process followed for making a release can be found at [Release Process](Release-Process.md).
## Minor releases
### Frequency
Minor releases are less frequent in order to provide a more stable baseline for users. They are currently
running on a sixteen weeks cadence. The release schedule can be seen on the
[release rotation wiki page](https://github.com/kata-containers/community/wiki/Release-Team-Rota).
### Compatibility
Kata guarantees compatibility between components that are within one minor release of each other.
This is critical for dependencies which cross between host (shimv2 runtime) and
the guest (hypervisor, rootfs and agent). For example, consider a cluster with a long-running
deployment, workload-never-dies, all on Kata version 2.1.3 components. If the operator updates
the Kata components to the next new minor release (i.e. 2.2.0), we need to guarantee that the 2.2.0
shimv2 runtime still communicates with 2.1.3 agent within workload-never-dies.
Handling live-update is out of the scope of this document. See this [`kata-runtime` issue](https://github.com/kata-containers/runtime/issues/492) for details.
During container's lifecycle, different Hooks can be executed to do custom actions. In Kata Containers, we support two types of Hooks, `OCI Hooks` and `Kata Hooks`.
### OCI Hooks
The OCI Spec stipulates six hooks that can be executed at different time points and namespaces, including `Prestart Hooks`, `CreateRuntime Hooks`, `CreateContainer Hooks`, `StartContainer Hooks`, `Poststart Hooks` and `Poststop Hooks`. We support these types of Hooks as compatible as possible in Kata Containers.
The path and arguments of these hooks will be passed to Kata for execution via `bundle/config.json`. For example:
```
...
"hooks": {
"prestart": [
{
"path": "/usr/bin/prestart-hook",
"args": ["prestart-hook", "arg1", "arg2"],
"env": [ "key1=value1"]
}
],
"createRuntime": [
{
"path": "/usr/bin/createRuntime-hook",
"args": ["createRuntime-hook", "arg1", "arg2"],
"env": [ "key1=value1"]
}
]
}
...
```
### Kata Hooks
In Kata, we support another three kinds of hooks executed in guest VM, including `Guest Prestart Hook`, `Guest Poststart Hook`, `Guest Poststop Hook`.
The executable files for Kata Hooks must be packaged in the *guest rootfs*. The file path to those guest hooks should be specified in the configuration file, and guest hooks must be stored in a subdirectory of `guest_hook_path` according to their hook type. For example:
+ In configuration file:
```
guest_hook_path="/usr/share/hooks"
```
+ In guest rootfs, prestart-hook is stored in `/usr/share/hooks/prestart/prestart-hook`.
## Execution
The table below summarized when and where those different hooks will be executed in Kata Containers:
| Hook Name | Hook Type | Hook Path | Exec Place | Exec Time |
|---|---|---|---|---|
| `Prestart(deprecated)` | OCI hook | host runtime namespace | host runtime namespace | After VM is started, before container is created. |
| `CreateRuntime` | OCI hook | host runtime namespace | host runtime namespace | After VM is started, before container is created, after `Prestart` hooks. |
| `CreateContainer` | OCI hook | host runtime namespace | host vmm namespace* | After VM is started, before container is created, after `CreateRuntime` hooks. |
| `StartContainer` | OCI hook | guest container namespace | guest container namespace | After container is created, before container is started. |
| `Poststart` | OCI hook | host runtime namespace | host runtime namespace | After container is started, before start operation returns. |
| `Poststop` | OCI hook | host runtime namespace | host runtime namespace | After container is deleted, before delete operation returns. |
| `Guest Prestart` | Kata hook | guest agent namespace | guest agent namespace | During start operation, before container command is executed. |
| `Guest Poststart` | Kata hook | guest agent namespace | guest agent namespace | During start operation, after container command is executed, before start operation returns. |
| `Guest Poststop` | Kata hook | guest agent namespace | guest agent namespace | During delete operation, after container is deleted, before delete operation returns. |
+ `Hook Path` specifies where hook's path be resolved.
+ `Exec Place` specifies in which namespace those hooks can be executed.
+ For `CreateContainer` Hooks, OCI requires to run them inside the container namespace while the hook executable path is in the host runtime, which is a non-starter for VM-based containers. So we design to keep them running in the *host vmm namespace.*
+ `Exec Time` specifies at which time point those hooks can be executed.
To safeguard the integrity of container images and prevent tampering from the host side, we propose guest image management. This method, employed for Confidential Containers, ensures container images remain unaltered and secure.
## Introduction to remote snapshot
Containerd 1.7 introduced `remote snapshotter` feature which is the foundation for pulling images in the guest for Confidential Containers.
While it's beyond the scope of this document to fully explain how the container rootfs is created to the point it can be executed, a fundamental grasp of the snapshot concept is essential. Putting it in a simple way, containerd fetches the image layers from an OCI registry into its local content storage. However, they cannot be mounted as is (e.g. the layer can be tar+gzip compressed) as well as they should be immutable so the content can be shared among containers. Thus containerd leverages snapshots of those layers to build the container's rootfs.
The role of `remote snapshotter` is to reuse snapshots that are stored in a remotely shared place, thus enabling containerd to prepare the container’s rootfs in a manner similar to that of a local `snapshotter`. The key behavior that makes this the building block of Kata's guest image management for Confidential Containers is that containerd will not pull the image layers from registry, instead it assumes that `remote snapshotter` and/or an external entity will perform that operation on his behalf.
Maybe the simplest example of `remote snapshotter` in Confidential Containers is the pulling of images in the guest VM. Once ensuring the VM is part of a Trusted Computing Base (TCB) and throughout a chain of delegations involving containerd, `remote snapshotter` and kata-runtime, it is possible for the kata-agent to pull the image directly.
## `Remote snapshotter` implementations
`Remote snapshotter` is containerd plug-ins that should implement the [Snapshot Interface](https://pkg.go.dev/github.com/containerd/containerd/v2/snapshots?utm_source=godoc#Snapshotter).
The following `remote snapshotter` is leveraged by Kata Containers:
-`nydus snapshotter`
### `Nydus snapshotter`
This `snapshotter` is implemented as an external containerd proxy plug-in for [`Nydus`](https://nydus.dev/).
Currently it supports a couple of runtime backend, notably, `FUSE`, `virtiofs`, and `EROFS`, being the former leveraged on the tested Kata Containers CI.
## `Nydus snapshotter` and containerd-shim-v2 integration
```go
// KataVirtualVolume encapsulates information for extra mount options and direct volumes.
The following sequence diagram depicted below offers a detailed overview of the messages/calls exchanged to pull an unencrypted unsigned image from an unauthenticated container registry. This involves the kata-runtime, kata-agent, and the guest-components’ image-rs to use the guest pull mechanism.
First and foremost, the guest pull code path is only activated when `nydus snapshotter` requires the handling of a volume which type is `image_guest_pull`, as can be seen on the message below:
In other words, `VolumeType` of `KataVirtualVolumeType` is set to `image_guest_pull`.
Next the `handleImageGuestPullBlockVolume()` is called to build the Storage object that will be attached to the message later sent to kata-agent via the `CreateContainerRequest()` RPC. It is in the `handleImageGuestPullBlockVolume()` that it will begin the handling of the pause image if the request is for a sandbox container type (see more about pause image below).
Below is an example of storage information packaged in the message sent to the kata-agent:
Next, the kata-agent's RPC module will handle the create container request which, among other things, involves adding storages to the sandbox. The storage module contains implementations of `StorageHandler` interface for various storage types, being the `ImagePullHandler` in charge of handling the storage object for the container image (the storage manager instantiates the handler based on the value of the "driver").
`ImagePullHandler` delegates the image pulling operation to the `ImageService.pull_image()` that is going to create the image's bundle directory on the guest filesystem and, in turn, class the image-rs to in fact fetch and uncompress the image's bundle.
> **Notes:**
> In this flow, `ImageService.pull_image()` parses the image metadata, looking for either the `io.kubernetes.cri.container-type: sandbox` or `io.kubernetes.cri-o.ContainerType: sandbox` (CRI-IO case) annotation, then it never calls the `image-rs.pull_image()` because the pause image is expected to already be inside the guest's filesystem, so instead `ImageService.unpack_pause_image()` is called.
References:
[1] [[RFC] Image management proposal for hosting sharing and peer pods](https://github.com/confidential-containers/confidential-containers/issues/137)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
