Before this patch there was a mismatch between the JSON path under which
the state of the rule evaluation is set in comparison to under which
it is retrieved.
This resulted in the behavior that each time the policy was evaluated,
it thought it was the _first_ time the policy was evaluated.
This also means that the consistency check for the `sandbox_name`
was ineffective.
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
With full debug logging enabled there might be around 1,500 redials
so log just ~15 of these redials to avoid flooding the log.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This PR ensures that OCICRYPT_CONFIG_PATH file is initialized only
when CDH socket exists. This prevents startup error if attestation
binaries are not installed in PodVM.
Fixes: https://github.com/kata-containers/kata-containers/issues/10568
Signed-off-by: Silenio Quarti <silenio_quarti@ca.ibm.com>
A new attribute named `devno` is added to DeviceVirtioScsi.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVhostUserFs.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to VhostVsock.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVirtioSerial.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
A new attribute named `devno` is added to DeviceVirtioBlk.
It will be used to specify a device number for a CCW bus type.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
To explicitly specify a device number on the QEMU command line
for the following devices using the CCW transport on s390x:
- SerialDevice
- BlockDevice
- VhostUserDevice
- SCSIController
- VSOCKDevice
this commit introduces a new structure CcwSubChannel and implements
the following methods:
- add_device()
- remove_device()
- address_format_ccw()
- set_addr()
You can see the detailed explanation for each method in the comment.
This resolves the 1st part of #10573.
Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
We need to clean-up any created files/dirs otherwise
we cause problems on self-hosted runners. Using tempdir which
will be removed automatically.
Signed-off-by: Zvonko Kaiser <zkaiser@nvidia.com>
Use regorous engine's add_data method to add state to the policy.
This data can later be accessed inside rego context through the data namespace.
Support state modifications (json-patches) that may be returned as a result from policy evaluation.
Also initialize a policy engine data slice "pstate" dedicated for storing state.
Fixes#10087
Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
This reverts commit f15e16b692, as we
don't have to do this since we're relying on the
`static_sandbox_resource_mgmt` feature, which gives us the correct
amount of memory and CPUs to be allocated.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
The qemu-coco-dev runtime class should be as close as possible to what
the TEEs runtime classes are doing, and this was one of the options that
ended up overlooked till now.
Shout out to Dan Mihai for noticing that!
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Most likely this was overlooked during the development / review, but
we're actually interested on the size rather than on the pagesize of the
hugepages.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This change is motivated by a problem in peerpod's podvms. In this setup
the lifecycle of guest components is managed by systemd. The current code
skips over init steps like setting the ocicrypt-rs env and initialization
of a CDH client in this case.
To address this the launch of the processes has been isolated into its
own fn.
Signed-off-by: Magnus Kulke <magnuskulke@microsoft.com>
This commit introduces changes to enable testing kata-agent's container
APIs of CreateContainer/StartContainer/RemoveContainer. The changeset
include:
- using confidential-containers image-rs crate to pull/unpack/mount a
container image. Currently supports only un-authenicated registry pull
- re-factor api handlers to reduce cmdline complexity and handle
request generation logic in tool
- introduce an OCI config template for container creation
- add test case
Fixes#9707
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
The current version of the oci-spec crate compiles RLimit structs only
for Linux and Solaris. Until this is fixed upstream, add compilation
conditions to the type converters for the affected structs.
Fixes: #10071
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The parse_json_string function is specific to parsing capability strings
out of ttRPC proto definitions and does not benefit from being available
to other crates. Moving it into the protocols crate allows removing
kata-sys-util as a dependency, which in turn enables compiling the
library on darwin.
Fixes: #10071
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The cgroups.rs source file was removed in
234d7bca04. With cgroups support handled
in runtime-rs, the cgroups dependency on kata-sys-util can be removed.
Signed-off-by: Markus Rudy <mr@edgeless.systems>
The cloud-api-adaptor needs to support different types of pod VM
instance.
We needs to pass some annotations like machine_type, default_vcpus and
default_memory to prepare the VMs.
Signed-off-by: Chasing1020 <643601464@qq.com>
This patch adds the support of the remote hypervisor type for runtime-rs.
The cloud-api-adaptor needs the annotations and network namespace path
to create the VMs.
The remote hypervisor opens a UNIX domain socket specified in the config
file, and sends ttrpc requests to a external process to control sandbox
VMs.
Fixes: #10350
Signed-off-by: Chasing1020 <643601464@qq.com>
Add GPU annotations for remote hypervisor to help
with the right instance selection based on number of GPUs
and model
Signed-off-by: Pradipta Banerjee <pradipta.banerjee@gmail.com>
We may decide to add this later on, but for now this is only targetting
TEEs and the confidential image / initrd.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Return of proper error to the initiator is not guaranteed.
Method StopVM could kill shim process together with VM pieces.
Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com>
Corrects device filemode permissions typo/regression in rustjail to `666` instead of `066`.
`666` is the standard and expected value for these devices in containers.
Fixes: #10454
Signed-off-by: Simon Kaegi <simon.kaegi@gmail.com>
When QEMU is terminated by signal 15, it deletes the PidFile.
Upon detecting that QEMU has exited, the shim executes the stopVM function.
If the PidFile is not found, the PID is set to 0.
Subsequently, the shim executes `kill -9 0`, which terminates the current process group.
This prevents any further logic from being executed, resulting in resources not being cleaned up.
Signed-off-by: wangyaqi54 <wangyaqi54@jd.com>
Semantics are lifted straight out of the go runtime for compatibility.
We introduce DeviceVirtioScsi to represent a virtio-scsi device and
instantiate it if block device driver in the configuration file is set
to virtio-scsi. We also introduce ObjectIoThread which is instantiated
if the configuration file additionally enables iothreads.
Signed-off-by: Pavel Mores <pmores@redhat.com>
When do update_container_namespaces updating namespaces, setting
all UTS(and IPC) namespace paths to None resulted in hostnames
set prior to the update becoming ineffective. This was primarily
due to an error made while aligning with the oci spec: in an attempt
to match empty strings with None values in oci-spec-rs, all paths
were incorrectly set to None.
Fixes#10325
Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
Use rstest for unit test rather than TestData arrays where
possible to make the code more compact, easier to read
and open the possibility to enhance test cases with a
description more easily.
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
While Kubernetes defines `binaryData` as `[]byte`,
when defined in a YAML file the raw bytes are
base64 encoded. Therefore, we need to read the YAML
value as `String` and not as `Vec<u8>`.
Fixes: #10410
Signed-off-by: Leonard Cohnen <lc@edgeless.systems>
The root cause is that the CDH client is a global variable, and unit tests `test_unseal_env` and `test_unseal_file`
share this lock-free global variable, leading to resource contention and destruction.
Merging the two unit tests into one test_sealed_secret will resolve this issue.
Fixes: #10403
Signed-off-by: ChengyuZhu6 <zhucy0405@gmail.com>
This reverts commit 31e09058af, as it's
breaking the agent unit tests CI.
This is a stop gap till Chengyu Zhu finds the time to properly address
the issue, avoiding the CI to be blocked for now.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
Refactor CDH-related operations into the cdh_handler function to make the `create_container` code clearer.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Users must set the mount path to `/sealed/<path>` for kata agent to detect the sealed secret mount
and handle it in createcontainer stage.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
Introduced `unseal_file` function to unseal secret as files:
- Implemented logic to handle symlinks and regular files within the sealed secret directory.
- For each entry, call CDH to unseal secrets and the unsealed contents are written to a new file, and a symlink is created to replace the sealed symlink.
Fixes: #8123
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Linda Yu <linda.yu@intel.com>
add_device() now checks if QEMU is running already by checking if we have
a QMP connection. If we do a new function hotplug_device() is called
which hotplugs the device if it's a network one.
Signed-off-by: Pavel Mores <pmores@redhat.com>
With the helpers from previous commit, the actual hotplugging
implementation, though lengthy, is mostly just assembling a QMP command
to hotplug the network device backend and then doing the same for the
corresponding frontend.
Note that hotplug_network_device() takes cmdline_generator types Netdev
and DeviceVirtioNet. This is intentional and aims to take advantage of
the similarity between parameter sets needed to coldplug and hotplug
devices reuse and simplify our code. To enable using the types from qmp,
accessors were added as needed.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Before adding network device hotplugging functionality itself we add
a couple of helpers in a separate commit since their functionality is
non-trivial.
To hotplug a device we need a free PCI slot. We add find_free_slot()
which can be called to obtain one. It looks for PCI bridges connected
to the root bridge and looks for an unoccupied slot on each of them. The
first found is returned to the caller. The algorithm explicitly doesn't
support any more complex bridge hierarchies since those are never produced
when coldplugging PCI bridges.
Sending netdev queue and vhost file descriptors to QEMU is slightly
involved and implemented in pass_fd(). The actual socket has to be passed
in an SCM_RIGHTS socket control message (also called ancillary data, see
man 3 cmsg) so we have to use the msghdr structure and sendmsg() call
(see man 2 sendmsg) to send the message. Since qapi-rs doesn't support
sending messages with ancillary data we have to do the sending sort of
"under it", manually, by retrieving qapi-rs's socket and using it directly.
Signed-off-by: Pavel Mores <pmores@redhat.com>
NetworkConfig::index has been used to generate an id for a network device
backend. However, it turns out that it's not unique (it's always zero
as confirmed by a comment at its definition) so it's not suitable to
generate an id that needs to be unique.
Use the host device name instead.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Network device hotplugging will use the same infrastructure (Netdev,
DeviceVirtioNet) as coldplugging, i.e. QemuCmdLine. To make the code
of network device setup visible outside of QemuCmdLine we factor it out
to a non-member function `get_network_device()` and make QemuCmdLine just
delegate to it.
Signed-off-by: Pavel Mores <pmores@redhat.com>
The function takes a whole QemuCmdLine but only actually uses
HypervisorConfig. We increase callability of the function by limiting
its interface to what it needs. This will come handy shortly.
Signed-off-by: Pavel Mores <pmores@redhat.com>
At least one PCI bridge is necessary to hotplug PCI devices. We only
support PCI (at this point at least) since that's what the go runtime
does (note that looking at the code in virtcontainers it might seem that
other bus types are supported, however when the bridge objects are passed
to govmm, all but PCI bridges are actually ignored). The entire logic of
bridge setup is lifted from runtime-go for compatibility's sake.
Signed-off-by: Pavel Mores <pmores@redhat.com>
Inorder to support sandbox api, intorduce the sandbox_config
struct and split the sandbox start stage from init process.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Docker cannot exit normally after the container process exits when
used with runtime-rs since it doesn't receive the exit event. This
commit enable runtime-rs to send TaskExit to containerd after process
exits.
Also, it moves "system_time_into" and "option_system_time_into" from
crates/runtimes/common/src/types/trans_into_shim.rs to a new utility
mod.
Signed-off-by: Sicheng Liu <lsc2001@outlook.com>
When the sandbox api was enabled, the pasue container
wouldn't be created, thus the shared sandbox pidns
should be fallbacked to the first container's init process,
instead of return any error here.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
When using network adapters that support SR-IOV, a VFIO device can be
plugged into a guest VM and claimed as a network interface. This can
significantly enhance network performance.
Fixes: #9758
Signed-off-by: Lei Huang <leih@nvidia.com>
rename the task_service to service, in order to
incopperate with the following added sandbox
services.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
In order to make different from sandbox request/response, this commit
changed the task request/response to TaskRequest/TaskResponse.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the wait_vm would be called before calling stop_vm,
which would take the reader lock, thus blocking the stop_vm
getting the writer lock, which would trigge the dead lock.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
Since the block_on would block on the current thread
which would prevent other async tasks to be run on this
worker thread, thus change it to use the async task for
this task.
Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
This PR introduces support for selectively compiling Dragonball in
runtime-rs. By default, Dragonball will continue to be compiled into
the containerd-shim-kata-v2 executable, but users now have the option
to disable Dragonball compilation.
Fixes#10310
Signed-off-by: sidney chang <2190206983@qq.com>
Add cdi devices including ContainerDevice definition and
annotation_container_device method to annotate vfio device
in OCI Spec annotations which is inserted into Guest with
its mapping of vendor-class and guest pci path.
Fixes#10145
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
We need vfio device's properties device, vendor and
class, but we can only get property device and vendor.
just extend it with class is ok.
Fixes#10145
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
As we don't have any CI, nor maintainer to keep ACRN code around, we
better have it removed than give users the expectation that it should or
would work at some point.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
This change technically affects the path for enabled guest selinux as well,
however since this is not implemented in runtime-rs anyway nothing should
break. When guest selinux support is added this change will come handy.
Signed-off-by: Pavel Mores <pmores@redhat.com>
If guest selinux is off the runtime has to ensure that container OCI spec
contains no selinux labels for the container rootfs and process. Failure
to do so causes kata agent to try and apply the labels which fails since
selinux is not enabled in guest, which in turn causes container launch
to fail.
This is largely inspired by golang runtime(*) with a slight deviation
in ordering of checks. This change simply checks the disable_guest_selinux
config setting and if it's true it clears both rootfs and process label if
necessary. Golang runtime, on the other hand, seems to first check if
process label is non-empty and only then it checks the config setting,
meaning that if process label is empty the rootfs label is not reset
even if it's non-empty. Frankly, this looks like a potential bug though
probably unlikely to manifest since it can be assumed that the labels are
either both empty, or both non-empty.
(*) 4fd4b02f2e/src/runtime/virtcontainers/kata_agent.go (L1005)
Signed-off-by: Pavel Mores <pmores@redhat.com>
In order to handle the setting we have to first parse it and make its
value available to the rest of the program.
The yes() function is added to comply with serde which seems to insist
on default values being returned from functions. Long term, this is
surely not the best place for this function to live, however given that
this is currently the first and only place where it's used it seems
appropriate to put it near its use. If it ends up being reused elsewhere
a better place will surely emerge.
Signed-off-by: Pavel Mores <pmores@redhat.com>
kata-shim was not reporting `inactive_file` in memory stat.
This memory is deducted by containerd when calculating the size of container working set, as it can be paged out by the operating
system under memory pressure. Without reporting `inactive_file`, containerd will over report container memory usage.
[Here](https://github.com/containerd/containerd/blob/v1.7.22/pkg/cri/server/container_stats_list_linux.go#L117) is where containerd
deducts `inactive_file` from memory usage.
Note that kata-shim correctly reports `total_inactive_file` for cgroup v1, but this was not implemented for cgroup v2.
This commit:
- Adds code in kata-shim to report "inactive_file" memory for cgroup v2
- Implements reporting of all available cgroup v2 memory stats to containerd
- Uses defensive coding to avoid assuming existence of any memory.stat fields
The list of available cgroup v2 memory stats defined by containerd can be found
[here](https://pkg.go.dev/github.com/containerd/cgroups/v2/stats#MemoryStat).
Fixes#10280
Signed-off-by: Alex Man <alexman@stripe.com>
This patch adds support to call kata agents SetPolicy
API. Also adds tests for SetPolicy API using agent-ctl.
Fixes#9711
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
agent built with policy feature initializes the policy engine using a
policy document from a default path, which is installed & linked during
UVM rootfs build. This commit adds support to provide a default agent
policy as environment variable.
This targets development/testing scenarios where kata-agent
is wanted to be started as a local process.
Fixes#10301
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
Add two parameters for enabling cosign signature image verification.
- `enable_signature_verification`: to activate signature verification
- `image_policy`: URI of the image policy
config
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
new version of the anyhow crate has changed the backtrace capture thus
unit tests of kata-agent that compares a raised error with an expected
one would fail. To fix this, we need only panics to have backtraces,
thus set `RUST_BACKTRACE=1` and `RUST_LIB_BACKTRACE=0` for tests due to
document
https://docs.rs/anyhow/latest/anyhow/
Signed-off-by: Xynnn007 <xynnn@linux.alibaba.com>
It's a prerequisite PR to make built-in vmm dragonball compilation
options configurable.
Extract TAP device-related code from dragonball's dbs_utils into a
separate library within the runtime-rs hypervisor module.
To enhance functionality and reduce dependencies, the extracted code
has been reimplemented using the libc crate and the ifreq structure.
Fixes#10182
Signed-off-by: sidney chang <2190206983@qq.com>
Remove the recently added default UID/GID values, because the genpolicy
design is to initialize those fields before this new code path gets
executed.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Some container images are configured such that the user (and group)
under which their entrypoint should run is not a number (or pair of
numbers), but a user name.
For example, in a Dockerfile, one might write:
> USER 185
indicating that the entrypoint should run under UID=185.
Some images, however, might have:
> RUN groupadd --system --gid=185 spark
> RUN useradd --system --uid=185 --gid=spark spark
> ...
> USER spark
indicating that the UID:GID pair should be resolved at runtime via
/etc/passwd.
To handle such images correctly, read through all /etc/passwd files in
all layers, find the latest version of it (i.e., the top-most layer with
such a file), and, in so doing, ensure that whiteouts of this file are
respected (i.e., if one layer adds the file and some subsequent layer
removes it, don't use it).
Signed-off-by: Hernan Gatta <hernan.gatta@opaque.co>
Disabling the UID Policy rule was a workaround for #9928. Re-enable
that rule here and add a new test/CI temporary workaround for this
issue. This new test workaround will be removed after fixing #9928.
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
kata-agent incorrectly reports CPU time for cgroup v2, causing 1000x underreporting.
For cgroup v2, kata-agent reads the cpu.stat file, which reports the time consumed by the processes in the cgroup in µs.
However, there was a bug in kata-agent where it returned this value in µs without converting it to ns.
This commit adds the necessary µs to ns conversion for cgroup v2, aligning it with v1 behavior and kata-shim's expectations.
This fixes#10278
Signed-off-by: Alex Man <alexman@stripe.com>
ITA stands for Intel Trust Authority, which is in the process to being
renamed to ITTS (Intel Tiber Trust Services).
As we've bumped guest-components on trustee, let's make sure we also
bump image-rs to the commit that brings ITA support in:
* 1db6c3a876
The reason we need to bump the dependency here is to avoid kbs_protocol
mismatch between the version used by the agent and the trustee one.
Signed-off-by: Fabiano Fidêncio <fabiano@fidencio.org>
In the existing implementation for the CopyFile subcommand,
- cmd line argument list is too long, including various metadata information.
- in case of a regular file, passing the actual data as bytes stream adds to the size and complexity of the input.
- the copy request will fail when the file size exceeds that of the allowed ttrpc max data length limit of 4Mb.
This change refactors the CopyFile handler and modifies the input to a known 'source' 'destination' syntax.
Fixes#9708
Signed-off-by: Sumedh Alok Sharma <sumsharma@microsoft.com>
It will panic when users do GPU vfio passthrough with cdi in runtime.
The root cause is that CustomSpec.Annotations is nil when new element
added.
To address this issue, initialization is introduced when it's nil.
Fixes#10266
Signed-off-by: Alex Lyn <alex.lyn@antgroup.com>
Move device code with nvdimm driver to nvdimm_device_handler, including
nvdimm device and pmem device.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Removed the `StorageHandlerManager` struct and its associated implementations and
introduced a type alias `StorageHandlerManager` for `HandlerManager` to simplify the code.
The new type alias maintains the same functionality while reducing redundancy.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Introduced `HandlerManager` struct to manage registered handlers, which will be used to storage and device management for kata-agent.
Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
