Commit Graph

12582 Commits

Author SHA1 Message Date
Fabiano Fidêncio
8332f3c684
kata-deploy: Fix the snapshotter config placement
In the way the script is without this patch, we're trying to set
```toml
[`$shim`]
snapshotter = $snapshotter
```

However, what we actually want to set is the full runtime table instead
of shim.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-26 08:26:38 -03:00
Fabiano Fidêncio
907f1ddb9e
kata-deploy: Fix shim check for snapshotter configuration
We want to check whether the shim is part of the "plain text" shims
passed to the daemonset (meaning, checking against `$SHIMS`).  Before
this fix we were checking against `$shims`, which is an array of shims
instead of a string, resulting on a broken check.

Fixes: #8732

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-26 07:42:36 -03:00
Tim Zhang
a4ad12a3d1
Merge pull request #8729 from liubin/fix/package-kata-monitor
kata-monitor: fix Dockerfile to build image
2023-12-26 18:30:15 +08:00
alex.lyn
3b317e69e2 runtime-rs: add README and user guide to deploy directvol CSI Driver
Fixes: #8602

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-26 18:00:35 +08:00
Bin Liu
23eb3042c7 kata-monitor: fix Dockerfile to build image
move `SKIP_GO_VERSION_CHECK` after `make` command to skip
checking golang version.

And also upgrade golang to 1.19.

Fixes: #8728

Signed-off-by: Bin Liu <bin@hyper.sh>
2023-12-26 15:11:13 +08:00
Xuewei Niu
1065ca6fa7
Merge pull request #8626 from justxuewei/vhost-user-endpoint 2023-12-26 12:52:21 +08:00
Xuewei Niu
36a4cbccf6 runtime-rs: Expand all DeviceType in match arms
The compiler will give a warning if a developer forget to add an arm for
a new variants defined.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00
Xuewei Niu
f2d08bc00f runtime-rs: Remove unused index from Endpoints
The affected `Endpoint`s are `VhostUserEndpoint` and `TapEndpoint`.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00
Xuewei Niu
60a42351e2 runtime-rs: DAN supports vhost-user-net device
DAN reads vhost-user-net device from JSON config. It only supports VMM
running as server right now.

Fixes: #8625

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00
Xuewei Niu
693a0cfbfd dragonball: Make vhost-user-net ready for VhostUserEndpoint
The changes involve:

- Expose VhostUserConfig struct to runtime-rs.
- Set a default value while num_queues or queue_size are 0.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:59 +08:00
Xuewei Niu
54df832407 runtime-rs: Support VhostUserEndpoint
This commit introduces VhostUserEndpoint and supports relative to
vhost-user-net devices for device manager. For now, Dragonball is able to
attach vhost-user-net devices.

Fixes: #8625

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-26 10:18:50 +08:00
Xuewei Niu
374c2f01aa runtime-rs: Simplify VhostUserType enum
Remove unused string parameter from each item.

Fixes: #8625

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-25 16:15:57 +08:00
Xuewei Niu
38eb4077a6
Merge pull request #8503 from justxuewei/vhost-user-net
dragonball: Support vhost-user-net device
2023-12-25 13:47:51 +08:00
Xuewei Niu
4c5de72863 dragonball: Wrap config space into set_config_space
Config space of network device is shared and accord with virtio 1.1 spec.
It is a good way to abstract the common part into one function.
`set_config_space()` implements this.

Plus, this patch removes `vq_pairs` from vhost-net devices, since there is
a possibility of data inconsistency. For example, some places read that
from `self.vq_pairs`, others read from `queue_sizes.len() / 2`.

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-25 10:47:34 +08:00
Alex.Lyn
3a3f39aa2d
Merge pull request #8668 from Apokleos/pci-path-refactor
runtime-rs: Refactor the code related to PCI paths and VFIO device driver initialize in DM.
2023-12-23 21:44:07 +08:00
Steve Horsman
1afce09858
Merge pull request #8721 from stevenhorsman/kata-deploy-typos
kata-deploy: snapshotter typo fixes
2023-12-22 21:26:03 +00:00
stevenhorsman
4a95c0d07f kata-deploy: snapshotter typo fixes
- Add spaces so that the if statements are valid

Fixes: #8720
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-12-22 16:32:02 +00:00
Dan Mihai
080541a0f2 genpolicy: add SPDX license header
Add SPDX license header to rules.rego.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Saul Paredes
7f126be67e genpolicy: Update oci_distribution to 0.10.0
Also support alternative media type and update samples

Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
9eb6fd4c24 docs: add agent policy and genpolicy docs
Add docs for the Agent Policy and for the genpolicy tool.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
57f93195ef genpolicy: add support for StatefulSet YAML input
Generate policy for K8s StatefulSet YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
35958ec9cc genpolicy: add support for ReplicationController
Generate policy for K8s ReplicationController YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
7da17099f2 genpolicy: add support for ReplicaSet YAML input
Generate policy for K8s ReplicaSet YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
d84300f1ee genpolicy: add support for List YAML input
Generate policy for K8s List YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
a03452637b genpolicy: add support for Job YAML input
Generate policy for K8s Job YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
2dbd01c80b genpolicy: add support for Deployment YAML input
Generate policy for K8s Deployment YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
a40a6003d0 genpolicy: add support for DaemonSet YAML input
Generate policy for K8s DaemonSet YAML.

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Dan Mihai
48829120b6 policy: initial genpolicy commit
Add application that infers K8s user's intentions based on user's
K8s YAML file, and generates a Rego/OPA based policy for that YAML.

Just Pod YAML files are supported as input using this initial source
code. Support for other types of YAML files will come with upcoming
commits.

Fixes: #7673

Signed-off-by: Dan Mihai <dmihai@microsoft.com>
2023-12-22 15:35:05 +00:00
Chao Wu
555136c1a5
Merge pull request #8662 from openanolis/pci/4-upstream
dragonball: introduce pci msi/msix interrupt
2023-12-22 18:08:31 +08:00
Steve Horsman
c5f939cdc1
Merge pull request #8655 from fidencio/topic/kata-deploy-add-snapshotter-support
kata-deploy: Allow setting up snapshotters per runtime handler
2023-12-22 09:16:07 +00:00
Chao Wu
8cf3bcefd8 dragonball: introduce pci msi/msix interrupt
introduce msi/msix mod to maintain information for PCI Message Signalled
Interrupt Extended Capability. It will be initialized when parsing pci
configuration space and used when getting interrupt capabilities.

fixes: #8661

Signed-off-by: Gerry Liu <gerry@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Shifang Feng <fengshifang@linux.alibaba.com>
Signed-off-by: Yang Su <yang.su@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Xin Lin <jingshan@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
2023-12-22 16:28:22 +08:00
Xuewei Niu
beadce54c5 dragonball: Support vhost-user-net devices
This PR introduces vhost-user-net devices to Dragonball. The devices are
allowed to run as server on the VMM side.

Fixes: #8502

Signed-off-by: Eric Ren <renzhen@linux.alibaba.com>
Signed-off-by: Liu Jiang <gerry@linux.alibaba.com>
Signed-off-by: Zha Bin <zhabin@linux.alibaba.com>
Signed-off-by: Chao Wu <chaowu@linux.alibaba.com>
Signed-off-by: Zizheng Bian <zizheng.bian@linux.alibaba.com>
Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-22 14:53:18 +08:00
Xuewei Niu
1f21d3cb2c dragonball: Introduce address space for MmioV2DeviceState
Vhost-user-net has a dependency on address space from `MmioV2DeviceState`.
The addition of the address space is introduced in this patch. Plus, it
makes sure all unit tests have the according parameter as well.

Fixes: #8502

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-22 14:53:18 +08:00
Fupan Li
dc9a0ac8ce
Merge pull request #8718 from justxuewei/enable-vhost
tests: Load vhost modules explicitly while Kata installing
2023-12-22 14:52:49 +08:00
Xuewei Niu
206ed6d77d tests: Load vhost modules explicitly while Kata installing
The default network backend of runtime-rs with Dragonball is vhost-net
after #8609 merged. The tests might be failed if vhost modules are not
loaded.

Fixes: #8717

Signed-off-by: Xuewei Niu <niuxuewei.nxw@antgroup.com>
2023-12-22 11:07:37 +08:00
alex.lyn
94c83cea84 runtime-rs: Refactor vfio driver implementation
It's important to ensure that these tasks which setup vfio
devices are completed before add_device.

So Moving vfio device setup code to a dedicated method at device
building time which does not affect the behavior of other code.

And this change makes it easier to understand the difference
between create and attach, and also makes the boundaries
clearer.

Fixes: #8665

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-22 10:37:40 +08:00
alex.lyn
82d3cfdeda runtime-rs: Make VhostUserConfig's field pci_path type more specific
Make VhostUserConfig pci_path's type more specific, change it
from Option<String> to Option<PciPath>.

Fixes: #8665

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-22 10:35:38 +08:00
alex.lyn
5cc2890a10 runtime-rs: refactor and re-implement pci path.
Do refactor and re-implement to make the pci path more "rusty".

Fixes: #8665

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-22 10:34:41 +08:00
Fabiano Fidêncio
32e1ba2525
Merge pull request #8714 from cmaf/libsh-update-loc
tests: Use function from Kata repo
2023-12-21 12:30:31 -03:00
Fabiano Fidêncio
6cc6ca5a7f
kata-deploy: Allow setting up snapshotters per runtime handler
Since containerd 1.7.0 we can easily set a specific snapshotter to be
used with a runtime handler, and we should take advantage of this,
mostly as it'll help setting up any runtime using devmapper or nydus
snapshotters.

This implementation here has a few caveats:
* The format expected for the SNAPSHOTTER_HANDLER_MAPPING is:
  `shim:snapshotter,shim:snapshotter,...`
* It only works with containerd 1.7 or newer
* We **never** change the default containerd snapshotter
* We don't do any check on our side to verify whether the snapshotter
  required is properly deployed
* Users will have to add an annotation to their pods, in order to use
  the snapshotter set up per runtime handler
  * Example:
    ```
    metadata:
      ...
      annotations:
        io.containerd.cri.runtime-handler: kata-fc
    ```

Fixes: #8615

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-21 07:20:10 -03:00
alex.lyn
1b5758c1f2 runtime-rs: Move the PciPath-related code to a dedicated file
Move the pciPath code to a new file pci_path.rs and update the
references.

Fixes: #8665

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-21 11:35:18 +08:00
alex.lyn
275de453d5 runtime-rs: remove useless get_host_guest_map and its test case
Fixes: #8665

Signed-off-by: alex.lyn <alex.lyn@antgroup.com>
2023-12-21 11:07:56 +08:00
Chelsea Mafrica
9f394f6e18 tests: Use function from Kata repo
Switch to use function from Kata repo in common.bash to reduce
dependency on the tests repo.

Fixes #8713

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2023-12-20 16:45:06 -08:00
Dan Mihai
d916da15dd
Merge pull request #8688 from microsoft/danmihai1/k8s-confidential
tests: retry connection to pod SSH server
2023-12-20 15:01:26 -08:00
Fabiano Fidêncio
3482256340
Merge pull request #8709 from fidencio/topic/update-jq-for-kata-deploy
kata-deploy: Update `jq` as part of the kata-deploy daemonset
2023-12-20 16:48:07 -03:00
Fabiano Fidêncio
c9e631dc0c
kata-deploy: Reapply "kata-deploy: Use tomlq to configure containerd"
This reverts commit ee5fa08a27.

This is perfectly fine to do as we narrwoed down the issue to be on the
version of `jq` provided by alpine, and we've already updated it in the
previous commit (in this very same series).

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-20 12:52:41 -03:00
Fabiano Fidêncio
41320c586e
kata-deploy: Install jq from GitHub
`jq` coming from alpine is in its 1.6 version, and that has a bug that
hits us quite hard, as it changes a float to an int whenever the number
is in the `x.0` format.

One example is:
```bash
/ # jq --version
jq-1.6
/ # echo '{"foo": 1.0}' | jq .foo
1
```

With this in mind, let's switch, at least for now, to using the `jq`
released directly on github, as it does address the issue we've been
hitting.
```bash
⋊> Downloads ./jq-linux-amd64 --version
jq-1.7
⋊> Downloads echo '{"foo": 1.0}' | jq .foo
1.0
```

Fixes: #8678

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2023-12-20 12:52:41 -03:00
Greg Kurz
ce094ecdc2
Merge pull request #8679 from stevenhorsman/kata-deploy-containerd-config-fix
gha: kata-deploy: Revert containerd config break
2023-12-20 12:58:56 +01:00
stevenhorsman
ee5fa08a27 Revert "kata-deploy: Use tomlq to configure containerd"
This reverts commit dd9f5b07b9.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-12-20 09:10:43 +00:00
stevenhorsman
9e718b4e23 gha: kata-deploy: Add containerd status check
After kata-deploy has installed, check that the worker nodes
are still in Ready state and don't have a containerd://Unknown
container runtime versions, identicating that container isn't working
to ensure that we didn't corrupt the containerd config during kata-deploy's edits

Fixes: #8678
Signed-off-by: stevenhorsman <steven@uk.ibm.com>
2023-12-20 09:10:43 +00:00