Commit Graph

9394 Commits

Author SHA1 Message Date
Eric Ernst
d2df1209a5 docs: describe kata handling for core-scheduling
Add initial documentation for core-scheduling.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 16:17:00 -07:00
Michael Crosby
22b6a94a84 shim: add support for core scheduling
In linux 5.14 and hopefully some backports, core scheduling allows processes to
be co scheduled within the same domain on SMT enabled systems.

Containerd impl sets the core sched domain when launching a shim. This
allows a clean way for each shim(container/pod) to be in its own domain and any
additional containers, (v2 pods) be be launched with the same domain as well as
any exec'd process added to the container.

kernel docs: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/core-scheduling.html

For Kata specifically, we will look for SCHED_CORE environment variable
to be set to indicate we shuold create a new schedule core domain.

This is equivalent to the containerd shim's PR: e48bbe8394

Fixes: #4309

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
2022-05-31 10:10:40 -07:00
Eric Ernst
af2ef3f7a5 agent-ctl: introduce handle for iptables get/set
Add support for the updated agent API for iptables

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
65f0cef16c kata-runtime: add iptables CLI to test http endpoint
While end users can connect directly to the shim, let's provide a way to
easily get/set iptables from kata-runtime itself.

Fixes: #4080
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
3201ad0830 shim-client: ensure we check resp status for Put/Post
Without this, potential errors are silently dropped. Let's ensure we
return the error code as well as potenial data from the response.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
0706fb28ac kata-runtime: shmgmt: make url usage consistent
Before, we had a mix of slash, etc. Unfortunately, when cleaning URL
paths, serve mux seems to mangle the request method, resulting in each
request being a GET (instead of PUT or POST).

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
2a09378dd9 shim-client: add support for DoPut
While at it, make sure we check for nil in DoPost

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
640173cfc2 shim-mgmt: Add endpoint handler for interacting with iptables
Add two endpoints: ip6tables, iptables.

Each url handler supports GET and PUT operations. PUT expects
the requests' data to be []bytes, and to contain iptable information in
format to be consumed by iptables-restore.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
0136be22ca virtcontainers: plumb iptable set/get from sandbox to agent
Introduce get/set iptable handling. We add a sandbox API for getting and
setting the IPTables within the guest. This routes it from sandbox
interface, through kata-agent, ultimately making requests to the guest
agent.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
bd50d463b2 agent: iptables: get/set handling for iptables
Initial support for getting and setting iptables in the guest.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:27:58 -07:00
Eric Ernst
7c4049aabb osbuilder: add iptables package
Since we are introducing an agent API for interacting with guest
iptables, let's ensure that our example rootfs' have iptables-save/restore
installed.

Fixes: #4356

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 09:21:02 -07:00
Eric Ernst
03176a9e09 proto: update generated code based on proto update
Update the generated agent.pb.go code based on proto update.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 08:45:59 -07:00
Eric Ernst
38ebbc705b proto: update to add set/get iptables
Update the agent protocol definition to introduce support for setting
and getting iptables from the guest.

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2022-05-31 08:45:59 -07:00
Bin Liu
78d45b434f agent: return mount file content if parse mountinfo failed
Include mount file content in error message when parsing
mountinfo failed for debug.

Fixes: #4246, #4103

Signed-off-by: Bin Liu <bin@hyper.sh>
2022-05-31 23:36:14 +08:00
Manabu Sugimoto
c7b3941c96 runk: Enable test for the agent built with standard-oci-runtime feature
This enables tests for the kata-agent for runk that is built
with standard-oci-runtime feature in CI.

Fixes: #4351

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-05-31 21:54:28 +09:00
Manabu Sugimoto
6dbce7c3de agent: Remove unused import in console test
Remove some unused imports in console test module
used by runk's test.

Fixes: #4351

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-05-31 21:54:02 +09:00
Xuewei Niu
6ecea84bc5 rustjail: get home dir using nix crate
Get user's home dir using `nix::unistd` crate instead of `utils` crate,
and remove useless code from agent.

Fixes: #4209

Signed-off-by: Xuewei Niu <justxuewei@apache.org>
2022-05-31 15:04:33 +08:00
Manabu Sugimoto
648b8d0aec runk: Return error when tty is used without console socket
runk always launches containers with detached mode,
so users have to use a console socket with run or
create operation when a terminal is used.
If users set `terminal` to `true` in `config.json` and
try to launch a container without specifying a console
socket, runk returns an error with a message early.

Fixes: #4324

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-05-31 09:55:39 +09:00
James O. D. Hunt
96c8df40b5
Merge pull request #4335 from ManaSugi/runk/fix-invalid-rootfs
runk: Handle rootfs path in config.json properly
2022-05-30 14:03:58 +01:00
Manabu Sugimoto
5205efd9b4 runk: Add Podman guide in README
runk can launch containers using Podman, so add the guide
in README.

Fixes: #4338

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-05-30 19:06:46 +09:00
James O. D. Hunt
d157f9b71e
Merge pull request #3871 from amshinde/update-containerd-docs
Update containerd docs
2022-05-30 08:38:07 +01:00
Manabu Sugimoto
d862ca0590 runk: Handle rootfs path in config.json properly
This commit enables runk to handle `root.path` in `config.json`
properly even if the path is specified by a relative path that
includes the single (`.`) or the double (`..`) dots.
For example, with a bundle at `/to/bundle` and a rootfs directly
under `/to/bundle` such as `/to/bundle/{bin,dev,etc,home,...}`,
the `root.path` value can be either `/to/bundle` or just `.`.
This behavior conforms to OCI runtime spec.
Accordingly, a bundle path managed by runk's status file
(`status.json`) always is statically stored as a canonical path.
Previously, a bundle path has been got by `oci_state()` of rustjail's
API that returns the path as the parent directory path of a rootfs
(`root.path`). In case of the kata-agent, this works properly because
the kata containers assume that the rootfs path is always
`/to/bundle/rootfs`. However in case of standard OCI runtimes,
a rootfs can be placed anywhere under a bundle, so the rootfs path
doesn't always have to be at a `/to/bundle/rootfs`.

Fixes: #4334

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2022-05-30 14:41:26 +09:00
snir911
d50937435d
Merge pull request #4318 from fidencio/topic/update-clh-to-v24.0
clh: Update to v24.0
2022-05-29 15:06:17 +03:00
James O. D. Hunt
56591804b3 docs: Improve snap build instructions
Make it clearer how to build the snap package manually.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-26 15:56:36 +01:00
James O. D. Hunt
cb2b30970d snap: Build using destructive mode
Destructive mode is required to build the Kata Containers snap. See:

```
.github/workflows/snap-release.yaml
.github/workflows/snap.yaml
```

Hence, update the last file that we forgot to update with
`--destructive-mode`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-26 15:56:36 +01:00
James O. D. Hunt
60823abb9c docs: Move snap README
Move the snap README to a subdirectory to resolve the warning given by
`snapcraft` (folded and reformatted slightly for clarity):

```
The 'snap' directory is meant specifically for snapcraft,
but it contains the following non-snapcraft-related paths,
which is unsupported and will cause unexpected behavior:

- README.md

If you must store these files within the 'snap' directory,
move them to 'snap/local', which is ignored by snapcraft.
```

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-26 15:56:36 +01:00
James O. D. Hunt
4134beee39
Merge pull request #4301 from jodh-intel/snap-package-rust-virtiofsd
snap: Build and package rust version of virtiofsd
2022-05-26 15:55:06 +01:00
Fabiano Fidêncio
fff832874e clh: Update to v24.0
This release has been tracked through the v24.0 project.

virtio-iommu specification describes how a device can be attached by default
to a bypass domain. This feature is particularly helpful for booting a VM with
guest software which doesn't support virtio-iommu but still need to access
the device. Now that Cloud Hypervisor supports this feature, it can boot a VM
with Rust Hypervisor Firmware or OVMF even if the virtio-block device exposing
the disk image is placed behind a virtual IOMMU.

Multiple checks have been added to the code to prevent devices with identical
identifiers from being created, and therefore avoid unexpected behaviors at boot
or whenever a device was hot plugged into the VM.

Sparse mmap support has been added to both VFIO and vfio-user devices. This
allows the device regions that are not fully mappable to be partially mapped.
And the more a device region can be mapped into the guest address space, the
fewer VM exits will be generated when this device is accessed. This directly
impacts the performance related to this device.

A new serial_number option has been added to --platform, allowing a user to
set a specific serial number for the platform. This number is exposed to the
guest through the SMBIOS.

* Fix loading RAW firmware (#4072)
* Reject compressed QCOW images (#4055)
* Reject virtio-mem resize if device is not activated (#4003)
* Fix potential mmap leaks from VFIO/vfio-user MMIO regions (#4069)
* Fix algorithm finding HOB memory resources (#3983)

* Refactor interrupt handling (#4083)
* Load kernel asynchronously (#4022)
* Only create ACPI memory manager DSDT when resizable (#4013)

Deprecated features will be removed in a subsequent release and users should
plan to use alternatives

* The mergeable option from the virtio-pmem support has been deprecated
(#3968)
* The dax option from the virtio-fs support has been deprecated (#3889)

Fixes: #4317

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-26 08:51:18 +00:00
James O. D. Hunt
49361749ed snap: Build and package rust version of virtiofsd
Update the snap config file to build the rust version of `virtiofsd` for
x86_64, but build QEMU's C version for other platforms.

Fixes: #4261.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 17:04:05 +01:00
James O. D. Hunt
27d903b76a snap: Put the yq binary in the staging bin directory
Rather than putting the `yq` binary in the staging directory itself,
put it in the `bin/` sub-directory.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
d7b4ce049e snap: Remove unused variable
Remove the unused `kata_url` variable and use the value in the `website`
YAML metadata instead.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
43de5440e5 snap: Fix unbound variable error
Don't assume `GITHUB_REF` is set.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
James O. D. Hunt
c9b291509d snap: Fix whitespace
Remove trailing space.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2022-05-25 09:40:09 +01:00
Fupan Li
62d1ed0651
Merge pull request #4290 from Tim-Zhang/remove-oci-kata-agent
runk: merge oci-kata-agent into runk
2022-05-25 11:31:25 +08:00
Fabiano Fidêncio
8a2b82ff51
Merge pull request #4276 from jongwu/build_rust_virtiofsd
virtiofsd: static build virtiofsd from rust code for non-x86
2022-05-24 14:57:21 +02:00
Eric Ernst
6d00701ec9
Merge pull request #4298 from yibozhuang/fix-direct-volume
Fix issues with direct-volume stats feature
2022-05-23 15:23:51 -07:00
Tim Zhang
122a85e222 agent: remove bin oci-kata-agent
Fixes: #4291

Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-05-23 16:55:16 +08:00
Tim Zhang
35619b45aa runk: merge oci-kata-agent into runk
Merge two bins into one.

Fixes: #4291

Signed-off-by: Tim Zhang <tim@hyper.sh>
2022-05-23 16:54:09 +08:00
Fabiano Fidêncio
b9315af092
Merge pull request #4294 from yibozhuang/direct-volume-stats
runtime: fix incorrect Action function for direct-volume stats
2022-05-23 10:22:29 +02:00
Jianyong Wu
10c13d719a qemu: remove virtiofsd option in qemu config
As virtiofsd will be built base on rust, "virtiofsd" option is no longer
needed in qemu.

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-23 12:57:59 +08:00
Jianyong Wu
d20bc5a4d2 virtiofsd: build rust based virtiofsd from source for non-x86_64
Based on @fidencio's opoinon,
On Arm: static build virtiofsd using musl lib;
on ppc64 & s390: static build virtiofsd using gnu lib;

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-23 12:57:59 +08:00
Archana Shinde
c95ba63c0c docs: Remove information related to Kata 1.x
Since Kata 2.x does not support runtime cli, remove information
related to it. Update the configuration snippet accordingly.

Fixes #3870

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-05-21 07:19:28 +05:30
Archana Shinde
34b80382b6 docs: Get rid of note related to networking.
One may want to use standalone containerd without k8s
and still have network enabled for the container.
Getting rid of note due to inaccuracy.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-05-21 07:19:28 +05:30
Archana Shinde
dfad5728a7 docs: Mention --cni flag while invoking ctr
Specify that the `--cni` flag needs to be passed to the `ctr` tool
while starting a container in order to have networking enabled for the
container. This flag allows containerd to call into the configured
network plugin which in turn creates a network interface for the
container.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2022-05-21 07:19:28 +05:30
Yibo Zhuang
8e7c5975c6 agent: fix direct-assigned volume stats
The current implementation of walking the
disks to match with the requested volume path
in agent doesn't work because the volume path
provided by the shim to the agent is the mount
path within the guest and not the device name.
The current logic is trying to match the
device name to the volume path which will never
match.

This change will simplify the
get_volume_capacity_stats and
get_volume_inode_stats to just call statfs and
get the bytes and inodes usage of the volume
path directly.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:43:27 -07:00
Yibo Zhuang
4428ceae16 runtime: direct-volume stats use correct name
Today the shim does a translation when doing
direct-volume stats where it takes the source and
returns the mount path within the guest.

The source for a direct-assigned volume is actually
the device path on the host and not the publish
volume path.

This change will perform a lookup of the mount info
during direct-volume stats to ensure that the
device path is provided to the shim for querying
the volume stats.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:42:47 -07:00
Yibo Zhuang
ffdc065b4c runtime: direct-volume stats update to use GET parameter
The go default http mux AFAIK doesn’t support pattern
routing so right now client is padding the url
for direct-volume stats with a subpath of the volume
path and this will always result in 404 not found returned
by the shim.

This change will update the shim to take the volume
path as a GET query parameter instead of a subpath.
If the parameter is missing or empty, then return
400 BadRequest to the client.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:41:51 -07:00
Yibo Zhuang
f295953183 runtime: fix incorrect Action function for direct-volume stats
The action function expects a function that returns error
but the current direct-volume stats Action returns
(string, error) which is invalid.

This change fixes the format and print out the stats from
the command instead.

Fixes: #4293

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 14:55:00 -07:00
Peng Tao
2c238c8504
Merge pull request #4213 from zvonkok/vfio
runtime: Adding the correct detection of mediated PCIe devices
2022-05-20 15:00:23 +08:00
Fabiano Fidêncio
811ac6a8ce
Merge pull request #4282 from r4f4/runtime-dedup-types-import
runtime: remove duplicate 'types' import
2022-05-19 22:15:36 +02:00