Commit Graph

8654 Commits

Author SHA1 Message Date
Michael Crosby
22b6a94a84 shim: add support for core scheduling
In linux 5.14 and hopefully some backports, core scheduling allows processes to
be co scheduled within the same domain on SMT enabled systems.

Containerd impl sets the core sched domain when launching a shim. This
allows a clean way for each shim(container/pod) to be in its own domain and any
additional containers, (v2 pods) be be launched with the same domain as well as
any exec'd process added to the container.

kernel docs: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/core-scheduling.html

For Kata specifically, we will look for SCHED_CORE environment variable
to be set to indicate we shuold create a new schedule core domain.

This is equivalent to the containerd shim's PR: e48bbe8394

Fixes: #4309

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Signed-off-by: Michael Crosby <michael@thepasture.io>
2022-05-31 10:10:40 -07:00
Fabiano Fidêncio
8a2b82ff51
Merge pull request #4276 from jongwu/build_rust_virtiofsd
virtiofsd: static build virtiofsd from rust code for non-x86
2022-05-24 14:57:21 +02:00
Eric Ernst
6d00701ec9
Merge pull request #4298 from yibozhuang/fix-direct-volume
Fix issues with direct-volume stats feature
2022-05-23 15:23:51 -07:00
Fabiano Fidêncio
b9315af092
Merge pull request #4294 from yibozhuang/direct-volume-stats
runtime: fix incorrect Action function for direct-volume stats
2022-05-23 10:22:29 +02:00
Jianyong Wu
10c13d719a qemu: remove virtiofsd option in qemu config
As virtiofsd will be built base on rust, "virtiofsd" option is no longer
needed in qemu.

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-23 12:57:59 +08:00
Jianyong Wu
d20bc5a4d2 virtiofsd: build rust based virtiofsd from source for non-x86_64
Based on @fidencio's opoinon,
On Arm: static build virtiofsd using musl lib;
on ppc64 & s390: static build virtiofsd using gnu lib;

Fixes: #4258
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2022-05-23 12:57:59 +08:00
Yibo Zhuang
8e7c5975c6 agent: fix direct-assigned volume stats
The current implementation of walking the
disks to match with the requested volume path
in agent doesn't work because the volume path
provided by the shim to the agent is the mount
path within the guest and not the device name.
The current logic is trying to match the
device name to the volume path which will never
match.

This change will simplify the
get_volume_capacity_stats and
get_volume_inode_stats to just call statfs and
get the bytes and inodes usage of the volume
path directly.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:43:27 -07:00
Yibo Zhuang
4428ceae16 runtime: direct-volume stats use correct name
Today the shim does a translation when doing
direct-volume stats where it takes the source and
returns the mount path within the guest.

The source for a direct-assigned volume is actually
the device path on the host and not the publish
volume path.

This change will perform a lookup of the mount info
during direct-volume stats to ensure that the
device path is provided to the shim for querying
the volume stats.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:42:47 -07:00
Yibo Zhuang
ffdc065b4c runtime: direct-volume stats update to use GET parameter
The go default http mux AFAIK doesn’t support pattern
routing so right now client is padding the url
for direct-volume stats with a subpath of the volume
path and this will always result in 404 not found returned
by the shim.

This change will update the shim to take the volume
path as a GET query parameter instead of a subpath.
If the parameter is missing or empty, then return
400 BadRequest to the client.

Fixes: #4297

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 18:41:51 -07:00
Yibo Zhuang
f295953183 runtime: fix incorrect Action function for direct-volume stats
The action function expects a function that returns error
but the current direct-volume stats Action returns
(string, error) which is invalid.

This change fixes the format and print out the stats from
the command instead.

Fixes: #4293

Signed-off-by: Yibo Zhuang <yibzhuang@gmail.com>
2022-05-20 14:55:00 -07:00
Peng Tao
2c238c8504
Merge pull request #4213 from zvonkok/vfio
runtime: Adding the correct detection of mediated PCIe devices
2022-05-20 15:00:23 +08:00
Fabiano Fidêncio
811ac6a8ce
Merge pull request #4282 from r4f4/runtime-dedup-types-import
runtime: remove duplicate 'types' import
2022-05-19 22:15:36 +02:00
Chelsea Mafrica
d8be0f8e9f
Merge pull request #4281 from r4f4/runtime-qemu-comments
runtime: sync docstrings with function names
2022-05-19 09:17:38 -07:00
Rafael Fonseca
7a5ccd1264 runtime: sync docstrings with function names
The functions were renamed but their docstrings were not.

Fixes #4006

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-19 14:31:47 +02:00
Greg Kurz
fa61bd43ee
Merge pull request #4238 from snir911/wip/legacy_console
qemu: allow using legacy serial device for the console
2022-05-19 14:30:59 +02:00
Rafael Fonseca
ce2e521a0f runtime: remove duplicate 'types' import
Fallout of 09f7962ff

Fixes #4285

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-19 13:49:47 +02:00
Snir Sheriber
834f93ce8a docs: fix annotations example
annotation value should always be quoted, regardless to its type

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-19 09:52:30 +03:00
GabyCT
d7aded7238
Merge pull request #4279 from GabyCT/topic/updateosbuilderreadme
docs: Remove clear containers reference in README
2022-05-18 14:26:56 -05:00
Snir Sheriber
f4994e486b runtime: allow annotation configuration to use_legacy_serial
and update the docs and test

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-18 18:58:21 +03:00
Gabriela Cervantes
24a2b0f6a2 docs: Remove clear containers reference in README
This PR removes the clear containers reference as this is not longer
being used and is deprecated at the rootfs builder README.

Fixes #4278

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-05-18 14:53:17 +00:00
Fabiano Fidêncio
c88a48be21
Merge pull request #4271 from r4f4/runtime-err-check-fix
runtime: do not check for EOF error in console watcher
2022-05-18 09:49:48 +02:00
GabyCT
9458cc0053
Merge pull request #4273 from GabyCT/topic/removenemuconf
kernel: Remove nemu.conf from packaging
2022-05-17 16:06:45 -05:00
Greg Kurz
42c64b3d2c
Merge pull request #4269 from r4f4/remove-unused-param-get_kata_deps
tools: delete unused param from get_from_kata_deps callers
2022-05-17 18:54:47 +02:00
Gabriela Cervantes
abad33eba0 kernel: Remove nemu.conf from packaging
This PR removes the nemu.conf as we are not longer using NEMU from
the kernel configurations.

Fixes #4272

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-05-17 16:23:17 +00:00
Chelsea Mafrica
04bd8f16f0
Merge pull request #4252 from Champ-Goblem/patch/fix-is-signal-handled
agent: Fix is_signal_handled failing parsing str to u64
2022-05-17 08:31:48 -07:00
GabyCT
12f0ab120a
Merge pull request #4191 from dgibson/go-test-script
Improve Go unit test script
2022-05-17 10:27:04 -05:00
Rafael Fonseca
e87eb13c4f tools: delete unused param from get_from_kata_deps callers
The param was deleted by a09e58fa80, so
update the callers not to use it.

Fixes #4245

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-17 15:18:41 +02:00
Rafael Fonseca
8052fe62fa runtime: do not check for EOF error in console watcher
The documentation of the bufio package explicitly says

"Err returns the first non-EOF error that was encountered by the
Scanner."

When io.EOF happens, `Err()` will return `nil` and `Scan()` will return
`false`.

Fixes #4079

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
2022-05-17 15:14:33 +02:00
Fabiano Fidêncio
5d43718494
Merge pull request #4267 from cmaf/packaging-config-add-numa
packaging: Add kernel config option for SGX in Gramine
2022-05-17 13:10:24 +02:00
Snir Sheriber
c67b9d2975 qemu: allow using legacy serial device for the console
This allows to get guest early boot logs which are usually
missed when virtconsole is used.
- It utilizes previous work on the govmm side:
https://github.com/kata-containers/govmm/pull/203
- unit test added

Fixes: #4237
Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-17 12:06:11 +03:00
Snir Sheriber
44814dce19 qemu: treat console kernel params within appendConsole
as it is tightly coupled with the appended console device
additionally have it tested

Signed-off-by: Snir Sheriber <ssheribe@redhat.com>
2022-05-17 12:05:31 +03:00
Fupan Li
856c8e81f1
Merge pull request #4220 from liubin/fix/4219
ci: Don't run Docs URL Alive Check workflow on forks
2022-05-17 12:19:55 +08:00
Chelsea Mafrica
4f586d2a91 packaging: Add kernel config option for SGX in Gramine
For the Gramine Shielded Containers guest kernel, CONFIG_NUMA must be
enabled.

Fixes  #4266

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2022-05-16 16:58:26 -07:00
Champ-Goblem
4b437d91f0 agent: Fix is_signal_handled failing parsing str to u64
In the is_signal_handled function, when parsing the hex string returned
from `/proc/<pid>/status` the space/tab character after the colon
is not removed.

This patch trims the result of SigCgt so that
all whitespace characters are removed. It also extends the existing
test cases to check for this scenario.

Fixes: #4250
Signed-off-by: Champ-Goblem <cameron@northflank.com>
2022-05-16 20:34:26 +02:00
Fabiano Fidêncio
6ffdebd202
Merge pull request #4255 from cmaf/tools-patch-qemu-sgx-numa
tools: Add QEMU patches for SGX numa support
2022-05-16 18:10:41 +02:00
Chelsea Mafrica
ee9ee77388
Merge pull request #4264 from GabyCT/topic/updatecontainerdrunt
docs: Update runc containerd runtime
2022-05-16 08:56:26 -07:00
Gabriela Cervantes
88fb9b72e2 docs: Update runc containerd runtime
As we are using a containerd version > 1.4 we need to update
the runc containerd runtime.

Fixes #4263

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2022-05-16 14:33:48 +00:00
Fabiano Fidêncio
d1f2852d8b tools: Stop building virtiofsd with qemu (for x86_64)
As we finally can move to using the rust virtiofs daemon, let's stop
bulding and packaging the C version of the virtiofsd for x86_64.

Fixes: #4249
Depends-on: github.com/kata-containers/tests#4785

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-16 09:30:24 +02:00
Fabiano Fidêncio
c39852e83f runtime: Use ${LIBEXEC}/virtiofsd as the default virtiofsd path
As now we build and ship the rust version of virtiofsd, which is not
tied to QEMU, we need to update its default location to match with where
we're installing this binary.

Fixes: #4249

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-16 09:30:24 +02:00
Chelsea Mafrica
b4b9068cb7 tools: Add QEMU patches for SGX numa support
There are a few patches for SGX numa support in QEMU added after the
6.2.0 release. Add them for SGX support in Kata.

Fixes #4254

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2022-05-13 16:34:57 -07:00
Fabiano Fidêncio
b780be99d7
Merge pull request #4233 from fidencio/topic/virtiofsd-switch-to-the-rust-version
Build and distribute the rust version of virtiofsd
2022-05-13 19:38:01 +02:00
Fabiano Fidêncio
a475956abd workflows: Add support for building virtiofsd
As already done for the other assets we rely on, let's build (well, pull
in this very specific case) the virtiofsd binary, as we're relying on
its standlone rust version from now on.

Fixes: #4234

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-13 11:37:36 +02:00
Fabiano Fidêncio
71f59f3a7b local-build: Add support for building virtiofsd
As done for the other binaries we release, let's add support for
"building" (or pulling down) the static binary we ship as part of the
kata-containers static tarball (the same one used by kata-deploy).

Right now the virtiofsd is installed in /opt/kata/libexec/virtiofsd, a
different path than the virtiofsd that comes with QEMU.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-13 11:37:36 +02:00
Fabiano Fidêncio
c7ac55b6d7 dockerbuild: Install unzip
As virtiofsd comes in the `zip` format, let's install unzip in the
containers and then be able to access the virtiofsd binary.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-13 11:37:36 +02:00
Fabiano Fidêncio
8e2042d055 tools: add script to pull virtiofsd
Right now this is very much x86_64 specific, but I'd like to count on
the maintainers of the other architectures to expand it.

Also, the name as it's now may be misleading, as we're actually only
pulling the binary that's statically built using `musl` and released as
part of virtiofsd official releases.  But we'll need to build it for the
other architectures, thus I'm following the naming of the scripts used
by the other components.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-13 11:37:21 +02:00
Fabiano Fidêncio
dbedea5086 versions: Add virtiofsd entry
As we're switching to using the rust version of the virtiofsd, let's
give it its own entry in the versions.yaml file, as it's no longer part
of QEMU.

It's important to mention that GitLab doesn't provide a well formed URL
for the releases.  Instead, it adds there a hash, leading us to have to
add the specific link for the tarball.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-05-13 11:23:39 +02:00
David Gibson
e73b70baff runtime: Don't run unit tests verbose by default
go-test.sh by default adds the -v option to 'go test' meaning that output
will be printed from all the passing tests as well as any failing ones.
This results in a lot of output in which it's often difficult to locate the
failing tests you're interested in.

So, remove -v from the default flags.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-13 13:22:31 +10:00
David Gibson
f24a6e761f runtime: Consolidate flags setting in unit tests script
One of the responsibilities of the go-test.sh script is setting up the
default flags for 'go test'.  This is constructed across several different
places in the script using several unneeded intermediate variables though.

Consolidate all the flag construction into one place.

fixes #4190

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-13 13:22:29 +10:00
David Gibson
cf465feb02 runtime: Don't change test behaviour based on $CI or $KATA_DEV_MODE
go-test.sh changes behaviour based on both the $CI and $KATA_DEV_MODE
variables, but not in a way that makes a lot of sense.

If either one is set it uses the test_coverage path, instead of the
test_local path.  That collects coverage information, as the name
suggests, but it also means it runs the tests twice as root and
non-root, which is very non-obvious.

It's not clear what use case the test_local path is for at all.
Developer local builds will typically have $KATA_DEV_MODE set and CI
builds will have $CI set.  There's essentially no downside to running
coverage all the time - it has little impact on the test runtime.

In addition, if *both* $CI and $KATA_DEV_MODE are set, the script
refuses to run things as root, considering it "unsafe".  While having
both set might be unwise in a general sense, there's not really any
way running sudo can be any more unsafe than it is with either one
set.

So, simplify everything by just always running the test_coverage path.
This leaves the test_local path unused, so we can remove it entirely.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-13 13:14:37 +10:00
David Gibson
34c4ac599c runtime: Remove redundant subcommands from go-test.sh
go-test.sh accepts subcommands, however invoking it in the usual way via
the Makefile doesn't use them.  In fact the only remaining subcommand is
"help" and we already have another way of getting the usage information
(-h or --help).  We don't need a second way, so just drop subcommand
handling.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-13 13:14:37 +10:00