Commit Graph

5966 Commits

Author SHA1 Message Date
Fupan Li
91ad176a06
Merge pull request #1434 from fidencio/wip/kata-deploy-cleanups
kata-deploy: stop mentioning qemu-virtiofsd, as the default qemu supports virtiofsd already (plus some cleanups)
2021-02-25 09:50:10 +08:00
Maksym Pavlenko
17e9a2cff5 agent: don't error of virtiofs share is already mounted
Port kata-containers/agent#883 to the Rust Agent.

In the event that the virtiofs device is already mounted at the
requested destination, don't error out. We'll check before attempting to
mount to see if the destination is already a mount point. If so, skip
doing the mount in the agent.

This facilitates mounting the sharedfs automatically in the guest before
the agent service starts.

Signed-off-by: Eric Ernst eric.g.ernst@gmail.com

Fixes: #1398

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-24 17:05:59 -08:00
Tim Zhang
947913f6e5 agent/protocols: Remove cargo:rerun-if-changed in build.rs
So that the build.rs will be re-runed if any file
within the package is changed.

Signed-off-by: Tim Zhang <tim@hyper.sh>
2021-02-24 21:50:47 +08:00
Tim Zhang
2c42be0ca6
Merge pull request #1447 from liubin/liubin/fix-1369
agent: Stop receive message from Receiver if got None
2021-02-24 16:48:24 +08:00
Bin Liu
974f926292
Merge pull request #1450 from fidencio/wip/release-notes-script-still-mentions-docker
Remove "Docker" & "[kata] runtime" references from the release scripts & process documentation
2021-02-24 15:59:36 +08:00
fupan.lfp
bc0ac526a2 shimv2: return the hypervisor's pid as the container pid
Since the kata's hypervisor process is in the network namespace,
which is close to container's process, and some host metrics
such as cadvisor can use this pid to access the network namespace
to get some network metrics. Thus this commit replace the shim's
pid with the hypervisor's pid.

Fixes: #1451

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2021-02-24 13:26:05 +08:00
Fabiano Fidêncio
10ed3da4eb release: Rename runtime-release-notes to release-notes
There's no runtime repo anymore, let's avoid making a reference to it,
which may end up confusing people reading the Release-Process file.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 17:25:40 +01:00
Fabiano Fidêncio
f5dab6af2e release: We're not compatible with Docker.
We don't support Docker as part of 2.x repository.

Fixes: #1449

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 17:25:40 +01:00
Fabiano Fidêncio
2c8ea0a8d1 kata-deploy: Add copyright to the kata-deploy's Dockerfile
All the work done on this file, apart from merging the 2.x repo, and now
removing unused lines, comes from Intel.

The reason it's being added is to silent a complaint from the static
checker.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:25:00 +01:00
Fabiano Fidêncio
4e494e34c9 packaging: Remove NEMU mentions
There's no more NEMU, for some time already.  Considering this, let's
just remove any mention to it as part of our project.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:24:54 +01:00
Fabiano Fidêncio
f21c54a985 kata-deploy: QEMU, for 2.x, already includes virtiofs
There's no reason to ship qemu & qemu-virtiofs when the former already
includes vitiofs support (and that's the default for 2.x deployments).

In case we will enable experimental qemu DAX support, we should add a
new target, a "qemu-experimental" target, as Carlos has been working on.

Fixes: #1424

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:24:48 +01:00
Fabiano Fidêncio
657bd789a3 kata-deploy: Get rid of references to the docker script
The docker script has been removed as part of
62cbaf4de4, but references to it were left
behind in the artifact-list.sh, release/kata-deploy-binaries.sh, and
kata-deploy/Dockerfile.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-23 14:20:39 +01:00
Fabiano Fidêncio
81109f89bd
Merge pull request #1423 from fidencio/wip/kata-deploy-crio-plus-shimv2
kata-deploy: Ensure CRI-O uses shimv2 & the "vm" runtime type
2021-02-22 20:28:30 +01:00
Manabu Sugimoto
dcea08697a rustjail: fix blkio conversion
BFQ weight controller is using the same BFQ weight scheme (i.e 1->1000).
Therefore, there is no need to do the conversion.

More details here: https://github.com/opencontainers/runc/pull/2786

Fixes: #1440

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-02-23 00:26:57 +09:00
bin
bc34cbbce5 agent: Stop receive message from Receiver if got None
If the container has exited, the sender in notifier watching OOM events
will be dropped after the loop exited, and recv() from the according
receiver will get None.

This will lead two problems for get_oom_event rpc all from agent:

- return an wrong OOM event.
- continuously return OOM events.

Fixes: #1369

Signed-off-by: bin <bin@hyper.sh>
2021-02-22 21:56:07 +08:00
Fabiano Fidêncio
01481d6ac0 kata-deploy: Ensure CRI-O uses the VM runtime type
For Kata Containers 2.x, CRI-O should always be using the
`containerd-shim-kata-v2` binary, and always be configured to use the
"vm" runtime type, developed specifically for the shimv2, instead of the
default "oci" runtime type.

I've taken the liberty to try to simplify the CRI-O script and make it
less error prone.  In the future, we can start dropping a configuration
file to /etc/crio/crio.conf.d and just removing it as part of the
cleanup, but that's for the future.

Fixes: #1357

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:27:33 +01:00
Fabiano Fidêncio
d1c717363d kata-deploy: Move the containerd workarounds to their own functions
Factoring those pieces of code to their own functions allows us to
easily re-use them when creating & cleaning up the CRI-O configuration
files, as CRI-O is also affected by the issues that are still opened.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:27:24 +01:00
Fabiano Fidêncio
5013634e23 kata-deploy: Stop shipping kata-{clh,fc,qemu,qemu-virtiofs} binaries
Those binaries are not revelant for 2.x deployments.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:24:12 +01:00
Fabiano Fidêncio
2270f19ee1 kata-deploy: Update README to reflect the current distributed artifacts
Our list was based on what we used to ship for Kata Containers 1.x, not
even taking into account the shimv2 binary.

Let's update it in order to reflect better what we currently distribute.

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-20 09:24:12 +01:00
Bin Liu
29d4abf23a
Merge pull request #1437 from jcvenegas/2021-02-18/self-documented-makefile
makefile: agent: Add self documented help
2021-02-20 10:04:30 +08:00
Carlos Venegas
a494c4de23 makefile: agent: Add self documented help
Add comments that allow self document variables and targets

Fixes: #1436

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-02-19 15:30:42 +00:00
Eric Ernst
4f67bf919d
Merge pull request #1344 from liubin/fix/1329-improvements-for-console-access
runtime: connect guest debug console bypass kata-monitor
2021-02-18 17:59:32 -08:00
David Gibson
a060b9a21b
Merge pull request #1190 from dgibson/pcipath
Clean up PCI path handling
2021-02-19 12:23:27 +11:00
David Gibson
72cb9287a0 vhost-user-blk: Use PciPath type for vhost user devices
VhostUserDeviceAttrs::PCIAddr didn't actually store a PCI address
(DDDD:BB:DD.F), but rather a PCI path.  Use the PciPath type and
rename things to make that clearer.

TestHandleBlockVolume previously used the bizarre value "0001:01"
which is neither a PCI address nor a PCI path for this value.  Change
it to a valid PCI path - it appears the actual value didn't matter for
that test, as long as it was consistent.

Forward port of
3596058c67

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
74f5b5febe runtime/block: Use PciPath type through block code
BlockDrive::PCIAddr doesn't actually store a PCI address
(DDDD:BB:DD.F) but a PCI path.  Use the PciPath type and rename things
to make that clearer.

TestHandleBlockVolume() previously used a bizarre value "0002:01" for
the "PCI address" which was neither an actual PCI address, nor a PCI
path.  Update it to use a PCI path - the actual value appears not to
matter in this test, as long as its consistent throughout.

Forward port of
64751f377b

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
32b40f5fe4 runtime/network: Use PciPath type through network handling
The "PCI address" returned by Endpoint::PciPath() isn't actually a PCI
address (DDDD:BB:DD.F), but rather a PCI path.  Rename and use the
PciPath type to clean this up and the various parts of the network
code connected to it.

Forward port of
3e589713cf

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
87c5823c4b agent/device: Add unit test for pcipath_to_sysfs()
Port this test from the Kata 1 Go agent to the Kata 2 Rust agent.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
066ce7ab51 agent/device: Pass root bus sysfs path to pcipath_to_sysfs()
Currently pcipath_to_sysfs() generates the path to the root bus node in
sysfs via create_pci_root_bus_path().  This is inconvenient for testing,
though, so instead make it take this as a parameter and generate the path
in the (single) caller.  As a bonus this will make life a bit easier when
we want to support machines with multiple PCI roots.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
fda48a9bf0 agent/device: Use pci::Path type, name things consistently
pcipath_to_sysfs takes a PCI path, with a particular format.  A number of
places implicitly need strings in that format, many of them repeat the
description.  To make things safer and briefer use the pci::Path type for
the purpose more widely, and just describe the string formatting of it at
the type definition.

Then, update variable names and comments throughout to call things in
this format "PCI path", rather than "PCI identifier", which is vague,
or "PCI address" which is just plain wrong.  Likewise we change names and
comments which incorrectly refer to sysfs paths as a "PCI address".

This changes the grpc proto definitions, but because it's just
changing the name of a field without changing the field number, it
shouldn't change the actual protocol.

A loose forward port of
da4bc1d184

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
c12b86dc82 agent/device: Generalize PCI path resolution to any number of bridges
Currently pcipath_to_sysfs(), which translates PCI paths into sysfs paths
accepts only pci paths with exactly 2 components; which represents PCI
devices separated from the root bus by exactly one PCI to PCI bridge (which
could be a virtual P2P bridge, such as a PCI-E root port).

There are cases we might reasonably want to support which have devices
either plugged directly into the root bus (zero bridges), or under
multiple layers of P2P bridge (a PCI-E switch would require at least 2
layers).

So, generalize pcipath_to_sysfs to support any number of components in the
PCI path.  We also make it use the new type for PCI paths internally rather
than plain strings.

This is a loose forward port of
9804b1e55d

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
3715c5775f agent/device: Rename and clarify semantics of get_pci_device_address()
get_pci_device_address() has pretty confusing semantics.  Both its input
and output are in other parts of the code described as a "PCI address", but
neither is *actually* a PCI address (in the standard DDDD:BB:DD.F format).

What it's really about is resolving a "PCI path" - that is way to locate a
PCI device by using it's slot number and the slot number of the bridge
leading to it - into a sysfs path.

Rename the function, and change a bunch of variable names to make those
semantics clearer.

Forward port of
https://github.com/kata-containers/agent/pull/855/commits/0eb612f06484

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
7e92831c7a protocols: Update PCI path names / terminology in agent protocol def
Now that we have types to represent PCI paths on both the agent and
runtime sides, we can update the protocol definitionto use clearer
terminology.

Note that this doesn't actually change the agent protocol, because it just
renames a field without changing its field ID or type.

While we're there fix a trivial rustfmt error in
src/agent/protocols/build.rs

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
8e5fd8ee84 runtime: Introduce PciSlot and PciPath types
This is a dedicated data type for representing PCI paths, that is, PCI
devices described by the slot numbers of the bridges we need to reach
them.

There are a number of places that uses strings with that structure for
things.  The plan is to use this data type to consolidate their
handling.  These are essentially Go equivalents of the pci::Slot and
pci::Path types introduced in the Rust agent.

Forward port of
185b3ab044

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:05 +11:00
David Gibson
7464d055a7 agent: PCI path type
Introduce a Rust type to represent a "PCI path" - that is a way of
locating a PCI device from a given root by listing the slots of all
the bridges leading to it and finally the slot of the device itself.

It's implemented as a vector of the previously added pci::Slot type,
and includes the necessary validation and conversions to/from strings.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
David Gibson
b22259ad9b agent: PCI slot type
Add a Rust type for representing a PCI slot on a single bus.  This is
essentially just an integer from 0..31 (inclusive), but includes the
code for converting from integers with appropriate validation and
formatting back to a string.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
David Gibson
8c2f9e6949 gitignore: Ignore *~ editor backup files
We ignore some other formats for backup files, but add this one, used by
emacs.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
Adams, Eric
b412e159f9 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-02-18 12:46:49 -08:00
James O. D. Hunt
a4e367506d
Merge pull request #1428 from cmaf/fix-shimv2-configpath
runtime: Create tracer later in shimv2
2021-02-18 16:04:36 +00:00
Chelsea Mafrica
a44b27291c runtime: Create tracer later in shimv2
Remove loading of configuration from New() because we do not know the
correct configuration file for the runtime until Create() and so that it
is not loaded more than once. Start tracer in create() so that it is
created after the runtime config is loaded in its original location.

Fixes #1411

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-02-17 19:45:48 -08:00
Fupan Li
6eba265734
Merge pull request #1407 from mxpv/json
Agent: OCI hooks return malformed json
2021-02-18 11:36:49 +08:00
Eric Ernst
0a9cc357c6
Merge pull request #1394 from egernst/custom-registry
osbuilder: Allow image registry to be customizable
2021-02-17 17:48:02 -08:00
Eric Ernst
49bdbac606 osbuilder: Allow image registry to be customizable
Give the user chance to specify their own registry in event the default
provided are not accessible, desirable.

Fixes: #1393

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:49:58 -08:00
GabyCT
be83b739df
Merge pull request #1414 from GabyCT/topic/updatelicensing
docs: Update licensing strategy to use kata 2.0 repository
2021-02-17 14:50:58 -06:00
Eric Ernst
78a5958a81
Merge pull request #1402 from egernst/clh-hooks
runtime: clh-config: add runtime hooks to the clh toml
2021-02-17 12:49:58 -08:00
Gabriela Cervantes
fdc573d500 docs: Update licensing strategy to use kata 2.0 repository
This PR updates the licensing strategy document to use the proper
tests repository for kata 2.0

Fixes #1413

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-02-15 11:10:01 -06:00
Eric Ernst
2e2749ad3f runtime: clh-config: add runtime hooks to the clh toml
Today hooks are only described in the QEMU toml. This shouldn't be VMM
specific -- let's make sure these are advertised for Cloud Hypervisor as
well.

Fixes: #1401

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-14 20:26:02 -08:00
Julio Montes
e830192fca
Merge pull request #1387 from devimc/2021-02-09/FixSnapCI
Fix Snap CI
2021-02-12 10:27:28 -06:00
Julio Montes
ef72926beb ci: snap: run snap CI on every pull request
Make sure a pull request doesn't break the snap packages,
run snap CI on every pull request.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:18:45 -06:00
Julio Montes
919d51274d snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:16:09 -06:00
Julio Montes
d054841430 ci: snap: build targets that not need sudo first
`sudo` is required to build the image, once the image has been built
the permission of some directories may change, let's build first the
targerts that not need `sudo`

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:05:42 -06:00