Commit Graph

6199 Commits

Author SHA1 Message Date
Carlos Venegas
a494c4de23 makefile: agent: Add self documented help
Add comments that allow self document variables and targets

Fixes: #1436

Signed-off-by: Carlos Venegas <jos.c.venegas.munoz@intel.com>
2021-02-19 15:30:42 +00:00
Eric Ernst
4f67bf919d
Merge pull request #1344 from liubin/fix/1329-improvements-for-console-access
runtime: connect guest debug console bypass kata-monitor
2021-02-18 17:59:32 -08:00
David Gibson
a060b9a21b
Merge pull request #1190 from dgibson/pcipath
Clean up PCI path handling
2021-02-19 12:23:27 +11:00
David Gibson
72cb9287a0 vhost-user-blk: Use PciPath type for vhost user devices
VhostUserDeviceAttrs::PCIAddr didn't actually store a PCI address
(DDDD:BB:DD.F), but rather a PCI path.  Use the PciPath type and
rename things to make that clearer.

TestHandleBlockVolume previously used the bizarre value "0001:01"
which is neither a PCI address nor a PCI path for this value.  Change
it to a valid PCI path - it appears the actual value didn't matter for
that test, as long as it was consistent.

Forward port of
3596058c67

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
74f5b5febe runtime/block: Use PciPath type through block code
BlockDrive::PCIAddr doesn't actually store a PCI address
(DDDD:BB:DD.F) but a PCI path.  Use the PciPath type and rename things
to make that clearer.

TestHandleBlockVolume() previously used a bizarre value "0002:01" for
the "PCI address" which was neither an actual PCI address, nor a PCI
path.  Update it to use a PCI path - the actual value appears not to
matter in this test, as long as its consistent throughout.

Forward port of
64751f377b

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
32b40f5fe4 runtime/network: Use PciPath type through network handling
The "PCI address" returned by Endpoint::PciPath() isn't actually a PCI
address (DDDD:BB:DD.F), but rather a PCI path.  Rename and use the
PciPath type to clean this up and the various parts of the network
code connected to it.

Forward port of
3e589713cf

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
87c5823c4b agent/device: Add unit test for pcipath_to_sysfs()
Port this test from the Kata 1 Go agent to the Kata 2 Rust agent.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
066ce7ab51 agent/device: Pass root bus sysfs path to pcipath_to_sysfs()
Currently pcipath_to_sysfs() generates the path to the root bus node in
sysfs via create_pci_root_bus_path().  This is inconvenient for testing,
though, so instead make it take this as a parameter and generate the path
in the (single) caller.  As a bonus this will make life a bit easier when
we want to support machines with multiple PCI roots.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
fda48a9bf0 agent/device: Use pci::Path type, name things consistently
pcipath_to_sysfs takes a PCI path, with a particular format.  A number of
places implicitly need strings in that format, many of them repeat the
description.  To make things safer and briefer use the pci::Path type for
the purpose more widely, and just describe the string formatting of it at
the type definition.

Then, update variable names and comments throughout to call things in
this format "PCI path", rather than "PCI identifier", which is vague,
or "PCI address" which is just plain wrong.  Likewise we change names and
comments which incorrectly refer to sysfs paths as a "PCI address".

This changes the grpc proto definitions, but because it's just
changing the name of a field without changing the field number, it
shouldn't change the actual protocol.

A loose forward port of
da4bc1d184

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
c12b86dc82 agent/device: Generalize PCI path resolution to any number of bridges
Currently pcipath_to_sysfs(), which translates PCI paths into sysfs paths
accepts only pci paths with exactly 2 components; which represents PCI
devices separated from the root bus by exactly one PCI to PCI bridge (which
could be a virtual P2P bridge, such as a PCI-E root port).

There are cases we might reasonably want to support which have devices
either plugged directly into the root bus (zero bridges), or under
multiple layers of P2P bridge (a PCI-E switch would require at least 2
layers).

So, generalize pcipath_to_sysfs to support any number of components in the
PCI path.  We also make it use the new type for PCI paths internally rather
than plain strings.

This is a loose forward port of
9804b1e55d

fixes #1040

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
3715c5775f agent/device: Rename and clarify semantics of get_pci_device_address()
get_pci_device_address() has pretty confusing semantics.  Both its input
and output are in other parts of the code described as a "PCI address", but
neither is *actually* a PCI address (in the standard DDDD:BB:DD.F format).

What it's really about is resolving a "PCI path" - that is way to locate a
PCI device by using it's slot number and the slot number of the bridge
leading to it - into a sysfs path.

Rename the function, and change a bunch of variable names to make those
semantics clearer.

Forward port of
https://github.com/kata-containers/agent/pull/855/commits/0eb612f06484

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
7e92831c7a protocols: Update PCI path names / terminology in agent protocol def
Now that we have types to represent PCI paths on both the agent and
runtime sides, we can update the protocol definitionto use clearer
terminology.

Note that this doesn't actually change the agent protocol, because it just
renames a field without changing its field ID or type.

While we're there fix a trivial rustfmt error in
src/agent/protocols/build.rs

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:08 +11:00
David Gibson
8e5fd8ee84 runtime: Introduce PciSlot and PciPath types
This is a dedicated data type for representing PCI paths, that is, PCI
devices described by the slot numbers of the bridges we need to reach
them.

There are a number of places that uses strings with that structure for
things.  The plan is to use this data type to consolidate their
handling.  These are essentially Go equivalents of the pci::Slot and
pci::Path types introduced in the Rust agent.

Forward port of
185b3ab044

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:56:05 +11:00
David Gibson
7464d055a7 agent: PCI path type
Introduce a Rust type to represent a "PCI path" - that is a way of
locating a PCI device from a given root by listing the slots of all
the bridges leading to it and finally the slot of the device itself.

It's implemented as a vector of the previously added pci::Slot type,
and includes the necessary validation and conversions to/from strings.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
David Gibson
b22259ad9b agent: PCI slot type
Add a Rust type for representing a PCI slot on a single bus.  This is
essentially just an integer from 0..31 (inclusive), but includes the
code for converting from integers with appropriate validation and
formatting back to a string.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
David Gibson
8c2f9e6949 gitignore: Ignore *~ editor backup files
We ignore some other formats for backup files, but add this one, used by
emacs.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-19 09:54:53 +11:00
Adams, Eric
b412e159f9 osbuilder: Port QAT Dockerfile to 2.0 repo
Update the Intel QAT Dockerfile to work with the 2.0 repos, fix some
bugs with building Debian/Ubuntu rootfs, and update the latest QAT
driver. Updated copyright.

Fixes: #1419

Signed-off-by: Adams, Eric <eric.adams@intel.com>
2021-02-18 12:46:49 -08:00
James O. D. Hunt
a4e367506d
Merge pull request #1428 from cmaf/fix-shimv2-configpath
runtime: Create tracer later in shimv2
2021-02-18 16:04:36 +00:00
Amulya Meka
5096103e7e osbuiler: fixing USE_DOCKER for ppc64le
For building rootfs with docker, glibc based rust target should be installed on ppc64le.
Additionally, protobuf-compiler would be required on ppc64le as it is not present by default.

Fixes: #1417

Signed-off-by: Amulya Meka <amulmek1@in.ibm.com>
2021-02-18 05:33:51 +00:00
Chelsea Mafrica
a44b27291c runtime: Create tracer later in shimv2
Remove loading of configuration from New() because we do not know the
correct configuration file for the runtime until Create() and so that it
is not loaded more than once. Start tracer in create() so that it is
created after the runtime config is loaded in its original location.

Fixes #1411

Signed-off-by: Chelsea Mafrica <chelsea.e.mafrica@intel.com>
2021-02-17 19:45:48 -08:00
Fupan Li
6eba265734
Merge pull request #1407 from mxpv/json
Agent: OCI hooks return malformed json
2021-02-18 11:36:49 +08:00
Eric Ernst
0a9cc357c6
Merge pull request #1394 from egernst/custom-registry
osbuilder: Allow image registry to be customizable
2021-02-17 17:48:02 -08:00
Eric Ernst
49bdbac606 osbuilder: Allow image registry to be customizable
Give the user chance to specify their own registry in event the default
provided are not accessible, desirable.

Fixes: #1393

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-17 16:49:58 -08:00
GabyCT
be83b739df
Merge pull request #1414 from GabyCT/topic/updatelicensing
docs: Update licensing strategy to use kata 2.0 repository
2021-02-17 14:50:58 -06:00
Eric Ernst
78a5958a81
Merge pull request #1402 from egernst/clh-hooks
runtime: clh-config: add runtime hooks to the clh toml
2021-02-17 12:49:58 -08:00
Gabriela Cervantes
fdc573d500 docs: Update licensing strategy to use kata 2.0 repository
This PR updates the licensing strategy document to use the proper
tests repository for kata 2.0

Fixes #1413

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2021-02-15 11:10:01 -06:00
Eric Ernst
2e2749ad3f runtime: clh-config: add runtime hooks to the clh toml
Today hooks are only described in the QEMU toml. This shouldn't be VMM
specific -- let's make sure these are advertised for Cloud Hypervisor as
well.

Fixes: #1401

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-14 20:26:02 -08:00
Julio Montes
e830192fca
Merge pull request #1387 from devimc/2021-02-09/FixSnapCI
Fix Snap CI
2021-02-12 10:27:28 -06:00
Julio Montes
ef72926beb ci: snap: run snap CI on every pull request
Make sure a pull request doesn't break the snap packages,
run snap CI on every pull request.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:18:45 -06:00
Julio Montes
919d51274d snap: fix kernel setup
kernel setup fails when `yes "n"` is used and `make oldconfig` doesn't
read anything from STDIN, `yes "n"` was added in the past as a
workaround to fix incomplete kernel configs.
Enable `build-kernel.sh` debug.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:16:09 -06:00
Julio Montes
d054841430 ci: snap: build targets that not need sudo first
`sudo` is required to build the image, once the image has been built
the permission of some directories may change, let's build first the
targerts that not need `sudo`

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:05:42 -06:00
Julio Montes
a115338ddd ci: snap: define proxy variables
define proxy variables before using them to fix `unbound variable`
error

fixes #1386

Signed-off-by: Julio Montes <julio.montes@intel.com>
2021-02-12 09:05:42 -06:00
Bo Chen
14bb24e4ca
Merge pull request #1406 from egernst/fix-ctr-cpuset
runtime: cpuset: when creating container, don't pass cpuset details
2021-02-11 22:31:44 -08:00
Maksym Pavlenko
df14d386a5 Agent: OCI hooks return malformed json
This PR fixes wrong serialization of OCI state object.
OCI hooks end up with a JSON string with double quotes in `state` field.

This happens because of confusion `Debug` and `Display` traits. Debug trait
returns a string representation with double quotes.

Ideally we should not use Debug as a part of serialization process, so a bit
more safer fix would be to move container states to `oci` crate and simply
disallow wrong values in that field.

`ContainerState` in go spec: https://github.com/opencontainers/runtime-spec/blob/master/specs-go/state.go#L4

Fixes: #1404

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-02-11 19:02:41 -08:00
Eric Ernst
3721351324 runtime: cpuset: when creating container, don't pass cpuset details
Today we only clear out the cpuset details when doing an update call on
existing container/pods. This works in the case of Kubernetes, but not
in the case where we are explicitly setting the cpuset details at boot
time. For example, if you are running a single container via docker ala:

docker run --cpuset-cpus 0-3 -it alpine sh

What would happen is the cpuset info would be passed in with the
container spec for create container request to the agent. At that point
in time, there'd only be the defualt number of CPUs available in the
guest (1), so you'd be left with cpusets set to 0. Next, we'd hotplug
the vCPUs, providing 0-4 CPUs in the guest, but the cpuset would never
be updated, leaving the application tied to CPU 0.

Ouch.

Until the day we support cpusets in the guest, let's make sure that we
start off clearing the cpuset fields.

Fixes: #1405

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-11 17:38:15 -08:00
Manabu Sugimoto
660b047306 oci: Update seccomp configuration
Seccomp configuration should be updated to prepare for the future seccomp support based on the latest OCI specification.

Add:
- flags which is used with seccomp(2) in struct LinuxSeccomp
- errnoRet which is errno return code in struct LinuxSyscall
- some new seccomp actions and an architecture

Fixes: #1391

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-02-11 22:37:02 +09:00
Fabiano Fidêncio
96196e102e
Merge pull request #1396 from dgibson/pointtopoint
agent: Remove bogus check from list_interfaces() unit test
2021-02-11 09:06:01 +01:00
David Gibson
c9c7c12440 agent: Remove bogus check from list_interfaces() unit test
The unit test for list_interfaces() checks that the hardware address
returned for each interface has non-zero length.  However, that need not be
the case.  Point-to-point devices, such as ppp, or tun devices in certain
configurations may not have a hardware address, which is represented as
a zero length address here.

This happens on my machine with a tun0 device created by OpenVPN.

fixes #1377

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-02-11 15:09:10 +11:00
Eric Ernst
cb6d2f3c40 osbuilder: alphabetize fields
Let's go ahead and list the usage info / fields in alphabetical order!

Signed-off-by: Eric Ernst <eric.g.ernst@gmail.com>
2021-02-10 12:39:10 -08:00
Fabiano Fidêncio
e2c8c7e603
Merge pull request #1031 from knittl/feature/kata-option-aliases
cli: Add aliases for `kata-` options
2021-02-10 16:22:13 +01:00
Fabiano Fidêncio
2009ef4872
Merge pull request #1364 from fidencio/wip/only-run-kata-deploy-test-on-pull-requests
github: Only run kata-deploy-test on pull-requests
2021-02-10 13:59:30 +01:00
Daniel Knittl-Frank
056d742c17 docs: Update documentation with new prefixless config options
Remove the old config options from the documentation and replace them
with the new form (without the redundant `kata-` prefix).

Signed-off-by: Daniel Knittl-Frank <knittl89+git@googlemail.com>
2021-02-10 07:55:18 +01:00
Daniel Knittl-Frank
fdcde7968a cli: use new prefixless config options in tools scripts
Update all tools/packaging scripts to prefer the new options over the
old ones (e.g. `--config` instead of `--kata-config`).

Signed-off-by: Daniel Knittl-Frank <knittl89+git@googlemail.com>
2021-02-10 07:55:18 +01:00
Daniel Knittl-Frank
02ee8b0b8a cli: Add aliases for kata- options
Remove `kata-` prefix from options `kata-config` and
`kata-show-default-config-paths`.

Fixes #1011

Signed-off-by: Daniel Knittl-Frank <knittl89+git@googlemail.com>
2021-02-10 07:55:18 +01:00
Daniel Knittl-Frank
c6bc43b697 docs: Fix broken link to fluentbit.io docs
Fix link to external website in fluentd how-to.

Signed-off-by: Daniel Knittl-Frank <knittl89+git@googlemail.com>
2021-02-10 07:55:18 +01:00
Julio Montes
61b448c7fb
Merge pull request #1385 from ManaSugi/change_virtiofsd_dir
docs: Fix the installation directory of virtiofsd
2021-02-09 10:49:33 -06:00
Manabu Sugimoto
20b27a16c9 docs: Fix the installation directory of virtiofsd
Change the installation directory of virtiofsd to kata-qemu.

Fixes: #1379

Signed-off-by: Manabu Sugimoto <Manabu.Sugimoto@sony.com>
2021-02-09 22:47:27 +09:00
Fabiano Fidêncio
f4fe97411e
Merge pull request #1267 from Jakob-Naucke/s390x-fix-docker-rootfs-build
osbuilder: Fix USE_DOCKER on s390x
2021-02-09 14:13:35 +01:00
Fabiano Fidêncio
8c1e0d3002 kernel: Enable OVERLAY_FS_{METACOPY,XINO_AUTO}
* CONFIG_OVERLAY_FS_METACOPY is needed to have reasonable performance
  for chmod and similar calls;
* CONFIG_OVERLAY_FS_XINO_AUTO is recommended for POSIX compliance.

Fixes: #1075

Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2021-02-09 13:01:01 +01:00
Jakob-Naucke
11fe6a3552
osbuilder: Fix USE_DOCKER on s390x
- Install the required protobuf-compiler on Ubuntu
- Install correct libc Rust target (glibc on s390x)
- Do not skip Rust installation on s390x

Fixes: #1266

Signed-off-by: Jakob Naucke <jakob.naucke@ibm.com>
2021-02-09 12:37:24 +01:00