Commit Graph

7360 Commits

Author SHA1 Message Date
James O. D. Hunt
baf4f76d97 docs: More detail on running tests as different users
Add some more detail to the unit test advice document about running
tests as different users.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
fcf45b0c92 docs: Use more idiomatic rust string check
Rather than comparing a string to a literal in the rust example,
use `.is_empty()` as that approach is more idiomatic and preferred.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
9fed7d0bde docs: Mention anyhow for error handling in UT doc
Add a comment stating that `anyhow` and `thiserror` should be used in
real rust code, rather than the unwieldy default `Result` handling
shown in the example.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
318b3f187b docs: No present continuous in UT advice doc
Change some headings to avoid using the present continuous tense which
should not be used for headings.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
e8bb6b2666 docs: Correct repo name usage
Change reference from "runtime repo" to "main repo" in unit test advice
document.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
c1111a1d2d docs: Use leading caps for lang names in UT advice doc
Use a capital letter when referring to Golang and Rust (and remove
unnecessary backticks for Rust).

> **Note:**
>
> We continue refer to "Go" as "Golang" since it's a common alias,
> but, crucially, familiarity with this name makes searching for
> information using this term possible: "Go" is too generic a word.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
597b239ef3 docs: Remove TOC in UT advice doc
Remove the table of contents in the Unit Test Advice document since
GitHub auto-generates these now.

See: https://github.com/kata-containers/kata-containers/pull/2023

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
cf360fad92 docs: Move unit test advice doc from tests repo
Unit tests necessarily need to be maintained with the code they test so
it makes sense to keep the Unit Test Advice document into the main repo
since that is where the majority of unit tests reside.

Note: The
[`Unit-Test-Advice.md` file](https://github.com/kata-containers/tests/blob/main/Unit-Test-Advice.md)
was copied from the `tests` repo when it's `HEAD` was
38855f1f40.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
James O. D. Hunt
bc9558149c docs: Move doc requirements section higher
Move the documentation requirements document link up so that it appears
immediately below the "How to Contribute" section.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-25 14:44:40 +00:00
Chelsea Mafrica
ed7eb26bff
Merge pull request #3113 from liubin/fix/3112-delete-netmon
runtime: delete netmon
2021-11-24 17:58:13 -08:00
Fupan Li
2938f60abb
Merge pull request #3012 from jodh-intel/agent-rm-unwraps
agent: Remove some unwrap and expect calls
2021-11-25 09:37:39 +08:00
Binbin Zhang
75bb340137 shimv2/service: fix defer funtions never run with os.Exit()
os.Exit() will terminate program immediately, the defer functions
won't be executed, so we add defer functions again before os.Exit().
Refer to https://pkg.go.dev/os#Exit

Fixes: #3059

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
2021-11-24 15:59:59 +01:00
James O. D. Hunt
bd3217daeb agent: Remove redundant returns
Remove an unnecessary `return` statement identified by clippy.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
adab64349c agent: Remove some unwrap and expect calls
Replace some `unwrap()` and `expect()` calls with code to return the
error to the caller.

Fixes: #3011.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
351cef7b6a agent: Remove unwrap from verify_cid()
Improved the `verify_cid()` function that validates container ID's by
removing the need for an `unwrap()`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
a7d1c70c4b agent: Improve baremount
Change `baremount()` to accept `Path` values rather than string values
since:

- `Path` is more natural given the function deals with paths.
- This minimises the caller having to convert between string and `Path`
  types, which simplifies the surrounding code.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
09abcd4dc6 agent-ctl: Remove some unwrap and expect calls
Replace some `unwrap()` and `expect()` calls with code to return the
error to the caller.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
35db75baa1 agent-ctl: Remove redundant returns
Remove a number of redundant `return`'s.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
46e459584d agent-ctl: Simplify main
Make the `main()` function simpler.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
James O. D. Hunt
c7349d0bf1 agent-ctl: Simplify error handling
Replace `ok_or().map_err()` combinations with the simpler `ok_or_else()`
construct.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-24 11:43:49 +00:00
bin
ddc68131df runtime: delete netmon
Netmon is not used anymore.

Fixes: #3112

Signed-off-by: bin <bin@hyper.sh>
2021-11-24 15:08:18 +08:00
Carlos Venegas
ac058b3897
Merge pull request #3105 from YchauWang/wyc-agent-make-02
agent: fixed the `make optimize` bug
2021-11-23 13:17:05 -06:00
Fabiano Fidêncio
181f876fdb
Merge pull request #3098 from fidencio/wip/move_kata-deploy-install-instruction_to_docs
docs: make kata-deploy more visible
2021-11-23 18:32:42 +01:00
João Vanzuita
705687dc42 docs: Add kata-deploy as part of the install docs
This PR links the kata-deloy installation instructions to the
docs/install folder.

Fixes: #2450

Signed-off-by: João Vanzuita <joao.vanzuita@de.bosch.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-23 13:57:22 +01:00
Fabiano Fidêncio
acece84906 docs: Use the default notation for "Note" on install README
Let's use the default GitHub notation for notes in documentation, as
describe here:
https://github.com/kata-containers/kata-containers/blob/main/docs/Documentation-Requir

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-23 13:27:35 +01:00
Fabiano Fidêncio
143fb27802 kata-deploy: Use the default notation for "Note"
Let's use the default GitHub notation for notes in documentation, as
describe here:
https://github.com/kata-containers/kata-containers/blob/main/docs/Documentation-Requirements.md#notes

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Suggested-by: James O. D. Hunt <james.o.hunt@intel.com>
2021-11-23 13:24:42 +01:00
Fabiano Fidêncio
45d76407aa kata-deploy: Don't mention arch specific binaries in the README
Although the binary name of the shipped binary is `qemu-system-x86_64`,
and we only ship kata-deploy for `x86_64`, we better leaving the
architecture specific name out of our README file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-23 13:21:37 +01:00
wangyongchao.bj
0c6c0735ec agent: fixed the make optimize bug
The unrecognized option: 'deny-warnings' args caused `make optimize` failed.
Fixed the Makefile of the agent project, make sure the `make optimize` command
execute correctly. This PR modify the rustc args from '--deny-warnings' to
'--deny warnings'.

Fixes: #3104

Signed-off-by: wangyongchao.bj <wangyongchao.bj@inspur.com>
2021-11-23 09:44:05 +08:00
Fabiano Fidêncio
0ae77e1232
Merge pull request #3102 from fidencio/wip/add-back-wrongly-removed-check-for-test-kata-deploy
workflows: Add back the checks for running test-kata-deploy
2021-11-22 22:36:03 +01:00
Fabiano Fidêncio
a7c08aa4b6 workflows: Add back the checks for running test-kata-deploy
Commit 3c9ae7f made /test_kata_deploy run
against HEAD, but it also mistakenly removed all the checks that ensure
/test_kata_deploy only runs when explicitly called.

Mea culpa on this, and let's add the tests back.

Fixes: #3101

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2021-11-22 18:33:10 +01:00
Carlos Venegas
3be15aed1c
Merge pull request #3071 from fidencio/wip/test-kata-deploy-should-use-the-latest-builds
kata-deploy: Ensure we test HEAD with `/test_kata_deploy`
2021-11-22 10:48:35 -06:00
Tim Zhang
cad279b37d
Merge pull request #3055 from liubin/fix/3054-update-spdk-doc
docs: update using-SPDK-vhostuser-and-kata.md
2021-11-22 15:47:02 +08:00
David Gibson
1b28d7180f
Merge pull request #2927 from dgibson/vfio-env-mangling
Update k8s SR-IOV plugin environment variables to work properly with Kata
2021-11-22 13:44:19 +11:00
Eric Ernst
a0919b0865
Merge pull request #2998 from egernst/fix-symlinks
watchers: don't dereference symlinks when copying files
2021-11-19 12:43:22 -08:00
Eric Ernst
b5dfcf2653 watcher: tests: ensure there is 20ms delay between fs writes
We noticed s390x test failures on several of the watcher unit tests.

Discovered that on s390 in particular, if we update a file in quick
sucecssion, the time stampe on the file would not be unique between the
writes. Through testing, we observe that a 20 millisecond delay is very
reliable for being able to observe the timestamp update. Let's ensure we
have this delay between writes for our tests so our tests are more
reliable.

In "the real world" we'll be polling for changes every 2 seconds, and
frequency of filesystem updates will be on order of minutes and days,
rather that microseconds.

Fixes: #2946

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2021-11-19 11:33:36 -08:00
Fabiano Fidêncio
d08bcde7aa
Merge pull request #3068 from fidencio/wip/kata-deploy-re-add-latest-and-stable-tags
kata-deploy: Add back stable & latest tags
2021-11-19 15:58:55 +01:00
David Gibson
78dff468bf agent/device: Adjust PCIDEVICE_* container environment variables for VM
The k8s SR-IOV plugin, when it assigns a VFIO device to a container, adds
an variable of the form PCIDEVICE_<identifier> to the container's
environment, so that the payload knows which device is which.  The contents
of the variable gives the PCI address of the device to use.

Kata allows VFIO devices to be passed in to a Kata container, however it
runs within a VM which has a different PCI topology.  In order for the
payload to find the right device, the environment variables therefore need
to be converted to list the guest PCI addresses instead of the host PCI
addresses.

fixes #2897

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 17:44:05 +11:00
David Gibson
4530e7df29 agent/device: Use simpler structure in update_spec_devices()
update_spec_devices() takes a bunch of updates for the device entries in
the OCI spec and applies them, adjusting things in both the linux.devices
and linux.resources.devices sections of the spec.

It's important that each entry in the spec only be updated once.  Currently
we ensure this by first creating an index of where the entries are, then
consulting that as we apply each update, so that earlier updates don't
cause us to incorrectly detect an entry as being relevant to a later
update.  This method works, but it's quite awkward.

This inverts the loop structure in update_spec_devices() to make this
clearer.  Instead of stepping through each update and finding the relevant
entries in the spec to change, we step through each entry in the spec and
find the relevant update.  This makes it structurally clear that we're only
updating each entry once.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 17:21:11 +11:00
Tim Zhang
653b461dc2
Merge pull request #3064 from lifupan/main
agent: fix the issue of missing create a new session for container
2021-11-19 11:28:54 +08:00
David Gibson
b60622786d agent/device: Correct misleading comment on test case
We have a test case commented as testing the case where linux.devices is
empty in the OCI spec.  While it's true that linux.devices is empth in this
example, the reason it fails isn't specifically because it's empty but
because it doesn't contain a device for the update we're trying to apply.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:25:04 +11:00
David Gibson
89ff700038 agent/device: Remove unnecessary check for empty container_path
update_spec_devices() explicitly checks for being called with an empty
container path and fails.  We have a unit test to verify this behaviour.

But while an empty container_path probably does mean something has gone
wrong elsewhere, that's also true of any number of other bad paths.  Having
an empty string here doesn't prevent what we're doing in this function
making sense - we can compare it to the strings in the OCI spec perfectly
well (though more likely we simply won't find it there).

So, there's no real reason to check this one particular odd case.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:25:03 +11:00
David Gibson
c855a312f0 agent/device: Make DevIndex local to update_spec_devices()
The DevIndex data structure keeps track of devices in the OCI
specification.  We used to carry it around to quite a lot of
functions, but it's now used only within update_spec_devices().  That
means we can simplify things a bit by just open coding the maps we
need, rather than declaring a special type.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:24:47 +11:00
David Gibson
084538d334 agent/device: Change update_spec_device to handle multiple devices at once
update_spec_device() adjusts the OCI spec for device differences
between the host and guest.  It is called repeatedly for each device
we need to alter.  These calls are now all in a single loop in
add_devices(), so it makes more sense to move the loop into a renamed
update_spec_devices() and process all the fixups in one call.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:23:58 +11:00
David Gibson
d6a3ebc496 agent/device: Obtain guest major/minor numbers when creating DevNumUpdate
Currently the DevNumUpdate structure is created with a path to a
device node in the VM, which is then used by update_spec_device().
However the only piece of information that update_spec_device()
actually needs is the VM side major and minor numbers for the device.
We can determine those when we create the DevNumUpdate structure.
This means we detect errors earlier and as a bonus we don't need to
make a copy of the vm path string.

Since that change requires updating 2 of the log statements, we take the
opportunity to update all the log statements to structured style.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:23:36 +11:00
David Gibson
f4982130e1 agent/device: Check for conflicting device updates
For each device in the OCI spec we need to update it to reflect the guest
rather than the host.  We do this with additional device information
provided by the runtime.  There should only be one update for each device
though, if there are multiple, something has gone horribly wrong.

Detect and report this situation, for safety.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:23:34 +11:00
David Gibson
f10e8c8165 agent/device: Batch changes to the OCI specification
As we process container devices in the agent, we repeatedly call
update_spec_device() to adjust the OCI spec as necessary for differences
between the host and the VM.  This means that for the whole of a pretty
complex call graph, the spec is in a partially-updated state - neither
fully as it was on the host, not fully as it will be for the container
within the VM.

Worse, it's not discernable from the contents itself which parts of the
spec have already been updated and which have not.  We used to have real
bugs because of this, until the DevIndex structure was introduced, but that
means a whole, fairly complex, parallel data structure needs to be passed
around this call graph just to keep track of the state we're in.

Start simplifying this by having the device handler functions not directly
update the spec, but instead return an update structure describing the
change they need.  Once all the devices are added, add_devices() will
process all the updates as a batch.

Note that collecting the updates in a HashMap, rather than a simple Vec
doesn't make a lot of sense in the current code, but will reduce churn
in future changes which make use of it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 14:23:15 +11:00
David Gibson
46a4020e9e agent/device: Types to represent update for a device in the OCI spec
Currently update_spec_device() takes parameters 'vm_path' and 'final_path'
to give it the information it needs to update a single device in the OCI
spec for the guest.  This bundles these parameters into a single structure
type describing the updates to a single device.  This doesn't accomplish
much immediately, but will allow a number of further cleanups.

At the same time we change the representation of vm_path from a Unicode
string to a std::path::Path, which is a bit more natural since we are
performing file operations on it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 12:27:52 +11:00
David Gibson
e7beed5430 agent/device: Remove unneeded clone() from several device handlers
virtio_blk_device_handler(), virtio_blk_ccw_device_handler() and
virtio_scsi_device_handler() all take a clone of their 'device' parameter.
They appear to do this in order to get a mutable copy in which they can
update the vm_path field.

However, the copy is dropped at the end of the function, so the only thing
that's used in it is the vm_path field passed to update_spec_device()
afterwards.

We can avoid the clone by just using a local variable for the vm_path.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 12:27:52 +11:00
David Gibson
2029eeebca agent/device: Improve update_spec_device() final_path handling
update_spec_device() takes a 'final_path' parameter which gives the
name the device should be given in the "inner" OCI spec.  We need this
for VFIO devices where the name the payload sees needs to match the
VM's IOMMU groups.  However, in all other cases (for now, and maybe
forever), this is the same as the original 'container_path' given in
the input OCI spec.  To make this clearer and simplify callers, make
this parameter an Option, and only update the device name if it is
non-None.

Additionally, update_spec_device() needs to call to_string() on
update_path to get an owned version.  Rust convention[0] is to let the
caller decide whether it should copy, or just give an existing owned
version to the function.  Change from &str to String to allow that; it
doesn't buy us anything right now, but will make some things a little
nicer in future.

[0] https://rust-lang.github.io/api-guidelines/flexibility.html?highlight=clone#caller-decides-where-to-copy-and-place-data-c-caller-control

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 12:27:52 +11:00
David Gibson
57541315db agent/device: Correct misleading parameter name in update_spec_device()
update_spec_device() takes a 'host_path' parameter which it uses to locate
the device to correct in the OCI spec.  Although this will usually be the
path of the device on the host, it doesn't have to be - a traditional
runtime like runc would create a device node of that name in the container
with the given (host) major and minor numbers.  To clarify that, rename it
to 'container_path'.

We also update the block comment to explain the distinctions more
carefully.  Finally we update some variable names in tests to match.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2021-11-19 12:27:52 +11:00