Commit Graph

4961 Commits

Author SHA1 Message Date
Peng Tao
9d90906546
Merge pull request #320 from dgibson/cleanups
Clean up some unnecessary data structures
2020-06-26 16:18:16 +08:00
Xu Wang
af4d85c257
Merge pull request #339 from bergwolf/2.0.0-alpha2
release: Kata Containers 2.0.0-alpha2
2020-06-26 15:50:29 +08:00
Peng Tao
fe5b54b38d release: Kata Containers 2.0.0-alpha2
Version bump no changes.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-25 19:02:52 -07:00
Xu Wang
f1945a4680
Merge pull request #335 from bergwolf/cleanup
cleanup: remove redundant files
2020-06-26 10:00:52 +08:00
Archana Shinde
b68d4e45ee shimv2: Removing function as no longer used
Function removeNamespace is no longer used. Get rid of
it.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2020-06-25 16:50:56 -07:00
Archana Shinde
f570a2cd40 shimv2 : Remove workaround for sharedPidNs
Removing code that existed as a workaround for a bug in
how shared process namespaces were handled in the agent.
That has been long fixed in the agent.
With this, sharedPidNs will now work with shimv2.

Fixes #337

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2020-06-25 16:50:39 -07:00
James O. D. Hunt
b2cc403eca build: Improve top-level Makefile
Define a set of functions that support the standard rules (build,
install, test, *etc*). Then simply add new components and tools to the
appropriate variable to support all the standard build semantics.

Fixes #331.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-06-25 11:19:12 +01:00
James O. D. Hunt
f2a19966b2 agent: Rename check rule to test
Changed the name of the rule that runs the tests to "test" for
consistency, but retained `check` for backwards compatibility
for now.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2020-06-25 11:18:23 +01:00
Peng Tao
a1ef594d2a cleanup: remove redundant files
And use top level VERSION for all components.

Fixes: #334
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-24 15:57:23 -07:00
Peng Tao
3bbb97add3
Merge pull request #312 from Pennyzct/network_throttle_on_qemu
rate-limiter: network I/O throttling on VM level
2020-06-25 04:59:44 +08:00
Peng Tao
bee02d47ed
Merge pull request #310 from fidencio/wip/forward_port_c3d_and_ted_yu_patches
[forward port] Bring to the development branch fixes provided by Christophe De Dinechin and Ted Yu.
2020-06-25 04:57:48 +08:00
Salvador Fuentes
5925ed491a
Merge pull request #330 from bergwolf/add-docs
merge documentation and packaging repository
2020-06-24 09:57:32 -05:00
David Gibson
ea1d799f79 qemu: Only one element of qemuPaths map is relevant
The qemuPaths field in qemuArchBase maps from machine type to the default
qemu path.  But, by the time we construct it, we already know the machine
type, so that entry ends up being the only one we care about.

So, collapse the map into a single path.  As a bonus, the qemuPath()
method can no longer fail.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-06-24 21:26:43 +10:00
David Gibson
5dffffd432 qemu: Remove useless table from qemuArchBase
The supportedQemuMachines array in qemuArchBase has a list of all the
qemu machine types supported for the architecture, with the options
for each.  But, the machineType field already tells us which of the
machine types we're actually using, and that's the only entry we
actually care about.

So, drop the table, and just have a single value with the machine type
we're actually using.  As a bonus that means the machine() method can
no longer fail, so no longer needs an error return.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-06-24 21:26:38 +10:00
David Gibson
97a02131c6 qemu: Detect and fail a bad machine type earlier
Currently, newQemuArch() doesn't return an error.  So, if passed an invalid
machine type, it will return a technically valid, but unusable qemuArch
object, which will probably fail with other errors shortly down the track.

Change this, to more cleanly fail the newQemuArch itself, letting us
detect a bad machine type earlier.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-06-24 21:07:33 +10:00
David Gibson
d6e7a58ac9 qemu: Clarify test with bad machine type
The last stanza of TestQemuAmd64Bridges is rather odd.  It tries to create
a qemu instance with a machine type of (QemuQ35 + QemuPC), or in other
words "q35pc", which isn't a thing.

What it's asserting about this is that the returned bridges list is empty
despite asking for bridges, so it looks like what this is really trying to
test is for sane behaviour when given a bad machine type.

So, split this out into a separate test, and make it explicit for clarity.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-06-24 21:02:17 +10:00
Peng Tao
e29546016e readme: fix reference to packaging tools
As we have merged it here.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-24 03:32:15 -07:00
Hui Zhu
b1a1d8e500
Merge pull request #328 from bergwolf/bump-golang
Bump golang version to 1.14.4
2020-06-24 17:52:46 +08:00
Penny Zheng
541fd58791 rate-limiter: add rate limiter unit test
add TestRxRateLimiter and TestTxRateLimiter unit tests

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:17:07 +00:00
Penny Zheng
d3098c56f6 rate-limiter: remove tc-based rate limiter
Removing tc-based rate limiter includes removing htb qdiscs, ifb
interfaces if created, etc.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:17:07 +00:00
Penny Zheng
08551287b1 rate-limiter: add tc-based tx rate limiter
Implement tc-based tx rate limiter to control network I/O outbound traffic
on VM level for hypervisors which don't support built-in rate limiter.
We take different actions, based on various inter-networking models.
For tcfilters as inter-networking model, we simply apply htb
qdisc discipline on the virtual netpair.
For other inter-networking models, such as macvtap, we resort to ifb,
by redirecting interface ingress traffic to ifb egress, and then apply htb
to ifb egress.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:17:07 +00:00
Penny Zheng
65a37b7d9c rate-limiter: add ifb interface
Ingress traffic shaping is very limited, and the htb
qdisc discipline couldn't be applied to interface ingress traffic.
Here, we import a new pseudo network interface, Intermediate Functional Block (ifb).
It is an alternative to tc filters for handling ingress traffic, by
redirecting interface ingress traffic to ifb and treat it as egress traffic there.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:17:07 +00:00
Penny Zheng
cfeb966763 rate-limiter: implement hypervisor-built-in rate limiter
As for hypervisors that support built-in rate limiter, like firecracker,
we use this built-in characteristics to implement rate limiter in kata.
kata-defined rate is in bits with scaling factors of 1000, otherwise fc-defined
rate is in bytes with scaling factors of 1024, so need reversion.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:16:58 +00:00
Penny Zheng
676ad989d7 rate-limiter: implement tc-based rx rate limiter
Implement tc-based rx rate limiter to control network I/O inbound traffic
on VM level for hypervisors which don't support built-in rate limiter.
In some detail, we use HTB(Hierarchical Token Bucket) qdisc shaping schemes
to control host interface egress traffic.
HTB shapes traffic based on the Token Bucket Filter algorithm, and one
fundamental part of the HTB qdisc is the borrowing mechanism.
Children classes borrow tokens from their parents once they have exceeded rate,
it will continue to attempt to borrow until it reaches ceil. See more details in
https://tldp.org/HOWTO/Traffic-Control-HOWTO/classful-qdiscs.html

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:14:59 +00:00
Penny Zheng
5a58ed29f1 rate-limiter: add getRateLimiter/setRateLimiter in endpoint
We use tc-based or built-in rate limiter to shape network I/O traffic
and they all must be tied to one specific interface/endpoint.
In order to tell whether we've ever added rate limiter to this interface/endpoint,
we create get/set func to reveal/store such info.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:14:51 +00:00
Penny Zheng
527c3f4634 test: Add unit test TestNewFirecrackerHypervisor
We have defined specific config file configuration-fc.toml for firecracker,
including specific features and requirements, but the related unit test
TestNewFirecrackerHypervisor is missing.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:14:42 +00:00
Penny Zheng
bd8658e362 rate-limiter: check if hypervisor supports built-in rate limiter
As for some hypervisors, like firecracker, they support built-in rate limiter
to control network I/O bandwidth on VMM level. And for some hypervisors, like qemu,
they don't.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:14:34 +00:00
Penny Zheng
c2645f5d5a rate-limiter: add rate limiter configuration/annotation on VM level
Add configuration/annotation about network I/O throttling on VM level.
rx_rate_limiter_max_rate is dedicated to control network inbound
bandwidth per pod.
tx_rate_limiter_max_rate is dedicated to control network outbound
bandwidth per pod.

Fixes: #250

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-06-24 06:14:04 +00:00
Peng Tao
782cd2ed10 packaging: merge packaging repository
git-subtree-dir: tools/packaging
git-subtree-mainline: f818b46a41
git-subtree-split: 1f22d72d5d

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 22:58:18 -07:00
Peng Tao
f818b46a41 readme: fix documentation reference link
Now it is inside the docs directory.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 21:27:56 -07:00
Peng Tao
a196c85e04 docs: merge documentation repository
Generated by
git subtree add --prefix=docs git@github.com:kata-containers/documentation.git master

git-subtree-dir: docs
git-subtree-mainline: ec146a1b39
git-subtree-split: 510287204b

Fixes: #329
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 21:27:23 -07:00
Peng Tao
c7b38e2fd7 golang: bump golang version to 1.14.4
That would help checking vendor and go.mod consistency.

Fixes: #327
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 21:07:13 -07:00
Peng Tao
66d385d7ed runtime: remove unneeded tests files
These are moved to the top directory.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 21:06:26 -07:00
Peng Tao
84b8260cfe runtime: fix vendor go.mod inconsistency
As reported by golang 1.14.3.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 21:01:11 -07:00
Peng Tao
ec146a1b39
Merge pull request #321 from dgibson/ppc64le
Don't use some x86 specific kernel and qemu options
2020-06-24 10:28:07 +08:00
Christophe de Dinechin
487520ff74 qemu: Report all errors on virtiofsd execution
The virtiofs daemon may run into errors other than the file
not existing, e.g. the file may not be executable.

Fixes: #2682

Message is now:
  virtiofs daemon /usr/local/bin/hello returned with error:
  fork/exec /usr/local/bin/virtiofsd: permission denied

instead of
  panic: runtime error: invalid memory address or nil

Fixes: #2582

Message is now:
  virtiofs daemon /usr/local/bin/hello-not-found returned with error:
  fork/exec /usr/local/bin/hello-not-found: no such file or directory

instead of:
  virtiofsd path (/usr/local/bin/hello-no-found) does not exist

Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2020-06-23 22:10:44 +02:00
Christophe de Dinechin
042426d73a katatestutils: Use the configured virtiofs daemon path
The current path is hardcoded as follows:
  virtio_fs_daemon = "/path/to/virtiofsd"

Switch to using the value of config.VirtioFSDaemon instead.

Fixes: #2686

Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2020-06-23 22:10:44 +02:00
Ted Yu
342bf3e949 virtcontainers: drop deferred func for GetAndSetSandboxBlockIndex
Fixes #2726

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2020-06-23 22:10:44 +02:00
Ted Yu
8e3bd358e5 shimv2: check correct error variable for deferred func in service#StartShim
In service#StartShim, there is no applicable error variable which is checked by deferred func because the err variable is redefined.
This PR fixes the error variable.

Fixes #2727

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com>
2020-06-23 22:10:44 +02:00
GabyCT
1f22d72d5d
Merge pull request #1081 from GabyCT/topic/enableu20
obs: Enable ubuntu 20.04
2020-06-23 14:47:09 -05:00
Julio Montes
ac9cc96a6f
Merge pull request #304 from fidencio/wip/forward_port_2703
[foward port] Add vIOMMU support to qemu q35
2020-06-23 12:20:52 -05:00
Julio Montes
98f99bb8ff
Merge pull request #326 from devimc/2020-06-23/fixLogsParser
virtcontainers: Fix structured logging in cgroups package
2020-06-23 10:39:23 -05:00
Gabriela Cervantes
8fb44eaffc obs: Enable ubuntu 20.04
This PR enables the obs packages for ubuntu 20.04.

Fixes #1080

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2020-06-23 10:18:25 -05:00
Julio Montes
0ca5983fdf virtcontainers: Fix structured logging in cgroups package
Call the `pkg/cgroups` package `SetLogger()` function to ensure all its log
records contain all required structured logging fields.

Fixes: #2782

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-06-23 07:41:12 -05:00
Julio Montes
7d2a67340b
Merge pull request #324 from bergwolf/runtime-1.11.1-for-2.0
port runtime stable commits
2020-06-23 07:12:44 -05:00
Archana Shinde
a976548fb2 shm: handle shm mount backed by empty-dir memory volumes
[cherry picked from runtime commit 3c4fe035e8041b44e1f3e06d5247938be9a1db15]

Check if shm mount is backed by empty-dir memory based volume.
If so let the logic to handle epehemeral volumes take care of this
mount, so that shm mount within the container is backed by tmpfs mount
within the the container in the VM.

Fixes: #323
Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 03:10:45 -07:00
Julio Montes
eed66021da virtcontainers: Fix structured logging in device/config package
[cherry picked from runtime commit d0dbd0485d2f4ec3760f6fa1252ded86a7709042]

Call the `device/config` package `SetLogger()` function to ensure all its log
records contain all required structured logging fields.

Signed-off-by: Julio Montes <julio.montes@intel.com>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 00:53:05 -07:00
Peng Tao
422768082d agent: update Cargo lock
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 00:52:46 -07:00
James O. D. Hunt
72283b86dd logging: Fix structured logging in store package
[ cherry-picked from runtime commit 13887bf89da9d2d7c215d77ca63129e1813e4c4a ]

Call the `store` packages `SetLogger()` function to ensure all its log
records contain all required structured logging fields.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 00:52:39 -07:00
Peng Tao
042135949a vc: make host shared path readonly
We need to make sure containers cannot modify host path unless it is explicitly shared to it. Right now we expose an additional top level shared directory to the guest and allow it to be modified. This is less ideal and can be enhanced by following method:
1. create two directories for each sandbox:
  -. /run/kata-containers/shared/sandboxes/$sbx_id/mounts/, a directory to hold all host/guest shared mounts
  -. /run/kata-containers/shared/sandboxes/$sbx_id/shared/, a host/guest shared directory (9pfs/virtiofs source dir)
2. /run/kata-containers/shared/sandboxes/$sbx_id/mounts/ is bind mounted readonly to /run/kata-containers/shared/sandboxes/$sbx_id/shared/, so guest cannot modify it
3. host-guest shared files/directories are mounted one-level under /run/kata-containers/shared/sandboxes/$sbx_id/mounts/ and thus present to guest at one level under /run/kata-containers/shared/sandboxes/$sbx_id/shared/

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-06-23 00:44:44 -07:00