Commit Graph

1442 Commits

Author SHA1 Message Date
Stefan Hajnoczi
46e2f885af devices: fix attach count for vhost-user-blk
Commit affd6e3216 ("devices: add reference
count for devices.") introduced an attach count for devices.  The
vhost-user-blk device increments the counter instead of decrementing it
when detaching.

Fixes: #1259
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-02-20 14:58:45 +00:00
Nitesh Konkar
6daefdb177 qemu: Cleanup Vm paths irrespective of Sandbox stop pass/fail
Sometimes qemu/qmp commands error out and VM files
get left behind on the host filesystem. Clen them up
irrespective of `stopSandbox` succeeds or fails.

Fixes: #1246

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2019-02-20 16:02:48 +05:30
Julio Montes
58d278560e virtcontainers: don't try to talk with the proxy when it's not running
To avoid long timeouts, the runtime shouldn't try to talk with the proxy
when it's not running.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:45 -06:00
Julio Montes
62c393c119 virtcontainers: change container's state to stop asap
container is killed by force, container's state MUST change its state to stop
immediately to avoid leaving it in a bad state.

fixes #1088

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Julio Montes
5201860bb0 virtcontainers: reimplement sandbox cgroup
All containers run in different cgroups even the sandbox, with this new
implementation the sandbox cpu cgroup wil be equal to the sum of all its
containers and the hypervisor process will be placed there impacting to the
containers running in the sandbox (VM). The default number of vcpus is
used when the sandbox has no constraints. For example, if default_vcpus
is 2, then quota will be 200000 and period 100000.

**c-ray test**
http://www.futuretech.blinkenlights.nl/c-ray.html

```
+=============================================+
|         | 6 threads 6cpus | 1 thread 1 cpu  |
+=============================================+
| current |   40 seconds    |   122 seconds   |
+==============================================
|   new   |   37 seconds    |   124 seconds   |
+==============================================
```

current = current cgroups implementation
new = new cgroups implementation

**workload**

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: c-ray
  annotations:
    io.kubernetes.cri.untrusted-workload: "true"
spec:
  restartPolicy: Never
  containers:
  - name: c-ray-1
    image: docker.io/devimc/c-ray:latest
    imagePullPolicy: IfNotPresent
    args: ["-t", "6", "-s", "1600x1200", "-r", "8", "-i",
          "/c-ray-1.1/sphfract", "-o", "/tmp/output.ppm"]
    resources:
      limits:
        cpu: 6
  - name: c-ray-2
    image: docker.io/devimc/c-ray:latest
    imagePullPolicy: IfNotPresent
    args: ["-t", "1", "-s", "1600x1200", "-r", "8", "-i",
          "/c-ray-1.1/sphfract", "-o", "/tmp/output.ppm"]
    resources:
      limits:
        cpu: 1
```

fixes #1153

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Julio Montes
9758cdba7c virtcontainers: move cpu cgroup implementation
cpu cgroups are container's specific hence all containers even the sandbox
should be able o create, delete and update their cgroups. The cgroup crated
matches with the cgroup path passed by the containers manager.

fixes #1117
fixes #1118
fixes #1021

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-19 13:13:44 -06:00
Peng Tao
22cee2d0cd
Merge pull request #1220 from jcvenegas/delete-force-not-fail-non-container
delete: force: Do not fail on non exiting container
2019-02-19 12:23:51 +08:00
Jose Carlos Venegas Munoz
c78d6b057e delete: force: Do not fail on non exiting container
When a container does not exist, runc does not fail.  Lets mimic this
behavior, sometimes kuberentes will try to force delete containers that
could not be created and gets confused if delete --force fails.

Fixes: #1219

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-02-18 11:39:25 -06:00
Li Wei
731ff7b13f
Merge pull request #1242 from teawater/golock
vendor: add missing comma to Gopkg.lock
2019-02-18 20:57:47 +08:00
Hui Zhu
a136999258 vendor: Add missing comma to Gopkg.lock and dep ensure
$ dep ensure
error while parsing /home/teawater/gopath/src/github.com/kata-containers/runtime/Gopkg.lock: Unable to parse the lock as TOML: (697, 6): missing comma

Add missing comma to Gopkg.lock and dep ensure to handle the issue.

Fixes: #1241

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-02-16 22:55:39 +08:00
Graham Whaley
816ea42840
Merge pull request #1202 from Pennyzct/unit-test
Unit test: fix bugs on a few unit tests on aarch64
2019-02-15 10:26:03 +00:00
Graham Whaley
7f2b2da7f7
Merge pull request #1239 from chavafg/topic/update-k8s
versions: update k8s, cri-o and containerd
2019-02-15 09:56:03 +00:00
Salvador Fuentes
409a8a5fbb versions: update k8s, cri-o and containerd
Update:
- k8s to version 1.13.3
- cri-o to version 1.13.0
- containerd to version 1.2.4, which
  according to its release notes, uses
  cri plugin version da0c016c830b2ea97fd1d737c49a568a816bf964

Fixes: #1238.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2019-02-14 16:09:12 -06:00
Julio Montes
268ddc595f
Merge pull request #1237 from chavafg/topic/update-qemu-lite-hash
versions: Update qemu-lite hash
2019-02-14 08:17:06 -06:00
Penny Zheng
23c554ee96 unit-test: refine func TestGetCPUDetails
refine struct testData in func TestGetCPUDetails to remove redundant
/unused struct field expectedVendor and expectedModel

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 15:14:27 +08:00
Penny Zheng
1b967a4a6a unit-test: add nolint comment to avoid unused warning
since all generic* could bring unused linter warnings, which lead to
CI crash, we add nolint comment to avoid them.

Fixes: #1200

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:42 +08:00
Penny Zheng
3ec56eaf9f runtime: add appendBridges for arm64
since generic func genericAppendBridges and genericBridges
is also applied for machine type QemuVirt, we use it as implementation
for appendBridges and bridges on aarch64.
since const defaultPCBridgeBus is used in generic func
genericAppendBridges for pc machine, we should define it once
in generic file, instead of redefining it in different
arch-specific files.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:42 +08:00
Penny Zheng
e93fb0b3a0 unit-test: test func for RunningOnVMM should be arch-dependent
original tests for func RunningOnVMM are sort of amd64-specific,
since all other archs don't support nested VMM for now.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 14:56:30 +08:00
Penny Zheng
0679f6fa59 unit-test: refine qemu_arm64_test.go
refine a set of test functions under qemu_arm64_test.go. e.g. test
func for memoryTopology shouldn't be the same one on amd64, since
for now, we don't support nvdimm on arm64.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 11:34:05 +08:00
Penny Zheng
44e2b9aa0a unit-test: struct TestDataa should be included in arch-indenpedent .go file
argument struct TestDataa in generic func genericTestGetCPUDetails is repeatedly
defined in almost all arch-dependent .go file, cli/kata-check_amd64_test.go,
cli/kata-check_ppc64le_test.go, etcm, except arm64. let's only declare it once in
cli/kata-check_test.go. change its name to testCPUDetail for better understanding.

Fixes: #1200

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-02-14 11:34:05 +08:00
Julio Montes
a1c85902f6 virtcontainers: add method to get hypervisor PID
hypervisor PID can be used to move the whole process and its
threads into a new cgroup.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-02-13 18:01:14 -06:00
Salvador Fuentes
c17dd11e01 versions: Update qemu-lite hash
We are using a newer hash for the packaged qemu-lite.
Update our record in versions.yaml to use the same version
in our CI.

Fixes: #1236.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2019-02-13 16:01:46 -06:00
Graham Whaley
136b188fd4
Merge pull request #1235 from nitkon/debug
virtcontainers: Improve debug messages when deleting files
2019-02-13 17:53:48 +00:00
Nitesh Konkar
a614273af5 virtcontainers: Improve debug when create/delete files
Adding debug messages which state which files
are being created/deleted could be helpful in
analyzing situations like leaky pod issues.

Fixes: #1234

Signed-off-by: Nitesh Konkar niteshkonkar@in.ibm.com
2019-02-13 20:51:33 +05:30
Graham Whaley
d07297c197
Merge pull request #1232 from jcvenegas/clear-rootfs-latest
versions: use clear linux latest rootfs.
2019-02-13 09:36:07 +00:00
Jose Carlos Venegas Munoz
84b7165e90 versions: use clear linux latest rootfs.
Update Clear Linux rootfs to latest.

Fixes: #1230

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-02-12 12:21:33 -06:00
Hui Zhu
d7b6b25059
Merge pull request #1205 from lifupan/asyncevents
shimv2: Send async task events to containerd/cri
2019-02-12 17:12:27 +08:00
Peng Tao
1c27ab79d9
Merge pull request #1225 from teawater/fixmakeinstall
Makefile: Set ARCH in GOPATH not set mode
2019-02-12 09:14:54 +08:00
Eric Ernst
6431f1f288
Merge pull request #1066 from sameo/topic/state-storage
Virtcontainers store
2019-02-11 14:42:20 -08:00
Graham Whaley
319a98fdf8
Merge pull request #1175 from nitkon/master
runtime: Do not error if only initrd/rootfs image installed
2019-02-11 17:48:49 +00:00
James O. D. Hunt
a3a3d1a53e
Merge pull request #1216 from grahamwhaley/20190206_remove_pullapprove
pullapprove: remove it
2019-02-11 10:55:07 +00:00
Hui Zhu
81c7a968ed Makefile: Set ARCH in GOPATH not set mode
In GOPATH not set mode got:
make: go: Command not found
Makefile:38: arch/-options.mk: No such file or directory
make: go: Command not found
Makefile:237: *** "ERROR: No hypervisors known for architecture  (looked for: firecracker qemu)".  Stop.

The root cause is GOPATH not set mode is not set ARCH.
Set it to fix the issue.

Fixes: #1224

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-02-11 17:06:16 +08:00
fupan
96e524d2a0 shimv2: Send task events to containerd/cri
The Runtime v2 supports an async event model. In order for the an upstream
caller (such as Docker) to get these events in the correct order a Runtime
v2 shim MUST implement some events.

For much more info, please see:
https://github.com/containerd/containerd/blob/master/runtime/v2/README.md#events

Fixes:#1204

Signed-off-by: fupan <lifupan@gmail.com>
2019-02-11 11:49:10 +08:00
Samuel Ortiz
bb99e4152b virtcontainers: Fix Store related vm factory leak
We are creating Store directories but never removing them.
Calling into a VM factory created vm Stop() will now clean the VM Store
artifacts up.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
7b0376f3d3 virtcontainers: Fix container.go cyclomatic complexity
With the Stores conversion, the newContainer() cyclomatic complexity
went over 15. We fix that by extracting the block devices creation
routine out of newContainer.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
f8e7e308c3 virtcontainers: Remove the resource storage original implementation
Now that we converted the virtcontainers code to the store package, we
can remove all the resource storage old code.

Fixes: #1099

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:33 +01:00
Samuel Ortiz
fad23ea54e virtcontainers: Conversion to Stores
We convert the whole virtcontainers code to use the store package
instead of the resource_storage one. The resource_storage removal will
happen in a separate change for a more logical split.

This change is fairly big but mostly does not change the code logic.
What really changes is when we create a store for a container or a
sandbox. We now need to explictly do so instead of just assigning a
filesystem{} instance. Other than that, the logic is kept intact.

Fixes: #1099

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-07 00:59:29 +01:00
Graham Whaley
962e1e6566 pullapprove: remove it
We are moving off pullapprove. Remove its config file.

Fixes: #1215

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-02-06 15:50:31 +00:00
Samuel Ortiz
2ecffda170 virtcontainers: store: Add a ItemLock API
The ItemLock API allows for taking shared and exclusive locks on all
items.
For virtcontainers, this is specialized into taking locks on the Lock
item, and will be used for sandbox locking.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
6e9256f483 virtcontainers: store: Add a Raw API
The Raw API creates a raw item, i.e. an item that must be handled
directly by the caller. A raw item is one that's not defined by the
store.Item enum, i.e. it is a custom, caller defined one.
The caller gets a URL back and is responsible for handling the item
directly from this URL.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
c25c60898b virtcontainers: store: Add a VC specific Store
This is basically a Store dispatcher, for storing items into their right
Store (either configuration or state).
There's very little logic here, except for finding out which store an
item belongs to in the virtcontainers context.

vc.go also provides virtcontainers specific utilities.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
ef11bf52a6 virtcontainers: store: Add a Delete API
It's going to be used to completely clean a Store away.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
f2ab58d841 virtcontainers: store: Implement the filesystem backend
new() only creates the backend and initialized the first layout.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
d22cdf2dd9 virtcontainers: store: Add an internal backend interface
All Store backends will have to implement that simple interface.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
6b87ecfc1b virtcontainers: store: Keep track of newly created Stores
When a component creates a new store from a given root path, we add it
to the store manager and return it back when another component asks for
it.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
efd50ecac9 virtcontainers: Add a Store manager
Each virtcontainers module/component should be able to get a handler on
a Store for loading component specific items. The Store manager is an
internal Store layer for tracking all created Stores.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Samuel Ortiz
4be76e9969 virtcontainers: Initial Store implementation
Store is a replacement for the current resource storage virtcontainers
implementation and the Manager is the front-end for it. The back-ends
will provide actual storage capabilities and the first one will be the
filesystem one, for storing virtcontainers Items on a local filesystem.

The main design goals for Store are the following ones:

- Simplicity: The API should be short and simple.
- Transparency: The core virtcontainers code should not care about
  the storage backend details.
- Extensibility: It should be easily extensible to add non local and in
  memory backends.

Manger provides a very short and simple API for the rest of the virtcontainers
code base to consume:

New: Creates a new Store, if needed.
Load: Loads an Item from a Store
Store: Stores an Item into a Store.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-06 14:19:18 +01:00
Eric Ernst
d3c63e66e3
Merge pull request #1212 from sameo/topic/ut-noise
Reduce virtcontainers unit test noise
2019-02-05 14:43:29 -08:00
Julio Montes
f92ca1d98d
Merge pull request #1199 from devimc/topic/snapBanner
snap: add snapcraft store banner
2019-02-05 14:34:25 -06:00
Samuel Ortiz
2affa1fe26 virtcontainers: Reduce hyperstart agent test noise
We need to pass a context to the filesystem handle.

Fixes: #1211

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-02-05 16:33:06 +01:00