Commit Graph

284 Commits

Author SHA1 Message Date
Eric Ernst
1382662c1b build: default to using qemu-lite-x86_64
Update the runtime to use qemu-lite by default. After a
build this will be observed as the default in configuration.toml

Fixes: #293
Depends-on: github.com/kata-containers/tests#308

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-17 08:50:32 -07:00
Frank Cao
32c734e10b
Merge pull request #305 from jodh-intel/add-gometalinter-to-versions-db
versions: Add gometalinter to versions database
2018-05-16 00:15:12 +08:00
Peng Tao
dfdad13e45
Merge pull request #311 from jodh-intel/env-remove-agent-version
kata-env: Remove static agent version
2018-05-15 21:22:36 +08:00
James O. D. Hunt
80bedb005e kata-env: Remove static agent version
Remove the agent version from the output of `kata-env`. The value was
always a static string (`<<unknown>>`) because the runtime cannot
determine the agent version without creating a container.

Note that agent details, including the version, *are* displayed when
the user runs `kata-collect-data.sh`.

Fixes #310.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-15 10:27:06 +01:00
Eric Ernst
90fc7e6d85
Merge pull request #296 from devimc/cpu/fixMemFootprint
virtcontainers/qemu: reduce memory footprint
2018-05-14 18:55:25 -07:00
Julio Montes
4527a8066a virtcontainers/qemu: honour CPU constrains
Don't fail if a new container with a CPU constraint was added to
a POD and no more vCPUs are available, instead apply the constraint
and let kernel balance the resources.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-14 17:33:31 -05:00
Julio Montes
07db945b09 virtcontainers/qemu: reduce memory footprint
There is a relation between the maximum number of vCPUs and the
memory footprint, if QEMU maxcpus option and kernel nr_cpus
cmdline argument are big, then memory footprint is big, this
issue only occurs if CPU hotplug support is enabled in the kernel,
might be because of kernel needs to allocate resources to watch all
sockets waiting for a CPU to be connected (ACPI event).

For example

```
+---------------+-------------------------+
|               | Memory Footprint (KB)   |
+---------------+-------------------------+
| NR_CPUS=240   | 186501                  |
+---------------+-------------------------+
| NR_CPUS=8     | 110684                  |
+---------------+-------------------------+
```

In order to do not affect CPU hotplug and allow to users to have containers
with the same number of physical CPUs, this patch tries to mitigate the
big memory footprint by using the actual number of physical CPUs as the
maximum number of vCPUs for each container if `default_maxvcpus` is <= 0 in
the runtime configuration file,  otherwise `default_maxvcpus` is used as the
maximum number of vCPUs.

Before this patch a container with 256MB of RAM

```
              total        used        free      shared  buff/cache   available
Mem:           195M         40M        113M         26M         41M        112M
Swap:            0B          0B          0B
```

With this patch

```
              total        used        free      shared  buff/cache   available
Mem:           236M         11M        188M         26M         36M        186M
Swap:            0B          0B          0B
```

fixes #295

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-14 17:33:31 -05:00
James O. D. Hunt
a0b10838ee versions: Add gometalinter to versions database
Our tests CI is dependent on `gometalinter` which are run by the static
checks script. However, `gometalinter` changes a lot
and when it does, it breaks (what were) valid PRs.

Add `gometalinter` to the versions database so we can pin the version
we use to a known good one.

Fixes #304.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-14 15:39:45 +01:00
Eric Ernst
90e3ba6027
Merge pull request #300 from kata-containers/version-0.2.0-update
build: update to version 0.2.0
2018-05-11 13:05:17 -07:00
Eric Ernst
8ce6cd53b0 release: update to version 0.2.0
bump version

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-11 10:45:33 -07:00
James O. D. Hunt
6227fa96c7
Merge pull request #278 from jcvenegas/k8s1.10
versions: Move to k8s 1.10
2018-05-11 07:59:53 +01:00
Eric Ernst
91e9ed0898
Merge pull request #294 from jodh-intel/vc-reduce-path-lengths
virtcontainers: Reduce path lengths
2018-05-09 20:40:59 -07:00
Eric Ernst
0c489d322c
Merge pull request #289 from amshinde/accept-empty-env-val
oci: Allow environment values to be empty
2018-05-09 11:45:10 -07:00
James O. D. Hunt
48e949476e
Merge pull request #269 from jodh-intel/enforce-socket-path-len
socket: Enforce socket length
2018-05-09 13:49:20 +01:00
Salvador Fuentes
5b9cebd25a versions: Add cri-o version for openshift v3.9.0
Since we want to test under kubernetes 1.10 and we also
want to supoort latest version of Openshift, which is v3.9.0,
we need to test with with different versions of CRI-O.

K8s 1.10 should use cri-o 1.10, while openshift v3.9.0
should use v1.9

Depends-on: github.com/kata-containers/tests#277

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-05-09 07:39:39 -05:00
James O. D. Hunt
6a47808580 virtcontainers: Reduce path lengths
Reduce the virtcontainers prefix path to avoid hitting the 107 byte
Unix domain socket path limit.

Related #268.

Fixes #290.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-09 11:37:15 +01:00
James O. D. Hunt
bce9edd277 socket: Enforce socket length
A Unix domain socket is limited to 107 usable bytes on Linux. However,
not all code creating socket paths was checking for this limits.

Created a new `utils.BuildSocketPath()` function (with tests) to
encapsulate the logic and updated all code creating sockets to use it.

Fixes #268.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-09 11:36:24 +01:00
Archana Shinde
b7674de3cf oci: Allow environment values to be empty
An empty string for an environment variable simply means that the
variable is unset. Do not error out if the env value is empty.

Fixes #288

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-08 09:27:35 -07:00
Eric Ernst
f6544a3524
Merge pull request #270 from devimc/command/update
cli: implement update command
2018-05-08 08:12:03 -07:00
Julio Montes
2b2aeabd89 vendor: update agent
This new version of kata-agent brings support for
updating resources and cpuset cgroups

Shortlog:
28cf91a grpc: implement update command
d96b8e1 grpc: update cpuset cgroup
4bcacdc network: Don't remove network routes or DNS when destroying sandbox
1f5cf20 network: Don't store the network info as pointers if slices used
8f828bb uevent: Fix netlink error while assigning pid in netlink client
093f61b agent: add grpc tracer UT
33bd601 agent: add server interceptor to log grpc requests
134d5d5 test: add start/stop grpc server UT
7e94246 agent: track grpc server
9fb8024 UT: add tests for channel
bea6183 agent: wait serial channel to be ready before reading
f8c8c4c agent: accept grpc connections multiple times

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:26:38 -05:00
Julio Montes
81f376920e cli: implement update command
Update command is used to update container's resources at run time.
All constraints are applied inside the VM to each container cgroup.
By now only CPU constraints are fully supported, vCPU are hot added
or removed depending of the new constraint.

fixes #189

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:26:38 -05:00
Julio Montes
0cea861f93 vendor: add docker/go-units
go-units package is used to convert integers to memory
units, for example 1024 -> 1G

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:13:23 -05:00
Eric Ernst
fa848ba436
Merge pull request #282 from WeiZhang555/device-manager
virtcontainers: refactor device.go to device manager
2018-05-07 21:06:28 -07:00
Zhang Wei
f4a453b86c virtcontainers: address some comments
* Move makeNameID() func to virtcontainers/utils file as it's a generic
function for making name and ID.
* Move bindDevicetoVFIO() and bindDevicetoHost() to vfio driver package.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
28de16a450 virtcontainers: fix typo
Fix typo.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
9acbcba967 virtcontainers: make CreateDevice func private
CreateDevice() is only used by `NewDevices()` so we can make it private and
there's no need to export it.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
366558ad5b virtcontainers: refactor device.go to device manager
Fixes #50

This is done for decoupling device management part from other parts.
It seperate device.go to several dirs and files:

```
virtcontainers/device
├── api
│   └── interface.go
├── config
│   └── config.go
├── drivers
│   ├── block.go
│   ├── generic.go
│   ├── utils.go
│   ├── vfio.go
│   ├── vhost_user_blk.go
│   ├── vhost_user.go
│   ├── vhost_user_net.go
│   └── vhost_user_scsi.go
└── manager
    ├── manager.go
    └── utils.go
```

* `api` contains interface definition of device management, so upper level caller
should import and use the interface, and lower level should implement the interface.
it's bridge to device drivers and callers.
* `config` contains structed exported data.
* `drivers` contains specific device drivers including block, vfio and vhost user
devices.
* `manager` exposes an external management package with a `DeviceManager`.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Archana Shinde
81503d7c69
Merge pull request #259 from bergwolf/sandbox_api_2
add sandbox process operation relay API support
2018-05-07 10:01:50 -07:00
Peng Tao
410e5e6abb hyperstart_agent: fix comments
As @egernst pointed out, it should be hyperstart_agent instead of
cc-agent.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-05 09:23:46 +08:00
Peng Tao
1bb6ab9e22 api: add sandbox iostream API
It returns stdin, stdout and stderr stream of the specified process in
the container.

Fixes: #258

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
bf4ef4324e API: add sandbox winsizeprocess api
It sends tty resize request to the agent to resize a process's tty
window.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
55dc0b2995 API: add sandbox signalprocess api
It sends the signal to a process of a container, or all processes
inside a container.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
45970ba796 API: add sandbox waitprocess api
It waits a process inside the container of a sandbox.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Eric Ernst
992c895eaa
Merge pull request #267 from amshinde/pass-pci-addr
Determine and pass PCI addresses for block devices
2018-05-03 16:20:47 -07:00
Archana Shinde
717bc4cd26 virtcontainers: Pass the PCI address for block based rootfs
Store the PCI address of rootfs in case the rootfs is block
based and passed using virtio-block.
This helps up get rid of prdicting the device name inside the
container for the block device. The agent will determine the device
node name using the PCI address.

Fixes #266

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
da08a65de3 device: Assign pci address to block device for kata_agent
Store PCI address for a block device on hotplugging it via
virtio-blk. This address will be passed by kata agent in the
device "Id" field. The agent within the guest can then use this
to identify the PCI slot in the guest and create the device node
based on it.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
85865f1a2c bridge: Store the bridge address to state
We need to store the bridge address to state to use it
for assigning addresses to devices attached to teh bridge.
So we need to make sure that the bridge pointer is assigned
the address.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
718dbd2a71 device: Assign pci address for block devices
Introduce a new field in Drive to store the PCI address if the drive is
attached using virtio-blk.
Assign PCI address in the format bridge-addr/device-addr.
Since we need to assign the address while hotplugging, pass Drive
by address.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
dd927921c1 qemu: Return bridge itself with addDeviceToBridge instead of bridge bus
Change the function to return the bridge itself that the
device is attached to. This will allow bridge address to be used
for determining the PCI slot of the device within the guest.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:08 -07:00
Salvador Fuentes
7bd63d59e5 versions: update openshift to v3.9.0
Update our supported openshift version
from v3.7.1 to v3.9.0

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-05-03 10:16:36 -05:00
Peng Tao
3d1b4a1595
Merge pull request #284 from stefanha/Makefile-add-install-dependency
Makefile: add missing dependencies to install targets
2018-05-03 12:18:25 +08:00
Stefan Hajnoczi
c032061bf7 Makefile: add missing dependencies to install targets
"make install" fails on a clean working directory:

  $ make install
  install: cannot stat ‘data/kata-collect-data.sh’: No such file or directory

This happens because install and install-scripts do not depend on the
runtime.  Make doesn't know it needs to build the runtime before it can
be installed.

Add the missing dependencies to the install targets so that "make
install" works on a clean working directory and rebuilds when source
files have been modified.

Note that SCRIPTS contains the generated kata-collect-data.sh script.
That file needs to be generated before it can be installed, so make
SCRIPTS a dependency of install-scripts.

Fixes: #283
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-05-02 15:23:39 +01:00
Sebastien Boeuf
87aa1d77ed
Merge pull request #252 from bergwolf/sandbox_api_1
API: support sandbox monitor operation
2018-05-01 10:01:17 -07:00
Peng Tao
9d1311d0ee kata_agent: refactor sendReq
CI complains about cyclomatic complexity in sendReq.

warning: cyclomatic complexity 16 of function (*kataAgent).sendReq() is
high (> 15) (gocyclo)

Refactor it a bit to avoid such error. I'm not a big fan of the new code
but it is done so because golang does not support generics.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 22:42:39 +08:00
Peng Tao
35ebadcedc api: add sandbox Monitor API
It monitors the sandbox status and returns an error channel to let
caller watch it.

Fixes: #251

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 22:42:33 +08:00
Eric Ernst
70b3c774f8
Merge pull request #263 from bergwolf/sandbox_pointer
virtcontainers: always pass sandbox as a pointer
2018-05-01 07:33:33 -07:00
Peng Tao
5fb4768f83 virtcontainers: always pass sandbox as a pointer
Currently we sometimes pass it as a pointer and other times not. As
a result, the view of sandbox across virtcontainers may not be the same
and it costs extra memory copy each time we pass it by value. Fix it
by ensuring sandbox is always passed by pointers.

Fixes: #262

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 20:50:07 +08:00
Jose Carlos Venegas Munoz
4d73637829 versions: Move to k8s 1.10
Move to k8s 1.10

Fixes: #277

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-05-01 02:15:11 -05:00
Sebastien Boeuf
8d897f407f
Merge pull request #238 from jodh-intel/collect-script-support-initrd+osbuilder-file
Tidy up and add support for initrd and osbuilder metadata file
2018-04-30 16:00:15 -07:00
Eric Ernst
ff9b2bd04e
Merge pull request #256 from sboeuf/improve_container_search_CLI
cli: Optimize container research
2018-04-30 14:41:40 -07:00