Commit Graph

4961 Commits

Author SHA1 Message Date
Eric Ernst
fa848ba436
Merge pull request #282 from WeiZhang555/device-manager
virtcontainers: refactor device.go to device manager
2018-05-07 21:06:28 -07:00
Eric Ernst
eb035e167f architecture: add OCI cmd diagrams
Create docker example UML flows.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 20:44:05 -07:00
Zhang Wei
f4a453b86c virtcontainers: address some comments
* Move makeNameID() func to virtcontainers/utils file as it's a generic
function for making name and ID.
* Move bindDevicetoVFIO() and bindDevicetoHost() to vfio driver package.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
28de16a450 virtcontainers: fix typo
Fix typo.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
9acbcba967 virtcontainers: make CreateDevice func private
CreateDevice() is only used by `NewDevices()` so we can make it private and
there's no need to export it.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
366558ad5b virtcontainers: refactor device.go to device manager
Fixes #50

This is done for decoupling device management part from other parts.
It seperate device.go to several dirs and files:

```
virtcontainers/device
├── api
│   └── interface.go
├── config
│   └── config.go
├── drivers
│   ├── block.go
│   ├── generic.go
│   ├── utils.go
│   ├── vfio.go
│   ├── vhost_user_blk.go
│   ├── vhost_user.go
│   ├── vhost_user_net.go
│   └── vhost_user_scsi.go
└── manager
    ├── manager.go
    └── utils.go
```

* `api` contains interface definition of device management, so upper level caller
should import and use the interface, and lower level should implement the interface.
it's bridge to device drivers and callers.
* `config` contains structed exported data.
* `drivers` contains specific device drivers including block, vfio and vhost user
devices.
* `manager` exposes an external management package with a `DeviceManager`.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Archana Shinde
81503d7c69
Merge pull request #259 from bergwolf/sandbox_api_2
add sandbox process operation relay API support
2018-05-07 10:01:50 -07:00
Eric Ernst
4602ef5a60 Clean up architecture .md
Remove more references to Clear in the images as well as text.

Reworked some sections for grammar/flow.

Immediate next steps:
1. At least the delete/kill command section needs to be cleaned
 up/clarified
2. Move CRI-O UML flow example to its own section, or subection of CRI-O
3. Carve up UML diagram for basic docker example case.
4. Add section describing initrd configuration
5. Add section detailing the gRPC protocol
6. Agent section needs cleaning around gRPC description.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 07:35:09 -07:00
Eric Ernst
e04609b1cf initial architecture documentation
There is still a lot in progress, but sharing a first pass.

To do includes:
-need updated pngs (s/cc/kata)
-'signifcant' cmd section could use (simple) UML
-Need better location, and possible split up the CRI UML example
-need description of CRI-containerd
-Missing API extensions and description

Should likely carve this up into smaller .mds, as no one should read
that much text, and I don't want to get more than 200 review comments.

Contributed to by: Julio Montes, Archana Shinde, Sebastien Boeuf, and
the original CC-3.0 doc.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 07:33:04 -07:00
Peng Tao
410e5e6abb hyperstart_agent: fix comments
As @egernst pointed out, it should be hyperstart_agent instead of
cc-agent.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-05 09:23:46 +08:00
Peng Tao
1bb6ab9e22 api: add sandbox iostream API
It returns stdin, stdout and stderr stream of the specified process in
the container.

Fixes: #258

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
bf4ef4324e API: add sandbox winsizeprocess api
It sends tty resize request to the agent to resize a process's tty
window.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
55dc0b2995 API: add sandbox signalprocess api
It sends the signal to a process of a container, or all processes
inside a container.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Peng Tao
45970ba796 API: add sandbox waitprocess api
It waits a process inside the container of a sandbox.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00
Eric Ernst
992c895eaa
Merge pull request #267 from amshinde/pass-pci-addr
Determine and pass PCI addresses for block devices
2018-05-03 16:20:47 -07:00
Archana Shinde
717bc4cd26 virtcontainers: Pass the PCI address for block based rootfs
Store the PCI address of rootfs in case the rootfs is block
based and passed using virtio-block.
This helps up get rid of prdicting the device name inside the
container for the block device. The agent will determine the device
node name using the PCI address.

Fixes #266

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
da08a65de3 device: Assign pci address to block device for kata_agent
Store PCI address for a block device on hotplugging it via
virtio-blk. This address will be passed by kata agent in the
device "Id" field. The agent within the guest can then use this
to identify the PCI slot in the guest and create the device node
based on it.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
85865f1a2c bridge: Store the bridge address to state
We need to store the bridge address to state to use it
for assigning addresses to devices attached to teh bridge.
So we need to make sure that the bridge pointer is assigned
the address.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
718dbd2a71 device: Assign pci address for block devices
Introduce a new field in Drive to store the PCI address if the drive is
attached using virtio-blk.
Assign PCI address in the format bridge-addr/device-addr.
Since we need to assign the address while hotplugging, pass Drive
by address.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:09 -07:00
Archana Shinde
dd927921c1 qemu: Return bridge itself with addDeviceToBridge instead of bridge bus
Change the function to return the bridge itself that the
device is attached to. This will allow bridge address to be used
for determining the PCI slot of the device within the guest.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-03 10:59:08 -07:00
Salvador Fuentes
7bd63d59e5 versions: update openshift to v3.9.0
Update our supported openshift version
from v3.7.1 to v3.9.0

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-05-03 10:16:36 -05:00
Peng Tao
3d1b4a1595
Merge pull request #284 from stefanha/Makefile-add-install-dependency
Makefile: add missing dependencies to install targets
2018-05-03 12:18:25 +08:00
Stefan Hajnoczi
c032061bf7 Makefile: add missing dependencies to install targets
"make install" fails on a clean working directory:

  $ make install
  install: cannot stat ‘data/kata-collect-data.sh’: No such file or directory

This happens because install and install-scripts do not depend on the
runtime.  Make doesn't know it needs to build the runtime before it can
be installed.

Add the missing dependencies to the install targets so that "make
install" works on a clean working directory and rebuilds when source
files have been modified.

Note that SCRIPTS contains the generated kata-collect-data.sh script.
That file needs to be generated before it can be installed, so make
SCRIPTS a dependency of install-scripts.

Fixes: #283
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-05-02 15:23:39 +01:00
Sebastien Boeuf
87aa1d77ed
Merge pull request #252 from bergwolf/sandbox_api_1
API: support sandbox monitor operation
2018-05-01 10:01:17 -07:00
Peng Tao
9d1311d0ee kata_agent: refactor sendReq
CI complains about cyclomatic complexity in sendReq.

warning: cyclomatic complexity 16 of function (*kataAgent).sendReq() is
high (> 15) (gocyclo)

Refactor it a bit to avoid such error. I'm not a big fan of the new code
but it is done so because golang does not support generics.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 22:42:39 +08:00
Peng Tao
35ebadcedc api: add sandbox Monitor API
It monitors the sandbox status and returns an error channel to let
caller watch it.

Fixes: #251

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 22:42:33 +08:00
Eric Ernst
70b3c774f8
Merge pull request #263 from bergwolf/sandbox_pointer
virtcontainers: always pass sandbox as a pointer
2018-05-01 07:33:33 -07:00
Peng Tao
5fb4768f83 virtcontainers: always pass sandbox as a pointer
Currently we sometimes pass it as a pointer and other times not. As
a result, the view of sandbox across virtcontainers may not be the same
and it costs extra memory copy each time we pass it by value. Fix it
by ensuring sandbox is always passed by pointers.

Fixes: #262

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-01 20:50:07 +08:00
Jose Carlos Venegas Munoz
4d73637829 versions: Move to k8s 1.10
Move to k8s 1.10

Fixes: #277

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-05-01 02:15:11 -05:00
Sebastien Boeuf
8d897f407f
Merge pull request #238 from jodh-intel/collect-script-support-initrd+osbuilder-file
Tidy up and add support for initrd and osbuilder metadata file
2018-04-30 16:00:15 -07:00
Eric Ernst
ff9b2bd04e
Merge pull request #256 from sboeuf/improve_container_search_CLI
cli: Optimize container research
2018-04-30 14:41:40 -07:00
Sebastien Boeuf
e6f066b828 cli: Optimize container research
This commit will allow for better performance regarding the time spent
to retrieve the sandbox ID related to a container ID.

The way it works is by relying on a specific mapping between container
IDs and sanbox IDs, meaning it allows to retrieve directly the sandbox
ID related to a container ID from the CLI. This lowers complexity from
O(n²) to O(1), because we don't need to call into ListPod() which was
parsing all the pods and all the containers on the system everytime
the CLI need to retrieve this mapping.

This commit also updates the whole unit tests as a consequence. This
is involving most of them since they were all relying on ListPod()
before.

Fixes #212

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-04-30 10:53:08 -07:00
Graham Whaley
f92d7dd1c1
Merge pull request #275 from sboeuf/fix_k8s_shim_killed
virtcontainers: Properly remove the container when shim gets killed
2018-04-30 16:51:34 +01:00
Sebastien Boeuf
e78941e3e5
Merge pull request #272 from amshinde/pass-bundle-in-hooks
hooks: Send the bundle path in the state that is sent with hooks
2018-04-30 07:28:27 -07:00
Sebastien Boeuf
789dbca6d6 virtcontainers: Properly remove the container when shim gets killed
Here is an interesting case I have been debugging. I was trying to
understand why a "kubeadm reset" was not working for kata-runtime
compared to runc. In this case, the only pod started with Kata is
the kube-dns pod. For some reasons, when this pod is stopped and
removed, its containers receive some signals, 2 of them being SIGTERM
signals, which seems the way to properly stop them, but the third
container receives a SIGCONT. Obviously, nothing happens in this
case, but apparently CRI-O considers this should be the end of the
container and after a few seconds, it kills the container process
(being the shim in Kata case). Because it is using a SIGKILL, the
signal does not get forwarded to the agent because the shim itself
is killed right away. After this happened, CRI-O calls into
"kata-runtime state", we detect the shim is not running anymore
and we try to stop the container. The code will eventually call
into agent.RemoveContainer(), but this will fail and return an
error because inside the agent, the container is still running.

The approach to solve this issue here is to send a SIGKILL signal
to the container after the shim has been waited for. This call does
not check for the error returned because most of the cases, regular
use cases, will end up returning an error because the shim itself
not being there actually represents the container inside the VM has
already terminated.
And in case the shim has been killed without the possibility to
forward the signal (like described in first paragraph), the SIGKILL
will work and will allow the following call to agent.stopContainer()
to proceed to the removal of the container inside the agent.

Fixes #274

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-04-27 18:36:27 -07:00
Archana Shinde
a301a9e641 hooks: Send the bundle path in the state that is sent with hooks
We currently just send the pid in the state. While OCI specifies
a few other fields as well, this commit just adds the bundle path
and the container id to the state. This should fix the errors seen
with hooks that rely on the bundle path.

Other fields like running "state" string have been left out. As this
would need sending the strings that OCI recognises. Hooks have been
implemented in virtcontainers and sending the state string would
require calling into OCI specific code in virtcontainers.

The bundle path again is OCI specific, but this can be accessed
using annotations. Hooks really need to be moved to the cli as they
are OCI specific. This however needs network hotplug to be implemented
first so that the hooks can be called from the cli after the
VM has been created.

Fixes #271

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-04-27 15:48:58 -07:00
James O. D. Hunt
d4225ede2f
Merge pull request #260 from jcvenegas/go1.10
versions: move to go 1.10
2018-04-27 14:50:24 +01:00
Peng Tao
49e3f814bc
Merge pull request #86 from Pennyzct/gopath
kata/osbuilder: Add support for multiple GOPATH directories
2018-04-27 19:11:23 +08:00
Penny Zheng
c3ac7180f8 rootfs: Add support for multiple GOPATH directories
Current rootfs.sh fails when GOPATH is a set of directories.We
simply choose the first one as the working directory, as go get
only works against the first item in the GOPATH.

Fixes: #87

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2018-04-27 02:44:43 +00:00
Jose Carlos Venegas Munoz
20432dd99f versions: ci: Move to go 1.10
Change the latest working go version for kata.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-04-26 21:26:51 -05:00
Eric Ernst
ff3518e3ec
Merge pull request #232 from sboeuf/fix_openshift_k8s
cli: Don't wait for OCI delete to stop the sandbox
2018-04-26 15:38:48 -07:00
Peng Tao
ec6628ef6b
Merge pull request #25 from bergwolf/cgroups
kernel-config: enable proper cgroups support
2018-04-27 01:22:53 +08:00
Jose Carlos Venegas Munoz
9830810684 vendor: update covertool.
Update covertool to allow run tests with go 1.10

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-04-26 11:38:15 -05:00
Sebastien Boeuf
644489b6e7 virtcontainers: Fix gofmt issues for Go 1.10
Now that our CI has moved to Go 1.10, we need to update one file
that is not formatted as the new gofmt (1.10) expects it to be
formatted.

Fixes #249

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-04-26 11:38:15 -05:00
Peng Tao
2d94e24c87 kernel-config: enable proper cgroups support
The kata-agent requires at least the blk/pids/net_prio cgroups.

Fixes: #24

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-04-26 23:53:22 +08:00
Julio Montes
902b1d32e5
Merge pull request #23 from jcvenegas/init-ci
ci: Add initial CI scripts
2018-04-26 09:56:04 -05:00
Jose Carlos Venegas Munoz
6f560c1c4d tag_repos: Clone using https
If ssh keys are not set ssh clone wont work.

Clone using https and push using ssh.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-04-26 09:42:46 -05:00
Jose Carlos Venegas Munoz
8ba7e3c72e release: Add tests for tag_repos.sh
- Add tests for release tool tag_repos.sh
- Toplevel makefile
- Add make test target for CI

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-04-26 09:42:46 -05:00
Jose Carlos Venegas Munoz
191089fa8f ci: Add scripts needed by CI
Add required files to execute CI.

Fixes: #21

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-04-26 09:42:46 -05:00
Julio Montes
31eb51ee7d
Merge pull request #244 from jodh-intel/backtrace-on-sigusr1
cli: Backtrace on SIGUSR1
2018-04-26 07:49:10 -05:00