Commit Graph

5350 Commits

Author SHA1 Message Date
Jose Carlos Venegas Munoz
793a47a790 qemu: fix build for fedora 28 and ubuntu 18.
This patch fix build for distros with new glib versions.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-05-15 11:41:00 -05:00
Jose Carlos Venegas Munoz
17d48e366f pkglib: Add debian.series to generated files.
If patches are found add debian.series to generated files.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-05-15 11:41:00 -05:00
Frank Cao
32c734e10b
Merge pull request #305 from jodh-intel/add-gometalinter-to-versions-db
versions: Add gometalinter to versions database
2018-05-16 00:15:12 +08:00
Julio Montes
7916e21edd developer-guide: update guest kernel image section
add steps to build kata kernel from source

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-15 09:08:23 -05:00
Peng Tao
dfdad13e45
Merge pull request #311 from jodh-intel/env-remove-agent-version
kata-env: Remove static agent version
2018-05-15 21:22:36 +08:00
James O. D. Hunt
80bedb005e kata-env: Remove static agent version
Remove the agent version from the output of `kata-env`. The value was
always a static string (`<<unknown>>`) because the runtime cannot
determine the agent version without creating a container.

Note that agent details, including the version, *are* displayed when
the user runs `kata-collect-data.sh`.

Fixes #310.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-15 10:27:06 +01:00
James O. D. Hunt
317b701499 docs: Explain SIGUSR1 usage in dev guide
Add details to the developer guide of how to obtain a backtrace by
sending a `SIGUSR1` signal to the component.

Fixes #70.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-15 09:32:21 +01:00
James O. D. Hunt
1a966503e5 docs: More log-parser detail in dev guide
Add brief details to the developer guide explaining that the
log parser can convert the format of the logs.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-15 09:32:18 +01:00
Eric Ernst
be388761a7
Merge pull request #75 from jodh-intel/fix-release-checklist-link
docs: Fix release checklist link in releases doc
2018-05-14 21:13:19 -07:00
Eric Ernst
90fc7e6d85
Merge pull request #296 from devimc/cpu/fixMemFootprint
virtcontainers/qemu: reduce memory footprint
2018-05-14 18:55:25 -07:00
Julio Montes
4527a8066a virtcontainers/qemu: honour CPU constrains
Don't fail if a new container with a CPU constraint was added to
a POD and no more vCPUs are available, instead apply the constraint
and let kernel balance the resources.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-14 17:33:31 -05:00
Julio Montes
07db945b09 virtcontainers/qemu: reduce memory footprint
There is a relation between the maximum number of vCPUs and the
memory footprint, if QEMU maxcpus option and kernel nr_cpus
cmdline argument are big, then memory footprint is big, this
issue only occurs if CPU hotplug support is enabled in the kernel,
might be because of kernel needs to allocate resources to watch all
sockets waiting for a CPU to be connected (ACPI event).

For example

```
+---------------+-------------------------+
|               | Memory Footprint (KB)   |
+---------------+-------------------------+
| NR_CPUS=240   | 186501                  |
+---------------+-------------------------+
| NR_CPUS=8     | 110684                  |
+---------------+-------------------------+
```

In order to do not affect CPU hotplug and allow to users to have containers
with the same number of physical CPUs, this patch tries to mitigate the
big memory footprint by using the actual number of physical CPUs as the
maximum number of vCPUs for each container if `default_maxvcpus` is <= 0 in
the runtime configuration file,  otherwise `default_maxvcpus` is used as the
maximum number of vCPUs.

Before this patch a container with 256MB of RAM

```
              total        used        free      shared  buff/cache   available
Mem:           195M         40M        113M         26M         41M        112M
Swap:            0B          0B          0B
```

With this patch

```
              total        used        free      shared  buff/cache   available
Mem:           236M         11M        188M         26M         36M        186M
Swap:            0B          0B          0B
```

fixes #295

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-14 17:33:31 -05:00
Jose Carlos Venegas Munoz
3295f8a5db
Merge pull request #20 from erick0z/packaging_scripts
packages: Add spec files and update scripts
2018-05-14 16:15:52 -05:00
Erick Cardona
235276fbff packages: Add spec files and update scripts
This commit adds the necessary spec files and scripts in order to be able to create
packages in OBS (Open Build System) and locally.

Fixes #15

Signed-off-by: Erick Cardona <erick.cardona.ruiz@intel.com>
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-05-14 16:13:59 -05:00
Sebastien Boeuf
8638297cae
Merge pull request #65 from egernst/architecture-docs
add initial kata architecture docs
2018-05-14 11:52:30 -07:00
Eric Ernst
4144a4ceea
Merge pull request #61 from GabyCT/topic/ubuntuinstall
docs: Add ubuntu installation guide
2018-05-14 11:25:24 -07:00
Eric Ernst
a7018665a7 Adding OCI created images, some minor cleanup
Will refactor history, but looking to have this as
a starting baseline to merge.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-14 10:15:26 -07:00
James O. D. Hunt
3b1eed3460
Merge pull request #63 from GabyCT/topic/fedorainstall
docs: Add fedora installation guide.
2018-05-14 18:01:07 +01:00
James O. D. Hunt
9a945c68b5 docs: Fix release checklist link in releases doc
The link to the release checklist in the Releases document was linking
to the parent document, not the separate checklist document.

Fixes #73.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-14 16:27:43 +01:00
James O. D. Hunt
a0b10838ee versions: Add gometalinter to versions database
Our tests CI is dependent on `gometalinter` which are run by the static
checks script. However, `gometalinter` changes a lot
and when it does, it breaks (what were) valid PRs.

Add `gometalinter` to the versions database so we can pin the version
we use to a known good one.

Fixes #304.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-14 15:39:45 +01:00
James O. D. Hunt
3151f35c86 rootfs: Add extra packages to summary file
Added the extra packages the user requested to the summary file as
previously only the default packages were listed.

Fixes #92.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-14 14:39:49 +01:00
Eric Ernst
90e3ba6027
Merge pull request #300 from kata-containers/version-0.2.0-update
build: update to version 0.2.0
2018-05-11 13:05:17 -07:00
Eric Ernst
8ce6cd53b0 release: update to version 0.2.0
bump version

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-11 10:45:33 -07:00
c00416947
f1f534c6ae vendor: update agent
This new version of kata-agent brings support for
stats of a certain container

Short logs:
077e6f9 grpc : Add the StatsContainer api for `events cli`
d29bf53 block: Get rid of device prediction for Storage as well
3b565ad block: Use PCI address to determine block device name

Signed-off-by: Haomin <caihaomin@huawei.com>
2018-05-11 19:19:57 +08:00
James O. D. Hunt
6227fa96c7
Merge pull request #278 from jcvenegas/k8s1.10
versions: Move to k8s 1.10
2018-05-11 07:59:53 +01:00
Gabriela Cervantes
872fe597a2 docs: Add ubuntu installation guide
Add ubuntu installation guide for kata containers packages.

Fixes #60

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2018-05-11 01:59:36 -05:00
Eric Ernst
91e9ed0898
Merge pull request #294 from jodh-intel/vc-reduce-path-lengths
virtcontainers: Reduce path lengths
2018-05-09 20:40:59 -07:00
Eric Ernst
03e93cd55a cleanup per jodh's feedback
Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-09 15:00:22 -07:00
Eric Ernst
0c489d322c
Merge pull request #289 from amshinde/accept-empty-env-val
oci: Allow environment values to be empty
2018-05-09 11:45:10 -07:00
James O. D. Hunt
48e949476e
Merge pull request #269 from jodh-intel/enforce-socket-path-len
socket: Enforce socket length
2018-05-09 13:49:20 +01:00
Salvador Fuentes
5b9cebd25a versions: Add cri-o version for openshift v3.9.0
Since we want to test under kubernetes 1.10 and we also
want to supoort latest version of Openshift, which is v3.9.0,
we need to test with with different versions of CRI-O.

K8s 1.10 should use cri-o 1.10, while openshift v3.9.0
should use v1.9

Depends-on: github.com/kata-containers/tests#277

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-05-09 07:39:39 -05:00
James O. D. Hunt
6a47808580 virtcontainers: Reduce path lengths
Reduce the virtcontainers prefix path to avoid hitting the 107 byte
Unix domain socket path limit.

Related #268.

Fixes #290.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-09 11:37:15 +01:00
James O. D. Hunt
bce9edd277 socket: Enforce socket length
A Unix domain socket is limited to 107 usable bytes on Linux. However,
not all code creating socket paths was checking for this limits.

Created a new `utils.BuildSocketPath()` function (with tests) to
encapsulate the logic and updated all code creating sockets to use it.

Fixes #268.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-05-09 11:36:24 +01:00
Gabriela Cervantes
bdb1b2d44f docs: Add fedora installation guide.
This will add fedora installation guide for kata containers packages.

Fixes #62

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2018-05-09 03:10:19 -05:00
Archana Shinde
b7674de3cf oci: Allow environment values to be empty
An empty string for an environment variable simply means that the
variable is unset. Do not error out if the env value is empty.

Fixes #288

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2018-05-08 09:27:35 -07:00
Eric Ernst
f6544a3524
Merge pull request #270 from devimc/command/update
cli: implement update command
2018-05-08 08:12:03 -07:00
Julio Montes
2b2aeabd89 vendor: update agent
This new version of kata-agent brings support for
updating resources and cpuset cgroups

Shortlog:
28cf91a grpc: implement update command
d96b8e1 grpc: update cpuset cgroup
4bcacdc network: Don't remove network routes or DNS when destroying sandbox
1f5cf20 network: Don't store the network info as pointers if slices used
8f828bb uevent: Fix netlink error while assigning pid in netlink client
093f61b agent: add grpc tracer UT
33bd601 agent: add server interceptor to log grpc requests
134d5d5 test: add start/stop grpc server UT
7e94246 agent: track grpc server
9fb8024 UT: add tests for channel
bea6183 agent: wait serial channel to be ready before reading
f8c8c4c agent: accept grpc connections multiple times

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:26:38 -05:00
Julio Montes
81f376920e cli: implement update command
Update command is used to update container's resources at run time.
All constraints are applied inside the VM to each container cgroup.
By now only CPU constraints are fully supported, vCPU are hot added
or removed depending of the new constraint.

fixes #189

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:26:38 -05:00
Julio Montes
0cea861f93 vendor: add docker/go-units
go-units package is used to convert integers to memory
units, for example 1024 -> 1G

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-05-08 07:13:23 -05:00
Eric Ernst
fa848ba436
Merge pull request #282 from WeiZhang555/device-manager
virtcontainers: refactor device.go to device manager
2018-05-07 21:06:28 -07:00
Eric Ernst
eb035e167f architecture: add OCI cmd diagrams
Create docker example UML flows.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 20:44:05 -07:00
Zhang Wei
f4a453b86c virtcontainers: address some comments
* Move makeNameID() func to virtcontainers/utils file as it's a generic
function for making name and ID.
* Move bindDevicetoVFIO() and bindDevicetoHost() to vfio driver package.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
28de16a450 virtcontainers: fix typo
Fix typo.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
9acbcba967 virtcontainers: make CreateDevice func private
CreateDevice() is only used by `NewDevices()` so we can make it private and
there's no need to export it.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Zhang Wei
366558ad5b virtcontainers: refactor device.go to device manager
Fixes #50

This is done for decoupling device management part from other parts.
It seperate device.go to several dirs and files:

```
virtcontainers/device
├── api
│   └── interface.go
├── config
│   └── config.go
├── drivers
│   ├── block.go
│   ├── generic.go
│   ├── utils.go
│   ├── vfio.go
│   ├── vhost_user_blk.go
│   ├── vhost_user.go
│   ├── vhost_user_net.go
│   └── vhost_user_scsi.go
└── manager
    ├── manager.go
    └── utils.go
```

* `api` contains interface definition of device management, so upper level caller
should import and use the interface, and lower level should implement the interface.
it's bridge to device drivers and callers.
* `config` contains structed exported data.
* `drivers` contains specific device drivers including block, vfio and vhost user
devices.
* `manager` exposes an external management package with a `DeviceManager`.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-05-08 10:24:26 +08:00
Archana Shinde
81503d7c69
Merge pull request #259 from bergwolf/sandbox_api_2
add sandbox process operation relay API support
2018-05-07 10:01:50 -07:00
Eric Ernst
4602ef5a60 Clean up architecture .md
Remove more references to Clear in the images as well as text.

Reworked some sections for grammar/flow.

Immediate next steps:
1. At least the delete/kill command section needs to be cleaned
 up/clarified
2. Move CRI-O UML flow example to its own section, or subection of CRI-O
3. Carve up UML diagram for basic docker example case.
4. Add section describing initrd configuration
5. Add section detailing the gRPC protocol
6. Agent section needs cleaning around gRPC description.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 07:35:09 -07:00
Eric Ernst
e04609b1cf initial architecture documentation
There is still a lot in progress, but sharing a first pass.

To do includes:
-need updated pngs (s/cc/kata)
-'signifcant' cmd section could use (simple) UML
-Need better location, and possible split up the CRI UML example
-need description of CRI-containerd
-Missing API extensions and description

Should likely carve this up into smaller .mds, as no one should read
that much text, and I don't want to get more than 200 review comments.

Contributed to by: Julio Montes, Archana Shinde, Sebastien Boeuf, and
the original CC-3.0 doc.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2018-05-07 07:33:04 -07:00
Peng Tao
410e5e6abb hyperstart_agent: fix comments
As @egernst pointed out, it should be hyperstart_agent instead of
cc-agent.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-05 09:23:46 +08:00
Peng Tao
1bb6ab9e22 api: add sandbox iostream API
It returns stdin, stdout and stderr stream of the specified process in
the container.

Fixes: #258

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-05-04 15:38:32 +08:00