Commit Graph

1760 Commits

Author SHA1 Message Date
Jose Carlos Venegas Munoz
0061e166d4 virtcontainers: move resource calculation to its own function
Make cpu and memory calculation in a different function
this help to reduce the function complexity and easy  unit test.

Fixes: #1296

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-11 12:17:01 -06:00
Jose Carlos Venegas Munoz
7504d9e50c unit-test: add TestSandboxUpdateResources
Add unit test for Update Resources

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-11 12:17:01 -06:00
Salvador Fuentes
f009a534c2 versions: update nemu to latest release
Latest release fixes entropy issue.

Fixes: #1251.
Fixes: #987.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2019-03-11 11:59:16 -06:00
James O. D. Hunt
ff7019999f
Merge pull request #1351 from jodh-intel/collect-script-add-more-details
Add crio and containerd details to collect script
2019-03-11 17:43:59 +00:00
James O. D. Hunt
cad58e8a2d
Merge pull request #1340 from devimc/topic/noJournald
pkg: reduce memory footprint
2019-03-11 14:37:48 +00:00
James O. D. Hunt
f2a506affa scripts: Add containerd details to collect script
Add a new sub-section for containerd details.

Fixes #1349.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-11 14:20:39 +00:00
James O. D. Hunt
7266d31813 scripts: Log crio config file in collect script
Add details of the crio config to the data collection script.

Partially fixes #1349.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-11 14:20:34 +00:00
James O. D. Hunt
30f9776e60 scripts: Create separate section for crio in collect script
Rather than folding crio under k8s, move it to a separate sub-section.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-11 14:18:10 +00:00
James O. D. Hunt
ae08ea3211 scripts: Add helper function to collect script
Add a `have_service()` helper function to the data collection script to
simplify the code.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-11 14:18:02 +00:00
Graham Whaley
ae4d8b453e versions: kernel: update to 4.19.28
Update kernel to latest version.

Fixes: #1288

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-03-11 11:44:37 +00:00
James O. D. Hunt
7aa48565d0
Merge pull request #1326 from ganeshmaharaj/fix-rootfs-mount
Fix rootfs mount assumptions
2019-03-11 11:32:57 +00:00
James O. D. Hunt
5b58e6a715
Merge pull request #1328 from alicefr/golint-fix
s390x: fix golint complain
2019-03-11 11:17:31 +00:00
Ace-Tang
c7ace4b4bc qemu: throw error when fail to get addr from bridges
Return error soon when addDeviceToBridge() can not get empty address
from bridges, or the error will thrown by qemu, this is not obvious.

Fixes: #1005

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-03-11 18:03:46 +08:00
James O. D. Hunt
81404baf1d
Merge pull request #1344 from caoruidong/rmphy
Network: remove Physical field in VethEndpoint
2019-03-11 10:01:32 +00:00
Wei Zhang
da80c70c0c config: enhance Feature structure
Fixes #1226

Add more fields to better describe an experimental feature.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-10 22:44:41 +08:00
Wei Zhang
111774c859 config: add docs for experimental
Fixes #1226

Add more docs for experimental features.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-12 11:03:28 +08:00
Wei Zhang
050f03bb36 config: Add config flag "experimental"
Fixes #1226

Add new flag "experimental" for supporting underworking features.
Some features are under developing which are not ready for release,
there're also some features which will break compatibility which is not
suitable to be merged into a kata minor release(x version in x.y.z)

For getting these features above merged earlier for more testing, we can
mark them as "experimental" features, and move them to formal features
when they are ready.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2019-03-12 11:03:28 +08:00
Haomin Tsai
7ff18192a4
Merge pull request #880 from jingxiaolu/rollback_when_creation_fail
test: add tests for sandbox creation rollback and cleanup
2019-03-09 14:12:22 +08:00
Julio Montes
2456ac52eb pkg: reduce memory footprint
Reduce memory footprint ~7% by disabling some systemd services like
systemd-journald and systemd-udevd, those services are just consuming memory
and are not needed. For example kata-agent logs the errors through the proxy.

fixes #1339

Signed-off-by: Julio Montes <julio.montes@intel.com>
2019-03-08 15:45:25 -06:00
Ruidong Cao
df9a4015a5 Network: remove Physical field in VethEndpoint
This a bool field. It is useless because veth endpoint is never a
physical endpoint.

Fixes #1343

Signed-off-by: Ruidong Cao <caoruidong@huawei.com>
2019-03-09 03:29:27 +08:00
Sebastien Boeuf
80cdf895c5
Merge pull request #1166 from teawater/vm_cache
VMCache: the new function that creates VMs as caches before using it
2019-03-08 07:52:11 -08:00
Hui Zhu
76d9db3e0b vendor: Add github.com/gogo/protobuf
VMCache code use github.com/gogo/protobuf.

Fixes: #52

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-08 20:59:00 +08:00
Ganesh Maharaj Mahalingam
45fe8700b8 runtime: Add unit tests
Add unit tests for the rootfs patch

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-03-07 23:50:24 -08:00
Hui Zhu
0f8b2ad007 VMCache: Update factory to run as a VMCache server
When VMCache is enabled, factory init will run as a VMcache server.

Fixes: #52

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-08 10:06:38 +08:00
Hui Zhu
90704c8bb6 VMCache: the core and the client
VMCache is a new function that creates VMs as caches before using it.
It helps speed up new container creation.
The function consists of a server and some clients communicating
through Unix socket.  The protocol is gRPC in protocols/cache/cache.proto.
The VMCache server will create some VMs and cache them by factory cache.
It will convert the VM to gRPC format and transport it when gets
requestion from clients.
Factory grpccache is the VMCache client.  It will request gRPC format
VM and convert it back to a VM.  If VMCache function is enabled,
kata-runtime will request VM from factory grpccache when it creates
a new sandbox.

VMCache has two options.
vm_cache_number specifies the number of caches of VMCache:
unspecified or == 0   --> VMCache is disabled
> 0                   --> will be set to the specified number
vm_cache_endpoint specifies the address of the Unix socket.

This commit just includes the core and the client of VMCache.

Currently, VM cache still cannot work with VM templating and vsock.
And just support qemu.

Fixes: #52

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-08 10:05:59 +08:00
Penny Zheng
d8bcddb3d1 qemu-arm64: add unit test for func appendImage on aarch64
Since we overrided the func appendImage for aarch64, we should also
provide related unit test.

Fixes: #843

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-03-07 18:38:40 +08:00
Penny Zheng
986e4dc7b8 qemu-arm64: Support nvdimm on arm64
Original guest image was reprensented as block device in qemu-aarch64,
and it will bring up write lock error when running multiple containers.
Thanks to the new expanded IPA_SIZE feature in kernel 4.20 and
Eric Auger's related patch set in qemu(which are still under upstream
review), we could fully support nvdimm on arm64.

Depends-on: github.com/kata-containers/packaging#377

Fixes: #843

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2019-03-07 18:08:11 +08:00
Alice Frosi
8ba27e14a1 s390x: remove pmu from test
Remove pmu option because it is not used and the test TestQemuS390xCPUModel
fails because the option is present

Fixes: #1329

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
2019-03-06 17:02:15 +01:00
Alice Frosi
6242af34e4 s390x: fix TestQemuS390xMemoryTopology
The memory parameters of the test are wrong

Fixes: #1329

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
2019-03-06 17:02:15 +01:00
Graham Whaley
ec6a1cc823
Merge pull request #1313 from jcvenegas/fix-unit-test
unit test: Fix local test
2019-03-06 10:18:51 +00:00
Alice Frosi
613edd5195 s390x: fix golint complain
Changed capital letter of VirtioSerialCCW to avoid golint complain

Fixes : #1327

Signed-off-by: Alice Frosi <afrosi@de.ibm.com>
2019-03-06 10:38:17 +01:00
Frank Cao
c986a08ad9
Merge pull request #1243 from grahamwhaley/20190214_uscan_urls
Add upstream version url regexp's to allow upto date checks
2019-03-06 17:08:19 +08:00
Ganesh Maharaj Mahalingam
27a92f94c8 runtime: Fix rootfs mount assumptions
This patch fixes the issue where various version of snapshotters,
overlay, block based graphdriver, containerd-shim-v2 overlay, block
based snapshotters mount & create rootfs differently and kata should be
able to handle them all.

The current version of the code always assumes that a folder named
'rootfs' exists within the mount device and that is the path the
container should start at. This patch checks the existing mount point
and if it is the same as the rootFs passed to the container, we no
longer add a suffix to the container's rootfs path.

Fixes: #1325

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
Co-Authored-by: Manohar Castelino <manohar.r.castelino@intel.com>
2019-03-05 13:41:37 -08:00
Jose Carlos Venegas Munoz
c964a26476 virtcontainers: makefile fix .ci path
If enter to vircontainers directory and do make check-go-test, the makefile
does not found the kata .ci directory use relative path to makefile.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-05 14:12:46 -06:00
Jose Carlos Venegas Munoz
fcee080a2d unit-test: Fix local test
Simplify empty string proxy type handling and cast invalid proxy type to
ProxyType.

Fixes: #1312

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-03-05 13:51:15 -06:00
Sebastien Boeuf
e2c17661b0
Merge pull request #1322 from lifupan/fixconsole
virtcontainer: watch the qemu's console when proxy's debug enabled
2019-03-05 06:50:57 -08:00
Graham Whaley
866da10f27
Merge pull request #1304 from Pennyzct/unit-test-2019/3/1
unit-test: fix undefined struct field SupportVSocks on arm64
2019-03-05 11:02:43 +00:00
Graham Whaley
8aedafad60
Merge pull request #1321 from teawater/makea64
Makefile: Fix aarch64 fail in No GO command or GOPATH not set mode
2019-03-05 10:12:49 +00:00
fupan
c4957ddd8d virtcontainer: watch the qemu's console when proxy's debug enabled
kata builtin proxy has always watched the qemu's console
whether proxy's debug is set or not, this is not aligned
with kata cli. This patch will change it and watch the
qemu's console only when proxy's debug is set in kata config.

Fixes: #1318

Signed-off-by: fupan <lifupan@gmail.com>
2019-03-05 17:09:26 +08:00
Eric Ernst
5dda0b7eea
Merge pull request #1317 from jodh-intel/fix-runtime-tracing
tracing: Fix tracing
2019-03-04 21:11:39 -08:00
l00397676
1e30673adc test: add tests for sandbox creation rollback and cleanup
Adding unit tests for rollback ops when sandbox creation failling.

Fixes: #1257

Signed-off-by: l00397676 <lujingxiao@huawei.com>
2019-03-05 11:00:35 +08:00
Xu Wang
886d859fbe
Merge pull request #1316 from lifupan/fixconfig
config: check the builtIn first when updating shim/proxy/agent
2019-03-05 09:52:39 +08:00
Hui Zhu
bdb34e7617 Makefile: Fix aarch64 fail in No GO command or GOPATH not set mode
Got aarch64 fail in No GO command or GOPATH not set mode:
Makefile:38: arch/aarch64-options.mk: No such file or directory
Makefile:237: *** "ERROR: No hypervisors known for architecture aarch64 (looked for: firecracker qemu)".  Stop.

The cause is the machine hardware name of aarch64 is aarch64 but
runtime's arch make file's name is arm64-options.mk.

This commit convert aarch64 to arm64 to fix the issue.

Fixes: #1320

Signed-off-by: Hui Zhu <teawater@hyper.sh>
2019-03-05 09:48:03 +08:00
Sebastien Boeuf
56a5accad0
Merge pull request #1307 from shaojiaxue/master
qemu: fix devID value error
2019-03-04 07:40:14 -08:00
James O. D. Hunt
c759cf5f37 tracing: Fix tracing
The store refactor (#1066) inadvertently broke runtime tracing as it
created new contexts containing trace spans.

Reworking the store changes to re-use the existing context resolves the
problem since runtime tracing assumes a single context.

Fixes #1277.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2019-03-04 11:02:31 +00:00
fupan
31232b4416 config: check the builtIn first when updating shim/proxy/agent
Check the "builtIn" first when updating the shim/proxy/agent,
thus can avoid checking the shim/proxy's binary files path which
is needless for "builtIn" type.

Fixes: #1314

Signed-off-by: fupan <lifupan@gmail.com>
2019-03-04 16:25:08 +08:00
fli
2af09d1d58
Merge pull request #1286 from teawater/nogomode
Makefile: Change "GOPATH not set" to "No GO command"
2019-03-04 09:37:56 +08:00
xueshaojia 00464843
03dd780ddd qemu: fix devID value error
reason: When excutes ExecuteNetCCWDeviceAdd, the DevID is always "virtio-".
If add-iface multy times, qemu may report "dumplicated id:virtio-".

Fixes: #1305

Signed-off-by: xueshaojia <xueshaojia@huawei.com>
2019-03-04 09:01:38 +08:00
Sebastien Boeuf
9f8d4e1291
Merge pull request #1311 from egernst/1.6.0-rc1-branch-bump
# Kata Containers 1.6.0-rc1
2019-03-01 13:15:22 -08:00
Eric Ernst
105dc2bd35 release: Kata Containers 1.6.0-rc1
- doc: Fix invalid URLs
- virtcontainers: fix vCPU calculation errors
- Network: add ipvlan unmarshal
- store: Add SetLogger API
- versions: Bump to kernel 4.19.24
- cgroups: fix failed to remove sandbox cgroup
- devices: fix attach count for vhost-user-blk
- qemu: Cleanup Vm paths irrespective of Sandbox stop pass/fail
- virtcontainers: reimplement sandbox cgroup
- delete: force: Do not fail on non exiting container
- vendor: add missing comma to Gopkg.lock
- Unit test: fix bugs on a few unit tests on aarch64
- versions: update k8s, cri-o and containerd
- versions: Update qemu-lite hash
- virtcontainers: Improve debug messages when deleting files
- versions: use clear linux latest rootfs.
- shimv2: Send async task events to containerd/cri
- Makefile: Set ARCH in GOPATH not set mode
- Virtcontainers store
- runtime: Do not error if only initrd/rootfs image installed
- pullapprove: remove it
- Reduce virtcontainers unit test noise
- snap: add snapcraft store banner
- kata-env: kata-env error out when there is no VERSION_ID.
- ppc64le: Fix hotplug issue
- Network interface removal and decoupling
- Makefile: Provide default hypervisor CMD settings
- ci: Add a CODEOWNERS file for github ack checks
- shimv2: use the runtime config file passed from containerd/cri
- cli: set config options before showing config paths
- makefile: honor DESDIR on install
- factory: do not destroy vm factory when checking status
- runtime: Fix TestCCCheckCLIFunction on P9 system

dc26508 virtcontainers: fix vCPU calculation errors
f540a80 store: Add SetLogger API
4357e85 doc: Fix invalid URLs
36141d2 Network: add ipvlan unmarshal
454775f cgroups: fix failed to remove sandbox cgroup
768658f versions: Update kernel version to 4.19.24
46e2f88 devices: fix attach count for vhost-user-blk
6daefdb qemu: Cleanup Vm paths irrespective of Sandbox stop pass/fail
58d2785 virtcontainers: don't try to talk with the proxy when it's not running
62c393c virtcontainers: change container's state to stop asap
5201860 virtcontainers: reimplement sandbox cgroup
9758cdb virtcontainers: move cpu cgroup implementation
c78d6b0 delete: force: Do not fail on non exiting container
a136999 vendor: Add missing comma to Gopkg.lock and dep ensure
409a8a5 versions: update k8s, cri-o and containerd
23c554e unit-test: refine func TestGetCPUDetails
1b967a4 unit-test: add nolint comment to avoid unused warning
3ec56ea runtime: add appendBridges for arm64
e93fb0b unit-test: test func for RunningOnVMM should be arch-dependent
0679f6f unit-test: refine qemu_arm64_test.go
44e2b9a unit-test: struct TestDataa should be included in arch-indenpedent .go file
a1c8590 virtcontainers: add method to get hypervisor PID
c17dd11 versions: Update qemu-lite hash
a614273 virtcontainers: Improve debug when create/delete files
84b7165 versions: use clear linux latest rootfs.
81c7a96 Makefile: Set ARCH in GOPATH not set mode
96e524d shimv2: Send task events to containerd/cri
bb99e41 virtcontainers: Fix Store related vm factory leak
7b0376f virtcontainers: Fix container.go cyclomatic complexity
f8e7e30 virtcontainers: Remove the resource storage original implementation
fad23ea virtcontainers: Conversion to Stores
962e1e6 pullapprove: remove it
2ecffda virtcontainers: store: Add a ItemLock API
6e9256f virtcontainers: store: Add a Raw API
c25c608 virtcontainers: store: Add a VC specific Store
ef11bf5 virtcontainers: store: Add a Delete API
f2ab58d virtcontainers: store: Implement the filesystem backend
d22cdf2 virtcontainers: store: Add an internal backend interface
6b87ecf virtcontainers: store: Keep track of newly created Stores
efd50ec virtcontainers: Add a Store manager
4be76e9 virtcontainers: Initial Store implementation
2affa1f virtcontainers: Reduce hyperstart agent test noise
a3eff87 virtcontainers: Make proxy startup sequence less noisy
f0312f6 virtcontainers: Reduce filesystem test noise
e402601 virtcontainers: Reduce sandbox test noise
799ac6e virtcontainers: Reduce qemu test noise
560902c virtcontainers: Reduce kata_agent test noise
2093fe6 virtcontainers: Reduce cc_proxy mock test noise
79ed088 virtcontainers: Reduce hyperstart mock test noise
be0726c runtime: Do not error if only initrd/rootfs image installed
7222f53 snap: add snapcraft store banner
4f51687 kata-env: kata-env error out when there is no VERSION_ID.
a48e614 ci: Add a CODEOWNERS file for github ack checks
b1c6667 vendor: Update govmm vendoring
b0986a5 ppc64le: Fix vCPU hotplug issue
68043b5 cli: set config options before showing config paths
f1a12ce shimv2: use the runtime config file passed from containerd/cri
18dcd2c virtcontainers: Decouple the network API from the sandbox one
b39cb1d virtcontainers: Remove the network interface
a02c39e Makefile: Set arch regardless of GOPATH state
d78a62d factory: do not destroy vm factory when checking status
718488b makefile: honor DESDIR on install
c93aa53 runtime: Check SMT=on/off only for P8 or lesser

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-03-01 10:45:10 -08:00