Commit Graph

2077 Commits

Author SHA1 Message Date
Peng Tao
b200163de9 kata_agent: send sandbox id in CreateSandbox request
And do not append sandbox id to kernel arguments since that
would fail qemu args comparison in vm factory.

Fixes: #523

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:18:44 +08:00
Peng Tao
e69ebe5add vendor: update agent grpc
To include SandboxId in CreateSandboxRequest.

Included commits:
745fa71 (origin/pr/309) agent: set sandbox id log field in CreateSandbox
a3321f4 (origin/pr/304) vendor: Revendor netlink to fix PID 1 segfault
a9ade15 (origin/pr/293) agent: Add sandbox level reference counter for sandbox storage
5f89c07 (origin/pr/263) yamux: disable yamux keep alive in server channel
815f831 (origin/pr/303) agent: do not quit on grpc serve errors
8abefd1 (origin/pr/301) protocols/client: close yamux session when closing the stream
67643cf (origin/pr/297) agent: Fix CPU tests for both initrd and rootfs image
cf8d3c8 (origin/pr/295) agent: disable yamux keep alive
ed656df (origin/pr/284) grpc: Ignore WriteStdin after the standard input has been closed
1a52204 grpc: Don't close the terminal on CloseStdin() call
04457e3 (origin/pr/292) logging: Add sandbox field
ba2a32a (origin/pr/286) create: Reduce cyclomatic complexity of CreateContainer
bce5f78 (origin/pr/289) logging: Redirect yamux warnings/errors to logger
23311ea namespace: Add check to make sure PID namespace is not received
7fe2eba (origin/pr/273) release: Kata Containers 1.1.0
4ef4971 (origin/pr/277) travis: Enable travis ci for ppc64le
57bf4e6 (origin/pr/279) namespace: do not create path using rand.
a589bdb (origin/pr/275) CI: update static-checks.sh in travis.yml
1372df2 (origin/pr/271) logging: Fix handling of non-string fields when PID 1
4018add (origin/pr/269) agent: honour CPU constraints when agent is the init process
1cff660 (origin/pr/268) mount: Remove redundant mkdir
cc74b5d (origin/pr/266) network: Setup localhost when running as init
dc06ec7 network: Add extra sanity checks
b580ae7 network: Check all function parameters
ee106e9 main: Exit(1) on error
5bf3909 network: Reduce cyclomatic complexity
cdf9bce namespace: Simplify code
0d7f36f (origin/pr/260) agent: update container cpuset cgroup parents
4cf1809 (origin/pr/258) agent: update container cpuset cgroup before setting the new configuration
d70e5bb agent: don't modify container's config
a30395a (origin/pr/253) grpc: signal frozen containers

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:18:44 +08:00
Peng Tao
0541a5fc77 cli: fix factory assignment
A stupid typo...

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:17:56 +08:00
Peng Tao
476f21f22c codecov: remove codecov.yml
Now that we have a global team yaml, let's use it.

Fixes: #540

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-08-01 11:08:13 +08:00
Peng Tao
fc45d2e3d1
Merge pull request #497 from jcvenegas/vsock-runtime5
Use VSOCK when is available in the host.
2018-08-01 09:55:05 +08:00
Julio Montes
33643797ad virtcontainers: Use vsock if host support it
When the hypervisor option `use_vsock` is true the runtime will check for vsock
support. If vsock is supported, not proxy will be used and the shims
will connect to the VM using VSOCKS. This flag is true by default, so will use
VSOCK when possible and no proxy will be started.

fixes #383

Signed-off-by: Jose Carlos Venegas Munoz jose.carlos.venegas.munoz@intel.com
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 15:38:45 -05:00
Julio Montes
3c15bc50d0 virtcontainers: remove parseVSOCKAddr function
parseVSOCKAddr function is no more needed since now agent config
contains a field to identify if vsocks should be used or not.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
052769196d virtcontainers: implement function to cold plug vsocks
`appendVSockPCI` function can be used to cold plug vocks, vhost file descriptor
holds the context ID and it's inherit by QEMU process, ID must be unique and
disable-modern prevents qemu from relying on fast MMIO.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
1515bd07a1 virtcontainers: update KataAgentConfig to support vsocks
add extra field in KataAgentConfig structure to specify if the
kata agent have to use a vsock instead of serial port.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
3adc8626e8 virtcontainers: log type of proxy started
In order to see what proxy was started or not, we should log
its type and the URL

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:44 -05:00
Julio Montes
4680e58e08 cli: add configuration option to enable/disable vsocks
Add `use_vsock` option to enable or disable the use of vsocks
for communication between host and guest.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 13:52:43 -05:00
Jose Carlos Venegas Munoz
f389b94d8a kata-agent: Remove GRPCSock unused variable.
We already save the URL used to connect to the agent in the `state.URL` this
variable is the used to connect the shim to agnet independently the socket type
(VSOCK or serial)

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2018-07-31 10:30:05 -05:00
Julio Montes
9b283254c3 virtcontainers: Implement function to get a free context ID
FindContextID generates a random number between 3 and max uint32
and uses it as context ID.
Using ioctl findContextID checks if the context ID is free, if
the context ID is being used by other process, this function
iterates from over all valid context IDs until one is available.

`/dev/vhost-vsock` is used to check what context IDs are free,
we need it to ensure we are using a unique context ID to
create the vsocks.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 10:30:05 -05:00
Julio Montes
2339ac3f93 virtcontainers/utils: Implement function to check vsocks support
Implement function to check if the system has support for vsocks.
This function looks for vsock and vhost-vsock devices returning
true if those exist, otherwise false.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 10:29:11 -05:00
Julio Montes
8ae28888e0 vendor: update govmm
add vhostfd and disable-modern to vhost-vsock-pci

shortlog:
3830b44 qemu: add vhostfd and disable-modern to vhost-vsock-pci
f700a97 qemu/qmp: implement function to hotplug vsock-pci

Signed-off-by: Julio Montes <julio.montes@intel.com>
2018-07-31 08:42:50 -05:00
Peng Tao
44a3a441aa qemu: wait on disconnected channel in qmp shutdown
That is how govmm ensures us that the qmp channel has been cleaned
up entirely.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-31 18:34:37 +08:00
James O. D. Hunt
17a2fb886f
Merge pull request #528 from jodh-intel/fix-log-fields
Ensure containerID and sandboxID fields logged as early as possible
2018-07-31 09:09:29 +01:00
Peng Tao
c8b4fabc37 qemu: clear qmp state before wait for qemu process
So that if there is any remaining state, we do not let it interfere
with the new one. This should fix the occasional vm factory hang.

Fixes: #535

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-31 11:48:40 +08:00
z00280905
44b9214141 devices: fix typo
Fix typo.

Signed-off-by: z00280905 <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
f474af1660 devices: add some test cases
Add test cases for device manager reworks.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
b7464899ec devices: address some comments
Address some review comments:
* remove unnecessary rollback logics
* add vfio hot unplug handling.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Zhang Wei
44c37bf774 devices: rename VFIODrive to VFIODev
Rename VFIODrive to VFIODev, also rename device interface "GetDeviceDrive()" to
"GetDeviceInfo()".

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Zhang Wei
7d9d66d782 devices: remove unused functions
cleanup: remove ununsed device interface function "GetDeviceInfo()"

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
a93e2d0421 unit-tests: fix unit tests
Fix #50

Fix unit tests

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 10:05:56 +08:00
Wei Zhang
f905c16f21 device-manager: refactor device manger
Fixes #50

This commit imports a big logic change:
* host device to be attached or appended now is sandbox level resources,
one device should bind to sandbox/hypervisor first, then container could
reference it via device's unique ID.
* attach or detach device should go through the device manager interface
instead of the device interface.
* allocate device ID in global device mapper to guarantee every device
has a uniq device ID and there won't be any ID collision.

With this change, there will some changes on data format on disk for sandbox
and container, these changes also make a breakage of backward compatibility.

New persist data format:
* every sandbox will get a new "devices.json" file under "/run/vc/sbs/<sid>/"
which saves detailed device information, this also conforms to the concept that
device should be sandbox level resource.
* every container uses a "devices.json" file but with new data format:
```
[
  {
    "ID": "b80d4736e70a471f",
    "ContainerPath": "/dev/zero"
  },
  {
    "ID": "6765a06e0aa0897d",
    "ContainerPath": "/dev/null"
  }
]
```
`ID` should reference to a device in a sandbox, `ContainerPath` indicates device
path inside a container.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-31 10:03:57 +08:00
Wei Zhang
eec7fa394f devices: don't use drivers package directly.
Instead of using drivers.XXXDevice directly, we should use exported
struct from device structure. package drivers should be internal struct
and other package should avoid read it's struct content directly.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 09:59:29 +08:00
Wei Zhang
5db5f42b71 devices: remove interface VhostUserDevice
The interface "VhostUserDevice" has duplicate functions and fields with
Device, so we can merge them into one interface and manage them with one
group of interfaces.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 09:59:29 +08:00
Wei Zhang
1194154309 devices: use device manager to manage all devices
Fixes #50

Previously the devices are created with device manager and laterly
attached to hypervisor with "device.Attach()", this could work, but
there's no way to remember the reference count for every device, which
means if we plug one device to hypervisor twice, it's truly inserted
twice, but actually we only need to insert once but use it in many
places.

Use device manager as a consolidated entrypoint of device management can
give us a way to handle many "references" to single device, because it
can save all devices and remember it's use count.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-31 09:59:29 +08:00
James O. D. Hunt
763a1b6265 logging: Remove unnecessary fields and use standard names
Ensure the entire codebase uses `"sandbox"` and `"container"` log
fields for the sandboxID and containerID respectively.

Simplify code where fields can be dropped.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
3323c087c5 logging: Add cid logging to update command
PR #468 neglected to update the `update` command.

Fixes #519.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
3d5ed6669c logging: Improve cid+sid logging
Refine the changes made on #468 by adding the containerID log field as
soon as possible (before *any* virtcontainers calls). This requires
that `setExternalLoggers()` be called more times, but it's essential to
ensure the correct log fields are available as early as possible.

Partially fixes #519.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
58448bbcb8 logging: Allow SetLogger to be called multiple times
Now that the `SetLogger()` functions accept a `logrus.Entry`, they can
access the fields that have already been set for the logger and
re-apply them if `SetLogger()` is called multiple times.

This fixes a bug whereby the logger functions -- which are necessarily
called multiple times [1] -- previously ended up applying any new fields
the specified logger contained, but erroneously removing any additional
fields added since `SetLogger()` was last called.

Partially fixes #519.

--
[1] - https://github.com/kata-containers/runtime/pull/468

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
029e7ca680 api: Change logger functions to accept a log entry
Rather than accepting a `logrus.FieldLogger` interface type, change all
the `SetLogger()` functions to accept a `logrus.Entry`.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
dfb758a82d logging: Remove duplicate arch field in vc
As of #521, the runtime now adds the `arch` log field so
`virtcontainers` doesn't need to set it too.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
acdd0b8e68 logging: Split logging source into two fields
Don't use slash-delimited values in log fields - create two separate
log fields (`source` and `subsystem`) for clarity.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
c59394d3ed network: Make better use of log fields
Add key log information as log fields rather than free-format text.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:41 +01:00
James O. D. Hunt
a0be57f64f network: Always call network logger function
Rather than using the virtcontainers logger, always call the network
logger function.

Signed-off-by: James O. D. Hunt <james.o.hunt@intel.com>
2018-07-30 15:32:40 +01:00
Eric Ernst
f4a7712795
Merge pull request #530 from kata-containers/revert-301-pre-addstorage-based-devmanager
revert: "virtcontainers: support pre-add storage for frakti"
2018-07-27 13:02:57 -07:00
Sebastien Boeuf
927487c142 revert: "virtcontainers: support pre-add storage for frakti"
This PR got merged while it had some issues with some shim processes
being left behind after k8s testing. And because those issues were
real issues introduced by this PR (not some random failures), now
the master branch is broken and new pull requests cannot get the
CI passing. That's the reason why this commit revert the changes
introduced by this PR so that we can fix the master branch.

Fixes #529

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2018-07-27 09:39:56 -07:00
Peng Tao
cfbc974fec
Merge pull request #521 from bergwolf/log
factory: add SetLogger API
2018-07-27 15:52:24 +08:00
zhangwei_cs
2c3215c018
Merge pull request #301 from WeiZhang555/pre-addstorage-based-devmanager
virtcontainers: support pre-add storage for frakti
2018-07-27 15:19:30 +08:00
z00280905
b3015dda26 devices: fix typo
Fix typo.

Signed-off-by: z00280905 <zhangwei555@huawei.com>
2018-07-27 09:33:50 +08:00
Eric Ernst
2a670ce022
Merge pull request #522 from chavafg/topic/update-docker-version
versions: Update docker-ce to 18.06
2018-07-26 16:15:50 -07:00
Salvador Fuentes
da77124898 versions: Update docker-ce to 18.06
Docker 18.06 was released last week, update our
supported docker to this new version.

Fixes: #510

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2018-07-26 10:52:43 -05:00
Sebastien Boeuf
c5075d08ed
Merge pull request #517 from jcvenegas/issue-516-timeout-centos
agent: Increase timeout for check request.
2018-07-26 06:59:37 -07:00
Peng Tao
9a497fedf5 factory: add SetLogger API
So that we actually use the same logger as other packages when being
invoked by CLI.

Fixes: #520

Signed-off-by: Peng Tao <bergwolf@gmail.com>
2018-07-26 20:56:31 +08:00
James O. D. Hunt
daa65a5526
Merge pull request #514 from gkennedy12/work
cli: add AMD support to kata-check
2018-07-26 13:28:14 +01:00
Wei Zhang
198a0695ab devices: add some test cases
Add test cases for device manager reworks.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00
Wei Zhang
8391b20805 devices: address some comments
Address some review comments:
* remove unnecessary rollback logics
* add vfio hot unplug handling.

Signed-off-by: Wei Zhang <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00
Zhang Wei
04f4f528f7 devices: rename VFIODrive to VFIODev
Rename VFIODrive to VFIODev, also rename device interface "GetDeviceDrive()" to
"GetDeviceInfo()".

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2018-07-26 14:15:52 +08:00