Commit Graph

14950 Commits

Author SHA1 Message Date
Peng Tao
8f0d7f6acb
Merge pull request #1 from jcvenegas/image-creation-support
Image creation support
2017-12-11 10:20:42 +08:00
Mark Ryan
367ac50fe8 Merge pull request #1624 from egernst/vhostuser-enabling
qemu: introduce vhost-user handling
2017-12-08 17:28:37 +00:00
Eric Ernst
48feb29fe5 qemu: introduce vhost-user handling
Add ability to add a vhostuser device to the
QEMU commandline.  We expect two different types of devices
to be connected through a vhostuser socket: SCSI and network.

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2017-12-08 09:03:34 -08:00
Julio Montes
b8ddd24400 qemu: Add function to list hotpluggable CPUs
ExecuteQueryHotpluggableCPUs returns the list of hotpluggable CPUs

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-12-08 10:01:06 -06:00
Julio Montes
8c428ed722 qemu: Add function to hotplug CPUs
ExecuteCPUDeviceAdd hot-adds a CPU to a running VM

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-12-07 14:16:00 -06:00
Julio Montes
24b14059b3 qemu: Add functions to process QMP response
Some QMP commands like ```query-hotpluggable-cpus``` returns a
response that needs to be processed and returned to the client as
a struct. This patch adds the function ```executeCommandWithResponse```
that returns the response of a QMP command.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-12-07 12:33:12 -06:00
Jose Carlos Venegas Munoz
d2f31dfbf7
Merge pull request #4 from sameo/topic/coc
Code Of Conduct and pullapprove support
2017-12-07 10:05:14 -06:00
Xu Wang
b3404d58f6
Initial commit 2017-12-06 23:01:13 -06:00
Jose Carlos Venegas Munoz
48820425eb Makefile: Add image target
Use as default base OS centos

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:03 -06:00
Jose Carlos Venegas Munoz
0e1b500fec docs: Add README
- Overview
- Terms

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:03 -06:00
Jose Carlos Venegas Munoz
d9fe322a5f image: Add README
Add readme do document how to use ./image-builder/image_builder.sh

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:03 -06:00
Jose Carlos Venegas Munoz
52d6b044b7 image: Add script to build image
Add script to build image based on a rootfs directory.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:03 -06:00
Jose Carlos Venegas Munoz
b284e42aef rootfs: Add docker support
Allow users build a rootfs using docker, this will make easy if they
dont have dependencies need to pull packages from a distro.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:32:01 -06:00
Jose Carlos Venegas Munoz
48992d3fdf rootfs: Add README
Add docs about how rootfs generation.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:24 -06:00
Jose Carlos Venegas Munoz
206db3d585 rootfs: Add template files for new distros
Add template to add new distros.

Added a Makefile to initialize new environment.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:24 -06:00
Jose Carlos Venegas Munoz
75a9d5eab7 rootfs: Add rootfs based on CentOS 7
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:24 -06:00
Jose Carlos Venegas Munoz
5b8478c4cf rootfs: Add script for Fedora base OS
Add scirpts to build a rootfs based on Fedora.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:24 -06:00
Jose Carlos Venegas Munoz
91bf410118 rootfs: Add script for Clear Linux base OS
Add script and config to build a rootfs based
in Clear Linux OS.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:24 -06:00
Jose Carlos Venegas Munoz
f1b8da340e rootfs: Add rootfs.sh script
Add script that will use helper bash
scripts that provide a way to build a rootfs
based in a OS.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2017-12-06 13:31:15 -06:00
Samuel Ortiz
5a3115e965 pullapprove: Let the builder team approve PRs
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2017-12-04 23:36:49 +01:00
Samuel Ortiz
6a0d61363a CoC: Add Code of Conduct
We follow the OpenStack Foundation CoC.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2017-12-04 23:35:38 +01:00
Julio Montes
e39da6ca47 qmp: Add support for hot plugging VFIO devices on PCI(E) bridges
This patch adds a new function to hot plug VFIO devices on PCI(E) bridges,
This change allows to hot plug N VFIO devices in Qemu PC and Q35

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-11-29 10:48:53 -06:00
Wang Xu
53901abe10 Initial commit 2017-11-21 17:03:45 +08:00
Mark Ryan
bc030d13d1 qemu: Add a SysProcAttr parameter to CreateCloudInitISO
This change adds an additional parameter to CreateCloudInitISO that
allows users more control over the newly created xorriso process.
They can for instance specify the user under which the new qemu process
should run and which capabilities should be retained in the child
xorriso process.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-11-20 17:27:02 +00:00
Mark Ryan
11977072ea qemu: Add a SysProcAttr parameter to LaunchCustomQemu
This change adds an additional parameter to LaunchCustomQemu that
allows users more control over the newly created process.  They can
for instance specify the user under which the new qemu process should
run and which capabilities should be retained in the child qemu
process.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-11-20 17:27:02 +00:00
Samuel Ortiz
6d07e2cddb
Initial commit 2017-11-13 12:24:56 +01:00
Archana Shinde
b639da45ed qemu: Add function to hotplug vfio device
Add ability to hotplug a pci device bound to
vfio-pci driver.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2017-11-09 18:04:33 -08:00
Manohar Castelino
7e5614b8a7 Networking: Add vhost fd support
Add vhost fd support. This is needed in the case of multi queue.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-11-02 13:02:33 -07:00
Julio Montes
14316ce0b1 qemu/qmp: Implement function to hot plug PCI devices
ExecutePCIDeviceAdd is a function that can be used to hot plug
devices directly on pci(e).0 or pci(e) bridges. ExecutePCIDeviceAdd
is PCI specific because unlike ExecuteDeviceAdd, it includes an
extra parameter to specify the device address on its parent bus.

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-10-24 09:01:12 -05:00
Julio Montes
83485dc9a4 qemu: Implement Bridge struct
Bridge struct represent pci bridges(pci-bridge) or
pcie bridges(pcie-pci-bridges), bridges can be used to
hot plug devices in pc and q35 machines

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-10-24 08:31:37 -05:00
Manohar Castelino
cfa8a995de Networking: Add support for handling macvtap interfaces
Add support for macvtap interfaces. This also brings in support
for generic multiqueue support in virt containers.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-10-12 09:59:03 -07:00
Julio Montes
83126d3e05 bios: add support for custom bios
Add Bios field into qemu Config struct, this allows to
start VM with custom bios

Partially fixes https://github.com/clearcontainers/runtime/issues/686

Signed-off-by: Julio Montes <julio.montes@intel.com>
2017-10-06 14:28:12 -05:00
Manohar Castelino
3da2ef9dea QEMU: Knobs: Huge Page Support: Add support for huge pages
Add support to launch virtual machines where the RAM is
allocated using huge pages. This is useful for running
with a user mode networking stack, and for custom setups
which require high performance and low latency.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-09-26 11:29:45 -07:00
Archana Shinde
9bfa792795 vfio: Add ability to pass VFIO devices to qemu
VFIO is meant for exposing exposing direct device access
to the virtual machine.
Add ability to append VFIO devices to qemu command line.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2017-09-22 16:02:27 -07:00
Mark Ryan
a70ffd1980 Build: Fix the build after repo move.
Ciao has recently moved from github.com/01org/ciao to
github.com/ciao-project/ciao.  This moves requires us to update our
import paths to build successfully.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-09-21 17:39:45 +01:00
Manohar Castelino
0c206170c4 Knobs: Modify the behaviour of the Mlock knob.
The Mlock knob is unfortunately tied to realtime.
Allow Mlock knob to implicitly enable realtime to get the
desired swapping behavior when swapping is desired.

Note: Realtime as implemented today can only be used to enable
swap, and as such does not really control realtime behaviour.
The knob is redundant but retained here just to ensure that
when more capabilities are added in future QEMU iterations
we can take advantage of the same.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-09-15 10:36:13 -07:00
Manohar Castelino
ddee41d553 QEMU: Enable realtime options
Enable realtime options in QEMU. Also add support to control memory
locking. Turning realtime on with memory locking disabled allows
memory to be swapped out, potentially increasing density of VMs.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-09-14 08:54:35 -07:00
Manohar Castelino
4ecb9de5b3 qemu: Add support for memory pre-allocation
Add support for pre-allocating all of the RAM.
This increases the memory footprint of QEMU and should be used
only when needed.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2017-09-12 15:45:16 -07:00
Archana Shinde
1fbe6c5d1d qmp: Update block device deletion for newer versions of qemu
blockdev-del command has been added in qemu 2.9 to replace
x-blockdev-del command used earlier for deleting block devices.
Update ExecuteXBlockdevDel() to use this updated qmp command.

Rename ExecuteXBlockdevDel to ExecuteBlockdevDel as this no longer
executes x-block-del command for qemu>=2.9.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2017-08-30 18:39:51 -07:00
Sebastien Boeuf
e74aeef1ad qemu: Add disable-modern option for virtio devices
For some cases, we have to disable the fast MMIO support, by disabling
virtio 1.0. The reason for this is that we want to be able to nest our
qemu VM inside a VM run by an hypervisor with no support for fast MMIO.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2017-08-17 08:47:50 -07:00
Sebastien Boeuf
8d617ff5b9 qemu: Update virtio-net-pci command line
In case of a network device, and specifically virtio-net-pci, we have
to update to what is expected by qemu. In this case, the driver name
should be prefixed with "driver=".

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2017-08-17 00:51:32 -07:00
Archana Shinde
25a2dc8f6e qemu: Update blockdev-add qmp command to support newer qemu versions
With qemu 2.9, the qmp block-dev command was updated  from:
{ "execute": "blockdev-add", "arguments": { "options": { ... } } }

to:
{ "execute": "blockdev-add", "arguments": { ... } }

Also, instead of id, blockdev-add now requires a node-name for the
root node(https://wiki.qemu.org/index.php/ChangeLog/2.9)

Store the version information with QMPStart and use that to issue
qmp command for adding block devices in the correct format.

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2017-08-08 08:53:05 -07:00
Rob Bradford
d4f77103be misc: Remove some of the code flagged by unused linter
Unfortunately the ununused linter is overzealous with some of the fields
that it things are unused as gophercloud relies on their values. So go
ahead with the most straightforward removals but do not enable unused on
travis builds.

ciao-image/datastore/datastore_test.go:28:5⚠️ var metaDsTables is unused (U1000) (unused)
ciao-controller/api/api_test.go:39:6⚠️ func myHostname is unused (U1000) (unused)
ciao-cli/identity.go:58:3⚠️ field Description is unused (U1000) (unused)
ciao-cli/identity.go:59:3⚠️ field DomainID is unused (U1000) (unused)
ciao-cli/identity.go:60:3⚠️ field Enabled is unused (U1000) (unused)
ciao-cli/identity.go:62:3⚠️ field ParentID is unused (U1000) (unused)
ciao-cli/identity.go:63:3⚠️ field Links is unused (U1000) (unused)
ciao-cli/identity.go:70:3⚠️ field Self is unused (U1000) (unused)
ciao-cli/identity.go:71:3⚠️ field Previous is unused (U1000) (unused)
ciao-cli/identity.go:72:3⚠️ field Next is unused (U1000) (unused)
ciao-cli/identity.go:207:3⚠️ field Next is unused (U1000) (unused)
ciao-cli/identity.go:208:3⚠️ field Previous is unused (U1000) (unused)
ciao-cli/identity.go:209:3⚠️ field Self is unused (U1000) (unused)
ciao-cli/identity.go:213:3⚠️ field Description is unused (U1000) (unused)
ciao-cli/identity.go:214:3⚠️ field DomainID is unused (U1000) (unused)
ciao-cli/identity.go:215:3⚠️ field Enabled is unused (U1000) (unused)
ciao-cli/identity.go:217:3⚠️ field Links is unused (U1000) (unused)
ciao-cli/identity.go:221:3⚠️ field ParentID is unused (U1000) (unused)
ciao-cli/main.go:105:6⚠️ type action is unused (U1000) (unused)
ciao-cli/volume.go:37:6⚠️ type customVolumeExt is unused (U1000) (unused)
ciao-cli/volume.go:39:2⚠️ field customVolumeExt is unused (U1000) (unused)
networking/ciao-cnci-agent/network.go:98:8⚠️ const maxKey is unused (U1000) (unused)
networking/libsnnet/tests/parallel/parallel_test.go:371:6⚠️ func dockerNetList is unused (U1000) (unused)
networking/libsnnet/tests/parallel/parallel_test.go:379:6⚠️ func dockerNetInfo is unused (U1000) (unused)
openstack/compute/api.go:308:2⚠️ const limit is unused (U1000) (unused)
openstack/compute/api.go:309:2⚠️ const marker is unused (U1000) (unused)
openstack/compute/api.go:312:6⚠️ type pager is unused (U1000) (unused)
openstack/compute/api.go:313:2⚠️ func pager.filter is unused (U1000) (unused)
openstack/compute/api.go:314:2⚠️ func pager.nextPage is unused (U1000) (unused)
openstack/compute/api_test.go:34:6⚠️ func myHostname is unused (U1000) (unused)
ciao-controller/api.go:72:2⚠️ const statusFilter is unused (U1000) (unused)
ciao-controller/api.go:75:6⚠️ type pager is unused (U1000) (unused)
ciao-controller/api.go:76:2⚠️ func pager.filter is unused (U1000) (unused)
ciao-controller/api.go:77:2⚠️ func pager.nextPage is unused (U1000) (unused)
ciao-controller/api.go:136:25⚠️ func (*nodePager).filter is unused (U1000) (unused)
ciao-controller/api.go:198:31⚠️ func (*nodeServerPager).filter is unused (U1000) (unused)
ciao-controller/controller_test.go:107:6⚠️ func addTestTenantNoCNCI is unused (U1000) (unused)
ciao-controller/controller_test.go:1104:6⚠️ func startTestWorkload is unused (U1000) (unused)
ciao-controller/controller_test.go:1123:6⚠️ func testStartWorkloadLaunchCNCI is unused (U1000) (unused)
ciao-controller/openstack_compute.go:552:5⚠️ field Links is unused (U1000) (unused)
qemu/qmp_test.go:493:3⚠️ const seconds is unused (U1000) (unused)
qemu/qmp_test.go:494:3⚠️ const microsecondsEv1 is unused (U1000) (unused)
qemu/qmp_test.go:495:3⚠️ const device is unused (U1000) (unused)
qemu/qmp_test.go:496:3⚠️ const path is unused (U1000) (unused)
templateutils/example_test.go:53:3⚠️ field hidden is unused (U1000) (unused)

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2017-07-19 10:23:46 +01:00
Rob Bradford
a1600dc15b misc: Remove unused fields identified by structcheck
Add structcheck to the list of linters used on travis runs.

ciao-cli/event.go:109:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.eventDeleteCommand.all (structcheck)
ciao-cli/event.go:110:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.eventDeleteCommand.tenant (structcheck)
ciao-cli/external_ips.go:636:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.poolAddCommand.ips (structcheck)
ciao-cli/node.go:43:2⚠️ unused struct field github.com/01org/ciao/ciao-cli.nodeListCommand.nodeID (structcheck)
ciao-controller/client_wrapper_test.go:29:2⚠️ unused struct field github.com/01org/ciao/ciao-controller.ssntpClientWrapper.ctl (structcheck)
qemu/qmp.go:111:2⚠️ unused struct field github.com/01org/ciao/qemu.qmpResult.data (structcheck)
ssntp/ssntp_test.go:193:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.evtTracedChannel (structcheck)
ssntp/ssntp_test.go:192:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.staTracedChannel (structcheck)
ssntp/ssntp_test.go:194:2⚠️ unused struct field github.com/01org/ciao/ssntp_test.ssntpClient.errTracedChannel (structcheck)
ssntp/server.go:75:2⚠️ unused struct field github.com/01org/ciao/ssntp.Server.roleVerify (structcheck)
networking/ciao-cnci-agent/client.go:97:2⚠️ unused struct field github.com/01org/ciao/networking/ciao-cnci-agent.agentClient.netCh (structcheck)
testutil/agent.go:37:2⚠️ unused struct field github.com/01org/ciao/testutil.SsntpTestClient.ticker (structcheck)

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2017-07-13 14:52:16 +01:00
Rob Bradford
58a835e6a6 misc: Remove unused variables identified by varcheck
And add varcheck to the list of linters used on travis runs (with an
increased deadline.)

ciao-launcher/qemu_test.go:31:5⚠️ unused variable or constant imageInfoTestGood (varcheck)
ciao-launcher/qemu_test.go:44:5⚠️ unused variable or constant imageInfoTestMissingBytes (varcheck)
ciao-launcher/qemu_test.go:57:5⚠️ unused variable or constant imageInfoTestMissingLine (varcheck)
ciao-launcher/qemu_test.go:69:5⚠️ unused variable or constant imageInfoTooBig (varcheck)
ciao-launcher/qemu_test.go:82:5⚠️ unused variable or constant imageInfoBadBytes (varcheck)
configuration/configuration_test.go:35:7⚠️ unused variable or constant glanceURL (varcheck)
ciao-controller/controller_test.go:1918:5⚠️ unused variable or constant testClients (varcheck)
qemu/qmp_test.go:44:2⚠️ unused variable or constant qmpSuccess (varcheck)
qemu/qmp_test.go:45:2⚠️ unused variable or constant qmpFailure (varcheck)

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2017-07-13 14:52:16 +01:00
Sebastien Boeuf
d48b5b5f48 qemu: Add PCI option to the NetDevice
The existing NetDevice relies on virtio-net driver, but there is a
useful PCI variant which was not available: virtio-net-pci.
This patch adds this new driver and adds two parameters specific to
this: "bus" and "addr".

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2017-03-09 14:54:11 -08:00
Mark Ryan
a84228ae99 qemu: Document how cancelling works.
The code that handles the serialization and cancelling of QMP commands
is a little complex and it took me some time to remember how it actually
works and why it works in this particular way.  For this reason I've
added some comments which will hopefully make the next bug fix in this
area a little less painful.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-02-16 09:49:44 +00:00
Mark Ryan
1e7202a5a6 qemu: Fix spelling error in qmp_test.go
Command only has two ms.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-02-16 09:49:44 +00:00
Mark Ryan
c6f334533a qemu: Fix command cancelling.
There was a bug with the cancelling of commands that meant that when
an attempt was made to cancel a command and then to issue a second
command, the first, cancelled command was re-issued.  This commit
fixes the issue and adds a new test case to check that cancelling
of commands does indeed work.  There was also an issue with the
test harness which meant that tests that issued more than one command
were not actually testing the second and third commands.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-02-16 09:49:44 +00:00
Mark Ryan
a8a798b0c0 qemu, ciao-launcher: Move ConfigDrive ISO creation code to qemu
Launcher's ConfigDrive ISO creation function, createCloudInitISO has
been moved to the qemu package so that it can be re-used by ciao-down.

Signed-off-by: Mark Ryan <mark.d.ryan@intel.com>
2017-02-06 12:16:09 +00:00