Files
kata-containers/virtcontainers/documentation/api/1.0/api.md
Peng Tao f886c0bf35 vc: drop container SetPid API
It is not used by anyone.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2019-07-22 19:27:52 -07:00

917 lines
26 KiB
Markdown

# virtcontainers 1.0 API
The virtcontainers 1.0 API operates on two high level objects:
[Sandboxes](#sandbox-api) and [containers](#container-api):
* [Sandbox API](#sandbox-api)
* [Container API](#container-api)
* [Examples](#examples)
## Sandbox API
The virtcontainers 1.0 sandbox API manages hardware virtualized
[sandbox lifecycles](#sandbox-functions). The virtcontainers sandbox
semantics strictly follow the
[Kubernetes](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/) ones.
The sandbox API allows callers to [create](#createsandbox), [delete](#deletesandbox),
[start](#startsandbox), [stop](#stopsandbox), [run](#runsandbox), [pause](#pausesandbox),
[resume](#resumesandbox) and [list](#listsandbox) VM (Virtual Machine) based sandboxes.
To initially create a sandbox, the API caller must prepare a
[`SandboxConfig`](#sandboxconfig) and pass it to either [`CreateSandbox`](#createsandbox)
or [`RunSandbox`](#runsandbox). Upon successful sandbox creation, the virtcontainers
API will return a [`VCSandbox`](#vcsandbox) interface back to the caller.
The `VCSandbox` interface is a sandbox abstraction hiding the internal and private
virtcontainers sandbox structure. It is a handle for API callers to manage the
sandbox lifecycle through the rest of the [sandbox API](#sandbox-functions).
* [Structures](#sandbox-structures)
* [Functions](#sandbox-functions)
### Sandbox Structures
* [`SandboxConfig`](#sandboxconfig)
* [`Resources`](#resources)
* [`HypervisorType`](#hypervisortype)
* [`HypervisorConfig`](#hypervisorconfig)
* [`AgentType`](#agenttype)
* [`ProxyType`](#proxytype)
* [`ProxyConfig`](#proxyconfig)
* [`ShimType`](#shimtype)
* [`NetworkConfig`](#networkconfig)
* [`NetInterworkingModel`](#netinterworkingmodel)
* [`Volume`](#volume)
* [`Sandbox ContainerConfig`](#sandbox-containerconfig)
* [`Sandbox Cmd`](#sandbox-cmd)
* [`Sandbox Mount`](#sandbox-mount)
* [`Sandbox DeviceInfo`](#sandbox-deviceinfo)
* [`VCSandbox`](#vcsandbox)
#### `SandboxConfig`
```Go
// SandboxConfig is a Sandbox configuration.
type SandboxConfig struct {
ID string
Hostname string
// Field specific to OCI specs, needed to setup all the hooks
Hooks Hooks
// VMConfig is the VM configuration to set for this sandbox.
VMConfig Resources
HypervisorType HypervisorType
HypervisorConfig HypervisorConfig
AgentType AgentType
AgentConfig interface{}
ProxyType ProxyType
ProxyConfig ProxyConfig
ShimType ShimType
ShimConfig interface{}
NetworkConfig NetworkConfig
// Volumes is a list of shared volumes between the host and the Sandbox.
Volumes []Volume
// Containers describe the list of containers within a Sandbox.
// This list can be empty and populated by adding containers
// to the Sandbox a posteriori.
Containers []ContainerConfig
// Annotations keys must be unique strings and must be name-spaced
// with e.g. reverse domain notation (org.clearlinux.key).
Annotations map[string]string
}
```
##### `Resources`
```Go
// Resources describes VM resources configuration.
type Resources struct {
// VCPUs is the number of available virtual CPUs.
VCPUs uint
// Memory is the amount of available memory in MiB.
Memory uint
}
```
##### `HypervisorType`
```Go
// HypervisorType describes an hypervisor type.
type HypervisorType string
const (
// QemuHypervisor is the QEMU hypervisor.
QemuHypervisor HypervisorType = "qemu"
// MockHypervisor is a mock hypervisor for testing purposes
MockHypervisor HypervisorType = "mock"
)
```
##### `HypervisorConfig`
```Go
// HypervisorConfig is the hypervisor configuration.
type HypervisorConfig struct {
// KernelPath is the guest kernel host path.
KernelPath string
// ImagePath is the guest image host path.
ImagePath string
// FirmwarePath is the bios host path
FirmwarePath string
// MachineAccelerators are machine specific accelerators
MachineAccelerators string
// HypervisorPath is the hypervisor executable host path.
HypervisorPath string
// DisableBlockDeviceUse disallows a block device from being used.
DisableBlockDeviceUse bool
// Shared file system type:
// - virtio-9p (default)
// - virtio-fs
SharedFS string
// VirtioFSDaemon is the virtio-fs vhost-user daemon path
VirtioFSDaemon string
// VirtioFSCacheSize is the virtio-fs DAX cache size in MiB
VirtioFSCacheSize uint32
// VirtioFSCache cache mode for fs version cache or "none"
VirtioFSCache string
// KernelParams are additional guest kernel parameters.
KernelParams []Param
// HypervisorParams are additional hypervisor parameters.
HypervisorParams []Param
// HypervisorMachineType specifies the type of machine being
// emulated.
HypervisorMachineType string
// Debug changes the default hypervisor and kernel parameters to
// enable debug output where available.
Debug bool
// NumVCPUs specifies default number of vCPUs for the VM.
// Sandbox configuration VMConfig.VCPUs overwrites this.
NumVCPUs uint32
// DefaultMem specifies default memory size in MiB for the VM.
// Sandbox configuration VMConfig.Memory overwrites this.
MemorySize uint32
// DefaultBridges specifies default number of bridges for the VM.
// Bridges can be used to hot plug devices
DefaultBridges uint32
// MemPrealloc specifies if the memory should be pre-allocated
MemPrealloc bool
// HugePages specifies if the memory should be pre-allocated from huge pages
HugePages bool
// Realtime Used to enable/disable realtime
Realtime bool
// Mlock is used to control memory locking when Realtime is enabled
// Realtime=true and Mlock=false, allows for swapping out of VM memory
// enabling higher density
Mlock bool
// DisableNestingChecks is used to override customizations performed
// when running on top of another VMM.
DisableNestingChecks bool
}
```
##### `AgentType`
```Go
// AgentType describes the type of guest agent a Sandbox should run.
type AgentType string
const (
// NoopAgentType is the No-Op agent.
NoopAgentType AgentType = "noop"
// KataContainersAgent is the Kata Containers agent.
KataContainersAgent AgentType = "kata"
// SocketTypeVSOCK is a VSOCK socket type for talking to an agent.
SocketTypeVSOCK = "vsock"
// SocketTypeUNIX is a UNIX socket type for talking to an agent.
// It typically means the agent is living behind a host proxy.
SocketTypeUNIX = "unix"
)
```
##### `ProxyType`
```Go
// ProxyType describes a proxy type.
type ProxyType string
const (
// NoopProxyType is the noopProxy.
NoopProxyType ProxyType = "noopProxy"
// NoProxyType is the noProxy.
NoProxyType ProxyType = "noProxy"
// KataProxyType is the kataProxy.
KataProxyType ProxyType = "kataProxy"
)
```
##### `ProxyConfig`
```Go
// ProxyConfig is a structure storing information needed from any
// proxy in order to be properly initialized.
type ProxyConfig struct {
Path string
Debug bool
}
```
##### `ShimType`
```Go
// ShimType describes a shim type.
type ShimType string
const (
// NoopShimType is the noopShim.
NoopShimType ShimType = "noopShim"
// KataShimType is the Kata Containers shim type.
KataShimType ShimType = "kataShim"
)
```
##### `NetworkConfig`
```Go
// NetworkConfig is the network configuration related to a network.
type NetworkConfig struct {
NetNSPath string
InterworkingModel NetInterworkingModel
}
```
###### `NetInterworkingModel`
```Go
// NetInterworkingModel defines the network model connecting
// the network interface to the virtual machine.
type NetInterworkingModel int
const (
// NetXConnectDefaultModel Ask to use DefaultNetInterworkingModel
NetXConnectDefaultModel NetInterworkingModel = iota
// NetXConnectBridgedModel uses a linux bridge to interconnect
// the container interface to the VM. This is the
// safe default that works for most cases except
// macvlan and ipvlan
NetXConnectBridgedModel
// NetXConnectMacVtapModel can be used when the Container network
// interface can be bridged using macvtap
NetXConnectMacVtapModel
// NetXConnectEnlightenedModel can be used when the Network plugins
// are enlightened to create VM native interfaces
// when requested by the runtime
// This will be used for vethtap, macvtap, ipvtap
NetXConnectEnlightenedModel
// NetXConnectInvalidModel is the last item to check valid values by IsValid()
NetXConnectInvalidModel
)
```
##### `Volume`
```Go
// Volume is a shared volume between the host and the VM,
// defined by its mount tag and its host path.
type Volume struct {
// MountTag is a label used as a hint to the guest.
MountTag string
// HostPath is the host filesystem path for this volume.
HostPath string
}
```
##### Sandbox `ContainerConfig`
```Go
// ContainerConfig describes one container runtime configuration.
type ContainerConfig struct {
ID string
// RootFs is the container workload image on the host.
RootFs string
// ReadOnlyRootfs indicates if the rootfs should be mounted readonly
ReadonlyRootfs bool
// Cmd specifies the command to run on a container
Cmd Cmd
// Annotations allow clients to store arbitrary values,
// for example to add additional status values required
// to support particular specifications.
Annotations map[string]string
Mounts []Mount
// Device configuration for devices that must be available within the container.
DeviceInfos []DeviceInfo
}
```
###### Sandbox `Cmd`
```Go
// Cmd represents a command to execute in a running container.
type Cmd struct {
Args []string
Envs []EnvVar
WorkDir string
// Note that these fields *MUST* remain as strings.
//
// The reason being that we want runtimes to be able to support CLI
// operations like "exec --user=". That option allows the
// specification of a user (either as a string username or a numeric
// UID), and may optionally also include a group (groupame or GID).
//
// Since this type is the interface to allow the runtime to specify
// the user and group the workload can run as, these user and group
// fields cannot be encoded as integer values since that would imply
// the runtime itself would need to perform a UID/GID lookup on the
// user-specified username/groupname. But that isn't practically
// possible given that to do so would require the runtime to access
// the image to allow it to interrogate the appropriate databases to
// convert the username/groupnames to UID/GID values.
//
// Note that this argument applies solely to the _runtime_ supporting
// a "--user=" option when running in a "standalone mode" - there is
// no issue when the runtime is called by a container manager since
// all the user and group mapping is handled by the container manager
// and specified to the runtime in terms of UID/GID's in the
// configuration file generated by the container manager.
User string
PrimaryGroup string
SupplementaryGroups []string
Interactive bool
Console string
Detach bool
NoNewPrivileges bool
Capabilities LinuxCapabilities
}
```
###### Sandbox `Mount`
```Go
// Mount describes a container mount.
type Mount struct {
Source string
Destination string
// Type specifies the type of filesystem to mount.
Type string
// Options list all the mount options of the filesystem.
Options []string
// HostPath used to store host side bind mount path
HostPath string
// ReadOnly specifies if the mount should be read only or not
ReadOnly bool
}
```
###### Sandbox `DeviceInfo`
```Go
// DeviceInfo is an embedded type that contains device data common to all types of devices.
type DeviceInfo struct {
// Device path on host
HostPath string
// Device path inside the container
ContainerPath string
// Type of device: c, b, u or p
// c , u - character(unbuffered)
// p - FIFO
// b - block(buffered) special file
// More info in mknod(1).
DevType string
// Major, minor numbers for device.
Major int64
Minor int64
// FileMode permission bits for the device.
FileMode os.FileMode
// id of the device owner.
UID uint32
// id of the device group.
GID uint32
// Hotplugged is used to store device state indicating if the
// device was hotplugged.
Hotplugged bool
// ID for the device that is passed to the hypervisor.
ID string
}
```
#### `VCSandbox`
```Go
// VCSandbox is the Sandbox interface
// (required since virtcontainers.Sandbox only contains private fields)
type VCSandbox interface {
Annotations(key string) (string, error)
GetAllContainers() []VCContainer
GetAnnotations() map[string]string
GetContainer(containerID string) VCContainer
ID() string
SetAnnotations(annotations map[string]string) error
}
```
### Sandbox Functions
* [`CreateSandbox`](#createsandbox)
* [`DeleteSandbox`](#deletesandbox)
* [`StartSandbox`](#startsandbox)
* [`StopSandbox`](#stopsandbox)
* [`RunSandbox`](#runsandbox)
* [`ListSandbox`](#listsandbox)
* [`StatusSandbox`](#statussandbox)
* [`PauseSandbox`](#pausesandbox)
* [`ResumeSandbox`](#resumesandbox)
#### `CreateSandbox`
```Go
// CreateSandbox is the virtcontainers sandbox creation entry point.
// CreateSandbox creates a sandbox and its containers. It does not start them.
func CreateSandbox(sandboxConfig SandboxConfig) (VCSandbox, error)
```
#### `DeleteSandbox`
```Go
// DeleteSandbox is the virtcontainers sandbox deletion entry point.
// DeleteSandbox will stop an already running container and then delete it.
func DeleteSandbox(sandboxID string) (VCSandbox, error)
```
#### `StartSandbox`
```Go
// StartSandbox is the virtcontainers sandbox starting entry point.
// StartSandbox will talk to the given hypervisor to start an existing
// sandbox and all its containers.
func StartSandbox(sandboxID string) (VCSandbox, error)
```
#### `StopSandbox`
```Go
// StopSandbox is the virtcontainers sandbox stopping entry point.
// StopSandbox will talk to the given agent to stop an existing sandbox
// and destroy all containers within that sandbox.
func StopSandbox(sandboxID string) (VCSandbox, error)
```
#### `RunSandbox`
```Go
// RunSandbox is the virtcontainers sandbox running entry point.
// RunSandbox creates a sandbox and its containers and then it starts them.
func RunSandbox(sandboxConfig SandboxConfig) (VCSandbox, error)
```
#### `ListSandbox`
```Go
// ListSandbox is the virtcontainers sandbox listing entry point.
func ListSandbox() ([]SandboxStatus, error)
```
#### `StatusSandbox`
```Go
// StatusSandbox is the virtcontainers sandbox status entry point.
func StatusSandbox(sandboxID string) (SandboxStatus, error)
```
#### `PauseSandbox`
```Go
// PauseSandbox is the virtcontainers pausing entry point which pauses an
// already running sandbox.
func PauseSandbox(sandboxID string) (VCSandbox, error)
```
#### `ResumeSandbox`
```Go
// ResumeSandbox is the virtcontainers resuming entry point which resumes
// (or unpauses) and already paused sandbox.
func ResumeSandbox(sandboxID string) (VCSandbox, error)
```
## Container API
The virtcontainers 1.0 container API manages sandbox
[container lifecycles](#container-functions).
A virtcontainers container is process running inside a containerized
environment, as part of a hardware virtualized context. In other words,
a virtcontainers container is just a regular container running inside a
virtual machine's guest OS.
A virtcontainers container always belong to one and only one
virtcontainers sandbox, again following the
[Kubernetes](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview).
logic and semantics.
The container API allows callers to [create](#createcontainer),
[delete](#deletecontainer), [start](#startcontainer), [stop](#stopcontainer),
[kill](#killcontainer) and [observe](#statuscontainer) containers. It also
allows for running [additional processes](#entercontainer) inside a
specific container.
As a virtcontainers container is always linked to a sandbox, the entire container
API always takes a sandbox ID as its first argument.
To create a container, the API caller must prepare a
[`ContainerConfig`](#sandbox-containerconfig) and pass it to
[`CreateContainer`](#createcontainer) together with a sandbox ID. Upon successful
container creation, the virtcontainers API will return a
[`VCContainer`](#vccontainer) interface back to the caller.
The `VCContainer` interface is a container abstraction hiding the internal
and private virtcontainers container structure. It is a handle for API callers
to manage the container lifecycle through the rest of the
[container API](#container-functions).
* [Structures](#container-structures)
* [Functions](#container-functions)
### Container Structures
* [Container `ContainerConfig`](#container-containerconfig)
* [Container `Cmd`](#container-cmd)
* [Container `Mount`](#container-mount)
* [Container `DeviceInfo`](#container-deviceinfo)
* [`Process`](#process)
* [`ContainerStatus`](#containerstatus)
* [`ProcessListOptions`](#processlistoptions)
* [`VCContainer`](#vccontainer)
#### Container `ContainerConfig`
```Go
// ContainerConfig describes one container runtime configuration.
type ContainerConfig struct {
ID string
// RootFs is the container workload image on the host.
RootFs string
// ReadOnlyRootfs indicates if the rootfs should be mounted readonly
ReadonlyRootfs bool
// Cmd specifies the command to run on a container
Cmd Cmd
// Annotations allow clients to store arbitrary values,
// for example to add additional status values required
// to support particular specifications.
Annotations map[string]string
Mounts []Mount
// Device configuration for devices that must be available within the container.
DeviceInfos []DeviceInfo
}
```
##### Container `Cmd`
```Go
// Cmd represents a command to execute in a running container.
type Cmd struct {
Args []string
Envs []EnvVar
WorkDir string
// Note that these fields *MUST* remain as strings.
//
// The reason being that we want runtimes to be able to support CLI
// operations like "exec --user=". That option allows the
// specification of a user (either as a string username or a numeric
// UID), and may optionally also include a group (groupame or GID).
//
// Since this type is the interface to allow the runtime to specify
// the user and group the workload can run as, these user and group
// fields cannot be encoded as integer values since that would imply
// the runtime itself would need to perform a UID/GID lookup on the
// user-specified username/groupname. But that isn't practically
// possible given that to do so would require the runtime to access
// the image to allow it to interrogate the appropriate databases to
// convert the username/groupnames to UID/GID values.
//
// Note that this argument applies solely to the _runtime_ supporting
// a "--user=" option when running in a "standalone mode" - there is
// no issue when the runtime is called by a container manager since
// all the user and group mapping is handled by the container manager
// and specified to the runtime in terms of UID/GID's in the
// configuration file generated by the container manager.
User string
PrimaryGroup string
SupplementaryGroups []string
Interactive bool
Console string
Detach bool
NoNewPrivileges bool
Capabilities LinuxCapabilities
}
```
##### Container `Mount`
```Go
// Mount describes a container mount.
type Mount struct {
Source string
Destination string
// Type specifies the type of filesystem to mount.
Type string
// Options list all the mount options of the filesystem.
Options []string
// HostPath used to store host side bind mount path
HostPath string
// ReadOnly specifies if the mount should be read only or not
ReadOnly bool
}
```
##### Container `DeviceInfo`
```Go
// DeviceInfo is an embedded type that contains device data common to all types of devices.
type DeviceInfo struct {
// Device path on host
HostPath string
// Device path inside the container
ContainerPath string
// Type of device: c, b, u or p
// c , u - character(unbuffered)
// p - FIFO
// b - block(buffered) special file
// More info in mknod(1).
DevType string
// Major, minor numbers for device.
Major int64
Minor int64
// FileMode permission bits for the device.
FileMode os.FileMode
// id of the device owner.
UID uint32
// id of the device group.
GID uint32
// Hotplugged is used to store device state indicating if the
// device was hotplugged.
Hotplugged bool
// ID for the device that is passed to the hypervisor.
ID string
}
```
#### `Process`
```Go
// Process gathers data related to a container process.
type Process struct {
// Token is the process execution context ID. It must be
// unique per sandbox.
// Token is used to manipulate processes for containers
// that have not started yet, and later identify them
// uniquely within a sandbox.
Token string
// Pid is the process ID as seen by the host software
// stack, e.g. CRI-O, containerd. This is typically the
// shim PID.
Pid int
StartTime time.Time
}
```
#### `ContainerStatus`
```Go
// ContainerStatus describes a container status.
type ContainerStatus struct {
ID string
State State
PID int
StartTime time.Time
RootFs string
// Annotations allow clients to store arbitrary values,
// for example to add additional status values required
// to support particular specifications.
Annotations map[string]string
}
```
#### `ProcessListOptions`
```Go
// ProcessListOptions contains the options used to list running
// processes inside the container
type ProcessListOptions struct {
// Format describes the output format to list the running processes.
// Formats are unrelated to ps(1) formats, only two formats can be specified:
// "json" and "table"
Format string
// Args contains the list of arguments to run ps(1) command.
// If Args is empty the agent will use "-ef" as options to ps(1).
Args []string
}
```
#### `VCContainer`
```Go
// VCContainer is the Container interface
// (required since virtcontainers.Container only contains private fields)
type VCContainer interface {
GetAnnotations() map[string]string
GetPid() int
GetToken() string
ID() string
Sandbox() VCSandbox
Process() Process
}
```
### Container Functions
* [`CreateContainer`](#createcontainer)
* [`DeleteContainer`](#deletecontainer)
* [`StartContainer`](#startcontainer)
* [`StopContainer`](#stopcontainer)
* [`EnterContainer`](#entercontainer)
* [`StatusContainer`](#statuscontainer)
* [`KillContainer`](#killcontainer)
* [`ProcessListContainer`](#processlistcontainer)
#### `CreateContainer`
```Go
// CreateContainer is the virtcontainers container creation entry point.
// CreateContainer creates a container on a given sandbox.
func CreateContainer(sandboxID string, containerConfig ContainerConfig) (VCSandbox, VCContainer, error)
```
#### `DeleteContainer`
```Go
// DeleteContainer is the virtcontainers container deletion entry point.
// DeleteContainer deletes a Container from a Sandbox. If the container is running,
// it needs to be stopped first.
func DeleteContainer(sandboxID, containerID string) (VCContainer, error)
```
#### `StartContainer`
```Go
// StartContainer is the virtcontainers container starting entry point.
// StartContainer starts an already created container.
func StartContainer(sandboxID, containerID string) (VCContainer, error)
```
#### `StopContainer`
```Go
// StopContainer is the virtcontainers container stopping entry point.
// StopContainer stops an already running container.
func StopContainer(sandboxID, containerID string) (VCContainer, error)
```
#### `EnterContainer`
```Go
// EnterContainer is the virtcontainers container command execution entry point.
// EnterContainer enters an already running container and runs a given command.
func EnterContainer(sandboxID, containerID string, cmd Cmd) (VCSandbox, VCContainer, *Process, error)
```
#### `StatusContainer`
```Go
// StatusContainer is the virtcontainers container status entry point.
// StatusContainer returns a detailed container status.
func StatusContainer(sandboxID, containerID string) (ContainerStatus, error)
```
#### `KillContainer`
```Go
// KillContainer is the virtcontainers entry point to send a signal
// to a container running inside a sandbox. If all is true, all processes in
// the container will be sent the signal.
func KillContainer(sandboxID, containerID string, signal syscall.Signal, all bool) error
```
#### `ProcessListContainer`
```Go
// ProcessListContainer is the virtcontainers entry point to list
// processes running inside a container
func ProcessListContainer(sandboxID, containerID string, options ProcessListOptions) (ProcessList, error)
```
## Examples
### Preparing and running a sandbox
```Go
// This example creates and starts a single container sandbox,
// using qemu as the hypervisor and kata as the VM agent.
func Example_createAndStartSandbox() {
envs := []vc.EnvVar{
{
Var: "PATH",
Value: "/bin:/usr/bin:/sbin:/usr/sbin",
},
}
cmd := vc.Cmd{
Args: strings.Split("/bin/sh", " "),
Envs: envs,
WorkDir: "/",
}
// Define the container command and bundle.
container := vc.ContainerConfig{
ID: "1",
RootFs: containerRootfs,
Cmd: cmd,
}
// Sets the hypervisor configuration.
hypervisorConfig := vc.HypervisorConfig{
KernelPath: "/usr/share/clear-containers/vmlinux.container",
ImagePath: "/usr/share/clear-containers/clear-containers.img",
HypervisorPath: "/usr/bin/qemu-system-x86_64",
}
// Use kata default values for the agent.
agConfig := vc.KataAgentConfig{}
// VM resources
vmConfig := vc.Resources{
VCPUs: 4,
Memory: 1024,
}
// The sandbox configuration:
// - One container
// - Hypervisor is QEMU
// - Agent is kata
sandboxConfig := vc.SandboxConfig{
VMConfig: vmConfig,
HypervisorType: vc.QemuHypervisor,
HypervisorConfig: hypervisorConfig,
AgentType: vc.KataContainersAgent
AgentConfig: agConfig,
Containers: []vc.ContainerConfig{container},
}
_, err := vc.RunSandbox(sandboxConfig)
if err != nil {
fmt.Printf("Could not run sandbox: %s", err)
}
return
}
```