From b27281954041e611dd921241c9eb7e76048463ca Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Mon, 8 Jan 2018 16:23:33 +0000 Subject: [PATCH 1/9] cmd/packet: Update the vendored packet.net go binding Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/vendor.conf | 2 +- .../github.com/docker/docker/hack/README.md | 60 ---- .../hack/integration-cli-on-swarm/README.md | 69 ---- .../agent/vendor.conf | 2 - .../github.com/packethost/packngo/README.md | 17 + .../github.com/packethost/packngo/devices.go | 177 ++++------ .../github.com/packethost/packngo/email.go | 8 +- .../packethost/packngo/facilities.go | 8 +- .../github.com/packethost/packngo/ip.go | 320 +++++++++--------- .../packethost/packngo/operatingsystems.go | 8 +- .../github.com/packethost/packngo/packngo.go | 70 +++- .../github.com/packethost/packngo/plans.go | 9 +- .../github.com/packethost/packngo/projects.go | 72 +--- .../packethost/packngo/spotmarket.go | 39 +++ .../github.com/packethost/packngo/sshkeys.go | 63 ++-- .../github.com/packethost/packngo/user.go | 10 +- .../github.com/packethost/packngo/utils.go | 21 -- .../github.com/packethost/packngo/volumes.go | 137 +++++--- 18 files changed, 468 insertions(+), 624 deletions(-) delete mode 100644 src/cmd/linuxkit/vendor/github.com/docker/docker/hack/README.md delete mode 100644 src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/README.md delete mode 100644 src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/agent/vendor.conf create mode 100644 src/cmd/linuxkit/vendor/github.com/packethost/packngo/spotmarket.go diff --git a/src/cmd/linuxkit/vendor.conf b/src/cmd/linuxkit/vendor.conf index b933bc8ce..b256832b7 100644 --- a/src/cmd/linuxkit/vendor.conf +++ b/src/cmd/linuxkit/vendor.conf @@ -31,7 +31,7 @@ github.com/moby/vpnkit 0e4293bb1058598c4b0a406ed171f52573ef414c github.com/opencontainers/go-digest 21dfd564fd89c944783d00d069f33e3e7123c448 github.com/opencontainers/image-spec v1.0.0 github.com/opencontainers/runtime-spec v1.0.0 -github.com/packethost/packngo 131798f2804a1b3e895ca98047d56f0d7e094e2a +github.com/packethost/packngo f1be085ecd6fca1b0a0e25eda71f208dcfcee5ab github.com/pkg/errors v0.8.0 github.com/pmezard/go-difflib v1.0.0 github.com/radu-matei/azure-sdk-for-go 3b12823551999669c9a325a32472508e0af7978e diff --git a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/README.md b/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/README.md deleted file mode 100644 index 9e588db25..000000000 --- a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/README.md +++ /dev/null @@ -1,60 +0,0 @@ -## About - -This directory contains a collection of scripts used to build and manage this -repository. If there are any issues regarding the intention of a particular -script (or even part of a certain script), please reach out to us. -It may help us either refine our current scripts, or add on new ones -that are appropriate for a given use case. - -## DinD (dind.sh) - -DinD is a wrapper script which allows Docker to be run inside a Docker -container. DinD requires the container to -be run with privileged mode enabled. - -## Generate Authors (generate-authors.sh) - -Generates AUTHORS; a file with all the names and corresponding emails of -individual contributors. AUTHORS can be found in the home directory of -this repository. - -## Make - -There are two make files, each with different extensions. Neither are supposed -to be called directly; only invoke `make`. Both scripts run inside a Docker -container. - -### make.ps1 - -- The Windows native build script that uses PowerShell semantics; it is limited -unlike `hack\make.sh` since it does not provide support for the full set of -operations provided by the Linux counterpart, `make.sh`. However, `make.ps1` -does provide support for local Windows development and Windows to Windows CI. -More information is found within `make.ps1` by the author, @jhowardmsft - -### make.sh - -- Referenced via `make test` when running tests on a local machine, -or directly referenced when running tests inside a Docker development container. -- When running on a local machine, `make test` to run all tests found in -`test`, `test-unit`, `test-integration`, and `test-docker-py` on -your local machine. The default timeout is set in `make.sh` to 60 minutes -(`${TIMEOUT:=60m}`), since it currently takes up to an hour to run -all of the tests. -- When running inside a Docker development container, `hack/make.sh` does -not have a single target that runs all the tests. You need to provide a -single command line with multiple targets that performs the same thing. -An example referenced from [Run targets inside a development container](https://docs.docker.com/opensource/project/test-and-docs/#run-targets-inside-a-development-container): `root@5f8630b873fe:/go/src/github.com/moby/moby# hack/make.sh dynbinary binary cross test-unit test-integration test-docker-py` -- For more information related to testing outside the scope of this README, -refer to -[Run tests and test documentation](https://docs.docker.com/opensource/project/test-and-docs/) - -## Release (release.sh) - -Releases any bundles built by `make` on a public AWS S3 bucket. -For information regarding configuration, please view `release.sh`. - -## Vendor (vendor.sh) - -A shell script that is a wrapper around Vndr. For information on how to use -this, please refer to [vndr's README](https://github.com/LK4D4/vndr/blob/master/README.md) diff --git a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/README.md b/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/README.md deleted file mode 100644 index 1cea52526..000000000 --- a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/README.md +++ /dev/null @@ -1,69 +0,0 @@ -# Integration Testing on Swarm - -IT on Swarm allows you to execute integration test in parallel across a Docker Swarm cluster - -## Architecture - -### Master service - - - Works as a funker caller - - Calls a worker funker (`-worker-service`) with a chunk of `-check.f` filter strings (passed as a file via `-input` flag, typically `/mnt/input`) - -### Worker service - - - Works as a funker callee - - Executes an equivalent of `TESTFLAGS=-check.f TestFoo|TestBar|TestBaz ... make test-integration-cli` using the bind-mounted API socket (`docker.sock`) - -### Client - - - Controls master and workers via `docker stack` - - No need to have a local daemon - -Typically, the master and workers are supposed to be running on a cloud environment, -while the client is supposed to be running on a laptop, e.g. Docker for Mac/Windows. - -## Requirement - - - Docker daemon 1.13 or later - - Private registry for distributed execution with multiple nodes - -## Usage - -### Step 1: Prepare images - - $ make build-integration-cli-on-swarm - -Following environment variables are known to work in this step: - - - `BUILDFLAGS` - - `DOCKER_INCREMENTAL_BINARY` - -Note: during the transition into Moby Project, you might need to create a symbolic link `$GOPATH/src/github.com/docker/docker` to `$GOPATH/src/github.com/moby/moby`. - -### Step 2: Execute tests - - $ ./hack/integration-cli-on-swarm/integration-cli-on-swarm -replicas 40 -push-worker-image YOUR_REGISTRY.EXAMPLE.COM/integration-cli-worker:latest - -Following environment variables are known to work in this step: - - - `DOCKER_GRAPHDRIVER` - - `DOCKER_EXPERIMENTAL` - -#### Flags - -Basic flags: - - - `-replicas N`: the number of worker service replicas. i.e. degree of parallelism. - - `-chunks N`: the number of chunks. By default, `chunks` == `replicas`. - - `-push-worker-image REGISTRY/IMAGE:TAG`: push the worker image to the registry. Note that if you have only single node and hence you do not need a private registry, you do not need to specify `-push-worker-image`. - -Experimental flags for mitigating makespan nonuniformity: - - - `-shuffle`: Shuffle the test filter strings - -Flags for debugging IT on Swarm itself: - - - `-rand-seed N`: the random seed. This flag is useful for deterministic replaying. By default(0), the timestamp is used. - - `-filters-file FILE`: the file contains `-check.f` strings. By default, the file is automatically generated. - - `-dry-run`: skip the actual workload - - `keep-executor`: do not auto-remove executor containers, which is used for running privileged programs on Swarm diff --git a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/agent/vendor.conf b/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/agent/vendor.conf deleted file mode 100644 index efd6d6d04..000000000 --- a/src/cmd/linuxkit/vendor/github.com/docker/docker/hack/integration-cli-on-swarm/agent/vendor.conf +++ /dev/null @@ -1,2 +0,0 @@ -# dependencies specific to worker (i.e. github.com/docker/docker/...) are not vendored here -github.com/bfirsh/funker-go eaa0a2e06f30e72c9a0b7f858951e581e26ef773 diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/README.md b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/README.md index d359a10d6..4080ab047 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/README.md +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/README.md @@ -7,3 +7,20 @@ Committing ---------- Before committing, it's a good idea to run `gofmt -w *.go`. ([gofmt](https://golang.org/cmd/gofmt/)) + +Acceptance Tests +---------------- + +If you want to run tests against the actual Packet API, you must set envvar `PACKET_TEST_ACTUAL_API` to non-empty string for the `go test`. The device tests wait for the device creation, so it's best to run a few in parallel. + +To run all the tests, you can do + +``` +$ PACKNGO_TEST_ACTUAL_API=1 go test -v -parallel 8 +``` + +It's also useful to run only single acceptance test at a time: + +``` +$ PACKNGO_TEST_ACTUAL_API=1 go test -v -run=TestAccDeviceBasic +``` diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/devices.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/devices.go index b67bb83a4..ead7c1f76 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/devices.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/devices.go @@ -24,24 +24,43 @@ type devicesRoot struct { // Device represents a Packet device type Device struct { - ID string `json:"id"` - Href string `json:"href,omitempty"` - Hostname string `json:"hostname,omitempty"` - State string `json:"state,omitempty"` - Created string `json:"created_at,omitempty"` - Updated string `json:"updated_at,omitempty"` - Locked bool `json:"locked,omitempty"` - BillingCycle string `json:"billing_cycle,omitempty"` - Tags []string `json:"tags,omitempty"` - Network []*IPAddress `json:"ip_addresses"` - OS *OS `json:"operating_system,omitempty"` - Plan *Plan `json:"plan,omitempty"` - Facility *Facility `json:"facility,omitempty"` - Project *Project `json:"project,omitempty"` - ProvisionPer float32 `json:"provisioning_percentage,omitempty"` - UserData string `json:"userdata",omitempty` - IPXEScriptUrl string `json:"ipxe_script_url,omitempty"` - AlwaysPXE bool `json:"always_pxe,omitempty"` + ID string `json:"id"` + Href string `json:"href,omitempty"` + Hostname string `json:"hostname,omitempty"` + State string `json:"state,omitempty"` + Created string `json:"created_at,omitempty"` + Updated string `json:"updated_at,omitempty"` + Locked bool `json:"locked,omitempty"` + BillingCycle string `json:"billing_cycle,omitempty"` + Storage map[string]interface{} `json:"storage,omitempty"` + Tags []string `json:"tags,omitempty"` + Network []*IPAddressAssignment `json:"ip_addresses"` + Volumes []*Volume `json:"volumes"` + OS *OS `json:"operating_system,omitempty"` + Plan *Plan `json:"plan,omitempty"` + Facility *Facility `json:"facility,omitempty"` + Project *Project `json:"project,omitempty"` + ProvisionEvents []*ProvisionEvent `json:"provisioning_events,omitempty"` + ProvisionPer float32 `json:"provisioning_percentage,omitempty"` + UserData string `json:"userdata,omitempty"` + RootPassword string `json:"root_password,omitempty"` + IPXEScriptURL string `json:"ipxe_script_url,omitempty"` + AlwaysPXE bool `json:"always_pxe,omitempty"` + HardwareReservation Href `json:"hardware_reservation,omitempty"` + SpotInstance bool `json:"spot_instance,omitempty"` + SpotPriceMax float64 `json:"spot_price_max,omitempty"` + TerminationTime *Timestamp `json:"termination_time,omitempty"` +} + +type ProvisionEvent struct { + ID string `json:"id"` + Body string `json:"body"` + CreatedAt *Timestamp `json:"created_at,omitempty"` + Href string `json:"href"` + Interpolated string `json:"interpolated"` + Relationships []Href `json:"relationships"` + State string `json:"state"` + Type string `json:"type"` } func (d Device) String() string { @@ -50,28 +69,33 @@ func (d Device) String() string { // DeviceCreateRequest type used to create a Packet device type DeviceCreateRequest struct { - HostName string `json:"hostname"` - Plan string `json:"plan"` - Facility string `json:"facility"` - OS string `json:"operating_system"` - BillingCycle string `json:"billing_cycle"` - ProjectID string `json:"project_id"` - UserData string `json:"userdata"` - Tags []string `json:"tags"` - IPXEScriptUrl string `json:"ipxe_script_url,omitempty"` - PublicIPv4SubnetSize int `json:"public_ipv4_subnet_size,omitempty"` - AlwaysPXE bool `json:"always_pxe,omitempty"` + Hostname string `json:"hostname"` + Plan string `json:"plan"` + Facility string `json:"facility"` + OS string `json:"operating_system"` + BillingCycle string `json:"billing_cycle"` + ProjectID string `json:"project_id"` + UserData string `json:"userdata"` + Storage string `json:"storage,omitempty"` + Tags []string `json:"tags"` + IPXEScriptURL string `json:"ipxe_script_url,omitempty"` + PublicIPv4SubnetSize int `json:"public_ipv4_subnet_size,omitempty"` + AlwaysPXE bool `json:"always_pxe,omitempty"` + HardwareReservationID string `json:"hardware_reservation_id,omitempty"` + SpotInstance bool `json:"spot_instance,omitempty"` + SpotPriceMax float64 `json:"spot_price_max,omitempty,string"` + TerminationTime *Timestamp `json:"termination_time,omitempty"` } // DeviceUpdateRequest type used to update a Packet device type DeviceUpdateRequest struct { - HostName string `json:"hostname"` + Hostname string `json:"hostname"` Description string `json:"description"` UserData string `json:"userdata"` Locked bool `json:"locked"` Tags []string `json:"tags"` AlwaysPXE bool `json:"always_pxe,omitempty"` - IPXEScriptUrl string `json:"ipxe_script_url,omitempty"` + IPXEScriptURL string `json:"ipxe_script_url,omitempty"` } func (d DeviceCreateRequest) String() string { @@ -95,14 +119,9 @@ type DeviceServiceOp struct { // List returns devices on a project func (s *DeviceServiceOp) List(projectID string) ([]Device, *Response, error) { path := fmt.Sprintf("%s/%s/devices?include=facility", projectBasePath, projectID) - - req, err := s.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - root := new(devicesRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", path, nil, root) if err != nil { return nil, resp, err } @@ -113,14 +132,9 @@ func (s *DeviceServiceOp) List(projectID string) ([]Device, *Response, error) { // Get returns a device by id func (s *DeviceServiceOp) Get(deviceID string) (*Device, *Response, error) { path := fmt.Sprintf("%s/%s?include=facility", deviceBasePath, deviceID) - - req, err := s.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - device := new(Device) - resp, err := s.client.Do(req, device) + + resp, err := s.client.DoRequest("GET", path, nil, device) if err != nil { return nil, resp, err } @@ -131,14 +145,9 @@ func (s *DeviceServiceOp) Get(deviceID string) (*Device, *Response, error) { // Create creates a new device func (s *DeviceServiceOp) Create(createRequest *DeviceCreateRequest) (*Device, *Response, error) { path := fmt.Sprintf("%s/%s/devices", projectBasePath, createRequest.ProjectID) - - req, err := s.client.NewRequest("POST", path, createRequest) - if err != nil { - return nil, nil, err - } - device := new(Device) - resp, err := s.client.Do(req, device) + + resp, err := s.client.DoRequest("POST", path, createRequest, device) if err != nil { return nil, resp, err } @@ -149,14 +158,9 @@ func (s *DeviceServiceOp) Create(createRequest *DeviceCreateRequest) (*Device, * // Update updates an existing device func (s *DeviceServiceOp) Update(deviceID string, updateRequest *DeviceUpdateRequest) (*Device, *Response, error) { path := fmt.Sprintf("%s/%s?include=facility", deviceBasePath, deviceID) - - req, err := s.client.NewRequest("PUT", path, updateRequest) - if err != nil { - return nil, nil, err - } - device := new(Device) - resp, err := s.client.Do(req, device) + + resp, err := s.client.DoRequest("PUT", path, updateRequest, device) if err != nil { return nil, resp, err } @@ -168,59 +172,31 @@ func (s *DeviceServiceOp) Update(deviceID string, updateRequest *DeviceUpdateReq func (s *DeviceServiceOp) Delete(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s", deviceBasePath, deviceID) - req, err := s.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("DELETE", path, nil, nil) } // Reboot reboots on a device func (s *DeviceServiceOp) Reboot(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s/actions", deviceBasePath, deviceID) - action := &DeviceActionRequest{Type: "reboot"} - req, err := s.client.NewRequest("POST", path, action) - if err != nil { - return nil, err - } - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("POST", path, action, nil) } // PowerOff powers on a device func (s *DeviceServiceOp) PowerOff(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s/actions", deviceBasePath, deviceID) - action := &DeviceActionRequest{Type: "power_off"} - req, err := s.client.NewRequest("POST", path, action) - if err != nil { - return nil, err - } - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("POST", path, action, nil) } // PowerOn powers on a device func (s *DeviceServiceOp) PowerOn(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s/actions", deviceBasePath, deviceID) - action := &DeviceActionRequest{Type: "power_on"} - req, err := s.client.NewRequest("POST", path, action) - if err != nil { - return nil, err - } - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("POST", path, action, nil) } type lockDeviceType struct { @@ -230,30 +206,15 @@ type lockDeviceType struct { // Lock sets a device to "locked" func (s *DeviceServiceOp) Lock(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s", deviceBasePath, deviceID) - action := lockDeviceType{Locked: true} - req, err := s.client.NewRequest("PATCH", path, action) - if err != nil { - return nil, err - } - - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("PATCH", path, action, nil) } // Unlock sets a device to "locked" func (s *DeviceServiceOp) Unlock(deviceID string) (*Response, error) { path := fmt.Sprintf("%s/%s", deviceBasePath, deviceID) - action := lockDeviceType{Locked: false} - req, err := s.client.NewRequest("PATCH", path, action) - if err != nil { - return nil, err - } - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("PATCH", path, action, nil) } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/email.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/email.go index ef53ee4bb..4c77d0f60 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/email.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/email.go @@ -26,13 +26,9 @@ type EmailServiceOp struct { // Get retrieves an email by id func (s *EmailServiceOp) Get(emailID string) (*Email, *Response, error) { - req, err := s.client.NewRequest("GET", emailBasePath, nil) - if err != nil { - return nil, nil, err - } - email := new(Email) - resp, err := s.client.Do(req, email) + + resp, err := s.client.DoRequest("GET", emailBasePath, nil, email) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/facilities.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/facilities.go index 42a3c7cb1..12aac9198 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/facilities.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/facilities.go @@ -41,13 +41,9 @@ type FacilityServiceOp struct { // List returns all available Packet facilities func (s *FacilityServiceOp) List() ([]Facility, *Response, error) { - req, err := s.client.NewRequest("GET", facilityBasePath, nil) - if err != nil { - return nil, nil, err - } - root := new(facilityRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", facilityBasePath, nil, root) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/ip.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/ip.go index d4f27de33..9293ab460 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/ip.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/ip.go @@ -1,110 +1,71 @@ package packngo -import "fmt" +import ( + "fmt" +) const ipBasePath = "/ips" -// IPService interface defines available IP methods -type IPService interface { - Assign(deviceID string, assignRequest *IPAddressAssignRequest) (*IPAddress, *Response, error) - Unassign(ipAddressID string) (*Response, error) - Get(ipAddressID string) (*IPAddress, *Response, error) +// DeviceIPService handles assignment of addresses from reserved blocks to instances in a project. +type DeviceIPService interface { + Assign(deviceID string, assignRequest *AddressStruct) (*IPAddressAssignment, *Response, error) + Unassign(assignmentID string) (*Response, error) + Get(assignmentID string) (*IPAddressAssignment, *Response, error) } -// IPAddress represents a ip address -type IPAddress struct { - ID string `json:"id"` - Address string `json:"address"` - Gateway string `json:"gateway"` - Network string `json:"network"` - AddressFamily int `json:"address_family"` - Netmask string `json:"netmask"` - Public bool `json:"public"` - Cidr int `json:"cidr"` - AssignedTo map[string]string `json:"assigned_to"` - Created string `json:"created_at,omitempty"` - Updated string `json:"updated_at,omitempty"` - Href string `json:"href"` - Facility Facility `json:"facility,omitempty"` -} - -// IPAddressAssignRequest represents the body if a ip assign request -type IPAddressAssignRequest struct { - Address string `json:"address"` -} - -func (i IPAddress) String() string { - return Stringify(i) -} - -// IPServiceOp implements IPService -type IPServiceOp struct { - client *Client -} - -// Get returns IpAddress by ID -func (i *IPServiceOp) Get(ipAddressID string) (*IPAddress, *Response, error) { - path := fmt.Sprintf("%s/%s", ipBasePath, ipAddressID) - - req, err := i.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - - ip := new(IPAddress) - resp, err := i.client.Do(req, ip) - if err != nil { - return nil, resp, err - } - - return ip, resp, err -} - -// Unassign unassigns an IP address record. This will remove the relationship between an IP -// and the device and will make the IP address available to be assigned to another device. -func (i *IPServiceOp) Unassign(ipAddressID string) (*Response, error) { - path := fmt.Sprintf("%s/%s", ipBasePath, ipAddressID) - - req, err := i.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := i.client.Do(req, nil) - return resp, err -} - -// Assign assigns an IP address to a device. The IP address must be in one of the IP ranges assigned to the device’s project. -func (i *IPServiceOp) Assign(deviceID string, assignRequest *IPAddressAssignRequest) (*IPAddress, *Response, error) { - path := fmt.Sprintf("%s/%s%s", deviceBasePath, deviceID, ipBasePath) - - req, err := i.client.NewRequest("POST", path, assignRequest) - - ip := new(IPAddress) - resp, err := i.client.Do(req, ip) - if err != nil { - return nil, resp, err - } - - return ip, resp, err -} - -// IP RESERVATIONS API - -// IPReservationService interface defines available IPReservation methods -type IPReservationService interface { - List(projectID string) ([]IPReservation, *Response, error) - RequestMore(projectID string, ipReservationReq *IPReservationRequest) (*IPReservation, *Response, error) - Get(ipReservationID string) (*IPReservation, *Response, error) +// ProjectIPService handles reservation of IP address blocks for a project. +type ProjectIPService interface { + Get(reservationID string) (*IPAddressReservation, *Response, error) + List(projectID string) ([]IPAddressReservation, *Response, error) + Request(projectID string, ipReservationReq *IPReservationRequest) (*IPAddressReservation, *Response, error) Remove(ipReservationID string) (*Response, error) + AvailableAddresses(ipReservationID string, r *AvailableRequest) ([]string, *Response, error) } -// IPReservationServiceOp implements the IPReservationService interface -type IPReservationServiceOp struct { - client *Client +type ipAddressCommon struct { + ID string `json:"id"` + Address string `json:"address"` + Gateway string `json:"gateway"` + Network string `json:"network"` + AddressFamily int `json:"address_family"` + Netmask string `json:"netmask"` + Public bool `json:"public"` + CIDR int `json:"cidr"` + Created string `json:"created_at,omitempty"` + Updated string `json:"updated_at,omitempty"` + Href string `json:"href"` + Management bool `json:"management"` + Manageable bool `json:"manageable"` + Project Href `json:"project"` } -// IPReservationRequest represents the body of a reservation request +// IPAddressReservation is created when user sends IP reservation request for a project (considering it's within quota). +type IPAddressReservation struct { + ipAddressCommon + Assignments []Href `json:"assignments"` + Facility Facility `json:"facility,omitempty"` + Available string `json:"available"` + Addon bool `json:"addon"` + Bill bool `json:"bill"` +} + +// AvailableResponse is a type for listing of available addresses from a reserved block. +type AvailableResponse struct { + Available []string `json:"available"` +} + +// AvailableRequest is a type for listing available addresses from a reserved block. +type AvailableRequest struct { + CIDR int `json:"cidr"` +} + +// IPAddressAssignment is created when an IP address from reservation block is assigned to a device. +type IPAddressAssignment struct { + ipAddressCommon + AssignedTo Href `json:"assigned_to"` +} + +// IPReservationRequest represents the body of a reservation request. type IPReservationRequest struct { Type string `json:"type"` Quantity int `json:"quantity"` @@ -112,95 +73,122 @@ type IPReservationRequest struct { Facility string `json:"facility"` } -// IPReservation represent an IP reservation for a single project -type IPReservation struct { - ID string `json:"id"` - Network string `json:"network"` - Address string `json:"address"` - AddressFamily int `json:"address_family"` - Netmask string `json:"netmask"` - Public bool `json:"public"` - Cidr int `json:"cidr"` - Management bool `json:"management"` - Manageable bool `json:"manageable"` - Addon bool `json:"addon"` - Bill bool `json:"bill"` - Assignments []map[string]string `json:"assignments"` - Created string `json:"created_at,omitempty"` - Updated string `json:"updated_at,omitempty"` - Href string `json:"href"` - Facility Facility `json:"facility,omitempty"` +// AddressStruct is a helper type for request/response with dict like {"address": ... } +type AddressStruct struct { + Address string `json:"address"` } -type ipReservationRoot struct { - IPReservations []IPReservation `json:"ip_addresses"` +func deleteFromIP(client *Client, resourceID string) (*Response, error) { + path := fmt.Sprintf("%s/%s", ipBasePath, resourceID) + + return client.DoRequest("DELETE", path, nil, nil) +} + +func (i IPAddressReservation) String() string { + return Stringify(i) +} + +func (i IPAddressAssignment) String() string { + return Stringify(i) +} + +// DeviceIPServiceOp is interface for IP-address assignment methods. +type DeviceIPServiceOp struct { + client *Client +} + +// Unassign unassigns an IP address from the device to which it is currently assigned. +// This will remove the relationship between an IP and the device and will make the IP +// address available to be assigned to another device. +func (i *DeviceIPServiceOp) Unassign(assignmentID string) (*Response, error) { + return deleteFromIP(i.client, assignmentID) +} + +// Assign assigns an IP address to a device. +// The IP address must be in one of the IP ranges assigned to the device’s project. +func (i *DeviceIPServiceOp) Assign(deviceID string, assignRequest *AddressStruct) (*IPAddressAssignment, *Response, error) { + path := fmt.Sprintf("%s/%s%s", deviceBasePath, deviceID, ipBasePath) + ipa := new(IPAddressAssignment) + + resp, err := i.client.DoRequest("POST", path, assignRequest, ipa) + if err != nil { + return nil, resp, err + } + + return ipa, resp, err +} + +// Get returns assignment by ID. +func (i *DeviceIPServiceOp) Get(assignmentID string) (*IPAddressAssignment, *Response, error) { + path := fmt.Sprintf("%s/%s", ipBasePath, assignmentID) + ipa := new(IPAddressAssignment) + + resp, err := i.client.DoRequest("GET", path, nil, ipa) + if err != nil { + return nil, resp, err + } + + return ipa, resp, err +} + +// ProjectIPServiceOp is interface for IP assignment methods. +type ProjectIPServiceOp struct { + client *Client +} + +// Get returns reservation by ID. +func (i *ProjectIPServiceOp) Get(reservationID string) (*IPAddressReservation, *Response, error) { + path := fmt.Sprintf("%s/%s", ipBasePath, reservationID) + ipr := new(IPAddressReservation) + + resp, err := i.client.DoRequest("GET", path, nil, ipr) + if err != nil { + return nil, resp, err + } + + return ipr, resp, err } // List provides a list of IP resevations for a single project. -func (i *IPReservationServiceOp) List(projectID string) ([]IPReservation, *Response, error) { +func (i *ProjectIPServiceOp) List(projectID string) ([]IPAddressReservation, *Response, error) { path := fmt.Sprintf("%s/%s%s", projectBasePath, projectID, ipBasePath) + reservations := new(struct { + Reservations []IPAddressReservation `json:"ip_addresses"` + }) - req, err := i.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - - reservations := new(ipReservationRoot) - resp, err := i.client.Do(req, reservations) + resp, err := i.client.DoRequest("GET", path, nil, reservations) if err != nil { return nil, resp, err } - return reservations.IPReservations, resp, err + return reservations.Reservations, resp, nil } -// RequestMore requests more IP space for a project in order to have additional IP addresses to assign to devices -func (i *IPReservationServiceOp) RequestMore(projectID string, ipReservationReq *IPReservationRequest) (*IPReservation, *Response, error) { +// Request requests more IP space for a project in order to have additional IP addresses to assign to devices. +func (i *ProjectIPServiceOp) Request(projectID string, ipReservationReq *IPReservationRequest) (*IPAddressReservation, *Response, error) { path := fmt.Sprintf("%s/%s%s", projectBasePath, projectID, ipBasePath) + ipr := new(IPAddressReservation) - req, err := i.client.NewRequest("POST", path, &ipReservationReq) - if err != nil { - return nil, nil, err - } - - ip := new(IPReservation) - resp, err := i.client.Do(req, ip) + resp, err := i.client.DoRequest("POST", path, ipReservationReq, ipr) if err != nil { return nil, resp, err } - return ip, resp, err -} - -// Get returns a single IP reservation object -func (i *IPReservationServiceOp) Get(ipReservationID string) (*IPReservation, *Response, error) { - path := fmt.Sprintf("%s/%s", ipBasePath, ipReservationID) - - req, err := i.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - - reservation := new(IPReservation) - resp, err := i.client.Do(req, reservation) - if err != nil { - return nil, nil, err - } - - return reservation, resp, err + return ipr, resp, err } // Remove removes an IP reservation from the project. -func (i *IPReservationServiceOp) Remove(ipReservationID string) (*Response, error) { - path := fmt.Sprintf("%s/%s", ipBasePath, ipReservationID) - - req, err := i.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := i.client.Do(req, nil) - if err != nil { - return nil, err - } - - return resp, err +func (i *ProjectIPServiceOp) Remove(ipReservationID string) (*Response, error) { + return deleteFromIP(i.client, ipReservationID) +} + +// AvailableAddresses lists addresses available from a reserved block +func (i *ProjectIPServiceOp) AvailableAddresses(ipReservationID string, r *AvailableRequest) ([]string, *Response, error) { + path := fmt.Sprintf("%s/%s/available", ipBasePath, ipReservationID) + ar := new(AvailableResponse) + + resp, err := i.client.DoRequest("GET", path, r, ar) + if err != nil { + return nil, resp, err + } + return ar.Available, resp, nil + } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/operatingsystems.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/operatingsystems.go index 4200a40cf..7fd7f27ad 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/operatingsystems.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/operatingsystems.go @@ -30,13 +30,9 @@ type OSServiceOp struct { // List returns all available operating systems func (s *OSServiceOp) List() ([]OS, *Response, error) { - req, err := s.client.NewRequest("GET", osBasePath, nil) - if err != nil { - return nil, nil, err - } - root := new(osRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", osBasePath, nil, root) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/packngo.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/packngo.go index 21a027941..44db0c0c6 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/packngo.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/packngo.go @@ -6,8 +6,11 @@ import ( "fmt" "io" "io/ioutil" + "log" "net/http" + "net/http/httputil" "net/url" + "os" "strconv" "strings" "time" @@ -18,6 +21,7 @@ const ( baseURL = "https://api.packet.net/" userAgent = "packngo/" + libraryVersion mediaType = "application/json" + debugEnvVar = "PACKNGO_DEBUG" headerRateLimit = "X-RateLimit-Limit" headerRateRemaining = "X-RateLimit-Remaining" @@ -42,6 +46,11 @@ type Response struct { Rate } +// Href is an API link +type Href struct { + Href string `json:"href"` +} + func (r *Response) populateRate() { // parse the rate limit headers and populate Response.Rate if limit := r.Header.Get(headerRateLimit); limit != "" { @@ -59,18 +68,20 @@ func (r *Response) populateRate() { // ErrorResponse is the http response used on errrors type ErrorResponse struct { - Response *http.Response - Errors []string `json:"errors"` + Response *http.Response + Errors []string `json:"errors"` + SingleError string `json:"error"` } func (r *ErrorResponse) Error() string { - return fmt.Sprintf("%v %v: %d %v", - r.Response.Request.Method, r.Response.Request.URL, r.Response.StatusCode, strings.Join(r.Errors, ", ")) + return fmt.Sprintf("%v %v: %d %v %v", + r.Response.Request.Method, r.Response.Request.URL, r.Response.StatusCode, strings.Join(r.Errors, ", "), r.SingleError) } // Client is the base API Client type Client struct { client *http.Client + debug bool BaseURL *url.URL @@ -81,17 +92,19 @@ type Client struct { RateLimit Rate // Packet Api Objects - Plans PlanService - Users UserService - Emails EmailService - SSHKeys SSHKeyService - Devices DeviceService - Projects ProjectService - Facilities FacilityService - OperatingSystems OSService - Ips IPService - IpReservations IPReservationService - Volumes VolumeService + Plans PlanService + Users UserService + Emails EmailService + SSHKeys SSHKeyService + Devices DeviceService + Projects ProjectService + Facilities FacilityService + OperatingSystems OSService + DeviceIPs DeviceIPService + ProjectIPs ProjectIPService + Volumes VolumeService + VolumeAttachments VolumeAttachmentService + SpotMarket SpotMarketService } // NewRequest inits a new http request with the proper headers @@ -140,6 +153,10 @@ func (c *Client) Do(req *http.Request, v interface{}) (*Response, error) { response := Response{Response: resp} response.populateRate() + if c.debug { + o, _ := httputil.DumpResponse(response.Response, true) + log.Printf("%s\n", string(o)) + } c.RateLimit = response.Rate err = checkResponse(resp) @@ -163,6 +180,19 @@ func (c *Client) Do(req *http.Request, v interface{}) (*Response, error) { return &response, err } +// DoRequest is a convenience method, it calls NewRequest follwed by Do +func (c *Client) DoRequest(method, path string, body, v interface{}) (*Response, error) { + req, err := c.NewRequest(method, path, body) + if c.debug { + o, _ := httputil.DumpRequestOut(req, true) + log.Printf("%s\n", string(o)) + } + if err != nil { + return nil, err + } + return c.Do(req, v) +} + // NewClient initializes and returns a Client, use this to get an API Client to operate on // N.B.: Packet's API certificate requires Go 1.5+ to successfully parse. If you are using // an older version of Go, pass in a custom http.Client with a custom TLS configuration @@ -171,6 +201,9 @@ func NewClient(consumerToken string, apiKey string, httpClient *http.Client) *Cl client, _ := NewClientWithBaseURL(consumerToken, apiKey, httpClient, baseURL) return client } + +// NewClientWithBaseURL returns a Client pointing to nonstandard API URL, e.g. +// for mocking the remote API func NewClientWithBaseURL(consumerToken string, apiKey string, httpClient *http.Client, apiBaseURL string) (*Client, error) { if httpClient == nil { // Don't fall back on http.DefaultClient as it's not nice to adjust state @@ -185,6 +218,7 @@ func NewClientWithBaseURL(consumerToken string, apiKey string, httpClient *http. } c := &Client{client: httpClient, BaseURL: u, UserAgent: userAgent, ConsumerToken: consumerToken, APIKey: apiKey} + c.debug = os.Getenv(debugEnvVar) != "" c.Plans = &PlanServiceOp{client: c} c.Users = &UserServiceOp{client: c} c.Emails = &EmailServiceOp{client: c} @@ -193,9 +227,11 @@ func NewClientWithBaseURL(consumerToken string, apiKey string, httpClient *http. c.Projects = &ProjectServiceOp{client: c} c.Facilities = &FacilityServiceOp{client: c} c.OperatingSystems = &OSServiceOp{client: c} - c.Ips = &IPServiceOp{client: c} - c.IpReservations = &IPReservationServiceOp{client: c} + c.DeviceIPs = &DeviceIPServiceOp{client: c} + c.ProjectIPs = &ProjectIPServiceOp{client: c} c.Volumes = &VolumeServiceOp{client: c} + c.VolumeAttachments = &VolumeAttachmentServiceOp{client: c} + c.SpotMarket = &SpotMarketServiceOp{client: c} return c, nil } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/plans.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/plans.go index 5b3fa86fd..148a2a5ce 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/plans.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/plans.go @@ -106,14 +106,9 @@ type PlanServiceOp struct { // List method returns all available plans func (s *PlanServiceOp) List() ([]Plan, *Response, error) { - req, err := s.client.NewRequest("GET", planBasePath, nil) - - if err != nil { - return nil, nil, err - } - root := new(planRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", planBasePath, nil, root) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/projects.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/projects.go index 9c539a8ce..219e753c8 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/projects.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/projects.go @@ -11,14 +11,9 @@ type ProjectService interface { Create(*ProjectCreateRequest) (*Project, *Response, error) Update(*ProjectUpdateRequest) (*Project, *Response, error) Delete(string) (*Response, error) - ListIPAddresses(string) ([]IPAddress, *Response, error) ListVolumes(string) ([]Volume, *Response, error) } -type ipsRoot struct { - IPAddresses []IPAddress `json:"ip_addresses"` -} - type volumesRoot struct { Volumes []Volume `json:"volumes"` } @@ -69,31 +64,11 @@ type ProjectServiceOp struct { client *Client } -func (s *ProjectServiceOp) ListIPAddresses(projectID string) ([]IPAddress, *Response, error) { - url := fmt.Sprintf("%s/%s/ips", projectBasePath, projectID) - req, err := s.client.NewRequest("GET", url, nil) - if err != nil { - return nil, nil, err - } - - root := new(ipsRoot) - resp, err := s.client.Do(req, root) - if err != nil { - return nil, resp, err - } - - return root.IPAddresses, resp, err -} - // List returns the user's projects func (s *ProjectServiceOp) List() ([]Project, *Response, error) { - req, err := s.client.NewRequest("GET", projectBasePath, nil) - if err != nil { - return nil, nil, err - } - root := new(projectsRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", projectBasePath, nil, root) if err != nil { return nil, resp, err } @@ -104,13 +79,9 @@ func (s *ProjectServiceOp) List() ([]Project, *Response, error) { // Get returns a project by id func (s *ProjectServiceOp) Get(projectID string) (*Project, *Response, error) { path := fmt.Sprintf("%s/%s", projectBasePath, projectID) - req, err := s.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - project := new(Project) - resp, err := s.client.Do(req, project) + + resp, err := s.client.DoRequest("GET", path, nil, project) if err != nil { return nil, resp, err } @@ -120,13 +91,9 @@ func (s *ProjectServiceOp) Get(projectID string) (*Project, *Response, error) { // Create creates a new project func (s *ProjectServiceOp) Create(createRequest *ProjectCreateRequest) (*Project, *Response, error) { - req, err := s.client.NewRequest("POST", projectBasePath, createRequest) - if err != nil { - return nil, nil, err - } - project := new(Project) - resp, err := s.client.Do(req, project) + + resp, err := s.client.DoRequest("POST", projectBasePath, createRequest, project) if err != nil { return nil, resp, err } @@ -137,13 +104,9 @@ func (s *ProjectServiceOp) Create(createRequest *ProjectCreateRequest) (*Project // Update updates a project func (s *ProjectServiceOp) Update(updateRequest *ProjectUpdateRequest) (*Project, *Response, error) { path := fmt.Sprintf("%s/%s", projectBasePath, updateRequest.ID) - req, err := s.client.NewRequest("PATCH", path, updateRequest) - if err != nil { - return nil, nil, err - } - project := new(Project) - resp, err := s.client.Do(req, project) + + resp, err := s.client.DoRequest("PATCH", path, updateRequest, project) if err != nil { return nil, resp, err } @@ -155,26 +118,15 @@ func (s *ProjectServiceOp) Update(updateRequest *ProjectUpdateRequest) (*Project func (s *ProjectServiceOp) Delete(projectID string) (*Response, error) { path := fmt.Sprintf("%s/%s", projectBasePath, projectID) - req, err := s.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("DELETE", path, nil, nil) } -// List returns Volumes for a project +// ListVolumes returns Volumes for a project func (s *ProjectServiceOp) ListVolumes(projectID string) ([]Volume, *Response, error) { url := fmt.Sprintf("%s/%s%s", projectBasePath, projectID, volumeBasePath) - req, err := s.client.NewRequest("GET", url, nil) - if err != nil { - return nil, nil, err - } - root := new(volumesRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", url, nil, root) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/spotmarket.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/spotmarket.go new file mode 100644 index 000000000..5dfb7d559 --- /dev/null +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/spotmarket.go @@ -0,0 +1,39 @@ +package packngo + +const spotMarketBasePath = "/market/spot/prices" + +// SpotMarketService expooses Spot Market methods +type SpotMarketService interface { + Prices() (PriceMap, *Response, error) +} + +// SpotMarketServiceOp implements SpotMarketService +type SpotMarketServiceOp struct { + client *Client +} + +// PriceMap is a map of [facility][plan]-> float Price +type PriceMap map[string]map[string]float64 + +// Prices gets current PriceMap from the API +func (s *SpotMarketServiceOp) Prices() (PriceMap, *Response, error) { + root := new(struct { + SMPs map[string]map[string]struct { + Price float64 `json:"price"` + } `json:"spot_market_prices"` + }) + + resp, err := s.client.DoRequest("GET", spotMarketBasePath, nil, root) + if err != nil { + return nil, resp, err + } + + prices := make(PriceMap) + for facility, planMap := range root.SMPs { + prices[facility] = map[string]float64{} + for plan, v := range planMap { + prices[facility][plan] = v.Price + } + } + return prices, resp, err +} diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/sshkeys.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/sshkeys.go index 474af7742..4ba120274 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/sshkeys.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/sshkeys.go @@ -2,11 +2,14 @@ package packngo import "fmt" -const sshKeyBasePath = "/ssh-keys" +const ( + sshKeyBasePath = "/ssh-keys" +) // SSHKeyService interface defines available device methods type SSHKeyService interface { List() ([]SSHKey, *Response, error) + ProjectList(string) ([]SSHKey, *Response, error) Get(string) (*SSHKey, *Response, error) Create(*SSHKeyCreateRequest) (*SSHKey, *Response, error) Update(*SSHKeyUpdateRequest) (*SSHKey, *Response, error) @@ -60,15 +63,10 @@ type SSHKeyServiceOp struct { client *Client } -// List returns a user's ssh keys -func (s *SSHKeyServiceOp) List() ([]SSHKey, *Response, error) { - req, err := s.client.NewRequest("GET", sshKeyBasePath, nil) - if err != nil { - return nil, nil, err - } - +func (s *SSHKeyServiceOp) list(url string) ([]SSHKey, *Response, error) { root := new(sshKeyRoot) - resp, err := s.client.Do(req, root) + + resp, err := s.client.DoRequest("GET", url, nil, root) if err != nil { return nil, resp, err } @@ -76,17 +74,23 @@ func (s *SSHKeyServiceOp) List() ([]SSHKey, *Response, error) { return root.SSHKeys, resp, err } +// ProjectList lists ssh keys of a project +func (s *SSHKeyServiceOp) ProjectList(projectID string) ([]SSHKey, *Response, error) { + return s.list(fmt.Sprintf("%s/%s%s", projectBasePath, projectID, sshKeyBasePath)) + +} + +// List returns a user's ssh keys +func (s *SSHKeyServiceOp) List() ([]SSHKey, *Response, error) { + return s.list(sshKeyBasePath) +} + // Get returns an ssh key by id func (s *SSHKeyServiceOp) Get(sshKeyID string) (*SSHKey, *Response, error) { path := fmt.Sprintf("%s/%s", sshKeyBasePath, sshKeyID) - - req, err := s.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - sshKey := new(SSHKey) - resp, err := s.client.Do(req, sshKey) + + resp, err := s.client.DoRequest("GET", path, nil, sshKey) if err != nil { return nil, resp, err } @@ -98,15 +102,11 @@ func (s *SSHKeyServiceOp) Get(sshKeyID string) (*SSHKey, *Response, error) { func (s *SSHKeyServiceOp) Create(createRequest *SSHKeyCreateRequest) (*SSHKey, *Response, error) { path := sshKeyBasePath if createRequest.ProjectID != "" { - path = "/projects/" + createRequest.ProjectID + sshKeyBasePath + path = fmt.Sprintf("%s/%s%s", projectBasePath, createRequest.ProjectID, sshKeyBasePath) } - req, err := s.client.NewRequest("POST", path, createRequest) - if err != nil { - return nil, nil, err - } - sshKey := new(SSHKey) - resp, err := s.client.Do(req, sshKey) + + resp, err := s.client.DoRequest("POST", path, createRequest, sshKey) if err != nil { return nil, resp, err } @@ -117,13 +117,9 @@ func (s *SSHKeyServiceOp) Create(createRequest *SSHKeyCreateRequest) (*SSHKey, * // Update updates an ssh key func (s *SSHKeyServiceOp) Update(updateRequest *SSHKeyUpdateRequest) (*SSHKey, *Response, error) { path := fmt.Sprintf("%s/%s", sshKeyBasePath, updateRequest.ID) - req, err := s.client.NewRequest("PATCH", path, updateRequest) - if err != nil { - return nil, nil, err - } - sshKey := new(SSHKey) - resp, err := s.client.Do(req, sshKey) + + resp, err := s.client.DoRequest("PATCH", path, updateRequest, sshKey) if err != nil { return nil, resp, err } @@ -135,12 +131,5 @@ func (s *SSHKeyServiceOp) Update(updateRequest *SSHKeyUpdateRequest) (*SSHKey, * func (s *SSHKeyServiceOp) Delete(sshKeyID string) (*Response, error) { path := fmt.Sprintf("%s/%s", sshKeyBasePath, sshKeyID) - req, err := s.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := s.client.Do(req, nil) - - return resp, err + return s.client.DoRequest("DELETE", path, nil, nil) } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/user.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/user.go index 36e03d909..80bf9dc84 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/user.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/user.go @@ -22,7 +22,7 @@ type User struct { Created string `json:"created_at,omitempty"` Updated string `json:"updated_at,omitempty"` TimeZone string `json:"timezone,omitempty"` - Emails []Email `json:"email,omitempty"` + Emails []Email `json:"emails,omitempty"` PhoneNumber string `json:"phone_number,omitempty"` URL string `json:"href,omitempty"` } @@ -38,13 +38,9 @@ type UserServiceOp struct { // Get method gets a user by userID func (s *UserServiceOp) Get(userID string) (*User, *Response, error) { - req, err := s.client.NewRequest("GET", userBasePath, nil) - if err != nil { - return nil, nil, err - } - user := new(User) - resp, err := s.client.Do(req, user) + + resp, err := s.client.DoRequest("GET", userBasePath, nil, user) if err != nil { return nil, resp, err } diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/utils.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/utils.go index a77d7b79e..57e5ef163 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/utils.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/utils.go @@ -17,27 +17,6 @@ func Stringify(message interface{}) string { return buf.String() } -// String allocates a new string value to store v and returns a pointer to it -func String(v string) *string { - p := new(string) - *p = v - return p -} - -// Int allocates a new int32 value to store v and returns a pointer to it, but unlike Int32 its argument value is an int. -func Int(v int) *int { - p := new(int) - *p = v - return p -} - -// Bool allocates a new bool value to store v and returns a pointer to it. -func Bool(v bool) *bool { - p := new(bool) - *p = v - return p -} - // StreamToString converts a reader to a string func StreamToString(stream io.Reader) string { buf := new(bytes.Buffer) diff --git a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/volumes.go b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/volumes.go index 96ab3b6e8..7b22d1504 100644 --- a/src/cmd/linuxkit/vendor/github.com/packethost/packngo/volumes.go +++ b/src/cmd/linuxkit/vendor/github.com/packethost/packngo/volumes.go @@ -2,33 +2,43 @@ package packngo import "fmt" -const volumeBasePath = "/storage" +const ( + volumeBasePath = "/storage" + attachmentsBasePath = "/attachments" +) // VolumeService interface defines available Volume methods type VolumeService interface { Get(string) (*Volume, *Response, error) Update(*VolumeUpdateRequest) (*Volume, *Response, error) Delete(string) (*Response, error) - Create(*VolumeCreateRequest) (*Volume, *Response, error) + Create(*VolumeCreateRequest, string) (*Volume, *Response, error) +} + +// VolumeAttachmentService defines attachment methdods +type VolumeAttachmentService interface { + Get(string) (*VolumeAttachment, *Response, error) + Create(string, string) (*VolumeAttachment, *Response, error) + Delete(string) (*Response, error) } // Volume represents a volume type Volume struct { - ID string `json:"id"` - Name string `json:"name,omitempty"` - Description string `json:"description,omitempty"` - Size int `json:"size,omitempty"` - State string `json:"state,omitempty"` - Locked bool `json:"locked,omitempty"` - BillingCycle string `json:"billing_cycle,omitempty"` - Created string `json:"created_at,omitempty"` - Updated string `json:"updated_at,omitempty"` - Href string `json:"href,omitempty"` - SnapshotPolicies []*SnapshotPolicy `json:"snapshot_policies,omitempty"` - Attachments []*Attachment `json:"attachments,omitempty"` - Plan *Plan `json:"plan,omitempty"` - Facility *Facility `json:"facility,omitempty"` - Project *Project `json:"project,omitempty"` + ID string `json:"id"` + Name string `json:"name,omitempty"` + Description string `json:"description,omitempty"` + Size int `json:"size,omitempty"` + State string `json:"state,omitempty"` + Locked bool `json:"locked,omitempty"` + BillingCycle string `json:"billing_cycle,omitempty"` + Created string `json:"created_at,omitempty"` + Updated string `json:"updated_at,omitempty"` + Href string `json:"href,omitempty"` + SnapshotPolicies []*SnapshotPolicy `json:"snapshot_policies,omitempty"` + Attachments []*VolumeAttachment `json:"attachments,omitempty"` + Plan *Plan `json:"plan,omitempty"` + Facility *Facility `json:"facility,omitempty"` + Project *Project `json:"project,omitempty"` } // SnapshotPolicy used to execute actions on volume @@ -39,12 +49,6 @@ type SnapshotPolicy struct { SnapshotCount int `json:"snapshot_count,omitempty"` } -// Attachment used to execute actions on volume -type Attachment struct { - ID string `json:"id"` - Href string `json:"href"` -} - func (v Volume) String() string { return Stringify(v) } @@ -71,10 +75,23 @@ type VolumeUpdateRequest struct { Plan string `json:"plan,omitempty"` } +// VolumeAttachment is a type from Packet API +type VolumeAttachment struct { + Href string `json:"href"` + ID string `json:"id"` + Volume Volume `json:"volume"` + Device Device `json:"device"` +} + func (v VolumeUpdateRequest) String() string { return Stringify(v) } +// VolumeAttachmentServiceOp implements VolumeService +type VolumeAttachmentServiceOp struct { + client *Client +} + // VolumeServiceOp implements VolumeService type VolumeServiceOp struct { client *Client @@ -83,13 +100,9 @@ type VolumeServiceOp struct { // Get returns a volume by id func (v *VolumeServiceOp) Get(volumeID string) (*Volume, *Response, error) { path := fmt.Sprintf("%s/%s?include=facility,snapshot_policies,attachments.device", volumeBasePath, volumeID) - req, err := v.client.NewRequest("GET", path, nil) - if err != nil { - return nil, nil, err - } - volume := new(Volume) - resp, err := v.client.Do(req, volume) + + resp, err := v.client.DoRequest("GET", path, nil, volume) if err != nil { return nil, resp, err } @@ -100,13 +113,9 @@ func (v *VolumeServiceOp) Get(volumeID string) (*Volume, *Response, error) { // Update updates a volume func (v *VolumeServiceOp) Update(updateRequest *VolumeUpdateRequest) (*Volume, *Response, error) { path := fmt.Sprintf("%s/%s", volumeBasePath, updateRequest.ID) - req, err := v.client.NewRequest("PATCH", path, updateRequest) - if err != nil { - return nil, nil, err - } - volume := new(Volume) - resp, err := v.client.Do(req, volume) + + resp, err := v.client.DoRequest("PATCH", path, updateRequest, volume) if err != nil { return nil, resp, err } @@ -118,29 +127,55 @@ func (v *VolumeServiceOp) Update(updateRequest *VolumeUpdateRequest) (*Volume, * func (v *VolumeServiceOp) Delete(volumeID string) (*Response, error) { path := fmt.Sprintf("%s/%s", volumeBasePath, volumeID) - req, err := v.client.NewRequest("DELETE", path, nil) - if err != nil { - return nil, err - } - - resp, err := v.client.Do(req, nil) - - return resp, err + return v.client.DoRequest("DELETE", path, nil, nil) } // Create creates a new volume for a project -func (v *VolumeServiceOp) Create(createRequest *VolumeCreateRequest) (*Volume, *Response, error) { - url := fmt.Sprintf("%s/%s%s", projectBasePath, createRequest.ProjectID, volumeBasePath) - req, err := v.client.NewRequest("POST", url, createRequest) - if err != nil { - return nil, nil, err - } - +func (v *VolumeServiceOp) Create(createRequest *VolumeCreateRequest, projectID string) (*Volume, *Response, error) { + url := fmt.Sprintf("%s/%s%s", projectBasePath, projectID, volumeBasePath) volume := new(Volume) - resp, err := v.client.Do(req, volume) + + resp, err := v.client.DoRequest("POST", url, createRequest, volume) if err != nil { return nil, resp, err } return volume, resp, err } + +// Attachments + +// Create Attachment, i.e. attach volume to a device +func (v *VolumeAttachmentServiceOp) Create(volumeID, deviceID string) (*VolumeAttachment, *Response, error) { + url := fmt.Sprintf("%s/%s%s", volumeBasePath, volumeID, attachmentsBasePath) + volAttachParam := map[string]string{ + "device_id": deviceID, + } + volumeAttachment := new(VolumeAttachment) + + resp, err := v.client.DoRequest("POST", url, volAttachParam, volumeAttachment) + if err != nil { + return nil, resp, err + } + return volumeAttachment, resp, nil +} + +// Get gets attachment by id +func (v *VolumeAttachmentServiceOp) Get(attachmentID string) (*VolumeAttachment, *Response, error) { + path := fmt.Sprintf("%s%s/%s", volumeBasePath, attachmentsBasePath, attachmentID) + volumeAttachment := new(VolumeAttachment) + + resp, err := v.client.DoRequest("GET", path, nil, volumeAttachment) + if err != nil { + return nil, resp, err + } + + return volumeAttachment, resp, nil +} + +// Delete deletes attachment by id +func (v *VolumeAttachmentServiceOp) Delete(attachmentID string) (*Response, error) { + path := fmt.Sprintf("%s%s/%s", volumeBasePath, attachmentsBasePath, attachmentID) + + return v.client.DoRequest("DELETE", path, nil, nil) +} From 007854f85edc9c186f331e2ce13f72e6cb38338e Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Mon, 8 Jan 2018 16:29:30 +0000 Subject: [PATCH 2/9] cmd/packet: Fix API changes in the new version of the go bindings Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/run_packet.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/cmd/linuxkit/run_packet.go b/src/cmd/linuxkit/run_packet.go index 9e2e4cbf5..2c022c881 100644 --- a/src/cmd/linuxkit/run_packet.go +++ b/src/cmd/linuxkit/run_packet.go @@ -172,7 +172,7 @@ func runPacket(args []string) { log.Debugf("%s\n", string(b)) req := packngo.DeviceUpdateRequest{ - HostName: hostname, + Hostname: hostname, UserData: userData, Locked: dev.Locked, Tags: dev.Tags, @@ -188,7 +188,7 @@ func runPacket(args []string) { } else { // Create a new device req := packngo.DeviceCreateRequest{ - HostName: hostname, + Hostname: hostname, Plan: plan, Facility: facility, OS: osType, From 50c4eb5461926a8bdb189e59ddb8e48a99b5d148 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Mon, 8 Jan 2018 16:37:46 +0000 Subject: [PATCH 3/9] cmd/packet: Actually use the -serve argument Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/run_packet.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cmd/linuxkit/run_packet.go b/src/cmd/linuxkit/run_packet.go index 2c022c881..47f5056cc 100644 --- a/src/cmd/linuxkit/run_packet.go +++ b/src/cmd/linuxkit/run_packet.go @@ -113,7 +113,7 @@ func runPacket(args []string) { var httpServer *http.Server if *serveFlag != "" { fs := serveFiles{[]string{fmt.Sprintf("%s-kernel", name), fmt.Sprintf("%s-initrd.img", name)}} - httpServer = &http.Server{Addr: ":8080", Handler: http.FileServer(fs)} + httpServer = &http.Server{Addr: *serveFlag, Handler: http.FileServer(fs)} go func() { log.Debugf("Listening on http://%s\n", *serveFlag) if err := httpServer.ListenAndServe(); err != nil { From 90510777ad347ec951910e7a878a0b0644055417 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Mon, 8 Jan 2018 17:20:31 +0000 Subject: [PATCH 4/9] cmd/packet: Tidy up the code prefix packet.net specific functions with 'packet' and make non-packet specific functions, such as validateHTTPURL() more generic so that they can easily be moved to 'utils.go' should they be needed elsewhere. Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/run_packet.go | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/src/cmd/linuxkit/run_packet.go b/src/cmd/linuxkit/run_packet.go index 47f5056cc..71c1a4914 100644 --- a/src/cmd/linuxkit/run_packet.go +++ b/src/cmd/linuxkit/run_packet.go @@ -152,8 +152,13 @@ func runPacket(args []string) { // Make sure the URL works initrdURL := fmt.Sprintf("%s/%s-initrd.img", url, name) kernelURL := fmt.Sprintf("%s/%s-kernel", url, name) - validateHTTPURL(kernelURL) - validateHTTPURL(initrdURL) + log.Infof("Validating URls: %s %s", initrdURL, kernelURL) + if err := validateHTTPURL(kernelURL); err != nil { + log.Fatalf("Invalid kernel URL %s: %v", kernelURL, err) + } + if err := validateHTTPURL(initrdURL); err != nil { + log.Fatalf("Invalid initrd URL %s: %v", initrdURL, err) + } client := packngo.NewClient("", apiKey, nil) tags := []string{} @@ -214,7 +219,7 @@ func runPacket(args []string) { sshHost := "sos." + dev.Facility.Code + ".packet.net" if *consoleFlag { // Connect to the serial console - if err := sshSOS(dev.ID, sshHost); err != nil { + if err := packetSOS(dev.ID, sshHost); err != nil { log.Fatal(err) } } else { @@ -250,19 +255,18 @@ func runPacket(args []string) { } // validateHTTPURL does a sanity check that a URL returns a 200 or 300 response -func validateHTTPURL(url string) { - log.Infof("Validating URL: %s", url) +func validateHTTPURL(url string) error { resp, err := http.Head(url) if err != nil { - log.Fatal(err) + return err } if resp.StatusCode >= 400 { - log.Fatal("Got a non 200- or 300- HTTP response code: %s", resp) + return fmt.Errorf("Got a non 200- or 300- HTTP response code: %s", resp) } - log.Debugf("OK: %d response code", resp.StatusCode) + return nil } -func sshSOS(user, host string) error { +func packetSOS(user, host string) error { log.Debugf("console: ssh %s@%s", user, host) hostKey, err := sshHostKey(host) From 8ae4f5bc86b4f55424ef03d95ad72e71d72499a7 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Mon, 8 Jan 2018 16:19:10 +0000 Subject: [PATCH 5/9] cmd/packet: Factor out the iPXE script creation Subsequent commits will use it in other places. Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/run_packet.go | 65 +++++++++++++++++++++------------- 1 file changed, 40 insertions(+), 25 deletions(-) diff --git a/src/cmd/linuxkit/run_packet.go b/src/cmd/linuxkit/run_packet.go index 71c1a4914..19dcc2276 100644 --- a/src/cmd/linuxkit/run_packet.go +++ b/src/cmd/linuxkit/run_packet.go @@ -122,31 +122,7 @@ func runPacket(args []string) { }() } - // Build the iPXE script - // Note, we *append* the -cmdline. iXPE booting will - // need the first set of "kernel-params" and we don't want to - // require these to be added to every YAML file. - userData := "#!ipxe\n\n" - userData += "dhcp\n" - userData += fmt.Sprintf("set base-url %s\n", url) - if *machineFlag != "baremetal_2a" { - var tty string - // x86_64 Packet machines have console on non standard ttyS1 which is not in most examples - if !strings.Contains(cmdline, "console=ttyS1") { - tty = "console=ttyS1,115200" - } - userData += fmt.Sprintf("set kernel-params ip=dhcp nomodeset ro serial %s %s\n", tty, cmdline) - userData += fmt.Sprintf("kernel ${base-url}/%s-kernel ${kernel-params}\n", name) - userData += fmt.Sprintf("initrd ${base-url}/%s-initrd.img\n", name) - } else { - // With EFI boot need to specify the initrd and root dev explicitly. See: - // http://ipxe.org/appnote/debian_preseed - // http://forum.ipxe.org/showthread.php?tid=7589 - userData += fmt.Sprintf("initrd --name initrd ${base-url}/%s-initrd.img\n", name) - userData += fmt.Sprintf("set kernel-params ip=dhcp nomodeset ro %s\n", cmdline) - userData += fmt.Sprintf("kernel ${base-url}/%s-kernel initrd=initrd root=/dev/ram0 ${kernel-params}\n", name) - } - userData += "boot" + userData := packetIPXEScript(name, url, cmdline, packetMachineToArch(*machineFlag)) log.Debugf("Using userData of:\n%s\n", userData) // Make sure the URL works @@ -254,6 +230,45 @@ func runPacket(args []string) { } } +// Convert machine type to architecture +func packetMachineToArch(machine string) string { + switch machine { + case "baremetal_2a", "baremetal_2a2": + return "aarch64" + default: + return "x86_64" + } +} + +// Build the iPXE script for packet machines +func packetIPXEScript(name, baseURL, cmdline, arch string) string { + // Note, we *append* the -cmdline. iXPE booting will + // need the first set of "kernel-params" and we don't want to + // require these to be added to every YAML file. + script := "#!ipxe\n\n" + script += "dhcp\n" + script += fmt.Sprintf("set base-url %s\n", baseURL) + if arch != "aarch64" { + var tty string + // x86_64 Packet machines have console on non standard ttyS1 which is not in most examples + if !strings.Contains(cmdline, "console=ttyS1") { + tty = "console=ttyS1,115200" + } + script += fmt.Sprintf("set kernel-params ip=dhcp nomodeset ro serial %s %s\n", tty, cmdline) + script += fmt.Sprintf("kernel ${base-url}/%s-kernel ${kernel-params}\n", name) + script += fmt.Sprintf("initrd ${base-url}/%s-initrd.img\n", name) + } else { + // With EFI boot need to specify the initrd and root dev explicitly. See: + // http://ipxe.org/appnote/debian_preseed + // http://forum.ipxe.org/showthread.php?tid=7589 + script += fmt.Sprintf("initrd --name initrd ${base-url}/%s-initrd.img\n", name) + script += fmt.Sprintf("set kernel-params ip=dhcp nomodeset ro %s\n", cmdline) + script += fmt.Sprintf("kernel ${base-url}/%s-kernel initrd=initrd root=/dev/ram0 ${kernel-params}\n", name) + } + script += "boot" + return script +} + // validateHTTPURL does a sanity check that a URL returns a 200 or 300 response func validateHTTPURL(url string) error { resp, err := http.Head(url) From 2431dd9950df1b87ec130f417c292f5557f79203 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Tue, 9 Jan 2018 15:35:23 +0000 Subject: [PATCH 6/9] cmd/packet: Use IPXEScriptURL instead of passing script via Userdata Using the userdat to pass the iPXE script precludes us from passing actual userdata to the instance. Instead pass a URL to the iPXE script. The script is expected to be a /-packet.ipxe. When starting a server also serve the iPXE script from memory (not the file system). The http server needed extending to differentiate between the serving files and the iPXE script and ServeMux is used to differentiate based on path. A subsequent commit will provide an option to write the iPXE script to disk. Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/run_packet.go | 73 ++++++++++++++++++++-------------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/src/cmd/linuxkit/run_packet.go b/src/cmd/linuxkit/run_packet.go index 19dcc2276..144e40c12 100644 --- a/src/cmd/linuxkit/run_packet.go +++ b/src/cmd/linuxkit/run_packet.go @@ -56,7 +56,7 @@ func runPacket(args []string) { fmt.Printf("Options:\n\n") flags.PrintDefaults() } - baseURLFlag := flags.String("base-url", "", "Base URL that the kernel and initrd are served from (or "+packetBaseURL+")") + baseURLFlag := flags.String("base-url", "", "Base URL that the kernel, initrd and iPXE script are served from (or "+packetBaseURL+")") zoneFlag := flags.String("zone", packetDefaultZone, "Packet Zone (or "+packetZoneVar+")") machineFlag := flags.String("machine", packetDefaultMachine, "Packet Machine Type (or "+packetMachineVar+")") apiKeyFlag := flags.String("api-key", "", "Packet API key (or "+packetAPIKeyVar+")") @@ -80,7 +80,7 @@ func runPacket(args []string) { url := getStringValue(packetBaseURL, *baseURLFlag, "") if url == "" { - log.Fatal("Need to specify a value for --base-url where the images are hosted. This URL should contain /%s-kernel and /%s-initrd.img") + log.Fatal("Need to specify a value for --base-url where the images are hosted. This URL should contain /%s-kernel, /%s-initrd.img and /%s-packet.ipxe") } facility := getStringValue(packetZoneVar, *zoneFlag, "") plan := getStringValue(packetMachineVar, *machineFlag, defaultMachine) @@ -101,19 +101,31 @@ func runPacket(args []string) { log.Fatalf("Combination of keep=%t and console=%t makes little sense", *keepFlag, *consoleFlag) } - // Read kernel command line - var cmdline string - if c, err := ioutil.ReadFile(prefix + "-cmdline"); err != nil { - log.Fatalf("Cannot open cmdline file: %v", err) - } else { - cmdline = string(c) - } + ipxeScriptName := fmt.Sprintf("%s-packet.ipxe", name) // Serve files with a local http server var httpServer *http.Server if *serveFlag != "" { + // Read kernel command line + var cmdline string + if c, err := ioutil.ReadFile(prefix + "-cmdline"); err != nil { + log.Fatalf("Cannot open cmdline file: %v", err) + } else { + cmdline = string(c) + } + + ipxeScript := packetIPXEScript(name, url, cmdline, packetMachineToArch(*machineFlag)) + log.Debugf("Using iPXE script:\n%s\n", ipxeScript) + + // Two handlers, one for the iPXE script and one for the kernel/initrd files + mux := http.NewServeMux() + mux.HandleFunc(fmt.Sprintf("/%s", ipxeScriptName), + func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, ipxeScript) + }) fs := serveFiles{[]string{fmt.Sprintf("%s-kernel", name), fmt.Sprintf("%s-initrd.img", name)}} - httpServer = &http.Server{Addr: *serveFlag, Handler: http.FileServer(fs)} + mux.Handle("/", http.FileServer(fs)) + httpServer = &http.Server{Addr: *serveFlag, Handler: mux} go func() { log.Debugf("Listening on http://%s\n", *serveFlag) if err := httpServer.ListenAndServe(); err != nil { @@ -122,16 +134,19 @@ func runPacket(args []string) { }() } - userData := packetIPXEScript(name, url, cmdline, packetMachineToArch(*machineFlag)) - log.Debugf("Using userData of:\n%s\n", userData) - - // Make sure the URL works + // Make sure the URLs work + ipxeURL := fmt.Sprintf("%s/%s", url, ipxeScriptName) initrdURL := fmt.Sprintf("%s/%s-initrd.img", url, name) kernelURL := fmt.Sprintf("%s/%s-kernel", url, name) - log.Infof("Validating URls: %s %s", initrdURL, kernelURL) + log.Infof("Validating URL: %s", ipxeURL) + if err := validateHTTPURL(ipxeURL); err != nil { + log.Fatalf("Invalid iPXE URL %s: %v", ipxeURL, err) + } + log.Infof("Validating URL: %s", kernelURL) if err := validateHTTPURL(kernelURL); err != nil { log.Fatalf("Invalid kernel URL %s: %v", kernelURL, err) } + log.Infof("Validating URL: %s", initrdURL) if err := validateHTTPURL(initrdURL); err != nil { log.Fatalf("Invalid initrd URL %s: %v", initrdURL, err) } @@ -153,11 +168,11 @@ func runPacket(args []string) { log.Debugf("%s\n", string(b)) req := packngo.DeviceUpdateRequest{ - Hostname: hostname, - UserData: userData, - Locked: dev.Locked, - Tags: dev.Tags, - AlwaysPXE: *alwaysPXE, + Hostname: hostname, + Locked: dev.Locked, + Tags: dev.Tags, + IPXEScriptURL: ipxeURL, + AlwaysPXE: *alwaysPXE, } dev, _, err = client.Devices.Update(*deviceFlag, &req) if err != nil { @@ -169,15 +184,15 @@ func runPacket(args []string) { } else { // Create a new device req := packngo.DeviceCreateRequest{ - Hostname: hostname, - Plan: plan, - Facility: facility, - OS: osType, - BillingCycle: billing, - ProjectID: projectID, - UserData: userData, - Tags: tags, - AlwaysPXE: *alwaysPXE, + Hostname: hostname, + Plan: plan, + Facility: facility, + OS: osType, + BillingCycle: billing, + ProjectID: projectID, + Tags: tags, + IPXEScriptURL: ipxeURL, + AlwaysPXE: *alwaysPXE, } dev, _, err = client.Devices.Create(&req) if err != nil { From 16ae50b593ad3b669022007f74aa0b773fef628a Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Tue, 9 Jan 2018 17:28:50 +0000 Subject: [PATCH 7/9] cmd/packet: Add linuxkit push packet support This currently just copies the kernel/initrd to a destination and writes the iPXE script there as well. The scheme is flexible enough to support scp or other means for pushing in the future. The kernel/initrd are conditionally decompressed (default for arm64). Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/push.go | 3 + src/cmd/linuxkit/push_packet.go | 147 ++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) create mode 100644 src/cmd/linuxkit/push_packet.go diff --git a/src/cmd/linuxkit/push.go b/src/cmd/linuxkit/push.go index a4048f673..b4ef59257 100644 --- a/src/cmd/linuxkit/push.go +++ b/src/cmd/linuxkit/push.go @@ -18,6 +18,7 @@ func pushUsage() { fmt.Printf(" azure\n") fmt.Printf(" gcp\n") fmt.Printf(" openstack\n") + fmt.Printf(" packet\n") fmt.Printf(" vcenter\n") fmt.Printf("\n") fmt.Printf("'options' are the backend specific options.\n") @@ -41,6 +42,8 @@ func push(args []string) { pushGcp(args[1:]) case "openstack": pushOpenstack(args[1:]) + case "packet": + pushPacket(args[1:]) case "vcenter": pushVCenter(args[1:]) case "help", "-h", "-help", "--help": diff --git a/src/cmd/linuxkit/push_packet.go b/src/cmd/linuxkit/push_packet.go new file mode 100644 index 000000000..a5dae9c1a --- /dev/null +++ b/src/cmd/linuxkit/push_packet.go @@ -0,0 +1,147 @@ +package main + +import ( + "compress/gzip" + "flag" + "fmt" + "io" + "io/ioutil" + "net/url" + "os" + "path/filepath" + "runtime" + + log "github.com/sirupsen/logrus" +) + +var ( + packetDefaultArch = "x86_64" + packetDefaultDecompress = false +) + +func init() { + if runtime.GOARCH == "arm64" { + packetDefaultArch = "aarch64" + // decompress on arm64. iPXE/kernel does not + // seem to grok compressed kernels/initrds. + packetDefaultDecompress = true + } +} + +// Process the run arguments and execute run +func pushPacket(args []string) { + flags := flag.NewFlagSet("packet", flag.ExitOnError) + invoked := filepath.Base(os.Args[0]) + flags.Usage = func() { + fmt.Printf("USAGE: %s push packet [options] [name]\n\n", invoked) + fmt.Printf("Options:\n\n") + flags.PrintDefaults() + } + baseURLFlag := flags.String("base-url", "", "Base URL that the kernel, initrd and iPXE script are served from (or "+packetBaseURL+")") + nameFlag := flags.String("img-name", "", "Overrides the prefix used to identify the files. Defaults to [name] (or "+packetNameVar+")") + archFlag := flags.String("arch", packetDefaultArch, "Image architecture (x86_64 or aarch64)") + decompressFlag := flags.Bool("decompress", packetDefaultDecompress, "Decompress kernel/initrd before pushing") + dstFlag := flags.String("destination", "", "URL where to push the image to. Currently only 'file' is supported as a scheme (which is also the default if omitted)") + + if err := flags.Parse(args); err != nil { + log.Fatal("Unable to parse args") + } + + remArgs := flags.Args() + prefix := "packet" + if len(remArgs) > 0 { + prefix = remArgs[0] + } + + baseURL := getStringValue(packetBaseURL, *baseURLFlag, "") + if baseURL == "" { + log.Fatal("Need to specify a value for --base-url from where the kernel, initrd and iPXE script will be loaded from.") + } + + if *dstFlag == "" { + log.Fatal("Need to specify the destination where to push to.") + } + + name := getStringValue(packetNameVar, *nameFlag, prefix) + + if _, err := os.Stat(fmt.Sprintf("%s-kernel", name)); os.IsNotExist(err) { + log.Fatalf("kernel file does not exist: %v", err) + } + if _, err := os.Stat(fmt.Sprintf("%s-initrd.img", name)); os.IsNotExist(err) { + log.Fatalf("initrd file does not exist: %v", err) + } + + // Read kernel command line + var cmdline string + if c, err := ioutil.ReadFile(prefix + "-cmdline"); err != nil { + log.Fatalf("Cannot open cmdline file: %v", err) + } else { + cmdline = string(c) + } + + ipxeScript := packetIPXEScript(name, baseURL, cmdline, *archFlag) + + // Parse the destination + dst, err := url.Parse(*dstFlag) + if err != nil { + log.Fatalf("Cannot parse destination: %v", err) + } + switch dst.Scheme { + case "", "file": + packetPushFile(dst, *decompressFlag, name, cmdline, ipxeScript) + default: + log.Fatalf("Unknown destination format: %s", dst.Scheme) + } +} + +func packetPushFile(dst *url.URL, decompress bool, name, cmdline, ipxeScript string) { + // Make sure the destination exists + dstPath := filepath.Clean(dst.Path) + if err := os.MkdirAll(dstPath, 0755); err != nil { + log.Fatalf("Could not create destination directory: %v", err) + } + + kernelName := fmt.Sprintf("%s-kernel", name) + if err := packetCopy(filepath.Join(dstPath, kernelName), kernelName, decompress); err != nil { + log.Fatalf("Error copying kernel: %v", err) + } + + initrdName := fmt.Sprintf("%s-initrd.img", name) + if err := packetCopy(filepath.Join(dstPath, initrdName), initrdName, decompress); err != nil { + log.Fatalf("Error copying initrd: %v", err) + } + + ipxeScriptName := fmt.Sprintf("%s-packet.ipxe", name) + if err := ioutil.WriteFile(filepath.Join(dstPath, ipxeScriptName), []byte(ipxeScript), 0644); err != nil { + log.Fatalf("Error writing iPXE script: %v", err) + } +} + +func packetCopy(dst, src string, decompress bool) error { + in, err := os.Open(src) + if err != nil { + return err + } + defer in.Close() + + var r io.Reader = in + if decompress { + if rd, err := gzip.NewReader(in); err != nil { + log.Warnf("%s does not seem to be gzip'ed (%v). Ignore decompress.", src, err) + } else { + r = rd + } + } + + out, err := os.Create(dst) + if err != nil { + return err + } + defer out.Close() + + _, err = io.Copy(out, r) + if err != nil { + return err + } + return out.Close() +} From 1cef947ee1f0404f55e31ed8805e90afe28dacb1 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Wed, 10 Jan 2018 17:37:17 +0000 Subject: [PATCH 8/9] cmd/serve: Add a new 'linuxkit serve' command This simply starts a web server serving the specified directory. It's useful for PXE booting. Signed-off-by: Rolf Neugebauer --- src/cmd/linuxkit/main.go | 3 +++ src/cmd/linuxkit/serve.go | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) create mode 100644 src/cmd/linuxkit/serve.go diff --git a/src/cmd/linuxkit/main.go b/src/cmd/linuxkit/main.go index fda3ea09d..7db9f9864 100644 --- a/src/cmd/linuxkit/main.go +++ b/src/cmd/linuxkit/main.go @@ -76,6 +76,7 @@ func main() { fmt.Printf(" pkg Package building\n") fmt.Printf(" push Push a VM image to a cloud or image store\n") fmt.Printf(" run Run a VM image on a local hypervisor or remote cloud\n") + fmt.Printf(" serve Run a local http server (for iPXE booting)\n") fmt.Printf(" version Print version information\n") fmt.Printf(" help Print this message\n") fmt.Printf("\n") @@ -124,6 +125,8 @@ func main() { push(args[1:]) case "run": run(args[1:]) + case "serve": + serve(args[1:]) case "version": printVersion() case "help": diff --git a/src/cmd/linuxkit/serve.go b/src/cmd/linuxkit/serve.go new file mode 100644 index 000000000..ebe16da05 --- /dev/null +++ b/src/cmd/linuxkit/serve.go @@ -0,0 +1,34 @@ +package main + +import ( + "flag" + "fmt" + "net/http" + "os" + "path/filepath" + + log "github.com/sirupsen/logrus" +) + +func logRequest(handler http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + log.Infof("%s %s", r.Method, r.URL) + handler.ServeHTTP(w, r) + }) +} + +// serve starts a local web server +func serve(args []string) { + flags := flag.NewFlagSet("serve", flag.ExitOnError) + invoked := filepath.Base(os.Args[0]) + flags.Usage = func() { + fmt.Printf("USAGE: %s serve [options]\n\n", invoked) + fmt.Printf("Options:\n\n") + flags.PrintDefaults() + } + portFlag := flags.String("port", ":8080", "Local port to serve on") + dirFlag := flags.String("directory", ".", "Directory to serve") + + http.Handle("/", http.FileServer(http.Dir(*dirFlag))) + log.Fatal(http.ListenAndServe(*portFlag, logRequest(http.DefaultServeMux))) +} From 293b2bb9822d950497e342bd317978b413e4ba97 Mon Sep 17 00:00:00 2001 From: Rolf Neugebauer Date: Wed, 10 Jan 2018 18:26:54 +0000 Subject: [PATCH 9/9] docs: Update packet.net documentation Signed-off-by: Rolf Neugebauer --- docs/platform-packet.md | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/docs/platform-packet.md b/docs/platform-packet.md index 8097602f4..d830b1afe 100644 --- a/docs/platform-packet.md +++ b/docs/platform-packet.md @@ -40,11 +40,17 @@ retry the boot typically fixes this. ## Boot -LinuxKit on Packet boots the `kernel+initrd` output from moby -via -[iPXE](https://help.packet.net/technical/infrastructure/custom-ipxe). iPXE -booting requires a HTTP server on which you can store your images. The -`-base-url` option specifies the URL to the HTTP server. +LinuxKit on Packet boots the `kernel+initrd` output from moby via +[iPXE](https://help.packet.net/technical/infrastructure/custom-ipxe) +which also requires a iPXE script. iPXE booting requires a HTTP server +on which you can store your images. The `-base-url` option specifies +the URL to a HTTP server from which `-kernel`, +`-initrd.img`, and `-packet.ipxe` can be downloaded during +boot. + +If you have your own HTTP server, you can use `linuxkit push packet` +to create the files (including the iPXE script) you need to make +available. If you don't have a public HTTP server at hand, you can use the `-serve` option. This will create a local HTTP server which can either @@ -62,9 +68,10 @@ PACKET_API_KEY= PACKET_PROJECT_ID= \ linuxkit run packet -serve :8080 -base-url packet ``` -To boot a `arm64` image for Type 2a machine (`-machine -baremetal_2a`) you currently need build using `linuxkit build packet.yml packet.arm64.yml` and then un-compress both the kernel and -the initrd before booting, e.g: +To boot a `arm64` image for Type 2a machine (`-machine baremetal_2a`) +you currently need to build using `linuxkit build packet.yml +packet.arm64.yml` and then un-compress both the kernel and the initrd +before booting, e.g: ```sh mv packet-initrd.img packet-initrd.img.gz && gzip -d packet-initrd.img.gz @@ -78,12 +85,16 @@ PACKET_API_KEY= PACKET_PROJECT_ID= \ linuxkit run packet -machine baremetal_2a -serve :8080 -base-url -base-url packet ``` +Alternatively, `linuxkit push packet` will uncompress the kernel and +initrd images on arm machines (or explicitly via the `-decompress` +flag. There is also a `linuxkit serve` command which will start a +local HTTP server serving the specified directory. + **Note**: It may take several minutes to deploy a new server. If you are attached to the console, you should see the BIOS and the boot messages. - ## Console By default, `linuxkit run packet ...` will connect to the