656bd87fd2...d9d2a91780d9d2a91 Merge pull request #193 from ijc/bugfix-191
307f13b Defer dockerRm until we are finished with the contents
ebd7228 Merge pull request #191 from ijc/reduce-memory-via-tempfiles
3045a80 Stream `docker export` directly to consumer
9f44acf Generate intermediate image into a temp file
9558740 Add cpu and mem profiling options
Reduces the memory usage substantially.
While here make some notes about the need to update src/cmd/linuxkit/build.go
where people might see them.
Signed-off-by: Ian Campbell <ijc@docker.com>
Rather than queueing up into a `bytes.Buffer`.
In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) compared with the previous patch from 2.8G to 110M. The
tar output case goes from 2.1G to 110M also. Overall allocations are ~715M in
both cases.
Signed-off-by: Ian Campbell <ijc@docker.com>
All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.
In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.
Signed-off-by: Ian Campbell <ijc@docker.com>
The log.Fatal* calls will leak the vpnkit process since
defer functions are not invoked when os.Exit(int) is invoked
We register an ExitHandler with logrus - that'll be invoke when
log.Fatal* is called
Signed-off-by: Steve Hiehn <shiehn@pivotal.io>
Signed-off-by: Dave Protasowski <dprotaso@gmail.com>
This allows the `linuxkit/kubernetes` "image-cache" packages to use a standard
`linuxkit pkg build` based flow rather than requiring surrounding scaffolding.
Fixes#2766. Compared with the original (actually, the second) proposal made in
issue #2766, the field is `docker-images` rather than `images` to allow for
future inclusion of e.g. `containerd-images`.
Signed-off-by: Ian Campbell <ijc@docker.com>
You can use `linuxkit run vbox ...` to run using Oracle VirtualBox.
ISO or raw should be supported, and you can specify EFI.
Some more options may be useful in future.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
vpnkit mode has one or two arguments (eth socket and port control socket).
If splitting into only 2 then
vpnkit,vpnkit-state/eth.sock,vpnkit-state/port.sock
becomes
2: vpnkit
1: vpnkit-state/eth.sock,vpnkit-state/port.sock
rather than
0: vpnkit
1: vpnkit-state/eth.sock
2: vpnkit-state/port.sock
as desired.
Signed-off-by: Ian Campbell <ijc@docker.com>
- ENA networking is what the very new (and bare metal) machines have
- sriov "simple" mode is Intel VF passthrough
I reflect the underlying APIs, which is bool for ENA and a string
for sriov even though there is currently only one valid sriov option...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This is a `moby.ImageConfig` struct which is marshalled into JSON and added as
the `org.mobyproject.config` label on the built image.
Convert `pkg/sysctl` as PoC.
Signed-off-by: Ian Campbell <ijc@docker.com>
The syntax used for the yaml definitions is changed by the need to include the
substruct in the struct literal.
For the label switch to `ImageConfig` directly, which is actually more correct
in that it avoids spurious `name` and `image` fields in the label.
Signed-off-by: Ian Campbell <ijc@docker.com>
Where "config-related" here means "ones you might find in the
"org.mobyproject.config" label on an image.
By making this new struct an anonymous member of the existing Image struct the
Go json parser does the right thing (i.e. inlines into the parent) when parsing
a complete image (from a yml assembly) by default. The Go yaml library which we
use requires a tag on the anonymous field to achieve the same.
Signed-off-by: Ian Campbell <ijc@docker.com>
This code was identical in the QEMU and HyperKit cases. Move it to util.go and
wrap it in a function, with minimal changes for returning an error.
Signed-off-by: Ian Campbell <ijc@docker.com>
This is not in most examples, but is in the Packet example, and
causes a duplicated console.
fix#2735
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
With the switch to github.com/satori/go.uuid this is no
longer needed. Strangely, it was not in vendor.conf...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
When hyperkit is used with VPNKit we can "publish" VM ports
on localhost by using an API to the VPNKit process.
This commit adds a "-publish" flag to the hyperkit backend
exposing this functionality. "-publish" can be used multiple
times and is used the same way as for the qemu backend.
Other changes:
- Use 'github.com/google/uuid' as the 'uuid' package since
VPKNkit uses it
- use 'vpnkit' prefix instead of 'vpnKit'
- Improve logging
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
