Generate intermediate image into a temp file

All of the `output*` functions took a `[]byte` and immediately wrapped it in a
`bytes.Buffer` to produce an `io.Reader`. Make them take an `io.Reader` instead
and satisfy this further up the call chain by directing `moby.Build` to output
to a temp file instead of another `bytes.Buffer`.

In my test case (building kube master image) this reduces Maximum RSS (as
measured by time(1)) from 6.7G to 2.8G and overall allocations from 9.7G to
5.3G. When building a tar (output to /dev/null) the Maximum RSS fell slightly
from 2.2G to 2.1G. Overall allocations remained stable at around 5.3G.

Signed-off-by: Ian Campbell <ijc@docker.com>
This commit is contained in:
Ian Campbell
2017-12-06 14:53:18 +00:00
parent 9558740c11
commit 9f44acf8e3
3 changed files with 54 additions and 31 deletions

View File

@@ -1,7 +1,6 @@
package moby
import (
"bytes"
"crypto/sha256"
"fmt"
"io"
@@ -58,9 +57,21 @@ func ensureLinuxkitImage(name string) error {
return err
}
// TODO pass through --pull to here
buf := new(bytes.Buffer)
Build(m, buf, false, "")
image := buf.Bytes()
tf, err := ioutil.TempFile("", "")
if err != nil {
return err
}
defer os.Remove(tf.Name())
Build(m, tf, false, "")
if err := tf.Close(); err != nil {
return err
}
image, err := os.Open(tf.Name())
if err != nil {
return err
}
defer image.Close()
kernel, initrd, cmdline, err := tarToInitrd(image)
if err != nil {
return fmt.Errorf("Error converting to initrd: %v", err)