github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-07-20 18:31:15 +00:00
Code Issues Projects Releases Wiki Activity

Change KUBE_PARALLEL_BUILD_MEMORY to 20 GiB

Browse Source 
A benchmark with go1.16 on amd64 resulted in an overall maximum memory
usage of 15GiB. This means we now lower the `KUBE_PARALLEL_BUILD_MEMORY`
to `20` to still have some room left.

The benchmark has been done with the following `Dockerfile`:

```dockerfile
FROM k8s.gcr.io/build-image/kube-cross:v1.16.0-1
RUN apt-get update && apt-get install -y time

WORKDIR /go/src/k8s.io/kubernetes
RUN git clone https://github.com/kubernetes/kubernetes
ENV KUBE_PARALLEL_BUILD_MEMORY=0
RUN /usr/bin/time -v make -C kubernetes cross-in-a-container
```

The value of interest from the output:

```
Maximum resident set size (kbytes): 1847108
```

The RSS is not the only value we have to consider. During the runtime I
measured the usage of the cached memory, which gets peaks up to 15GiB.

Test machine specs:
- CPU: 2 x Intel Scalable Gold 5120 28-Core Processor @ 2.2GHz
- RAM: 384GB
- Disk: 1 x 3.8TB NVME
- OS: Debian 10 (buster)

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
This commit is contained in:
Sascha Grunert 2021-03-04 11:09:39 +01:00
parent 930ca85613
commit abf85d70c2
No known key found for this signature in database
GPG Key ID: 09D97D153EF94D93
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
hack/lib/golang.sh
View File

@ -306,10 +306,11 @@ readonly KUBE_TEST_SERVER_BINARIES=("${KUBE_TEST_SERVER_TARGETS[@]##*/}")
readonly KUBE_TEST_SERVER_PLATFORMS=("${KUBE_SERVER_PLATFORMS[@]:+"${KUBE_SERVER_PLATFORMS[@]}"}") readonly KUBE_TEST_SERVER_PLATFORMS=("${KUBE_SERVER_PLATFORMS[@]:+"${KUBE_SERVER_PLATFORMS[@]}"}")
# Gigabytes necessary for parallel platform builds. # Gigabytes necessary for parallel platform builds.
# As of January 2018, RAM usage is exceeding 30G. # As of March 2021 (go 1.16/amd64), the RSS usage is 2GiB by using cached
# memory of 15GiB.
# This variable can be overwritten at your own risk. # This variable can be overwritten at your own risk.
# It's defaulting to 40G to provide some headroom # It's defaulting to 20G to provide some headroom.
readonly KUBE_PARALLEL_BUILD_MEMORY=${KUBE_PARALLEL_BUILD_MEMORY:-40} readonly KUBE_PARALLEL_BUILD_MEMORY=${KUBE_PARALLEL_BUILD_MEMORY:-20}
readonly KUBE_ALL_TARGETS=( readonly KUBE_ALL_TARGETS=(
"${KUBE_SERVER_TARGETS[@]}" "${KUBE_SERVER_TARGETS[@]}"