test images: Adds Windows support (part 1)

Adds Windows support to the test/images/image-util.sh script.

A Windows node with Docker installed is required to build Windows images.
The connection URL to it must be set in the REMOTE_DOCKER_URL env variable.
Additionally, the authentication to the remote docker node is done through
certificates, which must be found in ~/.docker.

By default, the REMOTE_DOCKER_URL env variable is set to "" in the Makefile,
and because of it, the image-util.sh script will skip building and pushing
Windows images.

Added GOOS argument to the go build process in order to be able to build
Windows binaries. Additionally, the OS env variable was added to the images
Makefiles (default value is "linux") in order to maintain default behaviour.

Some images require a different Dockerfile for Windows images, since they
have different ways of installing dependencies. Because of this, if a image
needs to be built for Windows, it will first check for a Dockerfile_windows
file instead of the default one. If there isn't one, it means that the
same Dockerfile can be used for both Windows and Linux.

All Windows images will be based on the image
"mcr.microsoft.com/windows/servercore:ltsc2019". There are a couple of features
that are needed from this image, especially powershell.

Added busybox image for Windows. Most Windows images will be based on it, which
will help reduce the command line differences between Linux and Windows, but
not entirely.

Added Windows support for agnhost image.

test/images/BUILD

@@ -13,6 +13,7 @@ filegroup(
         ":package-srcs",
         "//test/images/agnhost:all-srcs",
         "//test/images/apparmor-loader:all-srcs",
+        "//test/images/busybox:all-srcs",
         "//test/images/echoserver:all-srcs",
         "//test/images/metadata-concealment:all-srcs",
         "//test/images/nonewprivs:all-srcs",

test/images/Makefile

@@ -14,6 +14,7 @@
 
 REGISTRY ?= gcr.io/kubernetes-e2e-test-images
 GOARM ?= 7
+REMOTE_DOCKER_URL ?=
 QEMUVERSION=v2.9.1
 GOLANG_VERSION=1.13.6
 export

test/images/agnhost/BASEIMAGE

@@ -3,3 +3,4 @@ linux/arm=arm32v6/alpine:3.6
 linux/arm64=arm64v8/alpine:3.6
 linux/ppc64le=ppc64le/alpine:3.6
 linux/s390x=s390x/alpine:3.6
+windows/amd64=REGISTRY/busybox:1.29-windows-amd64

test/images/agnhost/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/agnhost/Dockerfile_windows

@@ -0,0 +1,64 @@
+# Copyright 2020 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+ARG BASEIMAGE
+FROM $BASEIMAGE
+
+# from dnsutils image
+# install necessary packages:
+# - bind-tools: contains dig, which can used in DNS tests.
+# - CoreDNS: used in some DNS tests.
+# from hostexec image
+# installed necessary packages:
+# - curl, nc: used by a lot of e2e tests (inherited from BASEIMAGE)
+# from iperf image
+# install necessary packages: iperf
+ENV chocolateyUseWindowsCompression false
+RUN powershell -Command "\
+    iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1')); \
+    choco feature disable --name showDownloadProgress; \
+    choco install bind-toolsonly --version 9.10.3 -y
+RUN powershell -Command "\
+    wget -uri 'https://github.com/coredns/coredns/releases/download/v1.5.0/coredns_1.5.0_windows_amd64.tgz' -OutFile C:\coredns.tgz;\
+    tar -xzvf C:\coredns.tgz;\
+    Remove-Item C:\coredns.tgz"
+
+RUN powershell -Command "\
+    wget -uri 'https://iperf.fr/download/windows/iperf-2.0.9-win64.zip' -OutFile C:\iperf.zip;\
+    Expand-Archive -Path C:\iperf.zip -DestinationPath C:\ -Force;\
+    Rename-Item C:\iperf-2.0.9-win64 C:\iperf;\
+    Remove-Item C:\iperf.zip"
+
+# PORT 80 needed by: test-webserver
+# PORT 8080 needed by: netexec, nettest
+# PORT 8081 needed by: netexec
+# PORT 9376 needed by: serve-hostname
+EXPOSE 80 8080 8081 9376
+
+# from netexec
+RUN mkdir C:\uploads
+
+# from porter
+ADD porter/localhost.crt localhost.crt
+ADD porter/localhost.key localhost.key
+
+ADD agnhost agnhost
+
+# needed for the entrypoint-tester related tests. Some of the entrypoint-tester related tests
+# overrides this image's entrypoint with agnhost-2 binary, and will verify that the correct
+# entrypoint is used by the containers.
+RUN mklink agnhost-2 agnhost
+
+ENTRYPOINT ["/agnhost"]
+CMD ["pause"]

test/images/agnhost/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS=agnhost
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/agnhost/README.md

@@ -40,7 +40,7 @@ For example, let's consider the following `pod.yaml` file:
       containers:
       - args:
         - dns-suffix
-        image: us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10
+        image: us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11
         name: agnhost
       dnsConfig:
         nameservers:
@@ -207,7 +207,7 @@ Usage:
 
 ```console
 guestbook="test/e2e/testing-manifests/guestbook"
-sed_expr="s|{{.AgnhostImage}}|us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10|"
+sed_expr="s|{{.AgnhostImage}}|us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11|"
 
 # create the services.
 kubectl create -f ${guestbook}/frontend-service.yaml
@@ -290,14 +290,14 @@ Examples:
 
 ```console
 docker run -i \
-  us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10 \
+  us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11 \
   logs-generator --log-lines-total 10 --run-duration 1s
 ```
 
 ```console
 kubectl run logs-generator \
   --generator=run-pod/v1 \
-  --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10 \
+  --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11 \
   --restart=Never \
   -- logs-generator -t 10 -d 1s
 ```
@@ -455,7 +455,7 @@ Usage:
 ```console
     kubectl run test-agnhost \
       --generator=run-pod/v1 \
-      --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10 \
+      --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11 \
       --restart=Never \
       --env "POD_IP=<POD_IP>" \
       --env "NODE_IP=<NODE_IP>" \
@@ -510,7 +510,7 @@ Usage:
 ```console
     kubectl run test-agnhost \
       --generator=run-pod/v1 \
-      --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10 \
+      --image=us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11 \
       --restart=Never \
       --env "BIND_ADDRESS=localhost" \
       --env "BIND_PORT=8080" \
@@ -626,11 +626,17 @@ Usage:
 
 ## Other tools
 
-The image contains `iperf`, `curl`, `dns-tools` (including `dig`), CoreDNS.
+The image contains `iperf`, `curl`, `dns-tools` (including `dig`), CoreDNS, for both Windows and Linux.
+
+For Windows, the image is based on `busybox`, meaning that most of the Linux common tools are also
+available on it, making it possible to run most Linux commands in the `agnhost` Windows container
+as well. Keep in mind that there might still be some differences though (e.g.: `wget` does not
+have the `-T` argument on Windows).
+
+The Windows `agnhost` image includes a `nc` binary that is 100% compliant with its Linux equivalent.
 
 
 ## Image
 
-The image can be found at `us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.10` for Linux
+The image can be found at `us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.11` for both Linux and
-containers, and `e2eteam/agnhost:2.10` for Windows containers. In the future, the same
+Windows containers (based on `mcr.microsoft.com/windows/servercore:ltsc2019`).
-repository can be used for both OSes.

test/images/apparmor-loader/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/apparmor-loader/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS=loader
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/busybox/BASEIMAGE

@@ -0,0 +1 @@
+windows/amd64=mcr.microsoft.com/windows/servercore:ltsc2019

test/images/busybox/BUILD

@@ -0,0 +1,31 @@
+package(default_visibility = ["//visibility:public"])
+
+load(
+    "@io_bazel_rules_go//go:def.bzl",
+    "go_binary",
+    "go_library",
+)
+
+go_binary(
+    name = "hostname",
+    embed = [":go_default_library"],
+)
+
+go_library(
+    name = "go_default_library",
+    srcs = ["hostname.go"],
+    importpath = "k8s.io/kubernetes/test/images/busybox",
+)
+
+filegroup(
+    name = "package-srcs",
+    srcs = glob(["**"]),
+    tags = ["automanaged"],
+    visibility = ["//visibility:private"],
+)
+
+filegroup(
+    name = "all-srcs",
+    srcs = [":package-srcs"],
+    tags = ["automanaged"],
+)

test/images/busybox/Dockerfile_windows

@@ -0,0 +1,34 @@
+# Copyright 2020 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+ARG BASEIMAGE
+from $BASEIMAGE as prep
+
+ENV CURL_VERSION 7.57.0
+WORKDIR /curl
+ADD https://skanthak.homepage.t-online.de/download/curl-$CURL_VERSION.cab curl.cab
+RUN expand /R curl.cab /F:* .
+
+FROM $BASEIMAGE
+
+COPY --from=prep /curl/AMD64 /curl
+COPY --from=prep /curl/CURL.LIC /curl
+ADD https://github.com/kubernetes-sigs/windows-testing/raw/master/images/busybox/busybox.exe /bin/busybox.exe
+ADD https://github.com/diegocr/netcat/raw/master/nc.exe /bin/nc.exe
+ADD hostname /bin/hostname.exe
+USER ContainerAdministrator
+RUN FOR /f "tokens=*" %i IN ('C:\bin\busybox --list') DO mklink C:\bin\%i.exe C:\bin\busybox.exe
+RUN setx /M PATH "C:\bin;C:\curl\;%PATH%" &\
+    mkdir C:\tmp
+ENTRYPOINT ["cmd.exe", "/s", "/c"]

test/images/busybox/Makefile

@@ -0,0 +1,27 @@
+# Copyright 2020 The Kubernetes Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+SRCS=hostname
+OS ?= linux
+ARCH ?= amd64
+TARGET ?= $(CURDIR)
+GOARM = 7
+GOLANG_VERSION ?= latest
+SRC_DIR = $(notdir $(shell pwd))
+export
+
+bin:
+	../image-util.sh bin $(SRCS)
+
+.PHONY: bin

test/images/busybox/VERSION

@@ -0,0 +1 @@
+1.29

test/images/busybox/hostname.go

@@ -0,0 +1,50 @@
+/*
+Copyright 2020 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"net"
+	"os"
+)
+
+func getOutboundIP() net.IP {
+	conn, err := net.Dial("udp", "8.8.8.8:80")
+	if err != nil {
+		log.Fatal(err)
+	}
+	defer conn.Close()
+	localAddr := conn.LocalAddr().(*net.UDPAddr)
+	return localAddr.IP
+}
+
+func main() {
+	flagIP := flag.Bool("i", false, "a string")
+	flag.Parse()
+	if *flagIP {
+		ip := getOutboundIP()
+		fmt.Print(ip.String())
+	} else {
+		hostname, err := os.Hostname()
+		if err != nil {
+			log.Fatal(err)
+		}
+		fmt.Print(hostname)
+	}
+}

test/images/cuda-vector-add/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/echoserver/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/image-util.sh

@@ -77,7 +77,7 @@ build() {
     if [[ -f ${image}/Makefile ]]; then
       # make bin will take care of all the prerequisites needed
       # for building the docker image
-      make -C "${image}" bin ARCH="${arch}" TARGET="${temp_dir}"
+      make -C "${image}" bin OS="${os_name}" ARCH="${arch}" TARGET="${temp_dir}"
     fi
     pushd "${temp_dir}"
     # image tag
@@ -85,8 +85,16 @@ build() {
 
     if [[ -f BASEIMAGE ]]; then
       BASEIMAGE=$(getBaseImage "${os_name}" "${arch}" | ${SED} "s|REGISTRY|${REGISTRY}|g")
-      ${SED} -i "s|BASEIMAGE|${BASEIMAGE}|g" Dockerfile
+
-      ${SED} -i "s|BASEARCH|${arch}|g" Dockerfile
+      # NOTE(claudiub): Some Windows images might require their own Dockerfile
+      # while simpler ones will not. If we're building for Windows, check if
+      # "Dockerfile_windows" exists or not.
+      dockerfile_name="Dockerfile"
+      if [[ "$os_name" = "windows" && -f "Dockerfile_windows" ]]; then
+        dockerfile_name="Dockerfile_windows"
+      fi
+
+      ${SED} -i "s|BASEARCH|${arch}|g" $dockerfile_name
     fi
 
     # copy the qemu-*-static binary to docker image to build the multi architecture image on x86 platform
@@ -109,8 +117,16 @@ build() {
       fi
     fi
 
-    docker build --pull -t "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}" .
+    if [[ "$os_name" = "linux" ]]; then
-
+      docker build --pull -t "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}" --build-arg BASEIMAGE="${BASEIMAGE}" .
+    elif [[ -n "${REMOTE_DOCKER_URL:-}" ]]; then
+      # NOTE(claudiub): We're using a remote Windows node to build the Windows Docker images.
+      # The node requires TLS authentication, and thus it is expected that the
+      # ca.pem, cert.pem, key.pem files can be found in the ~/.docker folder.
+      docker --tlsverify -H "${REMOTE_DOCKER_URL}" build --pull -t "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}" --build-arg BASEIMAGE="${BASEIMAGE}" -f $dockerfile_name .
+    else
+      echo "Cannot build the image '${image}' for ${os_arch}. REMOTE_DOCKER_URL should be set, containing the URL to a Windows docker daemon."
+    fi
     popd
   done
 }
@@ -132,6 +148,12 @@ push() {
   TAG=$(<"${image}"/VERSION)
   if [[ -f ${image}/BASEIMAGE ]]; then
     os_archs=$(listOsArchs "$image")
+    # NOTE(claudiub): if the REMOTE_DOCKER_URL var is not set, or it is an empty string, we must skip
+    # pushing the Windows image and including it into the manifest list.
+    if test -z "${REMOTE_DOCKER_URL:-}" && printf "%s\n" "$os_archs" | grep -q '^windows'; then
+      echo "Skipping pushing the image '${image}' for Windows. REMOTE_DOCKER_URL should be set, containing the URL to a Windows docker daemon."
+      os_archs=$(printf "%s\n" "$os_archs" | grep -v "^windows")
+    fi
   else
     # prepend linux/ to the QEMUARCHS items.
     os_archs=$(printf 'linux/%s\n' "${!QEMUARCHS[*]}")
@@ -145,7 +167,12 @@ push() {
       exit 1
     fi
 
+    if [[ "$os_name" = "linux" ]]; then
       docker push "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}"
+    else
+      # NOTE(claudiub): We're pushing the image we built on the remote Windows node.
+      docker --tlsverify -H "${REMOTE_DOCKER_URL}" push "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}"
+    fi
   done
 
   kube::util::ensure-gnu-sed
@@ -165,7 +192,7 @@ push() {
       echo "The BASEIMAGE file for the ${image} image is not properly formatted. Expected entries to start with 'os/arch', found '${os_arch}' instead."
       exit 1
     fi
-    docker manifest annotate --arch "${arch}" "${REGISTRY}/${image}:${TAG}" "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}"
+    docker manifest annotate --os "${os_name}" --arch "${arch}" "${REGISTRY}/${image}:${TAG}" "${REGISTRY}/${image}:${TAG}-${os_name}-${arch}"
   done
   docker manifest push --purge "${REGISTRY}/${image}:${TAG}"
 }
@@ -182,7 +209,7 @@ bin() {
         golang:"${GOLANG_VERSION}" \
         /bin/bash -c "\
                 cd /go/src/k8s.io/kubernetes/test/images/${SRC_DIR} && \
-                CGO_ENABLED=0 ${arch_prefix} GOARCH=${ARCH} go build -a -installsuffix cgo --ldflags '-w' -o ${TARGET}/${SRC} ./$(dirname "${SRC}")"
+                CGO_ENABLED=0 ${arch_prefix} GOOS=${OS} GOARCH=${ARCH} go build -a -installsuffix cgo --ldflags '-w' -o ${TARGET}/${SRC} ./$(dirname "${SRC}")"
   done
 }
 
@@ -195,7 +222,7 @@ if [[ "${WHAT}" == "all-conformance" ]]; then
   # Discussed during Conformance Office Hours Meeting (2019.12.17):
   # https://docs.google.com/document/d/1W31nXh9RYAb_VaYkwuPLd1hFxuRX3iU0DmaQ4lkCsX8/edit#heading=h.l87lu17xm9bh
   # echoserver image not included: https://github.com/kubernetes/kubernetes/issues/84158
-  conformance_images=("agnhost" "jessie-dnsutils" "kitten" "nautilus" "nonewprivs" "resource-consumer" "sample-apiserver")
+  conformance_images=("busybox" "agnhost" "jessie-dnsutils" "kitten" "nautilus" "nonewprivs" "resource-consumer" "sample-apiserver")
   for image in "${conformance_images[@]}"; do
     eval "${TASK}" "${image}"
   done

test/images/ipc-utils/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/jessie-dnsutils/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/kitten/Dockerfile

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 COPY html/kitten.jpg kitten.jpg
 COPY html/data.json data.json

test/images/metadata-concealment/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS=check_metadata_concealment
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/nautilus/Dockerfile

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 COPY html/nautilus.jpg nautilus.jpg
 COPY html/data.json data.json

test/images/node-perf/npb-ep/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE as build_node_perf_npb_ep
+ARG BASEIMAGE
+FROM $BASEIMAGE as build_node_perf_npb_ep
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 
@@ -33,7 +34,8 @@ RUN make EP CLASS=D
 # main container in the second build stage.
 RUN mkdir -p /lib-copy && find /usr/lib -name "*.so.*" -exec cp {} /lib-copy \;
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 COPY --from=build_node_perf_npb_ep /NPB3.3.1/NPB3.3-OMP/bin/ep.D.x /
 COPY --from=build_node_perf_npb_ep /lib-copy /lib-copy

test/images/node-perf/npb-is/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE as build_node_perf_npb_is
+ARG BASEIMAGE
+FROM $BASEIMAGE as build_node_perf_npb_is
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 
@@ -35,7 +36,8 @@ RUN make IS CLASS=D
 # main container in the second build stage.
 RUN mkdir -p /lib-copy && find /usr/lib -name "*.so.*" -exec cp {} /lib-copy \;
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 COPY --from=build_node_perf_npb_is /NPB3.3.1/NPB3.3-OMP/bin/is.D.x /
 COPY --from=build_node_perf_npb_is /lib-copy /lib-copy

test/images/node-perf/tf-wide-deep/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/nonewprivs/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/nonewprivs/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS = nnp
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/nonroot/Dockerfile

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 USER 1234

test/images/pets/peer-finder/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/pets/peer-finder/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS = peer-finder
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/pets/redis-installer/Dockerfile

@@ -14,7 +14,8 @@
 
 # TODO: get rid of bash dependency and switch to plain busybox.
 # The tar in busybox also doesn't seem to understand compression.
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/pets/redis-installer/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS = peer-finder
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/pets/zookeeper-installer/Dockerfile

@@ -14,7 +14,8 @@
 
 # TODO: get rid of bash dependency and switch to plain busybox.
 # The tar in busybox also doesn't seem to understand compression.
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/pets/zookeeper-installer/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS = peer-finder
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/redis/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/regression-issue-74839/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS=regression-issue-74839
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOARM ?= 7

test/images/resource-consumer/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/resource-consumer/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS = consumer consume-cpu/consume-cpu
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/sample-apiserver/Dockerfile

@@ -32,7 +32,8 @@ RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=BASEARCH go get k8s.io/sample
 # we can copy it out from this throw away container image from a standard location
 RUN find /go/bin -name sample-apiserver -exec cp {} / \;
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 COPY --from=build_k8s_1_17_sample_apiserver /sample-apiserver /sample-apiserver
 
 ENTRYPOINT ["/sample-apiserver"]

test/images/sample-device-plugin/Dockerfile

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 ADD sampledeviceplugin /sampledeviceplugin
 ENTRYPOINT ["/sampledeviceplugin", "-alsologtostderr"]

test/images/sample-device-plugin/Makefile

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 SRCS=sampledeviceplugin
+OS ?= linux
 ARCH ?= amd64
 TARGET ?= $(CURDIR)
 GOLANG_VERSION ?= latest

test/images/volume/gluster/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/volume/iscsi/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 

test/images/volume/nfs/Dockerfile

@@ -12,7 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/
 RUN yum -y install /usr/bin/ps nfs-utils && yum clean all

test/images/volume/rbd/Dockerfile

@@ -15,7 +15,8 @@
 # CEPH all in one
 # Based on image by Ricardo Rocha, ricardo@catalyst.net.nz
 
-FROM BASEIMAGE
+ARG BASEIMAGE
+FROM $BASEIMAGE
 
 CROSS_BUILD_COPY qemu-QEMUARCH-static /usr/bin/