Add ARM64 and cross-compilation support to the agent image (#659)

* modified Dockerfile to work for both amd64 (Intel) and arm64 (M1)

* added changelog

* Update `Dockerfile` to have `ARCH` build argument

* Remove `docs/CHANGES.md`

* Upgrade the Basenine version from `v0.3.0` to `v0.4.6`

* Update `publish.yml` to have `ARCH` build argument

* Switch `BasenineImageRepo` to Docker Hub

* Have separate build arguments for `ARCH` and `GOARCH`

* Upgrade the Basenine version from `v0.4.6` to `v0.4.10`

* Oops forgot to update the 10th duplicated shell script

* Fix the oopsie and reduce duplications

* Fix `Dockerfile`

* Fix the incompatibility issue between Go plugins and gold linker in Alpine inside `Dockerfile`

* Fix `asm: xxhash_amd64.s:120: when dynamic linking, R15 is clobbered by a global variable access` error

* Update `Dockerfile` to have cross-compilation on an AMD64 machine

Also revert changes in the shell scripts

* Delete `debug.Dockerfile`

* Create a custom base (`debian:buster-slim` based) image for the shipped image

* Replace `mertyildiran/debian-pcap` with `up9inc/debian-pcap`

* Upgrade Basenine version to `v0.4.12`

* Use `debian:stable-slim` as the base

* Fix the indentation in the `Dockerfile`

* Update `publish.yml`

* Enable `publish.yml` for `feature/multiarch_build` branch

* Tag correctly and set `ARCH` Docker argument

* Remove the lines that are forgotten to be removed from the shell scripts

* Add `MizuAgentImageRepo` constant and use it as default `AgentImage` value

* Bring back `Set up Cloud SDK` step to `Build the CLI and publish` job

* Build ARM64 CLI for Linux as well

* Revert "Enable `publish.yml` for `feature/multiarch_build` branch"

This reverts commit d30be4c1f0.

* Revert Go 1.17 upgrade

* Remove `build_extensions_debug.sh` as well

* Make the `Dockerfile` to compile the agent statically

* Statically link the protocol extensions

* Fix `Dockerfile`

* Bring back `-s -w` flags

* Verify the signatures of the downloads in `dockcross/linux-arm64-musl`

* Revert modifications in some shell scripts

* Make the `BUILDARCH` and `TARGETARCH` separation in the `Dockerfile`

* Separate cross-compilation builder image into a separate repo named `up9inc/linux-arm64-musl-go-libpcap`

* Fill the shell script and specify the tag for `dockcross/linux-arm64-musl`

* Remove the unnecessary dependencies from `builder-native-base`

* Improve the comments in the `Dockerfile`

* Upgrade Basenine version to `v0.4.13`

* Fix `Dockerfile`

* Revert "Revert "Enable `publish.yml` for `feature/multiarch_build` branch""

This reverts commit 303e466bdc.

* Revert "Revert "Revert "Enable `publish.yml` for `feature/multiarch_build` branch"""

This reverts commit 0fe252bbdb.

* Remove `push-docker-debug` from the `Makefile`

* Rename `publish.yml` to `release.yml`

Co-authored-by: Alex Haiut <alex@up9.com>

devops/build-push-featurebranch-debug.sh

@@ -1,28 +0,0 @@
-#!/bin/bash
-set -e
-
-GCP_PROJECT=up9-docker-hub
-REPOSITORY=gcr.io/$GCP_PROJECT
-SERVER_NAME=mizu
-GIT_BRANCH=$(git branch | grep \* | cut -d ' ' -f2 | tr '[:upper:]' '[:lower:]')
-
-DOCKER_REPO=$REPOSITORY/$SERVER_NAME/$GIT_BRANCH
-SEM_VER=${SEM_VER=0.0.0}
-
-DOCKER_TAGGED_BUILDS=("$DOCKER_REPO:latest" "$DOCKER_REPO:$SEM_VER")
-
-if [ "$GIT_BRANCH" = 'develop' -o "$GIT_BRANCH" = 'master' -o "$GIT_BRANCH" = 'main' ]
-then
-  echo "Pushing to $GIT_BRANCH is allowed only via CI"
-  exit 1
-fi
-
-echo "building ${DOCKER_TAGGED_BUILDS[@]}"
-DOCKER_TAGS_ARGS=$(echo ${DOCKER_TAGGED_BUILDS[@]/#/-t }) # "-t FIRST_TAG -t SECOND_TAG ..."
-docker build -f debug.Dockerfile $DOCKER_TAGS_ARGS --build-arg SEM_VER=${SEM_VER} --build-arg BUILD_TIMESTAMP=${BUILD_TIMESTAMP} --build-arg GIT_BRANCH=${GIT_BRANCH} --build-arg COMMIT_HASH=${COMMIT_HASH} .
-
-for DOCKER_TAG in "${DOCKER_TAGGED_BUILDS[@]}"
-do
-  echo pushing "$DOCKER_TAG"
-  docker push "$DOCKER_TAG"
-done

devops/build_extensions.sh

@@ -1,13 +0,0 @@
-#!/bin/bash
-
-for f in tap/extensions/*; do
-    if [ -d "$f" ]; then
-        echo Building extension: $f
-        extension=$(basename $f) && \
-        cd tap/extensions/${extension} && \
-        go build -buildmode=plugin -o ../${extension}.so .  && \
-        cd ../../..  && \
-        mkdir -p agent/build/extensions  && \
-        cp tap/extensions/${extension}.so agent/build/extensions
-    fi
-done

devops/build_extensions_debug.sh

@@ -1,12 +0,0 @@
-#!/bin/bash
-
-for f in tap/extensions/*; do
-    if [ -d "$f" ]; then
-        extension=$(basename $f) && \
-        cd tap/extensions/${extension} && \
-        go build -gcflags="all=-N -l" -buildmode=plugin -o ../${extension}.so .  && \
-        cd ../../..  && \
-        mkdir -p agent/build/extensions  && \
-        cp tap/extensions/${extension}.so agent/build/extensions
-    fi
-done

devops/linux-arm64-musl-go-libpcap/Dockerfile

@@ -0,0 +1,18 @@
+FROM dockcross/linux-arm64-musl:latest AS builder-from-amd64-to-arm64v8
+
+# Install Go
+RUN curl https://go.dev/dl/go1.16.13.linux-amd64.tar.gz -Lo ./go.linux-amd64.tar.gz
+RUN curl https://go.dev/dl/go1.16.13.linux-amd64.tar.gz.asc -Lo ./go.linux-amd64.tar.gz.asc
+RUN curl https://dl.google.com/dl/linux/linux_signing_key.pub  -Lo linux_signing_key.pub
+RUN gpg --import linux_signing_key.pub && gpg --verify ./go.linux-amd64.tar.gz.asc ./go.linux-amd64.tar.gz
+RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go.linux-amd64.tar.gz
+ENV PATH "$PATH:/usr/local/go/bin"
+
+# Compile libpcap from source
+RUN curl https://www.tcpdump.org/release/libpcap-1.10.1.tar.gz -Lo ./libpcap.tar.gz
+RUN curl https://www.tcpdump.org/release/libpcap-1.10.1.tar.gz.sig -Lo ./libpcap.tar.gz.sig
+RUN curl https://www.tcpdump.org/release/signing-key.asc -Lo ./signing-key.asc
+RUN gpg --import signing-key.asc && gpg --verify libpcap.tar.gz.sig libpcap.tar.gz
+RUN tar -xzf libpcap.tar.gz && mv ./libpcap-* ./libpcap
+RUN cd ./libpcap && ./configure --host=arm && make
+RUN cp /work/libpcap/libpcap.a /usr/xcc/aarch64-linux-musl-cross/lib/gcc/aarch64-linux-musl/*/

devops/linux-arm64-musl-go-libpcap/build-push.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+set -e
+
+docker build . -t up9inc/linux-arm64-musl-go-libpcap && docker push up9inc/linux-arm64-musl-go-libpcap