Replace entrypoint script with initcontainers (#718)

* build: install the multus binary in an init container

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* build: generate kubeconfig via go

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* build: generate multus cni configuration via golang

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* build: provide a docker img for daemon based deployments

We will have 2 different images (only on amd64 archs):
- legacy entrypoint script based
- daemonized process

The `image-build` docker action is updated, to build these 2 images.

There will be 2 different deployment specs, along with e2e test
lanes, one for each of the aforementioned alternatives.

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* build: delegate CNI config watch loop via golang

For the thick-plugin alternative, provide the watch loop for
configuration regeneration via a golang binary.

Over time, this binary is expected to run the control loop to watch
out for pod updates.

To enable current multus users to chose when they upgrade to this new
deployment setup, these changes are provided in separate multus images,
having a different yaml spec files. Both of these alternatives are
tested e2e, since a new lane is introduced.

The following libraries are introduced, along with the motivation for
adding them:
- dproxy: allows traversing the default network configuration arbitrarily,
  similar to what an X path / JSON path tool provides.
  Repo is available at [0].
- fsnotify: watch for changes in the default CNI configuration file.
  Repo is available at [1].

The config map providing the default network CNI configuration is not
copied over, since originally, the user was not required to install a
default network CNI plugin first, but, nowadays, this is a required
step of multus.

As such, it is no longer required to provide a default CNI
configuration.

[0] - https://github.com/koron/go-dproxy
[1] - https://github.com/fsnotify/fsnotify

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* run gofmt

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* refactor: make the builder pattern more idiomatic to golang

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

* build: update github actions to release new imgs

Signed-off-by: Miguel Duarte Barroso <mdbarroso@redhat.com>

.github/workflows/image-build.yml

@@ -1,8 +1,26 @@
 name: Image build
 on: [pull_request]
 jobs:
+  ep-build-amd64:
+    name: Image build/amd64 LEGACY entrypoint
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+
+      - name: Build container image
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          push: false
+          tags: ghcr.io/${{ github.repository }}:ep-latest-amd64
+          file: images/Dockerfile
+
   build-amd64:
-    name: Image build/amd64
+    name: Image build/amd64 daemonized alternative
     runs-on: ubuntu-latest
     steps:
       - name: Check out code into the Go module directory
@@ -17,7 +35,7 @@ jobs:
           context: .
           push: false
           tags: ghcr.io/${{ github.repository }}:latest-amd64
-          file: images/Dockerfile
+          file: images/Dockerfile.thick
 
   build-arm64:
     name: Image build/arm64

.github/workflows/image-push-master.yml

@@ -33,6 +33,16 @@ jobs:
             ghcr.io/${{ github.repository }}:snapshot-amd64
           file: images/Dockerfile
 
+      - name: Push container image for daemon based deployment
+        if: github.repository_owner == 'k8snetworkplumbingwg'
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          push: true
+          tags: |
+            ghcr.io/${{ github.repository }}:thick-amd64
+          file: images/Dockerfile.thick
+
   push-arm64:
     name: Image push/arm64
     runs-on: ubuntu-latest
@@ -223,3 +233,8 @@ jobs:
           docker manifest annotate ${{ env.REPOSITORY }}:latest ${{ env.REPOSITORY }}:latest-ppc64le --arch ppc64le
           docker manifest annotate ${{ env.REPOSITORY }}:latest ${{ env.REPOSITORY }}:latest-s390x --arch s390x
           docker manifest push ${{ env.REPOSITORY }}:latest
+
+          docker pull ${{ env.REPOSITORY }}:thick-amd64
+          docker manifest create ${{ env.REPOSITORY }}:thick ${{ env.REPOSITORY }}:thick-amd64
+          docker manifest annotate ${{ env.REPOSITORY }}:thick ${{ env.REPOSITORY }}:thick-amd64 --arch amd64
+          docker manifest push ${{ env.REPOSITORY }}:thick

.github/workflows/image-push-release.yml

@@ -40,6 +40,17 @@ jobs:
             ${{ steps.docker_meta.outputs.tags }}-amd64
           file: images/Dockerfile
 
+      - name: Push container image for daemon based deployment
+        if: github.repository_owner == 'k8snetworkplumbingwg'
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          push: true
+          tags: |
+            ghcr.io/${{ github.repository }}:thick-amd64
+            ${{ steps.docker_meta.outputs.tags }}-thick-amd64
+          file: images/Dockerfile.thick
+
   push-arm64:
     name: Image push/arm64
     runs-on: ubuntu-latest

.github/workflows/kind-e2e.yml

@@ -14,7 +14,7 @@ jobs:
         run: docker run -d --restart=always -p "5000:5000" --name "kind-registry" registry:2
   
       - name: Build latest-amd64
-        run: docker build -t localhost:5000/multus:e2e -f images/Dockerfile .
+        run: docker build -t localhost:5000/multus:e2e -f images/Dockerfile.thick .
  
       - name: Push to local registry
         run: docker push localhost:5000/multus:e2e

.github/workflows/legacy-kind-e2e.yml

@@ -0,0 +1,46 @@
+name: e2e-kind legacy installation with entrypoint script
+on: [push, pull_request]
+jobs:
+  e2e-kind:
+    runs-on: ubuntu-latest
+    if: >
+      (( github.event.pull_request.head.repo.owner.login != github.event.pull_request.base.repo.owner.login ) &&
+        github.event_name == 'pull_request' ) || (github.event_name == 'push' && github.event.commits != '[]' )
+    steps:
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v2
+
+      - name: Setup registry
+        run: docker run -d --restart=always -p "5000:5000" --name "kind-registry" registry:2
+
+      - name: Build latest-amd64
+        run: docker build -t localhost:5000/multus:e2e -f images/Dockerfile .
+
+      - name: Push to local registry
+        run: docker push localhost:5000/multus:e2e
+
+      - name: Get kind/kubectl/koko
+        working-directory: ./e2e
+        run: ./get_tools.sh
+
+      - name: Setup cluster
+        working-directory: ./e2e
+        run: ./setup_cluster.sh
+
+      - name: Test simple pod
+        working-directory: ./e2e
+        run: ./test-simple-pod.sh
+
+      - name: Test macvlan1
+        working-directory: ./e2e
+        run: ./test-simple-macvlan1.sh
+
+      - name: Test default route1
+        working-directory: ./e2e
+        run: ./test-default-route1.sh
+
+      - name: cleanup cluster and registry
+        run: |
+          kind delete cluster
+          docker kill kind-registry
+          docker rm kind-registry