Move chroot from multus main process to its child processes (#1161)

We used to run chroot in multus main process when calling other CNI
plugin binary. We also use a mutex to lock the access to pod files.
But this causes performance issues when facing heavy
CNI_ADD/CNI_DEL requests.

With this patch, we do chroot in the child processes instead. So
file operations in the main process will not be affected by chroot.

This change requires the multus thick plugin pod to mount CNI bin
directory to the same path in the container host.

Signed-off-by: Peng Liu <pliu@redhat.com>

deployments/multus-daemonset-thick.yml

@@ -167,6 +167,11 @@ spec:
           volumeMounts:
             - name: cni
               mountPath: /host/etc/cni/net.d
+            # multus-daemon expects that cni-bin path must be identical between pod and container host.
+            # e.g. if the cni bin is in '/opt/cni/bin' on the container host side, then it should be mount to '/opt/cni/bin' in multus-daemon,
+            # not to any other directory, like '/opt/bin' or '/usr/bin'.
+            - name: cni-bin
+              mountPath: /opt/cni/bin
             - name: host-run
               mountPath: /host/run
             - name: host-var-lib-cni-multus