github/multus-cni
1
0
Fork 1
mirror of https://github.com/k8snetworkplumbingwg/multus-cni.git synced 2025-08-29 05:10:40 +00:00
Code Issues Projects Releases Wiki Activity

Allows specifying "global namespaces" when using namespace isolation

Browse Source
This commit is contained in:
dougbtv 2020-10-29 13:10:12 -04:00 committed by Tomofumi Hayashi
parent e924318e18
commit 476c738c13
8 changed files with 194 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
doc/configuration.md
View File

@ -119,6 +119,8 @@ The functionality provided by the `namespaceIsolation` configuration option enab
**NOTE**: The default namespace is special in this scenario. Even with namespace isolation enabled, any pod, in any namespace is allowed to refer to `NetworkAttachmentDefinitions` in the default namespace. This allows you to create commonly used unprivileged `NetworkAttachmentDefinitions` without having to put them in all namespaces. For example, if you had a `NetworkAttachmentDefinition` named `foo` the default namespace, you may reference it in an annotation with: `default/foo`.
**NOTE**: You can also add additional namespaces which can be referred to globally using the `global-namespaces` option (see next section).
For example, if a pod is created in the namespace called `development`, Multus will not allow networks to be attached when defined by custom resources created in a different namespace, say in the `default` network.
Consider the situation where you have a system that has users of different privilege levels -- as an example, a platform which has two administrators: a Senior Administrator and a Junior Administrator. The Senior Administrator may have access to all namespaces, and some network configurations as used by Multus are considered to be privileged in that they allow access to some protected resources available on the network. However, the Junior Administrator has access to only a subset of namespaces, and therefore it should be assumed that the Junior Administrator cannot create pods in their limited subset of namespaces. The `namespaceIsolation` feature provides for this isolation, allowing pods created in given namespaces to only access custom resources in the same namespace as the pod.
@ -215,7 +217,7 @@ pod/samplepod created
You'll note that pod fails to spawn successfully. If you check the Multus logs, you'll see an entry such as:
```
2018-12-18T21:41:32Z [error] GetPodNetwork: namespace isolation violation: podnamespace: development / target namespace: privileged
2018-12-18T21:41:32Z [error] GetNetworkDelegates: namespace isolation enabled, annotation violates permission, pod is in namespace development but refers to target namespace privileged
```
This error expresses that the pod resides in the namespace named `development` but refers to a `NetworkAttachmentDefinition` outside of that namespace, in this case, the namespace named `privileged`.
@ -253,6 +255,16 @@ NAME READY STATUS RESTARTS AGE
samplepod 1/1 Running 0 31s
```
### Allow specific namespaces to be used across namespaces when using namespace isolation
The `globalNamespaces` configuration option is only used when `namespaceIsolation` is set to true. `globalNamespaces` specifies a comma-delimited list of namespaces which can be referred to from outside of any given namespace in which a pod resides.
```
"globalNamespaces": "default,namespace-a,namespace-b",
```
Note that when using `globalNamespaces` the `default` namespace must be specified in the list if you wish to use that namespace, when `globalNamespaces` is not set, the `default` namespace is implied to be used across namespaces.
### Specify default cluster network in Pod annotations
Users may also specify the default network for any given pod (via annotation), for cases where there are multiple cluster networks available within a Kubernetes cluster.

6
doc/how-to-use.md
View File

@ -567,7 +567,11 @@ This the directory in which the Multus binary will be installed.
--namespace-isolation=false
Setting this option to true enables the Namespace isolation feature, which insists that custom resources must be created in the same namespace as the pods, otherwise it will refuse to attach those definitions as additional interfaces.
Setting this option to true enables the Namespace isolation feature, which insists that custom resources must be created in the same namespace as the pods, otherwise it will refuse to attach those definitions as additional interfaces. See (the configuration guide for more information)[configuration.md].
--global-namespaces=default,foo,bar
The `--global-namespaces` works only when `--namespace-isolation=true`. This takes a comma-separated list of namespaces which can be referred to globally when namespace isolation is enabled. See (the configuration guide for more information)[configuration.md].
--multus-bin-file=/usr/src/multus-cni/bin/multus

11
images/entrypoint.sh
View File

@ -12,6 +12,7 @@ MULTUS_AUTOCONF_DIR="/host/etc/cni/net.d"
MULTUS_BIN_FILE="/usr/src/multus-cni/bin/multus"
MULTUS_KUBECONFIG_FILE_HOST="/etc/cni/net.d/multus.d/multus.kubeconfig"
MULTUS_NAMESPACE_ISOLATION=false
MULTUS_GLOBAL_NAMESPACES=""
MULTUS_LOG_LEVEL=""
MULTUS_LOG_FILE=""
MULTUS_READINESS_INDICATOR_FILE=""
@ -42,6 +43,7 @@ function usage()
echo -e "\t--skip-multus-binary-copy=$SKIP_BINARY_COPY"
echo -e "\t--multus-kubeconfig-file-host=$MULTUS_KUBECONFIG_FILE_HOST"
echo -e "\t--namespace-isolation=$MULTUS_NAMESPACE_ISOLATION"
echo -e "\t--global-namespaces=$MULTUS_GLOBAL_NAMESPACES (used only with --namespace-isolation=true)"
echo -e "\t--multus-autoconfig-dir=$MULTUS_AUTOCONF_DIR (used only with --multus-conf-file=auto)"
echo -e "\t--multus-log-level=$MULTUS_LOG_LEVEL (empty by default, used only with --multus-conf-file=auto)"
echo -e "\t--multus-log-file=$MULTUS_LOG_FILE (empty by default, used only with --multus-conf-file=auto)"
@ -98,6 +100,9 @@ while [ "$1" != "" ]; do
--namespace-isolation)
MULTUS_NAMESPACE_ISOLATION=$VALUE
;;
--global-namespaces)
MULTUS_GLOBAL_NAMESPACES=$VALUE
;;
--multus-log-level)
MULTUS_LOG_LEVEL=$VALUE
;;
@ -255,6 +260,11 @@ if [ "$MULTUS_CONF_FILE" == "auto" ]; then
ISOLATION_STRING="\"namespaceIsolation\": true,"
fi
GLOBAL_NAMESPACES_STRING=""
if [ ! -z "${MULTUS_GLOBAL_NAMESPACES// }" ]; then
GLOBAL_NAMESPACES_STRING="\"globalNamespaces\": \"$MULTUS_GLOBAL_NAMESPACES\","
fi
LOG_LEVEL_STRING=""
if [ ! -z "${MULTUS_LOG_LEVEL// }" ]; then
case "$MULTUS_LOG_LEVEL" in
@ -330,6 +340,7 @@ EOF
"type": "multus",
$NESTED_CAPABILITIES_STRING
$ISOLATION_STRING
$GLOBAL_NAMESPACES_STRING
$LOG_LEVEL_STRING
$LOG_FILE_STRING
$ADDITIONAL_BIN_DIR_STRING

24
k8sclient/k8sclient.go
View File

@ -337,7 +337,7 @@ func TryLoadPodDelegates(pod *v1.Pod, conf *types.NetConf, clientInfo *ClientInf
networks, err := GetPodNetwork(pod)
if networks != nil {
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, conf.ConfDir, conf.NamespaceIsolation, resourceMap)
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, conf, resourceMap)
if err != nil {
if _, ok := err.(*NoK8sNetworkError); ok {
@ -449,8 +449,9 @@ func GetPodNetwork(pod *v1.Pod) ([]*types.NetworkSelectionElement, error) {
}
// GetNetworkDelegates returns delegatenetconf from net-attach-def annotation in pod
func GetNetworkDelegates(k8sclient *ClientInfo, pod *v1.Pod, networks []*types.NetworkSelectionElement, confdir string, confnamespaceIsolation bool, resourceMap map[string]*types.ResourceInfo) ([]*types.DelegateNetConf, error) {
logging.Debugf("GetNetworkDelegates: %v, %v, %v, %v, %v, %v", k8sclient, pod, networks, confdir, confnamespaceIsolation, resourceMap)
func GetNetworkDelegates(k8sclient *ClientInfo, pod *v1.Pod, networks []*types.NetworkSelectionElement, conf *types.NetConf, resourceMap map[string]*types.ResourceInfo) ([]*types.DelegateNetConf, error) {
logging.Debugf("GetNetworkDelegates: %v, %v, %v, %v, %v", k8sclient, pod, networks, conf, resourceMap)
// Read all network objects referenced by 'networks'
var delegates []*types.DelegateNetConf
defaultNamespace := pod.ObjectMeta.Namespace
@ -459,16 +460,16 @@ func GetNetworkDelegates(k8sclient *ClientInfo, pod *v1.Pod, networks []*types.N
// The pods namespace (stored as defaultNamespace, does not equal the annotation's target namespace in net.Namespace)
// In the case that this is a mismatch when namespaceisolation is enabled, this should be an error.
if confnamespaceIsolation {
if conf.NamespaceIsolation {
if defaultNamespace != net.Namespace {
// There is an exception however, we always allow a reference to the default namespace.
if net.Namespace != "default" {
// We allow exceptions based on the specified list of non-isolated namespaces (and/or "default" namespace, by default)
if !isValidNamespaceReference(net.Namespace, conf.NonIsolatedNamespaces) {
return nil, logging.Errorf("GetNetworkDelegates: namespace isolation enabled, annotation violates permission, pod is in namespace %v but refers to target namespace %v", defaultNamespace, net.Namespace)
}
}
}
delegate, updatedResourceMap, err := getKubernetesDelegate(k8sclient, net, confdir, pod, resourceMap)
delegate, updatedResourceMap, err := getKubernetesDelegate(k8sclient, net, conf.ConfDir, pod, resourceMap)
if err != nil {
return nil, logging.Errorf("GetNetworkDelegates: failed getting the delegate: %v", err)
}
@ -479,6 +480,15 @@ func GetNetworkDelegates(k8sclient *ClientInfo, pod *v1.Pod, networks []*types.N
return delegates, nil
}
func isValidNamespaceReference(targetns string, allowednamespaces []string) bool {
for _, eachns := range allowednamespaces {
if eachns == targetns {
return true
}
}
return false
}
func getNetDelegate(client *ClientInfo, pod *v1.Pod, netname, confdir, namespace string, resourceMap map[string]*types.ResourceInfo) (*types.DelegateNetConf, map[string]*types.ResourceInfo, error) {
logging.Debugf("getNetDelegate: %v, %v, %v, %s", client, netname, confdir, namespace)
// option1) search CRD object for the network

98
k8sclient/k8sclient_test.go
View File

@ -54,10 +54,21 @@ func NewFakeClientInfo() *ClientInfo {
var _ = Describe("k8sclient operations", func() {
var tmpDir string
var err error
var genericConf string
BeforeEach(func() {
tmpDir, err = ioutil.TempDir("", "multus_tmp")
Expect(err).NotTo(HaveOccurred())
genericConf = `{
"name":"node-cni-network",
"type":"multus",
"delegates": [{
"name": "weave1",
"cniVersion": "0.2.0",
"type": "weave-net"
}],
"kubeconfig":"/etc/kubernetes/node-kubeconfig.yaml"
}`
})
AfterEach(func() {
@ -104,7 +115,9 @@ var _ = Describe("k8sclient operations", func() {
Expect(err).NotTo(HaveOccurred())
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
Expect(len(delegates)).To(Equal(2))
@ -139,7 +152,9 @@ var _ = Describe("k8sclient operations", func() {
Expect(err).NotTo(HaveOccurred())
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(len(delegates)).To(Equal(0))
Expect(err).To(MatchError("GetNetworkDelegates: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (net1) in namespace (test): network-attachment-definitions.k8s.cni.cncf.io \"net1\" not found"))
})
@ -188,7 +203,9 @@ var _ = Describe("k8sclient operations", func() {
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
Expect(len(delegates)).To(Equal(3))
@ -262,7 +279,9 @@ var _ = Describe("k8sclient operations", func() {
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
Expect(len(delegates)).To(Equal(3))
@ -306,7 +325,9 @@ var _ = Describe("k8sclient operations", func() {
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
Expect(len(delegates)).To(Equal(2))
@ -333,7 +354,9 @@ var _ = Describe("k8sclient operations", func() {
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
Expect(len(delegates)).To(Equal(1))
@ -368,7 +391,9 @@ var _ = Describe("k8sclient operations", func() {
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
delegates, err := GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(len(delegates)).To(Equal(0))
Expect(err).To(MatchError(fmt.Sprintf("GetNetworkDelegates: failed getting the delegate: GetCNIConfig: err in GetCNIConfigFromFile: Error loading CNI config file %s: error parsing configuration: invalid character 'a' looking for beginning of value", net2Name)))
})
@ -842,7 +867,6 @@ users:
"namespaceIsolation": true
}`
netConf, err := types.LoadNetConf([]byte(conf))
Expect(err).NotTo(HaveOccurred())
net1 := `{
@ -867,12 +891,64 @@ users:
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
_, err = GetNetworkDelegates(clientInfo, pod, networks, tmpDir, netConf.NamespaceIsolation, nil)
netConf, err := types.LoadNetConf([]byte(conf))
netConf.ConfDir = tmpDir
_, err = GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).To(HaveOccurred())
Expect(err).To(MatchError("GetNetworkDelegates: namespace isolation enabled, annotation violates permission, pod is in namespace test but refers to target namespace kube-system"))
})
It("Properly allows a specified namespace reference when namespace isolation is enabled", func() {
fakePod := testutils.NewFakePod("testpod", "kube-system/net1", "")
conf := `{
"name":"node-cni-network",
"type":"multus",
"delegates": [{
"name": "weave1",
"cniVersion": "0.2.0",
"type": "weave-net"
}],
"kubeconfig":"/etc/kubernetes/node-kubeconfig.yaml",
"namespaceIsolation": true,
"globalNamespaces": "kube-system,donkey-kong"
}`
Expect(err).NotTo(HaveOccurred())
net1 := `{
"name": "net1",
"type": "mynet",
"cniVersion": "0.2.0"
}`
args := &skel.CmdArgs{
Args: fmt.Sprintf("K8S_POD_NAME=%s;K8S_POD_NAMESPACE=%s", fakePod.ObjectMeta.Name, fakePod.ObjectMeta.Namespace),
}
clientInfo := NewFakeClientInfo()
_, err = clientInfo.AddPod(fakePod)
Expect(err).NotTo(HaveOccurred())
_, err = clientInfo.AddNetAttachDef(testutils.NewFakeNetAttachDef("kube-system", "net1", net1))
Expect(err).NotTo(HaveOccurred())
k8sArgs, err := GetK8sArgs(args)
Expect(err).NotTo(HaveOccurred())
pod, err := clientInfo.GetPod(string(k8sArgs.K8S_POD_NAMESPACE), string(k8sArgs.K8S_POD_NAME))
networks, err := GetPodNetwork(pod)
Expect(err).NotTo(HaveOccurred())
netConf, err := types.LoadNetConf([]byte(conf))
netConf.ConfDir = tmpDir
_, err = GetNetworkDelegates(clientInfo, pod, networks, netConf, nil)
Expect(err).NotTo(HaveOccurred())
})
Context("Error function", func() {
It("Returns proper error message", func() {
err := &NoK8sNetworkError{"no kubernetes network found"}
@ -966,7 +1042,9 @@ users:
networks, err := GetPodNetwork(fakePod)
Expect(err).NotTo(HaveOccurred())
_, err = GetNetworkDelegates(clientInfo, fakePod, networks, tmpDir, false, nil)
netConf, err := types.LoadNetConf([]byte(genericConf))
netConf.ConfDir = tmpDir
_, err = GetNetworkDelegates(clientInfo, fakePod, networks, netConf, nil)
Expect(err).To(HaveOccurred())
})
})

15
types/conf.go
View File

@ -19,6 +19,7 @@ import (
"encoding/json"
"fmt"
"net"
"strings"
"github.com/containernetworking/cni/libcni"
"github.com/containernetworking/cni/pkg/skel"
@ -34,6 +35,7 @@ const (
defaultBinDir = "/opt/cni/bin"
defaultReadinessIndicatorFile = ""
defaultMultusNamespace = "kube-system"
defaultNonIsolatedNamespace = "default"
)
// LoadDelegateNetConfList reads DelegateNetConf from bytes
@ -315,6 +317,19 @@ func LoadNetConf(bytes []byte) (*NetConf, error) {
netconf.MultusNamespace = defaultMultusNamespace
}
// setup namespace isolation
if netconf.RawNonIsolatedNamespaces == "" {
netconf.NonIsolatedNamespaces = []string{defaultNonIsolatedNamespace}
} else {
// Parse the comma separated list
nonisolated := strings.Split(netconf.RawNonIsolatedNamespaces, ",")
// Cleanup the whitespace
for i, nonv := range nonisolated {
nonisolated[i] = strings.TrimSpace(nonv)
}
netconf.NonIsolatedNamespaces = nonisolated
}
// get RawDelegates and put delegates field
if netconf.ClusterNetwork == "" {
// for Delegates

41
types/conf_test.go
View File

@ -135,6 +135,47 @@ var _ = Describe("config operations", func() {
Expect(netConf.LogFile).To(Equal("/var/log/multus.log"))
})
It("properly sets namespace isolation using the default namespace", func() {
conf := `{
"name": "node-cni-network",
"type": "multus",
"logLevel": "debug",
"logFile": "/var/log/multus.log",
"kubeconfig": "/etc/kubernetes/node-kubeconfig.yaml",
"namespaceIsolation": true,
"delegates": [{
"type": "weave-net"
}]
}`
netConf, err := LoadNetConf([]byte(conf))
Expect(err).NotTo(HaveOccurred())
Expect(netConf.NamespaceIsolation).To(Equal(true))
Expect(len(netConf.NonIsolatedNamespaces)).To(Equal(1))
Expect(netConf.NonIsolatedNamespaces[0]).To(Equal("default"))
})
It("properly sets namespace isolation using custom namespaces", func() {
conf := `{
"name": "node-cni-network",
"type": "multus",
"logLevel": "debug",
"logFile": "/var/log/multus.log",
"kubeconfig": "/etc/kubernetes/node-kubeconfig.yaml",
"namespaceIsolation": true,
"globalNamespaces": " foo,bar ,default",
"delegates": [{
"type": "weave-net"
}]
}`
netConf, err := LoadNetConf([]byte(conf))
Expect(err).NotTo(HaveOccurred())
Expect(netConf.NamespaceIsolation).To(Equal(true))
Expect(len(netConf.NonIsolatedNamespaces)).To(Equal(3))
Expect(netConf.NonIsolatedNamespaces[0]).To(Equal("foo"))
Expect(netConf.NonIsolatedNamespaces[1]).To(Equal("bar"))
Expect(netConf.NonIsolatedNamespaces[2]).To(Equal("default"))
})
It("prevResult with no errors", func() {
conf := `{
"name": "node-cni-network",

5
types/types.go
View File

@ -47,7 +47,10 @@ type NetConf struct {
// Default network readiness options
ReadinessIndicatorFile string `json:"readinessindicatorfile"`
// Option to isolate the usage of CR's to the namespace in which a pod resides.
NamespaceIsolation bool `json:"namespaceIsolation"`
NamespaceIsolation bool `json:"namespaceIsolation"`
RawNonIsolatedNamespaces string `json:"globalNamespaces"`
NonIsolatedNamespaces []string `json:"-"`
// Option to set system namespaces (to avoid to add defaultNetworks)
SystemNamespaces []string `json:"systemNamespaces"`
// Option to set the namespace that multus-cni uses (clusterNetwork/defaultNetworks)