diff --git a/examples/k8s_audit_config/README.md b/examples/k8s_audit_config/README.md index e024e66f..58156491 100644 --- a/examples/k8s_audit_config/README.md +++ b/examples/k8s_audit_config/README.md @@ -1,6 +1,8 @@ # Introduction -This page describes how to get K8s Audit Logging working with Falco. For now, we'll describe how to enable audit logging in k8s 1.11, where the audit configuration needs to be directly provided to the api server. In 1.13 there is a different mechanism that allows audit confguration to be managed like other k8s objects, but these instructions are for 1.11. +This page describes how to get K8s Audit Logging working with Falco for either K8s 1.11, using static audit policies/sinks, or 1.13, with dynamic audit policies/sinks using AuditSink objects. + +## K8s 1.11 Instructions The main steps are: @@ -9,11 +11,11 @@ The main steps are: 1. Restart the API Server to enable Audit Logging 1. Observe K8s audit events at falco -## Deploy Falco to your K8s cluster +### Deploy Falco to your K8s cluster Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset. -## Define your audit policy and webhook configuration +### Define your audit policy and webhook configuration The files in this directory can be used to configure k8s audit logging. The relevant files are: @@ -26,11 +28,11 @@ Run the following to fill in the template file with the ClusterIP ip address you FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < webhook-config.yaml.in > webhook-config.yaml ``` -## Restart the API Server to enable Audit Logging +### Restart the API Server to enable Audit Logging A script [enable-k8s-audit.sh](./enable-k8s-audit.sh) performs the necessary steps of enabling audit log support for the apiserver, including copying the audit policy/webhook files to the apiserver machine, modifying the apiserver command line to add `--audit-log-path`, `--audit-policy-file`, etc. arguments, etc. (For minikube, ideally you'd be able to pass all these options directly on the `minikube start` command line, but manual patching is necessary. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.) -It is run as `bash ./enable-k8s-audit.sh `. `` can be one of the following: +It is run as `bash ./enable-k8s-audit.sh static`. `` can be one of the following: * "minikube" * "kops" @@ -40,15 +42,65 @@ When running with variant="kops", you must either modify the script to specify t Its output looks like this: ``` -$ bash enable-k8s-audit.sh minikube +$ bash enable-k8s-audit.sh minikube static +***Copying apiserver config patch script to apiserver... +apiserver-config.patch.sh 100% 1190 1.2MB/s 00:00 ***Copying audit policy/webhook files to apiserver... audit-policy.yaml 100% 2519 1.2MB/s 00:00 webhook-config.yaml 100% 248 362.0KB/s 00:00 +***Modifying k8s apiserver config (will result in apiserver restarting)... +***Done! +$ +``` +### Observe K8s audit events at falco + +K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`. + +## K8s 1.13 Instructions + +The main steps are: + +1. Deploy Falco to your K8s cluster +1. Restart the API Server to enable Audit Logging +1. Deploy the AuditSink object for your audit policy and webhook configuration +1. Observe K8s audit events at falco + +### Deploy Falco to your K8s cluster + +Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset. + +### Restart the API Server to enable Audit Logging + +A script [enable-k8s-audit.sh](./enable-k8s-audit.sh) performs the necessary steps of enabling dynamic audit support for the apiserver by modifying the apiserver command line to add `--audit-dynamic-configuration`, `--feature-gates=DynamicAuditing=true`, etc. arguments, etc. (For minikube, ideally you'd be able to pass all these options directly on the `minikube start` command line, but manual patching is necessary. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.) + +It is run as `bash ./enable-k8s-audit.sh dynamic`. `` can be one of the following: + +* "minikube" +* "kops" + +When running with variant="kops", you must either modify the script to specify the kops apiserver hostname or set it via the environment: `APISERVER_HOST=api.my-kops-cluster.com bash ./enable-k8s-audit.sh kops` + +Its output looks like this: + +``` +$ bash enable-k8s-audit.sh minikube dynamic +***Copying apiserver config patch script to apiserver... apiserver-config.patch.sh 100% 1190 1.2MB/s 00:00 ***Modifying k8s apiserver config (will result in apiserver restarting)... ***Done! $ ``` -## Observe K8s audit events at falco + +### Deploy AuditSink objects + +[audit-sink.yaml.in](./audit-sink.yaml.in), in this directory, is a template audit sink configuration that defines the dynamic audit policy and webhook to route k8s audit events to Falco. + +Run the following to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable. + +``` +FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < audit-sink.yaml.in > audit-sink.yaml +``` + +### Observe K8s audit events at falco K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`. diff --git a/examples/k8s_audit_config/apiserver-config.patch.sh b/examples/k8s_audit_config/apiserver-config.patch.sh index 1011e4a5..82ad7500 100644 --- a/examples/k8s_audit_config/apiserver-config.patch.sh +++ b/examples/k8s_audit_config/apiserver-config.patch.sh @@ -4,10 +4,18 @@ IFS='' FILENAME=${1:-/etc/kubernetes/manifests/kube-apiserver.yaml} VARIANT=${2:-minikube} +AUDIT_TYPE=${3:-static} -if grep audit-webhook-config-file $FILENAME ; then - echo audit-webhook patch already applied - exit 0 +if [ $AUDIT_TYPE == "static" ]; then + if grep audit-webhook-config-file $FILENAME ; then + echo audit-webhook patch already applied + exit 0 + fi +else + if grep audit-dynamic-configuration $FILENAME ; then + echo audit-dynamic-configuration patch already applied + exit 0 + fi fi TMPFILE="/tmp/kube-apiserver.yaml.patched" @@ -26,19 +34,29 @@ do echo "$LINE" >> "$TMPFILE" case "$LINE" in *$APISERVER_LINE*) - echo "$APISERVER_PREFIX --audit-log-path=/var/lib/k8s_audit/audit.log" >> "$TMPFILE" - echo "$APISERVER_PREFIX --audit-policy-file=/var/lib/k8s_audit/audit-policy.yaml" >> "$TMPFILE" - echo "$APISERVER_PREFIX --audit-webhook-config-file=/var/lib/k8s_audit/webhook-config.yaml" >> "$TMPFILE" - echo "$APISERVER_PREFIX --audit-webhook-batch-max-wait=5s" >> "$TMPFILE" + if [ $AUDIT_TYPE == "static" ]; then + echo "$APISERVER_PREFIX --audit-log-path=/var/lib/k8s_audit/audit.log" >> "$TMPFILE" + echo "$APISERVER_PREFIX --audit-policy-file=/var/lib/k8s_audit/audit-policy.yaml" >> "$TMPFILE" + echo "$APISERVER_PREFIX --audit-webhook-config-file=/var/lib/k8s_audit/webhook-config.yaml" >> "$TMPFILE" + echo "$APISERVER_PREFIX --audit-webhook-batch-max-wait=5s" >> "$TMPFILE" + else + echo "$APISERVER_PREFIX --audit-dynamic-configuration" >> "$TMPFILE" + echo "$APISERVER_PREFIX --feature-gates=DynamicAuditing=true" >> "$TMPFILE" + echo "$APISERVER_PREFIX --runtime-config=auditregistration.k8s.io/v1alpha1=true" >> "$TMPFILE" + fi ;; *"volumeMounts:"*) - echo " - mountPath: /var/lib/k8s_audit/" >> "$TMPFILE" - echo " name: data" >> "$TMPFILE" + if [ $AUDIT_TYPE == "static" ]; then + echo " - mountPath: /var/lib/k8s_audit/" >> "$TMPFILE" + echo " name: data" >> "$TMPFILE" + fi ;; *"volumes:"*) - echo " - hostPath:" >> "$TMPFILE" - echo " path: /var/lib/k8s_audit" >> "$TMPFILE" - echo " name: data" >> "$TMPFILE" + if [ $AUDIT_TYPE == "static" ]; then + echo " - hostPath:" >> "$TMPFILE" + echo " path: /var/lib/k8s_audit" >> "$TMPFILE" + echo " name: data" >> "$TMPFILE" + fi ;; esac diff --git a/examples/k8s_audit_config/audit-sink.yaml.in b/examples/k8s_audit_config/audit-sink.yaml.in new file mode 100644 index 00000000..ee38ae0f --- /dev/null +++ b/examples/k8s_audit_config/audit-sink.yaml.in @@ -0,0 +1,15 @@ +apiVersion: auditregistration.k8s.io/v1alpha1 +kind: AuditSink +metadata: + name: falco-audit-sink +spec: + policy: + level: RequestResponse + stages: + - ResponseComplete + webhook: + throttle: + qps: 10 + burst: 15 + clientConfig: + url: "http://$FALCO_SERVICE_CLUSTERIP:8765/k8s_audit" diff --git a/examples/k8s_audit_config/enable-k8s-audit.sh b/examples/k8s_audit_config/enable-k8s-audit.sh index 6e210767..73b920c8 100644 --- a/examples/k8s_audit_config/enable-k8s-audit.sh +++ b/examples/k8s_audit_config/enable-k8s-audit.sh @@ -3,6 +3,7 @@ set -euo pipefail VARIANT=${1:-minikube} +AUDIT_TYPE=${2:-static} if [ $VARIANT == "minikube" ]; then APISERVER_HOST=$(minikube ip) @@ -23,14 +24,18 @@ if [ $VARIANT == "kops" ]; then fi fi -echo "***Copying audit policy/webhook files to apiserver..." +echo "***Copying apiserver config patch script to apiserver..." ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo mkdir -p /var/lib/k8s_audit && sudo chown $SSH_USER /var/lib/k8s_audit" -scp -i $SSH_KEY audit-policy.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit -scp -i $SSH_KEY webhook-config.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit scp -i $SSH_KEY apiserver-config.patch.sh $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit +if [ $AUDIT_TYPE == "static" ]; then + echo "***Copying audit policy/webhook files to apiserver..." + scp -i $SSH_KEY audit-policy.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit + scp -i $SSH_KEY webhook-config.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit +fi + echo "***Modifying k8s apiserver config (will result in apiserver restarting)..." -ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo bash /var/lib/k8s_audit/apiserver-config.patch.sh $MANIFEST $VARIANT" +ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo bash /var/lib/k8s_audit/apiserver-config.patch.sh $MANIFEST $VARIANT $AUDIT_TYPE" echo "***Done!"