mirror of
https://github.com/falcosecurity/falco.git
synced 2025-06-28 15:47:25 +00:00
Add k8s audit instructions for kops (#535)
Also reorder the steps so falco is fully deployed before modifying the apiserver configuration.
This commit is contained in:
Mark Stemm
GitHub
parent
79d408f472
commit
c1b7356ac3
@ -1,38 +1,54 @@
|
||||
# Introduction
|
||||
|
||||
The files in this directory can be used to configure k8s audit logging. The relevant files are:
|
||||
This page describes how to get K8s Audit Logging working with Falco. For now, we'll describe how to enable audit logging in k8s 1.11, where the audit configuration needs to be directly provided to the api server. In 1.13 there is a different mechanism that allows audit confguration to be managed like other k8s objects, but these instructions are for 1.11.
|
||||
|
||||
* [audit-policy.yaml](./audit-policy.yaml): The k8s audit log configuration we used to create the rules in [k8s_audit_rules.yaml](../../rules/k8s_audit_rules.yaml). You may find it useful as a reference when creating your own K8s Audit Log configuration.
|
||||
* [webhook-config.yaml.in](./webhook-config.yaml.in): A (templated) webhook configuration that sends audit events to an ip associated with the falco service, port 8765. It is templated in that the *actual* ip is defined in an environment variable `FALCO_SERVICE_CLUSTERIP`, which can be plugged in using a program like `envsubst`. You may find it useful as a starting point when deciding how to route audit events to the embedded webserver within falco.
|
||||
The main steps are:
|
||||
|
||||
These files are only needed when using Minikube, which doesn't currently
|
||||
have the ability to provide an audit config/webhook config directly
|
||||
from the minikube commandline. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.
|
||||
1. Deploy Falco to your K8s cluster
|
||||
1. Define your audit policy and webhook configuration
|
||||
1. Restart the API Server to enable Audit Logging
|
||||
1. Observe K8s audit events at falco
|
||||
|
||||
* [apiserver-config.patch.sh](./apiserver-config.patch.sh): A script that changes the configuration file `/etc/kubernetes/manifests/kube-apiserver.yaml` to add necessary config options and mounts for the kube-apiserver container that runs within the minikube vm.
|
||||
|
||||
A way to use these files with minikube to run falco and enable audit logging would be the following:
|
||||
|
||||
#### Start Minikube with Audit Logging Enabled
|
||||
|
||||
Run the following to start minikube with Audit Logging Enabled:
|
||||
|
||||
```
|
||||
minikube start --kubernetes-version v1.11.0 --mount --mount-string $PWD:/tmp/k8s_audit_config --feature-gates AdvancedAuditing=true
|
||||
```
|
||||
|
||||
#### Create a Falco DaemonSet and Supporting Accounts/Services
|
||||
## Deploy Falco to your K8s cluster
|
||||
|
||||
Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset.
|
||||
|
||||
#### Configure Audit Logging with a Policy and Webhook
|
||||
## Define your audit policy and webhook configuration
|
||||
|
||||
Run the following commands to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above, and configure audit logging to use a policy and webhook that directs the right events to the falco daemonset. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable.
|
||||
The files in this directory can be used to configure k8s audit logging. The relevant files are:
|
||||
|
||||
* [audit-policy.yaml](./audit-policy.yaml): The k8s audit log configuration we used to create the rules in [k8s_audit_rules.yaml](../../rules/k8s_audit_rules.yaml).
|
||||
* [webhook-config.yaml.in](./webhook-config.yaml.in): A (templated) webhook configuration that sends audit events to an ip associated with the falco service, port 8765. It is templated in that the *actual* ip is defined in an environment variable `FALCO_SERVICE_CLUSTERIP`, which can be plugged in using a program like `envsubst`.
|
||||
|
||||
Run the following to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable.
|
||||
|
||||
```
|
||||
FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < webhook-config.yaml.in > webhook-config.yaml
|
||||
minikube ssh sudo bash /tmp/k8s_audit_config/apiserver-config.patch.sh
|
||||
```
|
||||
|
||||
K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`.
|
||||
## Restart the API Server to enable Audit Logging
|
||||
|
||||
A script [enable-k8s-audit.sh](./enable-k8s-audit.sh) performs the necessary steps of enabling audit log support for the apiserver, including copying the audit policy/webhook files to the apiserver machine, modifying the apiserver command line to add `--audit-log-path`, `--audit-policy-file`, etc. arguments, etc. (For minikube, ideally you'd be able to pass all these options directly on the `minikube start` command line, but manual patching is necessary. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.)
|
||||
|
||||
It is run as `bash ./enable-k8s-audit.sh <variant>`. `<variant>` can be one of the following:
|
||||
|
||||
* "minikube"
|
||||
* "kops"
|
||||
|
||||
When running with variant="kops", you must either modify the script to specify the kops apiserver hostname or set it via the environment: `APISERVER_HOST=api.my-kops-cluster.com bash ./enable-k8s-audit.sh kops`
|
||||
|
||||
Its output looks like this:
|
||||
|
||||
```
|
||||
$ bash enable-k8s-audit.sh minikube
|
||||
***Copying audit policy/webhook files to apiserver...
|
||||
audit-policy.yaml 100% 2519 1.2MB/s 00:00
|
||||
webhook-config.yaml 100% 248 362.0KB/s 00:00
|
||||
apiserver-config.patch.sh 100% 1190 1.2MB/s 00:00
|
||||
***Modifying k8s apiserver config (will result in apiserver restarting)...
|
||||
***Done!
|
||||
$
|
||||
```
|
||||
## Observe K8s audit events at falco
|
||||
|
||||
K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`.
|
||||
|
||||
@ -2,7 +2,8 @@
|
||||
|
||||
IFS=''
|
||||
|
||||
FILENAME="/etc/kubernetes/manifests/kube-apiserver.yaml"
|
||||
FILENAME=${1:-/etc/kubernetes/manifests/kube-apiserver.yaml}
|
||||
VARIANT=${2:-minikube}
|
||||
|
||||
if grep audit-webhook-config-file $FILENAME ; then
|
||||
echo audit-webhook patch already applied
|
||||
@ -12,23 +13,31 @@ fi
|
||||
TMPFILE="/tmp/kube-apiserver.yaml.patched"
|
||||
rm -f "$TMPFILE"
|
||||
|
||||
APISERVER_PREFIX=" -"
|
||||
APISERVER_LINE="- kube-apiserver"
|
||||
|
||||
if [ $VARIANT == "kops" ]; then
|
||||
APISERVER_PREFIX=" "
|
||||
APISERVER_LINE="/usr/local/bin/kube-apiserver"
|
||||
fi
|
||||
|
||||
while read LINE
|
||||
do
|
||||
echo "$LINE" >> "$TMPFILE"
|
||||
case "$LINE" in
|
||||
*"- kube-apiserver"*)
|
||||
echo " - --audit-log-path=/tmp/k8s_audit_config/audit.log" >> "$TMPFILE"
|
||||
echo " - --audit-policy-file=/tmp/k8s_audit_config/audit-policy.yaml" >> "$TMPFILE"
|
||||
echo " - --audit-webhook-config-file=/tmp/k8s_audit_config/webhook-config.yaml" >> "$TMPFILE"
|
||||
echo " - --audit-webhook-batch-max-wait=5s" >> "$TMPFILE"
|
||||
*$APISERVER_LINE*)
|
||||
echo "$APISERVER_PREFIX --audit-log-path=/var/lib/k8s_audit/audit.log" >> "$TMPFILE"
|
||||
echo "$APISERVER_PREFIX --audit-policy-file=/var/lib/k8s_audit/audit-policy.yaml" >> "$TMPFILE"
|
||||
echo "$APISERVER_PREFIX --audit-webhook-config-file=/var/lib/k8s_audit/webhook-config.yaml" >> "$TMPFILE"
|
||||
echo "$APISERVER_PREFIX --audit-webhook-batch-max-wait=5s" >> "$TMPFILE"
|
||||
;;
|
||||
*"volumeMounts:"*)
|
||||
echo " - mountPath: /tmp/k8s_audit_config/" >> "$TMPFILE"
|
||||
echo " - mountPath: /var/lib/k8s_audit/" >> "$TMPFILE"
|
||||
echo " name: data" >> "$TMPFILE"
|
||||
;;
|
||||
*"volumes:"*)
|
||||
echo " - hostPath:" >> "$TMPFILE"
|
||||
echo " path: /tmp/k8s_audit_config" >> "$TMPFILE"
|
||||
echo " path: /var/lib/k8s_audit" >> "$TMPFILE"
|
||||
echo " name: data" >> "$TMPFILE"
|
||||
;;
|
||||
|
||||
|
||||
36
examples/k8s_audit_config/enable-k8s-audit.sh
Normal file
36
examples/k8s_audit_config/enable-k8s-audit.sh
Normal file
@ -0,0 +1,36 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VARIANT=${1:-minikube}
|
||||
|
||||
if [ $VARIANT == "minikube" ]; then
|
||||
APISERVER_HOST=$(minikube ip)
|
||||
SSH_KEY=$(minikube ssh-key)
|
||||
SSH_USER=docker
|
||||
MANIFEST="/etc/kubernetes/manifests/kube-apiserver.yaml"
|
||||
fi
|
||||
|
||||
if [ $VARIANT == "kops" ]; then
|
||||
# APISERVER_HOST=api.your-kops-cluster-name.com
|
||||
SSH_KEY=~/.ssh/id_rsa
|
||||
SSH_USER=admin
|
||||
MANIFEST=/etc/kubernetes/manifests/kube-apiserver.manifest
|
||||
|
||||
if [ -z "${APISERVER_HOST+xxx}" ]; then
|
||||
echo "***You must specify APISERVER_HOST with the name of your kops api server"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "***Copying audit policy/webhook files to apiserver..."
|
||||
ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo mkdir -p /var/lib/k8s_audit && sudo chown $SSH_USER /var/lib/k8s_audit"
|
||||
scp -i $SSH_KEY audit-policy.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
|
||||
scp -i $SSH_KEY webhook-config.yaml $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
|
||||
scp -i $SSH_KEY apiserver-config.patch.sh $SSH_USER@$APISERVER_HOST:/var/lib/k8s_audit
|
||||
|
||||
echo "***Modifying k8s apiserver config (will result in apiserver restarting)..."
|
||||
|
||||
ssh -i $SSH_KEY $SSH_USER@$APISERVER_HOST "sudo bash /var/lib/k8s_audit/apiserver-config.patch.sh $MANIFEST $VARIANT"
|
||||
|
||||
echo "***Done!"
|
||||
Loading…
Reference in New Issue
Block a user