Merge branch 'dev' into agent-master

Merge pull request #245 from draios/prepare-for-0-7-0
Update for 0.7.0.
2026-03-20 03:32:09 +00:00 · 2017-06-06 10:13:23 -07:00 · 2017-05-30 12:52:44 -07:00 · 2017-05-30 09:35:24 -07:00 · 2017-05-25 13:51:12 -07:00 · 2017-05-25 12:15:35 -07:00
39 changed files with 1664 additions and 392 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -12,7 +12,7 @@ test/results*.json.*
 userspace/falco/lua/re.lua
 userspace/falco/lua/lpeg.so

-docker/event-generator/event-generator
+docker/event-generator/event_generator
 docker/event-generator/mysqld
 docker/event-generator/httpd
 docker/event-generator/sha1sum
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,9 @@ language: c
 env:
    - BUILD_TYPE=Debug
    - BUILD_TYPE=Release
+sudo: required
+services:
+    - docker
 before_install:
    - sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
    - sudo apt-get update
@@ -9,12 +12,12 @@ install:
    - sudo apt-get --force-yes install g++-4.8
    - sudo apt-get install rpm linux-headers-$(uname -r)
    - git clone https://github.com/draios/sysdig.git ../sysdig
-    - sudo apt-get install -y python-pip libvirt-dev jq
+    - sudo apt-get install -y python-pip libvirt-dev jq dkms
    - cd ..
    - curl -Lo avocado-36.0-tar.gz https://github.com/avocado-framework/avocado/archive/36.0lts.tar.gz
    - tar -zxvf avocado-36.0-tar.gz
    - cd avocado-36.0lts
-    - sudo pip install -r requirements-travis.txt
+    - sudo -H pip install -r requirements.txt
    - sudo python setup.py install
    - cd ../falco
 before_script:
@@ -35,7 +38,10 @@ script:
    - cmake .. -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DDRAIOS_DEBUG_FLAGS="-D_DEBUG -DNDEBUG"
    - make VERBOSE=1
    - make package
-    - cd ..
+    - cp falco*.deb ../docker/local
+    - cd ../docker/local
+    - docker build -t sysdig/falco:test .
+    - cd ../..
    - sudo test/run_regression_tests.sh $TRAVIS_BRANCH
 notifications:
  webhooks:
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,87 @@

 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).

+## v0.7.0
+
+Released 2016-05-30
+
+### Major Changes
+
+* Update the priorities of falco rules to use a wider range of priorities rather than just ERROR/WARNING. More info on the use of priorities in the ruleset can be found [here](https://github.com/draios/falco/wiki/Falco-Rules#rule-priorities). [[#244](https://github.com/draios/falco/pull/244)]
+
+### Minor Changes
+
+None.
+
+### Bug Fixes
+
+* Fix typos in various markdown files. Thanks @sublimino! [[#241](https://github.com/draios/falco/pull/241)]
+
+### Rule Changes
+
+* Add gitlab-mon as a gitlab binary, which allows it to run shells, etc. Thanks @dkerwin! [[#237](https://github.com/draios/falco/pull/237)]
+* A new rule Terminal shell in container" that looks for shells spawned in a container with an attached terminal. [[#242](https://github.com/draios/falco/pull/242)]
+* Fix some FPs related to the sysdig monitor agent. [[#243](https://github.com/draios/falco/pull/243)]
+* Fix some FPs related to stating containers combined with missed events [[#243](https://github.com/draios/falco/pull/243)]
+
+## v0.6.1
+
+Released 2016-05-15
+
+### Major Changes
+
+None
+
+### Minor Changes
+
+* Small changes to token bucket used to throttle falco events [[#234](https://github.com/draios/falco/pull/234)] [[#235](https://github.com/draios/falco/pull/235)] [[#236](https://github.com/draios/falco/pull/236)] [[#238](https://github.com/draios/falco/pull/238)]
+
+### Bug Fixes
+
+* Update the falco driver to work with kernel 4.11 [[#829](https://github.com/draios/sysdig/pull/829)]
+
+### Rule Changes
+
+* Don't allow apache2 to spawn shells in containers [[#231](https://github.com/draios/falco/issues/231)] [[#232](https://github.com/draios/falco/pull/232)]
+
+## v0.6.0
+
+Released 2016-03-29
+
+### Major Changes
+
+* Add the notion of tagged falco rules. Full documentation for this feature is available on the [wiki](https://github.com/draios/falco/wiki/Falco-Rules#rule-tags). [[#58](https://github.com/draios/falco/issues/58)] [[#59](https://github.com/draios/falco/issues/59)] [[#60](https://github.com/draios/falco/issues/60)] [[#206](https://github.com/draios/falco/pull/206)]
+* Falco now has its own dedicated kernel module. Previously, it would depend on sysdig being installed and would use sysdig's `sysdig-probe` kernel module. This ensures you can upgrade sysdig and falco without kernel driver compatibility problems. More details on the kernel module and its installation are on the [wiki](https://github.com/draios/falco/wiki/Falco-Kernel-Module). [[#215](https://github.com/draios/falco/issues/215)] [[#223](https://github.com/draios/falco/issues/223)] [[#224](https://github.com/draios/falco/pull/224)]
+* When providing multiple rules files by specifying `-r' multiple times, make sure that you can override rules/lists/macros. Previously, a list/macro/rule specified in an earlier file could not be overridden in a later file. [[#176](https://github.com/draios/falco/issues/176)] [[#177](https://github.com/draios/falco/pull/177)]
+* Add example k8s yaml files that show how to run falco as a k8s DaemonSet, and how to run falco-event-generator as a deployment running on one node. [[#222](https://github.com/draios/falco/pull/222)] [[#225](https://github.com/draios/falco/issues/225)] [[#226](https://github.com/draios/falco/pull/226)]
+* Update third party libraries to address security vulnerabilities. [[#182](https://github.com/draios/falco/pull/182)]
+* Falco can now be built on OSX. Like sysdig, on OSX it is limited to reading existing trace files. [[#210](https://github.com/draios/falco/pull/210)]
+
+### Minor Changes
+* Several changes to [falco-event-generator](https://github.com/draios/falco/wiki/Generating-Sample-Events) to improve usability. [[#205](https://github.com/draios/falco/pull/205)]
+* Switch to a formatter cache provided by sysdig code instead of using our own. [[#212](https://github.com/draios/falco/pull/212)]
+* Add automated tests that use locally-built docker images. [[#188](https://github.com/draios/falco/issues/188)]
+
+### Bug Fixes
+
+* Make sure output strings are not truncated when a given %field expression has a NULL value. [[#180](https://github.com/draios/falco/issues/180)] [[#181](https://github.com/draios/falco/pull/181)]
+* Allow ASSERTs when running travisci tests. [[#199](https://github.com/draios/falco/pull/199)]
+* Fix make dependencies for lyaml. [[#204](https://github.com/draios/falco/pull/204)] [[#130](https://github.com/draios/falco/issues/130)]
+* (This was a change in sysdig, but affected falco). Prevent hangs when traversing malformed parent thread state. [[#208](https://github.com/draios/falco/issues/208)]
+
+### Rule Changes
+
+* Add confd as a program that can write files below /etc and fleetctl as a program that can spawn shells. [[#175](https://github.com/draios/falco/pull/175)]
+* Add [exechealthz](https://github.com/kubernetes/contrib/tree/master/exec-healthz), a k8s liveness checking utility, to the list of shell spawners. [[#190](https://github.com/draios/falco/pull/190)]
+* Eliminate FPs related to weekly ubuntu cron jobs. [[#192](https://github.com/draios/falco/pull/192)]
+* Allow shells spawned by ansible, and eliminate FPs when managing machines via ansible. [[#193](https://github.com/draios/falco/pull/193)] [[#196](https://github.com/draios/falco/pull/196)] [[#202](https://github.com/draios/falco/pull/202)]
+* Eliminate FPs related to use of other security products. Thanks to @juju4 for the useful rule updates. [[#200](https://github.com/draios/falco/pull/200)]
+* Add additional possible locations for denyhosts, add [PM2](http://pm2.keymetrics.io/) as a shell spawner. [[#202](https://github.com/draios/falco/pull/202)]
+* Add flanneld as a privileged container, improve grouping for the "x running y" macros, allow denyhosts to spawn shells. [[#207](https://github.com/draios/falco/pull/207)]
+* Handle systemd changing its name to "(systemd)", add sv (part of [runit](http://smarden.org/runit/)) as a program that can write below /etc, allow writing to all `/dev/tty*` files. [[#209](https://github.com/draios/falco/pull/209)]
+* Add erl_child_setup as a shell spawner. Thanks to @dkerwin for the useful rule updates. [[#218](https://github.com/draios/falco/pull/218)]  [[#221](https://github.com/draios/falco/pull/221)]
+* Add support for gitlab omnibus containers/pods. Thanks to @dkerwin for the useful rule updates. [[#220](https://github.com/draios/falco/pull/220)]
+
 ## v0.5.0

 Released 2016-12-22
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -29,7 +29,9 @@ set(CMAKE_CXX_FLAGS_RELEASE "-O3 -fno-strict-aliasing -DNDEBUG")

 add_definitions(-DPLATFORM_NAME="${CMAKE_SYSTEM_NAME}")
 add_definitions(-DK8S_DISABLE_THREAD)
-add_definitions(-DHAS_CAPTURE)
+if(CMAKE_SYSTEM_NAME MATCHES "Linux")
+	add_definitions(-DHAS_CAPTURE)
+endif()

 if(CMAKE_BUILD_TYPE STREQUAL "Debug")
 	set(KBUILD_FLAGS "${DRAIOS_DEBUG_FLAGS} ${DRAIOS_FEATURE_FLAGS}")
@@ -39,13 +41,17 @@ endif()

 set(PACKAGE_NAME "falco")
 set(PROBE_VERSION "${FALCO_VERSION}")
-set(PROBE_NAME "sysdig-probe")
-set(PROBE_DEVICE_NAME "sysdig")
+set(PROBE_NAME "falco-probe")
+set(PROBE_DEVICE_NAME "falco")
 set(CMAKE_INSTALL_PREFIX /usr)

 set(CMD_MAKE make)

 set(SYSDIG_DIR "${PROJECT_SOURCE_DIR}/../sysdig")
+# make luaJIT work on OS X
+if(APPLE)
+	set(CMAKE_EXE_LINKER_FLAGS "-pagezero_size 10000 -image_base 100000000")
+endif()

 include(ExternalProject)

@@ -295,14 +301,18 @@ if(NOT USE_BUNDLED_LPEG)
 else()
 	set(LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
 	set(LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
+	set(LPEG_DEPENDENCIES "")
+	if(USE_BUNDLED_LUAJIT)
+		list(APPEND LPEG_DEPENDENCIES "luajit")
+	endif()
 	ExternalProject_Add(lpeg
-                DEPENDS luajit
+		DEPENDS ${LPEG_DEPENDENCIES}
 		URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
-                URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
-                BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
-                BUILD_IN_SOURCE 1
+		URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
+		BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
+		BUILD_IN_SOURCE 1
 		CONFIGURE_COMMAND ""
-                INSTALL_COMMAND "")
+		INSTALL_COMMAND "")
 endif()

 #
@@ -332,11 +342,11 @@ else()
 	set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
 	ExternalProject_Add(libyaml
 		URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
-                URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
-                BUILD_COMMAND ${CMD_MAKE}
-                BUILD_IN_SOURCE 1
+		URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
+		BUILD_COMMAND ${CMD_MAKE}
+		BUILD_IN_SOURCE 1
 		CONFIGURE_COMMAND ./bootstrap && ./configure
-                INSTALL_COMMAND "")
+		INSTALL_COMMAND "")
 endif()

 #
@@ -357,8 +367,15 @@ if(NOT USE_BUNDLED_LYAML)
 else()
 	set(LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
 	set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
+	set(LYAML_DEPENDENCIES "")
+	if(USE_BUNDLED_LUAJIT)
+		list(APPEND LYAML_DEPENDENCIES "luajit")
+	endif()
+	if(USE_BUNDLED_LIBYAML)
+		list(APPEND LYAML_DEPENDENCIES "libyaml")
+	endif()
 	ExternalProject_Add(lyaml
-		DEPENDS libyaml luajit
+		DEPENDS ${LYAML_DEPENDENCIES}
 		URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
                URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
                BUILD_COMMAND ${CMD_MAKE}
@@ -370,7 +387,9 @@ endif()
 install(FILES falco.yaml
 	DESTINATION "${FALCO_ETC_DIR}")

-add_subdirectory("${SYSDIG_DIR}/driver" "${PROJECT_BINARY_DIR}/driver")
+if(CMAKE_SYSTEM_NAME MATCHES "Linux")
+	add_subdirectory("${SYSDIG_DIR}/driver" "${PROJECT_BINARY_DIR}/driver")
+endif()
 add_subdirectory("${SYSDIG_DIR}/userspace/libscap" "${PROJECT_BINARY_DIR}/userspace/libscap")
 add_subdirectory("${SYSDIG_DIR}/userspace/libsinsp" "${PROJECT_BINARY_DIR}/userspace/libsinsp")

@@ -396,12 +415,12 @@ set(CPACK_GENERATOR DEB RPM TGZ)
 set(CPACK_DEBIAN_PACKAGE_MAINTAINER "Sysdig <support@sysdig.com>")
 set(CPACK_DEBIAN_PACKAGE_SECTION "utils")
 set(CPACK_DEBIAN_PACKAGE_HOMEPAGE "http://www.sysdig.org")
-set(CPACK_DEBIAN_PACKAGE_DEPENDS "sysdig")
-set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${PROJECT_SOURCE_DIR}/scripts/debian/postinst;${PROJECT_SOURCE_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm")
+set(CPACK_DEBIAN_PACKAGE_DEPENDS "dkms (>= 2.1.0.0)")
+set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/scripts/debian/postinst;${CMAKE_BINARY_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm")

 set(CPACK_RPM_PACKAGE_LICENSE "GPLv2")
 set(CPACK_RPM_PACKAGE_URL "http://www.sysdig.org")
-set(CPACK_RPM_PACKAGE_REQUIRES "sysdig")
+set(CPACK_RPM_PACKAGE_REQUIRES "dkms, gcc, make, kernel-devel, perl")
 set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/postinstall")
 set(CPACK_RPM_PRE_UNINSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/preuninstall")
 set(CPACK_RPM_POST_UNINSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/postuninstall")
--- a/README.md
+++ b/README.md
@@ -1,8 +1,8 @@
 # Sysdig Falco

-####Latest release
+#### Latest release

-**v0.5.0**
+**v0.7.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)

 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -29,13 +29,13 @@ One of the questions we often get when we talk about Sysdig Falco is “How does

 Documentation
 ---
-[Visit the wiki] (https://github.com/draios/falco/wiki) for full documentation on falco.
+[Visit the wiki](https://github.com/draios/falco/wiki) for full documentation on falco.

 Join the Community
 ---
-* Contact the [official mailing list] (https://groups.google.com/forum/#!forum/falco) for support and to talk with other users.
-* Follow us on [Twitter] (https://twitter.com/sysdig) for general falco and sysdig news.
-* This is our [blog] (https://sysdig.com/blog/), where you can find the latest [falco](https://sysdig.com/blog/tag/falco/) posts.
+* Contact the [official mailing list](https://groups.google.com/forum/#!forum/falco) for support and to talk with other users.
+* Follow us on [Twitter](https://twitter.com/sysdig) for general falco and sysdig news.
+* This is our [blog](https://sysdig.com/blog/), where you can find the latest [falco](https://sysdig.com/blog/tag/falco/) posts.
 * Join our [Public Slack](https://sysdig.slack.com) channel for sysdig and falco announcements and discussions.

 License Terms
@@ -44,7 +44,7 @@ Falco is licensed to you under the [GPL 2.0](./COPYING) open source license.

 Contributor License Agreements
 ---
-###Background
+### Background
 As we did for sysdig, we are formalizing the way that we accept contributions of code from the contributing community. We must now ask that contributions to falco be provided subject to the terms and conditions of a [Contributor License Agreement (CLA)](./cla). The CLA comes in two forms, applicable to contributions by individuals, or by legal entities such as corporations and their employees. We recognize that entering into a CLA with us involves real consideration on your part, and we’ve tried to make this process as clear and simple as possible.

 We’ve modeled our CLA off of industry standards, such as [the CLA used by Kubernetes](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md). Note that this agreement is not a transfer of copyright ownership, this simply is a license agreement for contributions, intended to clarify the intellectual property license granted with contributions from any person or entity. It is for your protection as a contributor as well as the protection of falco; it does not change your rights to use your own contributions for any other purpose.
@@ -57,7 +57,7 @@ Contributor License Agreements

 As always, we are grateful for your past and present contributions to falco.

-###What do I need to do in order to contribute code?
+### What do I need to do in order to contribute code?

 **Individual contributions**: Individuals who wish to make contributions must review the [Individual Contributor License Agreement](./cla/falco_contributor_agreement.txt) and indicate agreement by adding the following line to every GIT commit message:

--- a/docker/dev/docker-entrypoint.sh
+++ b/docker/dev/docker-entrypoint.sh
@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
        ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
    done

-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi

 exec "$@"
--- a/docker/local/Dockerfile
+++ b/docker/local/Dockerfile
@@ -25,7 +25,7 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
 	gcc \
 	gcc-5 \
 	gcc-4.9 \
-	sysdig && rm -rf /var/lib/apt/lists/*
+	dkms && rm -rf /var/lib/apt/lists/*

 # Since our base Debian image ships with GCC 5.0 which breaks older kernels, revert the
 # default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
--- a/docker/local/docker-entrypoint.sh
+++ b/docker/local/docker-entrypoint.sh
@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
        ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
    done

-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi

 exec "$@"
--- a/docker/stable/docker-entrypoint.sh
+++ b/docker/stable/docker-entrypoint.sh
@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
        ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
    done

-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi

 exec "$@"
--- a/examples/k8s-using-daemonset/README.md
+++ b/examples/k8s-using-daemonset/README.md
@@ -0,0 +1,5 @@
+# Example K8s Services for Falco
+
+The yaml file in this directory installs the following:
+ - Open Source Falco, as a DaemonSet. Falco is configured to communicate with the K8s API server via its service account, and changes its output to be K8s-friendly. It also sends to a slack webhook for the `#demo-falco-alerts` channel on our [public slack](https://sysdig.slack.com/messages/demo-falco-alerts/).
+ - The [Falco Event Generator](https://github.com/draios/falco/wiki/Generating-Sample-Events), as a deployment that ensures it runs on exactly 1 node.
--- a/examples/k8s-using-daemonset/falco-daemonset.yaml
+++ b/examples/k8s-using-daemonset/falco-daemonset.yaml
@@ -0,0 +1,59 @@
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: falco
+  labels:
+    name: falco-daemonset
+    app: demo
+spec:
+  template:
+    metadata:
+      labels:
+        name: falco
+        app: demo
+        role: security
+    spec:
+      containers:
+        - name: falco
+          image: sysdig/falco:latest
+          securityContext:
+            privileged: true
+          args: [ "/usr/bin/falco", "-K", "/var/run/secrets/kubernetes.io/serviceaccount/token", "-k", "https://kubernetes", "-pk", "-o", "json_output=true", "-o", "program_output.enabled=true", "-o",  "program_output.program=jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/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"]
+          volumeMounts:
+            - mountPath: /host/var/run/docker.sock
+              name: docker-socket
+              readOnly: true
+            - mountPath: /host/dev
+              name: dev-fs
+              readOnly: true
+            - mountPath: /host/proc
+              name: proc-fs
+              readOnly: true
+            - mountPath: /host/boot
+              name: boot-fs
+              readOnly: true
+            - mountPath: /host/lib/modules
+              name: lib-modules
+              readOnly: true
+            - mountPath: /host/usr
+              name: usr-fs
+              readOnly: true
+      volumes:
+        - name: docker-socket
+          hostPath:
+            path: /var/run/docker.sock
+        - name: dev-fs
+          hostPath:
+            path: /dev
+        - name: proc-fs
+          hostPath:
+            path: /proc
+        - name: boot-fs
+          hostPath:
+            path: /boot
+        - name: lib-modules
+          hostPath:
+            path: /lib/modules
+        - name: usr-fs
+          hostPath:
+            path: /usr
--- a/examples/k8s-using-daemonset/falco-event-generator-deployment.yaml
+++ b/examples/k8s-using-daemonset/falco-event-generator-deployment.yaml
@@ -0,0 +1,17 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: falco-event-generator-deployment
+  labels:
+    name: falco-event-generator-deployment
+    app: demo
+spec:
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: falco-event-generator
+    spec:
+      containers:
+      - name: falco-event-generator
+        image: sysdig/falco-event-generator:latest
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -76,7 +76,7 @@

 # dpkg -L login | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
 - list: login_binaries
-  items: [login, systemd, systemd-logind, su, nologin, faillog, lastlog, newgrp, sg]
+  items: [login, systemd, '"(systemd)"', systemd-logind, su, nologin, faillog, lastlog, newgrp, sg]

 # dpkg -L passwd | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
 - list: passwd_binaries
@@ -113,6 +113,9 @@
 - list: db_server_binaries
  items: [mysqld]

+- list: gitlab_binaries
+  items: [gitlab-shell, gitlab-mon, git]
+
 - macro: server_procs
  condition: proc.name in (http_server_binaries, db_server_binaries, docker_binaries, sshd)

@@ -219,22 +222,28 @@
 # scope the python scripts to a specific directory (namely, your
 # configured remote_tmp directory).
 - macro: parent_ansible_running_python
-  condition: proc.pname in (python, pypy) and proc.pcmdline contains ansible
+  condition: (proc.pname in (python, pypy) and proc.pcmdline contains ansible)

 - macro: ansible_running_python
-  condition: proc.name in (python, pypy) and proc.cmdline contains ansible
+  condition: (proc.name in (python, pypy) and proc.cmdline contains ansible)

 - macro: python_running_denyhosts
-  condition: proc.name=python and (proc.cmdline contains /usr/sbin/denyhosts or proc.cmdline contains /usr/local/bin/denyhosts.py)
+  condition: (proc.name=python and (proc.cmdline contains /usr/sbin/denyhosts or proc.cmdline contains /usr/local/bin/denyhosts.py))
+
+- macro: parent_python_running_denyhosts
+  condition: (proc.pname=python and (proc.pcmdline contains /usr/sbin/denyhosts or proc.pcmdline contains /usr/local/bin/denyhosts.py))

 - macro: parent_bro_running_python
-  condition: proc.pname=python and proc.cmdline contains /usr/share/broctl
+  condition: (proc.pname=python and proc.cmdline contains /usr/share/broctl)

 # As a part of kernel upgrades, dpkg will spawn a perl script with the
 # name linux-image-N.N. This macro matches that.
 - macro: parent_linux_image_upgrade_script
  condition: proc.pname startswith linux-image-

+- macro: java_running_sdjagent
+  condition: proc.name=java and proc.cmdline contains sdjagent.jar
+
 ###############
 # General Rules
 ###############
@@ -243,7 +252,8 @@
  desc: an attempt to write to any file below a set of binary directories
  condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
  output: "File below a known binary directory opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem]

 - macro: write_etc_common
  condition: >
@@ -252,7 +262,7 @@
                          package_mgmt_binaries, ssl_mgmt_binaries, dhcp_binaries,
                          ldconfig.real, ldconfig, confd, gpg, insserv,
                          apparmor_parser, update-mime, tzdata.config, tzdata.postinst,
-                          systemd-machine, debconf-show, rollerd, bind9.postinst)
+                          systemd-machine, debconf-show, rollerd, bind9.postinst, sv)
    and not proc.pname in (sysdigcloud_binaries)
    and not fd.directory in (/etc/cassandra, /etc/ssl/certs/java)
    and not ansible_running_python
@@ -262,7 +272,8 @@
  desc: an attempt to write to any file below /etc, not in a pipe installer session
  condition: write_etc_common and not proc.sname=fbash
  output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem]

 # Within a fbash session, the severity is lowered to INFO
 - rule: Write below etc in installer
@@ -270,6 +281,7 @@
  condition: write_etc_common and proc.sname=fbash
  output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name) within pipe installer session"
  priority: INFO
+  tags: [filesystem]

 - macro: cmp_cp_by_passwd
  condition: proc.name in (cmp, cp) and proc.pname=passwd
@@ -279,6 +291,7 @@
  condition: sensitive_files and open_read and server_procs and not proc_is_new and proc.name!="sshd"
  output: "Sensitive file opened for reading by trusted program after startup (user=%user.name command=%proc.cmdline file=%fd.name)"
  priority: WARNING
+  tags: [filesystem]

 - list: read_sensitive_file_binaries
  items: [iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, sshd, vsftpd, systemd]
@@ -293,31 +306,36 @@
    and not proc.cmdline contains /usr/bin/mandb
  output: "Sensitive file opened for reading by non-trusted program (user=%user.name name=%proc.name command=%proc.cmdline file=%fd.name)"
  priority: WARNING
+  tags: [filesystem]

 # Only let rpm-related programs write to the rpm database
 - rule: Write below rpm database
  desc: an attempt to write to the rpm database by any non-rpm related program
  condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (dnf,rpm,rpmkey,yum) and not ansible_running_python
  output: "Rpm database opened for writing by a non-rpm program (command=%proc.cmdline file=%fd.name)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem, software_mgmt]

 - rule: DB program spawned process
  desc: a database-server related program spawned a new process other than itself. This shouldn\'t occur and is a follow on from some SQL injection attacks.
  condition: proc.pname in (db_server_binaries) and spawned_process and not proc.name in (db_server_binaries)
  output: "Database-related program spawned process other than itself (user=%user.name program=%proc.cmdline parent=%proc.pname)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [process, database]

 - rule: Modify binary dirs
  desc: an attempt to modify any file below a set of binary directories.
  condition: bin_dir_rename and modify and not package_mgmt_procs
  output: "File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline operation=%evt.type file=%fd.name %evt.args)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem]

 - rule: Mkdir binary dirs
  desc: an attempt to create a directory below a set of binary directories.
  condition: mkdir and bin_dir_mkdir and not package_mgmt_procs
  output: "Directory below known binary directory created (user=%user.name command=%proc.cmdline directory=%evt.arg.path)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem]

 # Don't load shared objects coming from unexpected places
 # Commenting this out for now--there are lots of shared library
@@ -340,9 +358,12 @@
  condition: >
    evt.type = setns
    and not proc.name in (docker_binaries, k8s_binaries, lxd_binaries, sysdigcloud_binaries, sysdig, nsenter)
+    and not proc.name startswith "runc:"
    and not proc.pname in (sysdigcloud_binaries)
+    and not java_running_sdjagent
  output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline parent=%proc.pname %container.info)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [process]

 - list: known_shell_spawn_binaries
  items: [
@@ -352,7 +373,7 @@
    logrotate, ansible, less, adduser, pycompile, py3compile,
    pyclean, py3clean, pip, pip2, ansible-playboo, man-db,
    init, pluto, mkinitramfs, unattended-upgr, watch, sysdig,
-    landscape-sysin, nessusd, PM2, syslog-summary
+    landscape-sysin, nessusd, PM2, syslog-summary, erl_child_setup
    ]

 - rule: Run shell untrusted
@@ -366,9 +387,11 @@
                           monitoring_binaries)
    and not parent_ansible_running_python
    and not parent_bro_running_python
+    and not parent_python_running_denyhosts
    and not parent_linux_image_upgrade_script
  output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline pcmdline=%proc.pcmdline)"
-  priority: WARNING
+  priority: DEBUG
+  tags: [host, shell]

 - macro: trusted_containers
  condition: (container.image startswith sysdig/agent or
@@ -376,13 +399,15 @@
               not container.image startswith sysdig/falco-event-generator) or
              container.image startswith sysdig/sysdig or
              container.image startswith gcr.io/google_containers/hyperkube or
+              container.image startswith quay.io/coreos/flannel or
              container.image startswith gcr.io/google_containers/kube-proxy)

 - rule: File Open by Privileged Container
  desc: Any open by a privileged container. Exceptions are made for known trusted images.
  condition: (open_read or open_write) and container and container.privileged=true and not trusted_containers
  output: File opened for read/write by privileged container (user=%user.name command=%proc.cmdline %container.info file=%fd.name)
-  priority: WARNING
+  priority: INFO
+  tags: [container, cis]

 - macro: sensitive_mount
  condition: (container.mount.dest[/proc*] != "N/A")
@@ -391,7 +416,8 @@
  desc: Any open by a container that has a mount from a sensitive host directory (i.e. /proc). Exceptions are made for known trusted images.
  condition: (open_read or open_write) and container and sensitive_mount and not trusted_containers
  output: File opened for read/write by container mounting sensitive directory (user=%user.name command=%proc.cmdline %container.info file=%fd.name)
-  priority: WARNING
+  priority: INFO
+  tags: [container, cis]

 # Anything run interactively by root
 # - condition: evt.type != switch and user.name = root and proc.name != sshd and interactive
@@ -402,7 +428,17 @@
  desc: an attempt to run interactive commands by a system (i.e. non-login) user
  condition: spawned_process and system_users and interactive
  output: "System user ran an interactive command (user=%user.name command=%proc.cmdline)"
-  priority: WARNING
+  priority: INFO
+  tags: [users]
+
+- rule: Terminal shell in container
+  desc: A shell was spawned by a program in a container with an attached terminal.
+  condition: >
+    spawned_process and container
+    and shell_procs and proc.tty != 0
+  output: "A shell was spawned in a container with an attached terminal (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty)"
+  priority: NOTICE
+  tags: [container, shell]

 - rule: Run shell in container
  desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
@@ -411,17 +447,19 @@
    and shell_procs
    and proc.pname exists
    and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, lxd_binaries, aide_wrapper_binaries, nids_binaries,
-                           monitoring_binaries, initdb, pg_ctl, awk, apache2, falco, cron)
+                           monitoring_binaries, gitlab_binaries, initdb, pg_ctl, awk, falco, cron, erl_child_setup)
    and not trusted_containers
  output: "Shell spawned in a container other than entrypoint (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [container, shell]

 # sockfamily ip is to exclude certain processes (like 'groups') that communicate on unix-domain sockets
 - rule: System procs network activity
  desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
  condition: (fd.sockfamily = ip and system_procs) and (inbound or outbound)
  output: "Known system binary sent/received network traffic (user=%user.name command=%proc.cmdline connection=%fd.name)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [network]

 # With the current restriction on system calls handled by falco
 # (e.g. excluding read/write/sendto/recvfrom/etc, this rule won't
@@ -437,16 +475,18 @@
  desc: an attempt to change users by calling setuid. sudo/su are excluded. user "root" is also excluded, as setuid calls typically involve dropping privileges.
  condition: evt.type=setuid and evt.dir=> and not user.name=root and not proc.name in (userexec_binaries, mail_binaries, sshd, dbus-daemon-lau, ping, ping6, critical-stack-)
  output: "Unexpected setuid call by non-sudo, non-root program (user=%user.name parent=%proc.pname command=%proc.cmdline uid=%evt.arg.uid)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [users]

 - rule: User mgmt binaries
  desc: activity by any programs that can manage users, passwords, or permissions. sudo and su are excluded. Activity in containers is also excluded--some containers create custom users on top of a base linux distribution at startup.
  condition: spawned_process and proc.name in (user_mgmt_binaries) and not proc.name in (su, sudo) and not container and not proc.pname in (cron_binaries, systemd, run-parts)
  output: "User management binary command run outside of container (user=%user.name command=%proc.cmdline parent=%proc.pname)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [host, users]

 - list: allowed_dev_files
-  items: [/dev/null, /dev/stdin, /dev/stdout, /dev/stderr, /dev/tty, /dev/random, /dev/urandom, /dev/console]
+  items: [/dev/null, /dev/stdin, /dev/stdout, /dev/stderr, /dev/random, /dev/urandom, /dev/console]

 # (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
 - rule: Create files below dev
@@ -456,27 +496,32 @@
    (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT))
    and not proc.name in (dev_creation_binaries)
    and not fd.name in (allowed_dev_files)
+    and not fd.name startswith /dev/tty
  output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
-  priority: WARNING
+  priority: ERROR
+  tags: [filesystem]

 # fbash is a small shell script that runs bash, and is suitable for use in curl <curl> | fbash installers.
 - rule: Installer bash starts network server
  desc: an attempt by a program in a pipe installer session to start listening for network connections
  condition: evt.type=listen and proc.sname=fbash
  output: "Unexpected listen call by a process in a fbash session (command=%proc.cmdline)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [network]

 - rule: Installer bash starts session
  desc: an attempt by a program in a pipe installer session to start a new session
  condition: evt.type=setsid and proc.sname=fbash
  output: "Unexpected setsid call by a process in fbash session (command=%proc.cmdline)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [process]

 - rule: Installer bash non https connection
  desc: an attempt by a program in a pipe installer session to make an outgoing connection on a non-http(s) port
  condition: proc.sname=fbash and outbound and not fd.sport in (80, 443, 53)
  output: "Outbound connection on non-http(s) port by a process in a fbash session (command=%proc.cmdline connection=%fd.name)"
-  priority: WARNING
+  priority: NOTICE
+  tags: [network]

 # It'd be nice if we could warn when processes in a fbash session try
 # to download from any nonstandard location? This is probably blocked
@@ -490,6 +535,7 @@
  condition: evt.type=execve and proc.name in (chkconfig, systemctl) and proc.sname=fbash
  output: "Service management program run by process in a fbash session (command=%proc.cmdline)"
  priority: INFO
+  tags: [software_mgmt]

 # Notice when processes try to run any package management binary within a fbash session.
 # Note: this is not a WARNING, as you'd expect some package management
@@ -499,6 +545,7 @@
  condition: evt.type=execve and package_mgmt_procs and proc.sname=fbash
  output: "Package management program run by process in a fbash session (command=%proc.cmdline)"
  priority: INFO
+  tags: [software_mgmt]

 ###########################
 # Application-Related Rules
--- a/scripts/CMakeLists.txt
+++ b/scripts/CMakeLists.txt
@@ -1,5 +1,12 @@
+configure_file(debian/postinst.in debian/postinst)
+configure_file(debian/prerm.in debian/prerm)
+
 file(COPY "${PROJECT_SOURCE_DIR}/scripts/debian/falco"
 	DESTINATION "${PROJECT_BINARY_DIR}/scripts/debian")

 file(COPY "${PROJECT_SOURCE_DIR}/scripts/rpm/falco"
 	DESTINATION "${PROJECT_BINARY_DIR}/scripts/rpm")
+
+install(PROGRAMS ${SYSDIG_DIR}/scripts/sysdig-probe-loader
+	DESTINATION bin
+	RENAME falco-probe-loader)
--- a/scripts/debian/postinst
+++ b/scripts/debian/postinst
@@ -1,9 +0,0 @@
-#!/bin/sh
-set -e
-
-NAME=falco
-
-if [ -x "/etc/init.d/$NAME" ]; then
-        update-rc.d $NAME defaults >/dev/null
-fi
-
--- a/scripts/debian/postinst.in
+++ b/scripts/debian/postinst.in
@@ -0,0 +1,32 @@
+#!/bin/sh
+set -e
+
+DKMS_PACKAGE_NAME="@PACKAGE_NAME@"
+DKMS_VERSION="@PROBE_VERSION@"
+NAME="@PACKAGE_NAME@"
+
+postinst_found=0
+
+case "$1" in
+	configure)
+		for DKMS_POSTINST in /usr/lib/dkms/common.postinst /usr/share/$DKMS_PACKAGE_NAME/postinst; do
+			if [ -f $DKMS_POSTINST ]; then
+				$DKMS_POSTINST $DKMS_PACKAGE_NAME $DKMS_VERSION /usr/share/$DKMS_PACKAGE_NAME "" $2
+				postinst_found=1
+				break
+			fi
+		done
+		if [ "$postinst_found" -eq 0 ]; then
+			echo "ERROR: DKMS version is too old and $DKMS_PACKAGE_NAME was not"
+			echo "built with legacy DKMS support."
+			echo "You must either rebuild $DKMS_PACKAGE_NAME with legacy postinst"
+			echo "support or upgrade DKMS to a more current version."
+			exit 1
+		fi
+	;;
+esac
+
+if [ -x "/etc/init.d/$NAME" ]; then
+        update-rc.d $NAME defaults >/dev/null
+fi
+
--- a/scripts/debian/prerm
+++ b/scripts/debian/prerm
@@ -1,13 +0,0 @@
-#!/bin/sh
-set -e
-
-NAME=falco
-
-if [ -x "/etc/init.d/$NAME" ]; then
-		if [ -x "`which invoke-rc.d 2>/dev/null`" ]; then
-				invoke-rc.d $NAME stop || exit $?
-		else
-				/etc/init.d/$NAME stop || exit $?
-		fi
-fi
-
--- a/scripts/debian/prerm.in
+++ b/scripts/debian/prerm.in
@@ -0,0 +1,23 @@
+#!/bin/sh
+set -e
+
+NAME="@PACKAGE_NAME@"
+
+if [ -x "/etc/init.d/$NAME" ]; then
+		if [ -x "`which invoke-rc.d 2>/dev/null`" ]; then
+				invoke-rc.d $NAME stop || exit $?
+		else
+				/etc/init.d/$NAME stop || exit $?
+		fi
+fi
+
+DKMS_PACKAGE_NAME="@PACKAGE_NAME@"
+DKMS_VERSION="@PROBE_VERSION@"
+
+case "$1" in
+	remove|upgrade|deconfigure)
+		if [  "$(dkms status -m $DKMS_PACKAGE_NAME -v $DKMS_VERSION)" ]; then
+			dkms remove -m $DKMS_PACKAGE_NAME -v $DKMS_VERSION --all
+		fi
+	;;
+esac
--- a/scripts/rpm/postinstall
+++ b/scripts/rpm/postinstall
@@ -1 +1,15 @@
+dkms add -m falco -v %{version} --rpm_safe_upgrade
+if [ `uname -r | grep -c "BOOT"` -eq 0 ] && [ -e /lib/modules/`uname -r`/build/include ]; then
+	dkms build -m falco -v %{version}
+	dkms install --force -m falco -v %{version}
+elif [ `uname -r | grep -c "BOOT"` -gt 0 ]; then
+	echo -e ""
+	echo -e "Module build for the currently running kernel was skipped since you"
+	echo -e "are running a BOOT variant of the kernel."
+else
+	echo -e ""
+	echo -e "Module build for the currently running kernel was skipped since the"
+	echo -e "kernel source for this kernel does not seem to be installed."
+fi
+
 /sbin/chkconfig --add falco
--- a/scripts/rpm/preuninstall
+++ b/scripts/rpm/preuninstall
@@ -2,3 +2,5 @@ if [ $1 = 0 ]; then
 	/sbin/service falco stop > /dev/null 2>&1
 	/sbin/chkconfig --del falco
 fi
+
+dkms remove -m falco -v %{version} --all --rpm_safe_upgrade
--- a/test/falco_test.py
+++ b/test/falco_test.py
@@ -4,6 +4,9 @@ import os
 import re
 import json
 import sets
+import glob
+import shutil
+import subprocess

 from avocado import Test
 from avocado.utils import process
@@ -21,9 +24,9 @@ class FalcoTest(Test):
        self.stderr_contains = self.params.get('stderr_contains', '*', default='')
        self.exit_status = self.params.get('exit_status', '*', default=0)
        self.should_detect = self.params.get('detect', '*', default=False)
-        self.trace_file = self.params.get('trace_file', '*')
+        self.trace_file = self.params.get('trace_file', '*', default='')

-        if not os.path.isabs(self.trace_file):
+        if self.trace_file and not os.path.isabs(self.trace_file):
            self.trace_file = os.path.join(self.basedir, self.trace_file)

        self.json_output = self.params.get('json_output', '*', default=False)
@@ -43,6 +46,8 @@ class FalcoTest(Test):
        if not os.path.isabs(self.conf_file):
            self.conf_file = os.path.join(self.basedir, self.conf_file)

+        self.run_duration = self.params.get('run_duration', '*', default='')
+
        self.disabled_rules = self.params.get('disabled_rules', '*', default='')

        if self.disabled_rules == '':
@@ -89,15 +94,23 @@ class FalcoTest(Test):
            if not isinstance(self.detect_level, list):
                self.detect_level = [self.detect_level]

-        # Doing this in 2 steps instead of simply using
-        # module_is_loaded to avoid logging lsmod output to the log.
-        lsmod_output = process.system_output("lsmod", verbose=False)
+        self.package = self.params.get('package', '*', default='None')

-        if linux_modules.parse_lsmod_for_module(lsmod_output, 'sysdig_probe') == {}:
-            self.log.debug("Loading sysdig kernel module")
-            process.run('sudo insmod {}/driver/sysdig-probe.ko'.format(self.falcodir))
+        if self.package == 'None':
+            # Doing this in 2 steps instead of simply using
+            # module_is_loaded to avoid logging lsmod output to the log.
+            lsmod_output = process.system_output("lsmod", verbose=False)

-        self.str_variant = self.trace_file
+            if linux_modules.parse_lsmod_for_module(lsmod_output, 'falco_probe') == {}:
+                self.log.debug("Loading falco kernel module")
+                process.run('insmod {}/driver/falco-probe.ko'.format(self.falcodir), sudo=True)
+
+        self.addl_docker_run_args = self.params.get('addl_docker_run_args', '*', default='')
+
+        self.copy_local_driver = self.params.get('copy_local_driver', '*', default=False)
+
+        # Used by possibly_copy_local_driver as well as docker run
+        self.module_dir = os.path.expanduser("~/.sysdig")

        self.outputs = self.params.get('outputs', '*', default='')

@@ -111,8 +124,26 @@ class FalcoTest(Test):
                    output['file'] = item2[0]
                    output['line'] = item2[1]
                    outputs.append(output)
+                    filedir = os.path.dirname(output['file'])
+                    # Create the parent directory for the trace file if it doesn't exist.
+                    if not os.path.isdir(filedir):
+                        os.makedirs(filedir)
            self.outputs = outputs

+        self.disable_tags = self.params.get('disable_tags', '*', default='')
+
+        if self.disable_tags == '':
+            self.disable_tags=[]
+
+        self.run_tags = self.params.get('run_tags', '*', default='')
+
+        if self.run_tags == '':
+            self.run_tags=[]
+
+    def tearDown(self):
+        if self.package != 'None':
+            self.uninstall_package()
+
    def check_rules_warnings(self, res):

        found_warning = sets.Set()
@@ -180,13 +211,18 @@ class FalcoTest(Test):
        triggered_rules = match.group(1)

        for rule, count in self.detect_counts.iteritems():
-            expected_line = '{}: {}'.format(rule, count)
-            match = re.search(expected_line, triggered_rules)
+            expected = '{}: (\d+)'.format(rule)
+            match = re.search(expected, triggered_rules)

            if match is None:
-                self.fail("Could not find a line '{}' in triggered rule counts '{}'".format(expected_line, triggered_rules))
+                actual_count = 0
            else:
-                self.log.debug("Found expected count for {}: {}".format(rule, match.group()))
+                actual_count = int(match.group(1))
+
+            if actual_count != count:
+                self.fail("Different counts for rule {}: expected={}, actual={}".format(rule, count, actual_count))
+            else:
+                self.log.debug("Found expected count for rule {}: {}".format(rule, count))

    def check_outputs(self):
        for output in self.outputs:
@@ -216,12 +252,112 @@ class FalcoTest(Test):
                        if not attr in obj:
                            self.fail("Falco JSON object {} does not contain property \"{}\"".format(line, attr))

+    def install_package(self):
+
+        if self.package.startswith("docker:"):
+
+            image = self.package.split(":", 1)[1]
+            # Remove an existing falco-test container first. Note we don't check the output--docker rm
+            # doesn't have an -i equivalent.
+            res = process.run("docker rm falco-test", ignore_status=True)
+            rules_dir = os.path.abspath(os.path.join(self.basedir, "./rules"))
+            conf_dir = os.path.abspath(os.path.join(self.basedir, "../"))
+            traces_dir = os.path.abspath(os.path.join(self.basedir, "./trace_files"))
+            self.falco_binary_path = "docker run -i -t --name falco-test --privileged " \
+                                     "-v {}:/host/rules -v {}:/host/conf -v {}:/host/traces " \
+                                     "-v /var/run/docker.sock:/host/var/run/docker.sock " \
+                                     "-v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro " \
+                                     "-v /lib/modules:/host/lib/modules:ro -v {}:/root/.sysdig:ro -v " \
+                                     "/usr:/host/usr:ro {} {} falco".format(
+                                         rules_dir, conf_dir, traces_dir,
+                                         self.module_dir, self.addl_docker_run_args, image)
+
+        elif self.package.endswith(".deb"):
+            self.falco_binary_path = '/usr/bin/falco';
+
+            package_glob = "{}/{}".format(self.falcodir, self.package)
+
+            matches = glob.glob(package_glob)
+
+            if len(matches) != 1:
+                self.fail("Package path {} did not match exactly 1 file. Instead it matched: {}", package_glob, ",".join(matches))
+
+            package_path = matches[0]
+
+            cmdline = "dpkg -i {}".format(package_path)
+            self.log.debug("Installing debian package via \"{}\"".format(cmdline))
+            res = process.run(cmdline, timeout=120, sudo=True)
+
+    def uninstall_package(self):
+
+        if self.package.startswith("docker:"):
+            # Remove the falco-test image. Here we *do* check the return value
+            res = process.run("docker rm falco-test")
+
+        elif self.package.endswith(".deb"):
+            cmdline = "dpkg -r falco"
+            self.log.debug("Uninstalling debian package via \"{}\"".format(cmdline))
+            res = process.run(cmdline, timeout=120, sudo=True)
+
+    def possibly_copy_driver(self):
+        # Remove the contents of ~/.sysdig regardless of
+        # copy_local_driver.
+        self.log.debug("Checking for module dir {}".format(self.module_dir))
+        if os.path.isdir(self.module_dir):
+            self.log.info("Removing files below directory {}".format(self.module_dir))
+            for rmfile in glob.glob(self.module_dir + "/*"):
+                self.log.debug("Removing file {}".format(rmfile))
+                os.remove(rmfile)
+
+        if self.copy_local_driver:
+            verstr = subprocess.check_output([self.falco_binary_path, "--version"]).rstrip()
+            self.log.info("verstr {}".format(verstr))
+            falco_version = verstr.split(" ")[2]
+            self.log.info("falco_version {}".format(falco_version))
+            arch = subprocess.check_output(["uname", "-m"]).rstrip()
+            self.log.info("arch {}".format(arch))
+            kernel_release = subprocess.check_output(["uname", "-r"]).rstrip()
+            self.log.info("kernel release {}".format(kernel_release))
+
+            # sysdig-probe-loader has a more comprehensive set of ways to
+            # find the config hash. We only look at /boot/config-<kernel release>
+            md5_output = subprocess.check_output(["md5sum", "/boot/config-{}".format(kernel_release)]).rstrip()
+            config_hash = md5_output.split(" ")[0]
+
+            probe_filename = "falco-probe-{}-{}-{}-{}.ko".format(falco_version, arch, kernel_release, config_hash)
+            driver_path = os.path.join(self.falcodir, "driver", "falco-probe.ko")
+            module_path = os.path.join(self.module_dir, probe_filename)
+            self.log.debug("Copying {} to {}".format(driver_path, module_path))
+            shutil.copyfile(driver_path, module_path)
+
    def test(self):
        self.log.info("Trace file %s", self.trace_file)

-        # Run the provided trace file though falco
-        cmd = '{}/userspace/falco/falco {} {} -c {} -e {} -o json_output={} -v'.format(
-            self.falcodir, self.rules_args, self.disabled_args, self.conf_file, self.trace_file, self.json_output)
+        self.falco_binary_path = '{}/userspace/falco/falco'.format(self.falcodir)
+
+        self.possibly_copy_driver()
+
+        if self.package != 'None':
+            # This sets falco_binary_path as a side-effect.
+            self.install_package()
+
+        trace_arg = self.trace_file
+
+        if self.trace_file:
+            trace_arg = "-e {}".format(self.trace_file)
+
+        # Run falco
+        cmd = '{} {} {} -c {} {} -o json_output={} -v'.format(
+            self.falco_binary_path, self.rules_args, self.disabled_args, self.conf_file, trace_arg, self.json_output)
+
+        for tag in self.disable_tags:
+            cmd += ' -T {}'.format(tag)
+
+        for tag in self.run_tags:
+            cmd += ' -t {}'.format(tag)
+
+        if self.run_duration:
+            cmd += ' -M {}'.format(self.run_duration)

        self.falco_proc = process.SubProcess(cmd)

--- a/test/falco_tests.yaml
+++ b/test/falco_tests.yaml
@@ -0,0 +1,519 @@
+trace_files: !mux
+
+  docker_package:
+    package: docker:sysdig/falco:test
+    detect: True
+    detect_level: WARNING
+    rules_file: /host/rules/rule_names_with_spaces.yaml
+    trace_file: /host/traces/cat_write.scap
+    conf_file: /host/conf/falco.yaml
+
+  # This uses a volume mount to overwrite and prevent /usr/sbin/dkms
+  # from being run. As a result, it will force falco-probe-loader to
+  # fall back to loading the driver from ~/.sysdig. Setting
+  # copy_local_driver to True copied the driver to ~/.sysdig, so it
+  # will be available. In this case, we're running live for 5 seconds
+  # just to see if falco can load the driver.
+
+  docker_package_local_driver:
+    package: docker:sysdig/falco:test
+    addl_docker_run_args: -v /dev/null:/usr/sbin/dkms
+    copy_local_driver: True
+    detect: False
+    detect_level: WARNING
+    rules_file: /host/rules/tagged_rules.yaml
+    conf_file: /host/conf/falco.yaml
+    run_duration: 5
+
+  debian_package:
+    package: falco*.deb
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/rule_names_with_spaces.yaml
+    trace_file: trace_files/cat_write.scap
+
+  builtin_rules_no_warnings:
+    detect: False
+    trace_file: trace_files/empty.scap
+    rules_warning: False
+
+  test_warnings:
+    detect: False
+    trace_file: trace_files/empty.scap
+    rules_file: rules/falco_rules_warnings.yaml
+    rules_warning:
+      - no_evttype
+      - evttype_not_equals
+      - leading_not
+      - not_equals_at_end
+      - not_at_end
+      - not_before_trailing_evttype
+      - not_equals_before_trailing_evttype
+      - not_equals_and_not
+      - not_equals_before_in
+      - not_before_in
+      - not_in_before_in
+      - leading_in_not_equals_before_evttype
+      - leading_in_not_equals_at_evttype
+      - not_with_evttypes
+      - not_with_evttypes_addl
+      - not_equals_before_evttype
+      - not_equals_before_in_evttype
+      - not_before_evttype
+      - not_before_evttype_using_in
+    rules_events:
+      - no_warnings: [execve]
+      - no_evttype: [all]
+      - evttype_not_equals: [all]
+      - leading_not: [all]
+      - not_equals_after_evttype: [execve]
+      - not_after_evttype: [execve]
+      - leading_trailing_evttypes: [execve,open]
+      - leading_multtrailing_evttypes: [connect,execve,open]
+      - leading_multtrailing_evttypes_using_in: [connect,execve,open]
+      - not_equals_at_end: [all]
+      - not_at_end: [all]
+      - not_before_trailing_evttype: [all]
+      - not_equals_before_trailing_evttype: [all]
+      - not_equals_and_not: [all]
+      - not_equals_before_in: [all]
+      - not_before_in: [all]
+      - not_in_before_in: [all]
+      - evttype_in: [execve,open]
+      - evttype_in_plus_trailing: [connect,execve,open]
+      - leading_in_not_equals_before_evttype: [all]
+      - leading_in_not_equals_at_evttype: [all]
+      - not_with_evttypes: [all]
+      - not_with_evttypes_addl: [all]
+      - not_equals_before_evttype: [all]
+      - not_equals_before_in_evttype: [all]
+      - not_before_evttype: [all]
+      - not_before_evttype_using_in: [all]
+      - repeated_evttypes: [open]
+      - repeated_evttypes_with_in: [open]
+      - repeated_evttypes_with_separate_in: [open]
+      - repeated_evttypes_with_mix: [open]
+
+  rule_names_with_spaces:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/rule_names_with_spaces.yaml
+    trace_file: trace_files/cat_write.scap
+
+  multiple_rules_first_empty:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/empty_rules.yaml
+      - rules/single_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
+  multiple_rules_last_empty:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/empty_rules.yaml
+    trace_file: trace_files/cat_write.scap
+
+  multiple_rules:
+    detect: True
+    detect_level:
+      - WARNING
+      - INFO
+      - ERROR
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/double_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
+  multiple_rules_overriding:
+    detect: False
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/override_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
+  macro_overriding:
+    detect: False
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/override_macro.yaml
+    trace_file: trace_files/cat_write.scap
+
+  list_overriding:
+    detect: False
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/override_list.yaml
+    trace_file: trace_files/cat_write.scap
+
+  nested_list_overriding:
+    detect: False
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/override_nested_list.yaml
+    trace_file: trace_files/cat_write.scap
+
+  invalid_rule_output:
+    exit_status: 1
+    stderr_contains: "Runtime error: Error loading rules:.* Invalid output format 'An open was seen %not_a_real_field': 'invalid formatting token not_a_real_field'. Exiting."
+    rules_file:
+      - rules/invalid_rule_output.yaml
+    trace_file: trace_files/cat_write.scap
+
+  disabled_rules:
+    detect: False
+    rules_file:
+      - rules/empty_rules.yaml
+      - rules/single_rule.yaml
+    disabled_rules:
+      - open_from_cat
+    trace_file: trace_files/cat_write.scap
+
+  disabled_rules_using_regex:
+    detect: False
+    rules_file:
+      - rules/empty_rules.yaml
+      - rules/single_rule.yaml
+    disabled_rules:
+      - "open.*"
+    trace_file: trace_files/cat_write.scap
+
+  disabled_rules_using_enabled_flag:
+    detect: False
+    rules_file:
+      - rules/single_rule_enabled_flag.yaml
+    trace_file: trace_files/cat_write.scap
+
+  disabled_and_enabled_rules_1:
+    exit_status: 1
+    stderr_contains: "Runtime error: You can not specify both disabled .-D/-T. and enabled .-t. rules. Exiting."
+    disable_tags: [a]
+    run_tags: [a]
+    rules_file:
+      - rules/single_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
+  disabled_and_enabled_rules_2:
+    exit_status: 1
+    stderr_contains: "Runtime error: You can not specify both disabled .-D/-T. and enabled .-t. rules. Exiting."
+    disabled_rules:
+      - "open.*"
+    run_tags: [a]
+    rules_file:
+      - rules/single_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
+  null_output_field:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/null_output_field.yaml
+    trace_file: trace_files/cat_write.scap
+    stdout_contains: "Warning An open was seen .cport=<NA> command=cat /dev/null."
+
+  file_output:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/single_rule.yaml
+    conf_file: confs/file_output.yaml
+    trace_file: trace_files/cat_write.scap
+    outputs:
+      - /tmp/falco_outputs/file_output.txt: Warning An open was seen
+
+  program_output:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/single_rule.yaml
+    conf_file: confs/program_output.yaml
+    trace_file: trace_files/cat_write.scap
+    outputs:
+      - /tmp/falco_outputs/program_output.txt: Warning An open was seen
+
+  detect_counts:
+    detect: True
+    detect_level: WARNING
+    trace_file: traces-positive/falco-event-generator.scap
+    detect_counts:
+      - "Write below binary dir": 1
+      - "Read sensitive file untrusted": 3
+      - "Run shell in container": 1
+      - "Write below rpm database": 1
+      - "Write below etc": 1
+      - "System procs network activity": 1
+      - "Mkdir binary dirs": 1
+      - "System user interactive": 1
+      - "DB program spawned process": 1
+      - "Non sudo setuid": 1
+      - "Create files below dev": 1
+      - "Modify binary dirs": 2
+      - "Change thread namespace": 2
+
+  disabled_tags_a:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    disable_tags: [a]
+    detect_counts:
+      - open_1: 0
+      - open_2: 1
+      - open_3: 1
+      - open_4: 0
+      - open_5: 0
+      - open_6: 1
+      - open_7: 0
+      - open_8: 0
+      - open_9: 0
+      - open_10: 0
+      - open_11: 1
+      - open_12: 1
+      - open_13: 1
+
+  disabled_tags_b:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    disable_tags: [b]
+    detect_counts:
+      - open_1: 1
+      - open_2: 0
+      - open_3: 1
+      - open_4: 0
+      - open_5: 1
+      - open_6: 0
+      - open_7: 0
+      - open_8: 0
+      - open_9: 1
+      - open_10: 0
+      - open_11: 1
+      - open_12: 1
+      - open_13: 1
+
+  disabled_tags_c:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    disable_tags: [c]
+    detect_counts:
+      - open_1: 1
+      - open_2: 1
+      - open_3: 0
+      - open_4: 1
+      - open_5: 0
+      - open_6: 0
+      - open_7: 0
+      - open_8: 1
+      - open_9: 0
+      - open_10: 0
+      - open_11: 1
+      - open_12: 1
+      - open_13: 1
+
+  disabled_tags_ab:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    disable_tags: [a, b]
+    detect_counts:
+      - open_1: 0
+      - open_2: 0
+      - open_3: 1
+      - open_4: 0
+      - open_5: 0
+      - open_6: 0
+      - open_7: 0
+      - open_8: 0
+      - open_9: 0
+      - open_10: 0
+      - open_11: 1
+      - open_12: 1
+      - open_13: 1
+
+  disabled_tags_abc:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    disable_tags: [a, b, c]
+    detect_counts:
+      - open_1: 0
+      - open_2: 0
+      - open_3: 0
+      - open_4: 0
+      - open_5: 0
+      - open_6: 0
+      - open_7: 0
+      - open_8: 0
+      - open_9: 0
+      - open_10: 0
+      - open_11: 1
+      - open_12: 1
+      - open_13: 1
+
+  run_tags_a:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [a]
+    detect_counts:
+      - open_1: 1
+      - open_2: 0
+      - open_3: 0
+      - open_4: 1
+      - open_5: 1
+      - open_6: 0
+      - open_7: 1
+      - open_8: 1
+      - open_9: 1
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_b:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [b]
+    detect_counts:
+      - open_1: 0
+      - open_2: 1
+      - open_3: 0
+      - open_4: 1
+      - open_5: 0
+      - open_6: 1
+      - open_7: 1
+      - open_8: 1
+      - open_9: 0
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_c:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [c]
+    detect_counts:
+      - open_1: 0
+      - open_2: 0
+      - open_3: 1
+      - open_4: 0
+      - open_5: 1
+      - open_6: 1
+      - open_7: 1
+      - open_8: 0
+      - open_9: 1
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_ab:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [a, b]
+    detect_counts:
+      - open_1: 1
+      - open_2: 1
+      - open_3: 0
+      - open_4: 1
+      - open_5: 1
+      - open_6: 1
+      - open_7: 1
+      - open_8: 1
+      - open_9: 1
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_bc:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [b, c]
+    detect_counts:
+      - open_1: 0
+      - open_2: 1
+      - open_3: 1
+      - open_4: 1
+      - open_5: 1
+      - open_6: 1
+      - open_7: 1
+      - open_8: 1
+      - open_9: 1
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_abc:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [a, b, c]
+    detect_counts:
+      - open_1: 1
+      - open_2: 1
+      - open_3: 1
+      - open_4: 1
+      - open_5: 1
+      - open_6: 1
+      - open_7: 1
+      - open_8: 1
+      - open_9: 1
+      - open_10: 1
+      - open_11: 0
+      - open_12: 0
+      - open_13: 0
+
+  run_tags_d:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/tagged_rules.yaml
+    trace_file: trace_files/open-multiple-files.scap
+    run_tags: [d]
+    detect_counts:
+      - open_1: 0
+      - open_2: 0
+      - open_3: 0
+      - open_4: 0
+      - open_5: 0
+      - open_6: 0
+      - open_7: 0
+      - open_8: 0
+      - open_9: 0
+      - open_10: 0
+      - open_11: 1
+      - open_12: 0
+      - open_13: 0
--- a/test/falco_tests.yaml.in
+++ b/test/falco_tests.yaml.in
@@ -1,202 +0,0 @@
-trace_files: !mux
-  builtin_rules_no_warnings:
-    detect: False
-    trace_file: trace_files/empty.scap
-    rules_warning: False
-
-  test_warnings:
-    detect: False
-    trace_file: trace_files/empty.scap
-    rules_file: rules/falco_rules_warnings.yaml
-    rules_warning:
-      - no_evttype
-      - evttype_not_equals
-      - leading_not
-      - not_equals_at_end
-      - not_at_end
-      - not_before_trailing_evttype
-      - not_equals_before_trailing_evttype
-      - not_equals_and_not
-      - not_equals_before_in
-      - not_before_in
-      - not_in_before_in
-      - leading_in_not_equals_before_evttype
-      - leading_in_not_equals_at_evttype
-      - not_with_evttypes
-      - not_with_evttypes_addl
-      - not_equals_before_evttype
-      - not_equals_before_in_evttype
-      - not_before_evttype
-      - not_before_evttype_using_in
-    rules_events:
-      - no_warnings: [execve]
-      - no_evttype: [all]
-      - evttype_not_equals: [all]
-      - leading_not: [all]
-      - not_equals_after_evttype: [execve]
-      - not_after_evttype: [execve]
-      - leading_trailing_evttypes: [execve,open]
-      - leading_multtrailing_evttypes: [connect,execve,open]
-      - leading_multtrailing_evttypes_using_in: [connect,execve,open]
-      - not_equals_at_end: [all]
-      - not_at_end: [all]
-      - not_before_trailing_evttype: [all]
-      - not_equals_before_trailing_evttype: [all]
-      - not_equals_and_not: [all]
-      - not_equals_before_in: [all]
-      - not_before_in: [all]
-      - not_in_before_in: [all]
-      - evttype_in: [execve,open]
-      - evttype_in_plus_trailing: [connect,execve,open]
-      - leading_in_not_equals_before_evttype: [all]
-      - leading_in_not_equals_at_evttype: [all]
-      - not_with_evttypes: [all]
-      - not_with_evttypes_addl: [all]
-      - not_equals_before_evttype: [all]
-      - not_equals_before_in_evttype: [all]
-      - not_before_evttype: [all]
-      - not_before_evttype_using_in: [all]
-      - repeated_evttypes: [open]
-      - repeated_evttypes_with_in: [open]
-      - repeated_evttypes_with_separate_in: [open]
-      - repeated_evttypes_with_mix: [open]
-
-  rule_names_with_spaces:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/rule_names_with_spaces.yaml
-    trace_file: trace_files/cat_write.scap
-
-  multiple_rules_first_empty:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/empty_rules.yaml
-      - rules/single_rule.yaml
-    trace_file: trace_files/cat_write.scap
-
-  multiple_rules_last_empty:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/empty_rules.yaml
-    trace_file: trace_files/cat_write.scap
-
-  multiple_rules:
-    detect: True
-    detect_level:
-      - WARNING
-      - INFO
-      - ERROR
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/double_rule.yaml
-    trace_file: trace_files/cat_write.scap
-
-  multiple_rules_overriding:
-    detect: False
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/override_rule.yaml
-    trace_file: trace_files/cat_write.scap
-
-  macro_overriding:
-    detect: False
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/override_macro.yaml
-    trace_file: trace_files/cat_write.scap
-
-  list_overriding:
-    detect: False
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/override_list.yaml
-    trace_file: trace_files/cat_write.scap
-
-  nested_list_overriding:
-    detect: False
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/override_nested_list.yaml
-    trace_file: trace_files/cat_write.scap
-
-  invalid_rule_output:
-    exit_status: 1
-    stderr_contains: "Runtime error: Error loading rules:.* Invalid output format 'An open was seen %not_a_real_field': 'invalid formatting token not_a_real_field'. Exiting."
-    rules_file:
-      - rules/invalid_rule_output.yaml
-    trace_file: trace_files/cat_write.scap
-
-  disabled_rules:
-    detect: False
-    rules_file:
-      - rules/empty_rules.yaml
-      - rules/single_rule.yaml
-    disabled_rules:
-      - open_from_cat
-    trace_file: trace_files/cat_write.scap
-
-  disabled_rules_using_regex:
-    detect: False
-    rules_file:
-      - rules/empty_rules.yaml
-      - rules/single_rule.yaml
-    disabled_rules:
-      - "open.*"
-    trace_file: trace_files/cat_write.scap
-
-  disabled_rules_using_enabled_flag:
-    detect: False
-    rules_file:
-      - rules/single_rule_enabled_flag.yaml
-    trace_file: trace_files/cat_write.scap
-
-  null_output_field:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/null_output_field.yaml
-    trace_file: trace_files/cat_write.scap
-    stdout_contains: "Warning An open was seen .cport=<NA> command=cat /dev/null."
-
-  file_output:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/single_rule.yaml
-    conf_file: confs/file_output.yaml
-    trace_file: trace_files/cat_write.scap
-    outputs:
-      - /tmp/falco_outputs/file_output.txt: Warning An open was seen
-
-  program_output:
-    detect: True
-    detect_level: WARNING
-    rules_file:
-      - rules/single_rule.yaml
-    conf_file: confs/program_output.yaml
-    trace_file: trace_files/cat_write.scap
-    outputs:
-      - /tmp/falco_outputs/program_output.txt: Warning An open was seen
-
-  detect_counts:
-    detect: True
-    detect_level: WARNING
-    trace_file: traces-positive/falco-event-generator.scap
-    detect_counts:
-      - "Write below binary dir": 1
-      - "Read sensitive file untrusted": 3
-      - "Run shell in container": 1
-      - "Write below rpm database": 1
-      - "Write below etc": 1
-      - "System procs network activity": 1
-      - "Mkdir binary dirs": 1
-      - "System user interactive": 1
-      - "DB program spawned process": 1
-      - "Non sudo setuid": 1
-      - "Create files below dev": 1
-      - "Modify binary dirs": 2
-      - "Change thread namespace": 2
--- a/test/falco_traces.yaml.in
+++ b/test/falco_traces.yaml.in
@@ -0,0 +1,203 @@
+has_json_output: !mux
+  yes:
+    json_output: True
+  no:
+    json_output: False
+
+traces: !mux
+  change-thread-namespace:
+    trace_file: traces-positive/change-thread-namespace.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Change thread namespace": 2
+
+  container-privileged:
+    trace_file: traces-positive/container-privileged.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "File Open by Privileged Container": 19
+
+  container-sensitive-mount:
+    trace_file: traces-positive/container-sensitive-mount.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "Sensitive Mount by Container": 19
+
+  create-files-below-dev:
+    trace_file: traces-positive/create-files-below-dev.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Create files below dev": 1
+
+  db-program-spawned-process:
+    trace_file: traces-positive/db-program-spawned-process.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "DB program spawned process": 1
+
+  falco-event-generator:
+    trace_file: traces-positive/falco-event-generator.scap
+    detect: True
+    detect_level: [ERROR, WARNING, INFO, NOTICE]
+    detect_counts:
+      - "Write below binary dir": 1
+      - "Read sensitive file untrusted": 3
+      - "Run shell in container": 1
+      - "Write below rpm database": 1
+      - "Write below etc": 1
+      - "System procs network activity": 1
+      - "Mkdir binary dirs": 1
+      - "System user interactive": 1
+      - "DB program spawned process": 1
+      - "Non sudo setuid": 1
+      - "Create files below dev": 1
+      - "Modify binary dirs": 2
+      - "Change thread namespace": 2
+
+  installer-fbash-manages-service:
+    trace_file: traces-info/installer-fbash-manages-service.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "Installer bash manages service": 4
+
+  installer-bash-non-https-connection:
+    trace_file: traces-positive/installer-bash-non-https-connection.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Installer bash non https connection": 1
+
+  installer-fbash-runs-pkgmgmt:
+    trace_file: traces-info/installer-fbash-runs-pkgmgmt.scap
+    detect: True
+    detect_level: [NOTICE, INFO]
+    detect_counts:
+      - "Installer bash runs pkgmgmt program": 4
+      - "Installer bash non https connection": 4
+
+  installer-bash-starts-network-server:
+    trace_file: traces-positive/installer-bash-starts-network-server.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Installer bash starts network server": 2
+      - "Installer bash non https connection": 3
+
+  installer-bash-starts-session:
+    trace_file: traces-positive/installer-bash-starts-session.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Installer bash starts session": 1
+      - "Installer bash non https connection": 3
+
+  mkdir-binary-dirs:
+    trace_file: traces-positive/mkdir-binary-dirs.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Mkdir binary dirs": 1
+
+  modify-binary-dirs:
+    trace_file: traces-positive/modify-binary-dirs.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Modify binary dirs": 1
+
+  modify-package-repo-list-installer:
+    trace_file: traces-info/modify-package-repo-list-installer.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "Write below etc in installer": 1
+
+  non-sudo-setuid:
+    trace_file: traces-positive/non-sudo-setuid.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Non sudo setuid": 1
+
+  read-sensitive-file-after-startup:
+    trace_file: traces-positive/read-sensitive-file-after-startup.scap
+    detect: True
+    detect_level: WARNING
+    detect_counts:
+      - "Read sensitive file untrusted": 1
+
+  read-sensitive-file-untrusted:
+    trace_file: traces-positive/read-sensitive-file-untrusted.scap
+    detect: True
+    detect_level: WARNING
+    detect_counts:
+      - "Read sensitive file untrusted": 1
+
+  run-shell-untrusted:
+    trace_file: traces-positive/run-shell-untrusted.scap
+    detect: True
+    detect_level: DEBUG
+    detect_counts:
+      - "Run shell untrusted": 1
+
+  shell-in-container:
+    trace_file: traces-positive/shell-in-container.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "Run shell in container": 1
+
+  system-binaries-network-activity:
+    trace_file: traces-positive/system-binaries-network-activity.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "System procs network activity": 1
+
+  system-user-interactive:
+    trace_file: traces-positive/system-user-interactive.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "System user interactive": 1
+
+  user-mgmt-binaries:
+    trace_file: traces-positive/user-mgmt-binaries.scap
+    detect: True
+    detect_level: NOTICE
+    detect_counts:
+      - "User mgmt binaries": 1
+
+  write-binary-dir:
+    trace_file: traces-positive/write-binary-dir.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Write below binary dir": 4
+
+  write-etc:
+    trace_file: traces-positive/write-etc.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Write below etc": 1
+
+  write-etc-installer:
+    trace_file: traces-info/write-etc-installer.scap
+    detect: True
+    detect_level: INFO
+    detect_counts:
+      - "Write below etc in installer": 1
+
+  write-rpm-database:
+    trace_file: traces-positive/write-rpm-database.scap
+    detect: True
+    detect_level: ERROR
+    detect_counts:
+      - "Write below rpm database": 1
--- a/test/rules/tagged_rules.yaml
+++ b/test/rules/tagged_rules.yaml
@@ -0,0 +1,93 @@
+- macro: open_read
+  condition: (evt.type=open or evt.type=openat) and evt.is_open_read=true and fd.typechar='f'
+
+- rule: open_1
+  desc: open one
+  condition: open_read and fd.name=/tmp/file-1
+  output: Open one (file=%fd.name)
+  priority: WARNING
+  tags: [a]
+
+- rule: open_2
+  desc: open two
+  condition: open_read and fd.name=/tmp/file-2
+  output: Open two (file=%fd.name)
+  priority: WARNING
+  tags: [b]
+
+- rule: open_3
+  desc: open three
+  condition: open_read and fd.name=/tmp/file-3
+  output: Open three (file=%fd.name)
+  priority: WARNING
+  tags: [c]
+
+- rule: open_4
+  desc: open four
+  condition: open_read and fd.name=/tmp/file-4
+  output: Open four (file=%fd.name)
+  priority: WARNING
+  tags: [a, b]
+
+- rule: open_5
+  desc: open file
+  condition: open_read and fd.name=/tmp/file-5
+  output: Open file (file=%fd.name)
+  priority: WARNING
+  tags: [a, c]
+
+- rule: open_6
+  desc: open six
+  condition: open_read and fd.name=/tmp/file-6
+  output: Open six (file=%fd.name)
+  priority: WARNING
+  tags: [b, c]
+
+- rule: open_7
+  desc: open seven
+  condition: open_read and fd.name=/tmp/file-7
+  output: Open seven (file=%fd.name)
+  priority: WARNING
+  tags: [a, b, c]
+
+- rule: open_8
+  desc: open eight
+  condition: open_read and fd.name=/tmp/file-8
+  output: Open eight (file=%fd.name)
+  priority: WARNING
+  tags: [b, a]
+
+- rule: open_9
+  desc: open nine
+  condition: open_read and fd.name=/tmp/file-9
+  output: Open nine (file=%fd.name)
+  priority: WARNING
+  tags: [c, a]
+
+- rule: open_10
+  desc: open ten
+  condition: open_read and fd.name=/tmp/file-10
+  output: Open ten (file=%fd.name)
+  priority: WARNING
+  tags: [b, c, a]
+
+- rule: open_11
+  desc: open eleven
+  condition: open_read and fd.name=/tmp/file-11
+  output: Open eleven (file=%fd.name)
+  priority: WARNING
+  tags: [d]
+
+- rule: open_12
+  desc: open twelve
+  condition: open_read and fd.name=/tmp/file-12
+  output: Open twelve (file=%fd.name)
+  priority: WARNING
+  tags: []
+
+- rule: open_13
+  desc: open thirteen
+  condition: open_read and fd.name=/tmp/file-13
+  output: Open thirteen (file=%fd.name)
+  priority: WARNING
+
--- a/test/run_regression_tests.sh
+++ b/test/run_regression_tests.sh
@@ -2,7 +2,6 @@

 SCRIPT=$(readlink -f $0)
 SCRIPTDIR=$(dirname $SCRIPT)
-MULT_FILE=$SCRIPTDIR/falco_tests.yaml
 BRANCH=$1

 function download_trace_files() {
@@ -19,56 +18,59 @@ function prepare_multiplex_fileset() {

    dir=$1
    detect=$2
-    detect_level=$3
-    json_output=$4

    for trace in $SCRIPTDIR/$dir/*.scap ; do
 	[ -e "$trace" ] || continue
 	NAME=`basename $trace .scap`
-	cat << EOF >> $MULT_FILE
-  $NAME-detect-$detect-json-$json_output:
+
+	# falco_traces.yaml might already have an entry for this trace
+	# file, with specific detection levels and counts. If so, skip
+	# it. Otherwise, add a generic entry showing whether or not to
+	# detect anything.
+	grep -q "$NAME:" $SCRIPTDIR/falco_traces.yaml && continue
+
+	cat << EOF >> $SCRIPTDIR/falco_traces.yaml
+  $NAME:
    detect: $detect
-    detect_level: $detect_level
+    detect_level: WARNING
    trace_file: $trace
-    json_output: $json_output
 EOF
    done
 }

 function prepare_multiplex_file() {
-    cp $SCRIPTDIR/falco_tests.yaml.in $MULT_FILE
+    cp $SCRIPTDIR/falco_traces.yaml.in $SCRIPTDIR/falco_traces.yaml

-    prepare_multiplex_fileset traces-positive True WARNING False
-    prepare_multiplex_fileset traces-negative False WARNING True
-    prepare_multiplex_fileset traces-info True INFO False
+    prepare_multiplex_fileset traces-positive True
+    prepare_multiplex_fileset traces-negative False
+    prepare_multiplex_fileset traces-info True

-    prepare_multiplex_fileset traces-positive True WARNING True
-    prepare_multiplex_fileset traces-info True INFO True
-
-    echo "Contents of $MULT_FILE:"
-    cat $MULT_FILE
+    echo "Contents of $SCRIPTDIR/falco_traces.yaml:"
+    cat $SCRIPTDIR/falco_traces.yaml
 }

-function run_tests() {
-    rm -rf /tmp/falco_outputs
-    mkdir /tmp/falco_outputs
-    CMD="avocado run --multiplex $MULT_FILE --job-results-dir $SCRIPTDIR/job-results -- $SCRIPTDIR/falco_test.py"
-    echo "Running: $CMD"
-    $CMD
-    TEST_RC=$?
-}
-
-
 function print_test_failure_details() {
    echo "Showing full job logs for any tests that failed:"
    jq '.tests[] | select(.status != "PASS") | .logfile' $SCRIPTDIR/job-results/latest/results.json  | xargs cat
 }

+function run_tests() {
+    rm -rf /tmp/falco_outputs
+    mkdir /tmp/falco_outputs
+    TEST_RC=0
+    for mult in $SCRIPTDIR/falco_traces.yaml $SCRIPTDIR/falco_tests.yaml; do
+	CMD="avocado run --multiplex $mult --job-results-dir $SCRIPTDIR/job-results -- $SCRIPTDIR/falco_test.py"
+	echo "Running: $CMD"
+	$CMD
+	RC=$?
+	TEST_RC=$((TEST_RC+$RC))
+	if [ $RC -ne 0 ]; then
+	    print_test_failure_details
+	fi
+    done
+}
+
 download_trace_files
 prepare_multiplex_file
 run_tests
-if [ $TEST_RC -ne 0 ]; then
-   print_test_failure_details
-fi
-
 exit $TEST_RC
--- a/test/trace_files/open-multiple-files.scap
+++ b/test/trace_files/open-multiple-files.scap
--- a/userspace/engine/falco_common.cpp
+++ b/userspace/engine/falco_common.cpp
@@ -24,6 +24,10 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 falco_common::falco_common()
 {
 	m_ls = lua_open();
+	if(!m_ls)
+	{
+		throw falco_exception("Cannot open lua");
+	}
 	luaL_openlibs(m_ls);
 }

--- a/userspace/engine/falco_engine.cpp
+++ b/userspace/engine/falco_engine.cpp
@@ -40,7 +40,8 @@ string lua_print_stats = "print_stats";
 using namespace std;

 falco_engine::falco_engine(bool seed_rng)
-	: m_rules(NULL), m_sampling_ratio(1), m_sampling_multiplier(0),
+	: m_rules(NULL), m_next_ruleset_id(0),
+	  m_sampling_ratio(1), m_sampling_multiplier(0),
 	  m_replace_container_info(false)
 {
 	luaopen_lpeg(m_ls);
@@ -55,6 +56,8 @@ falco_engine::falco_engine(bool seed_rng)
 	{
 		srandom((unsigned) getpid());
 	}
+
+	m_default_ruleset_id = find_ruleset_id(m_default_ruleset);
 }

 falco_engine::~falco_engine()
@@ -107,20 +110,52 @@ void falco_engine::load_rules_file(const string &rules_filename, bool verbose, b
 	load_rules(rules_content, verbose, all_events);
 }

-void falco_engine::enable_rule(string &pattern, bool enabled)
+void falco_engine::enable_rule(const string &pattern, bool enabled, const string &ruleset)
 {
-	m_evttype_filter->enable(pattern, enabled);
+	uint16_t ruleset_id = find_ruleset_id(ruleset);
+
+	m_evttype_filter->enable(pattern, enabled, ruleset_id);
 }

-unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
+void falco_engine::enable_rule(const string &pattern, bool enabled)
 {
+	enable_rule(pattern, enabled, m_default_ruleset);
+}

+void falco_engine::enable_rule_by_tag(const set<string> &tags, bool enabled, const string &ruleset)
+{
+	uint16_t ruleset_id = find_ruleset_id(ruleset);
+
+	m_evttype_filter->enable_tags(tags, enabled, ruleset_id);
+}
+
+void falco_engine::enable_rule_by_tag(const set<string> &tags, bool enabled)
+{
+	enable_rule_by_tag(tags, enabled, m_default_ruleset);
+}
+
+uint16_t falco_engine::find_ruleset_id(const std::string &ruleset)
+{
+	auto it = m_known_rulesets.lower_bound(ruleset);
+
+	if(it == m_known_rulesets.end() ||
+	   it->first != ruleset)
+	{
+		it = m_known_rulesets.emplace_hint(it,
+						   std::make_pair(ruleset, m_next_ruleset_id++));
+	}
+
+	return it->second;
+}
+
+unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev, uint16_t ruleset_id)
+{
 	if(should_drop_evt())
 	{
 		return unique_ptr<struct rule_result>();
 	}

-	if(!m_evttype_filter->run(ev))
+	if(!m_evttype_filter->run(ev, ruleset_id))
 	{
 		return unique_ptr<struct rule_result>();
 	}
@@ -155,6 +190,11 @@ unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
 	return res;
 }

+unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
+{
+	return process_event(ev, m_default_ruleset_id);
+}
+
 void falco_engine::describe_rule(string *rule)
 {
 	return m_rules->describe_rule(rule);
@@ -182,10 +222,11 @@ void falco_engine::print_stats()
 }

 void falco_engine::add_evttype_filter(string &rule,
-				      list<uint32_t> &evttypes,
+				      set<uint32_t> &evttypes,
+				      set<string> &tags,
 				      sinsp_filter* filter)
 {
-	m_evttype_filter->add(rule, evttypes, filter);
+	m_evttype_filter->add(rule, evttypes, tags, filter);
 }

 void falco_engine::clear_filters()
--- a/userspace/engine/falco_engine.h
+++ b/userspace/engine/falco_engine.h
@@ -20,6 +20,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #include <string>
 #include <memory>
+#include <set>

 #include "sinsp.h"
 #include "filter.h"
@@ -47,9 +48,24 @@ public:
 	void load_rules(const std::string &rules_content, bool verbose, bool all_events);

 	//
-	// Enable/Disable any rules matching the provided pattern (regex).
+	// Enable/Disable any rules matching the provided pattern
+	// (regex). When provided, enable/disable these rules in the
+	// context of the provided ruleset. The ruleset (id) can later
+	// be passed as an argument to process_event(). This allows
+	// for different sets of rules being active at once.
 	//
-	void enable_rule(std::string &pattern, bool enabled);
+	void enable_rule(const std::string &pattern, bool enabled, const std::string &ruleset);
+
+	// Wrapper that assumes the default ruleset
+	void enable_rule(const std::string &pattern, bool enabled);
+
+	//
+	// Enable/Disable any rules with any of the provided tags (set, exact matches only)
+	//
+	void enable_rule_by_tag(const std::set<std::string> &tags, bool enabled, const std::string &ruleset);
+
+	// Wrapper that assumes the default ruleset
+	void enable_rule_by_tag(const std::set<std::string> &tags, bool enabled);

 	struct rule_result {
 		sinsp_evt *evt;
@@ -58,12 +74,30 @@ public:
 		std::string format;
 	};

+	//
+	// Return the ruleset id corresponding to this ruleset name,
+	// creating a new one if necessary. If you provide any ruleset
+	// to enable_rule/enable_rule_by_tag(), you should look up the
+	// ruleset id and pass it to process_event().
+	//
+	uint16_t find_ruleset_id(const std::string &ruleset);
+
 	//
 	// Given an event, check it against the set of rules in the
 	// engine and if a matching rule is found, return details on
 	// the rule that matched. If no rule matched, returns NULL.
 	//
-	// the reutrned rule_result is allocated and must be delete()d.
+	// When ruleset_id is provided, use the enabled/disabled status
+	// associated with the provided ruleset. This is only useful
+	// when you have previously called enable_rule/enable_rule_by_tag
+	// with a ruleset string.
+	//
+	// the returned rule_result is allocated and must be delete()d.
+	std::unique_ptr<rule_result> process_event(sinsp_evt *ev, uint16_t ruleset_id);
+
+	//
+	// Wrapper assuming the default ruleset
+	//
 	std::unique_ptr<rule_result> process_event(sinsp_evt *ev);

 	//
@@ -78,11 +112,12 @@ public:
 	void print_stats();

 	//
-	// Add a filter, which is related to the specified list of
+	// Add a filter, which is related to the specified set of
 	// event types, to the engine.
 	//
 	void add_evttype_filter(std::string &rule,
-				list<uint32_t> &evttypes,
+				std::set<uint32_t> &evttypes,
+				std::set<std::string> &tags,
 				sinsp_filter* filter);

 	// Clear all existing filters.
@@ -120,6 +155,8 @@ private:
 	inline bool should_drop_evt();

 	falco_rules *m_rules;
+	uint16_t m_next_ruleset_id;
+	std::map<string, uint16_t> m_known_rulesets;
 	std::unique_ptr<sinsp_evttype_filter> m_evttype_filter;

 	//
@@ -146,6 +183,8 @@ private:
 	double m_sampling_multiplier;

 	std::string m_lua_main_filename = "rule_loader.lua";
+	std::string m_default_ruleset = "falco-default-ruleset";
+	uint32_t m_default_ruleset_id;

 	std::string m_extra;
 	bool m_replace_container_info;
--- a/userspace/engine/formats.cpp
+++ b/userspace/engine/formats.cpp
@@ -24,12 +24,14 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.


 sinsp* falco_formats::s_inspector = NULL;
-bool s_json_output = false;
+bool falco_formats::s_json_output = false;
+sinsp_evt_formatter_cache *falco_formats::s_formatters = NULL;

 const static struct luaL_reg ll_falco [] =
 {
 	{"formatter", &falco_formats::formatter},
 	{"free_formatter", &falco_formats::free_formatter},
+	{"free_formatters", &falco_formats::free_formatters},
 	{"format_event", &falco_formats::format_event},
 	{NULL,NULL}
 };
@@ -38,6 +40,10 @@ void falco_formats::init(sinsp* inspector, lua_State *ls, bool json_output)
 {
 	s_inspector = inspector;
 	s_json_output = json_output;
+	if(!s_formatters)
+	{
+		s_formatters = new sinsp_evt_formatter_cache(s_inspector);
+	}

 	luaL_openlib(ls, "formats", ll_falco, 0);
 }
@@ -73,22 +79,43 @@ int falco_formats::free_formatter(lua_State *ls)
 	return 0;
 }

+int falco_formats::free_formatters(lua_State *ls)
+{
+	if(s_formatters)
+	{
+		delete(s_formatters);
+		s_formatters = NULL;
+	}
+	return 0;
+}
+
 int falco_formats::format_event (lua_State *ls)
 {
 	string line;

-	if (!lua_islightuserdata(ls, -1) ||
+	if (!lua_isstring(ls, -1) ||
 	    !lua_isstring(ls, -2) ||
 	    !lua_isstring(ls, -3) ||
 	    !lua_islightuserdata(ls, -4)) {
-		throw falco_exception("Invalid arguments passed to format_event()\n");
+		lua_pushstring(ls, "Invalid arguments passed to format_event()");
+		lua_error(ls);
 	}
 	sinsp_evt* evt = (sinsp_evt*)lua_topointer(ls, 1);
 	const char *rule = (char *) lua_tostring(ls, 2);
 	const char *level = (char *) lua_tostring(ls, 3);
-	sinsp_evt_formatter* formatter = (sinsp_evt_formatter*)lua_topointer(ls, 4);
+	const char *format = (char *) lua_tostring(ls, 4);

-	formatter->tostring(evt, &line);
+	string sformat = format;
+
+	try {
+		s_formatters->tostring(evt, sformat, &line);
+	}
+	catch (sinsp_exception& e)
+	{
+		string err = "Invalid output format '" + sformat + "': '" + string(e.what()) + "'";
+		lua_pushstring(ls, err.c_str());
+		lua_error(ls);
+	}

 	// For JSON output, the formatter returned just the output
 	// string containing the format text and values. Use this to
--- a/userspace/engine/formats.h
+++ b/userspace/engine/formats.h
@@ -39,8 +39,13 @@ class falco_formats
 	// falco.free_formatter(formatter)
 	static int free_formatter(lua_State *ls);

+	// falco.free_formatters()
+	static int free_formatters(lua_State *ls);
+
 	// formatted_string = falco.format_event(evt, formatter)
 	static int format_event(lua_State *ls);

 	static sinsp* s_inspector;
+	static sinsp_evt_formatter_cache *s_formatters;
+	static bool s_json_output;
 };
--- a/userspace/engine/lua/rule_loader.lua
+++ b/userspace/engine/lua/rule_loader.lua
@@ -308,8 +308,12 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac

 	 install_filter(filter_ast.filter.value)

+	 if (v['tags'] == nil) then
+	    v['tags'] = {}
+	 end
+
 	 -- Pass the filter and event types back up
-	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes)
+	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes, v['tags'])

 	 -- Rule ASTs are merged together into one big AST, with "OR" between each
 	 -- rule.
--- a/userspace/engine/rules.cpp
+++ b/userspace/engine/rules.cpp
@@ -49,7 +49,8 @@ int falco_rules::clear_filters(lua_State *ls)
 {
 	if (! lua_islightuserdata(ls, -1))
 	{
-		throw falco_exception("Invalid arguments passed to clear_filters()\n");
+		lua_pushstring(ls, "Invalid arguments passed to clear_filters()");
+		lua_error(ls);
 	}

 	falco_rules *rules = (falco_rules *) lua_topointer(ls, -1);
@@ -65,42 +66,56 @@ void falco_rules::clear_filters()

 int falco_rules::add_filter(lua_State *ls)
 {
-	if (! lua_islightuserdata(ls, -3) ||
-	    ! lua_isstring(ls, -2) ||
+	if (! lua_islightuserdata(ls, -4) ||
+	    ! lua_isstring(ls, -3) ||
+	    ! lua_istable(ls, -2) ||
 	    ! lua_istable(ls, -1))
 	{
-		throw falco_exception("Invalid arguments passed to add_filter()\n");
+		lua_pushstring(ls, "Invalid arguments passed to add_filter()");
+		lua_error(ls);
 	}

-	falco_rules *rules = (falco_rules *) lua_topointer(ls, -3);
-	const char *rulec = lua_tostring(ls, -2);
+	falco_rules *rules = (falco_rules *) lua_topointer(ls, -4);
+	const char *rulec = lua_tostring(ls, -3);

-	list<uint32_t> evttypes;
+	set<uint32_t> evttypes;
+
+	lua_pushnil(ls);  /* first key */
+	while (lua_next(ls, -3) != 0) {
+                // key is at index -2, value is at index
+                // -1. We want the keys.
+		evttypes.insert(luaL_checknumber(ls, -2));
+
+		// Remove value, keep key for next iteration
+		lua_pop(ls, 1);
+	}
+
+	set<string> tags;

 	lua_pushnil(ls);  /* first key */
 	while (lua_next(ls, -2) != 0) {
                // key is at index -2, value is at index
                // -1. We want the keys.
-		evttypes.push_back(luaL_checknumber(ls, -2));
+		tags.insert(lua_tostring(ls, -1));

 		// Remove value, keep key for next iteration
 		lua_pop(ls, 1);
 	}

 	std::string rule = rulec;
-	rules->add_filter(rule, evttypes);
+	rules->add_filter(rule, evttypes, tags);

 	return 0;
 }

-void falco_rules::add_filter(string &rule, list<uint32_t> &evttypes)
+void falco_rules::add_filter(string &rule, set<uint32_t> &evttypes, set<string> &tags)
 {
 	// While the current rule was being parsed, a sinsp_filter
 	// object was being populated by lua_parser. Grab that filter
 	// and pass it to the engine.
 	sinsp_filter *filter = m_lua_parser->get_filter(true);

-	m_engine->add_evttype_filter(rule, evttypes, filter);
+	m_engine->add_evttype_filter(rule, evttypes, tags, filter);
 }

 int falco_rules::enable_rule(lua_State *ls)
@@ -109,7 +124,8 @@ int falco_rules::enable_rule(lua_State *ls)
 	    ! lua_isstring(ls, -2) ||
 	    ! lua_isnumber(ls, -1))
 	{
-		throw falco_exception("Invalid arguments passed to enable_rule()\n");
+		lua_pushstring(ls, "Invalid arguments passed to enable_rule()");
+		lua_error(ls);
 	}

 	falco_rules *rules = (falco_rules *) lua_topointer(ls, -3);
--- a/userspace/engine/rules.h
+++ b/userspace/engine/rules.h
@@ -18,7 +18,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #pragma once

-#include <list>
+#include <set>

 #include "sinsp.h"

@@ -42,7 +42,7 @@ class falco_rules

 private:
 	void clear_filters();
-	void add_filter(string &rule, list<uint32_t> &evttypes);
+	void add_filter(string &rule, std::set<uint32_t> &evttypes, std::set<string> &tags);
 	void enable_rule(string &rule, bool enabled);

 	lua_parser* m_lua_parser;
--- a/userspace/engine/token_bucket.cpp
+++ b/userspace/engine/token_bucket.cpp
@@ -31,20 +31,30 @@ token_bucket::~token_bucket()
 {
 }

-void token_bucket::init(uint32_t rate, uint32_t max_tokens)
+void token_bucket::init(double rate, double max_tokens, uint64_t now)
 {
 	m_rate = rate;
 	m_max_tokens = max_tokens;
 	m_tokens = max_tokens;
-	m_last_seen = sinsp_utils::get_current_time_ns();
+
+	if(now == 0)
+	{
+		now = sinsp_utils::get_current_time_ns();
+	}
+
+	m_last_seen = now;
 }

 bool token_bucket::claim()
 {
-	// Determine the number of tokens gained. Delta between
-	// last_seen and now, divided by the rate.
 	uint64_t now = sinsp_utils::get_current_time_ns();
-	uint64_t tokens_gained = (now - m_last_seen) / (m_rate * 1000000000);
+
+	return claim(1, now);
+}
+
+bool token_bucket::claim(double tokens, uint64_t now)
+{
+	double tokens_gained = m_rate * ((now - m_last_seen) / (1000000000.0));
 	m_last_seen = now;

 	m_tokens += tokens_gained;
@@ -58,14 +68,24 @@ bool token_bucket::claim()
 	}

 	//
-	// If tokens is < 1, can't claim.
+	// If m_tokens is < tokens, can't claim.
 	//
-	if(m_tokens < 1)
+	if(m_tokens < tokens)
 	{
 		return false;
 	}

-	m_tokens--;
+	m_tokens -= tokens;

 	return true;
 }
+
+double token_bucket::get_tokens()
+{
+	return m_tokens;
+}
+
+uint64_t token_bucket::get_last_seen()
+{
+	return m_last_seen;
+}
--- a/userspace/engine/token_bucket.h
+++ b/userspace/engine/token_bucket.h
@@ -31,30 +31,42 @@ public:
 	//
 	// Initialize the token bucket and start accumulating tokens
 	//
-	void init(uint32_t rate, uint32_t max_tokens);
+	void init(double rate, double max_tokens, uint64_t now = 0);

 	//
-	// Returns true if a token can be claimed. Also updates
-	// internal metrics.
+	// Try to claim tokens tokens from the token bucket, using a
+	// timestamp of now. Returns true if the tokens could be
+	// claimed. Also updates internal metrics.
 	//
+	bool claim(double tokens, uint64_t now);
+
+	// Simpler version of claim that claims a single token and
+	// uses the current time for now
 	bool claim();
+
+	// Return the current number of tokens available
+	double get_tokens();
+
+	// Return the last time someone tried to claim a token.
+	uint64_t get_last_seen();
+
 private:

 	//
 	// The number of tokens generated per second.
 	//
-	uint64_t m_rate;
+	double m_rate;

 	//
 	// The maximum number of tokens that can be banked for future
 	// claim()s.
 	//
-	uint64_t m_max_tokens;
+	double m_max_tokens;

 	//
 	// The current number of tokens
 	//
-	uint64_t m_tokens;
+	double m_tokens;

 	//
 	// The last time claim() was called (or the object was created).
--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -53,6 +53,7 @@ static void signal_callback(int signal)
 static void usage()
 {
    printf(
+	   "falco version " FALCO_VERSION "\n"
 	   "Usage: falco [options]\n\n"
 	   "Options:\n"
 	   " -h, --help                    Print this page\n"
@@ -60,6 +61,7 @@ static void usage()
 	   " -A                            Monitor all events, including those with EF_DROP_FALCO flag.\n"
 	   " -d, --daemon                  Run as a daemon\n"
 	   " -D <pattern>                  Disable any rules matching the regex <pattern>. Can be specified multiple times.\n"
+	   "                               Can not be specified with -t.\n"
           " -e <events_file>              Read the events from <events_file> (in .scap format) instead of tapping into live.\n"
 	   " -k <url>, --k8s-api=<url>\n"
 	   "                               Enable Kubernetes support by connecting to the API server\n"
@@ -85,6 +87,7 @@ static void usage()
 	   "                               Marathon url is optional and defaults to Mesos address, port 8080.\n"
 	   "                               The API servers can also be specified via the environment variable\n"
 	   "                               FALCO_MESOS_API.\n"
+	   " -M <num_seconds>              Stop collecting after <num_seconds> reached.\n"
 	   " -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.\n"
 	   "                               <key> can be a two-part <key>.<subkey>\n"
 	   " -p <output_format>, --print=<output_format>\n"
@@ -100,7 +103,12 @@ static void usage()
 	   "                               Can be specified multiple times to read from multiple files.\n"
 	   " -s <stats_file>               If specified, write statistics related to falco's reading/processing of events\n"
 	   "                               to this file. (Only useful in live mode).\n"
+	   " -T <tag>                      Disable any rules with a tag=<tag>. Can be specified multiple times.\n"
+	   "                               Can not be specified with -t.\n"
+	   " -t <tag>                      Only run those rules with a tag=<tag>. Can be specified multiple times.\n"
+	   "                               Can not be specified with -T/-D.\n"
 	   " -v                            Verbose output.\n"
+           " --version                     Print version number.\n"
 	   "\n"
    );
 }
@@ -128,12 +136,14 @@ std::list<string> cmdline_options;
 uint64_t do_inspect(falco_engine *engine,
 		    falco_outputs *outputs,
 		    sinsp* inspector,
+		    uint64_t duration_to_tot_ns,
 		    string &stats_filename)
 {
 	uint64_t num_evts = 0;
 	int32_t res;
 	sinsp_evt* ev;
 	StatsFileWriter writer;
+	uint64_t duration_start = 0;

 	if (stats_filename != "")
 	{
@@ -177,6 +187,17 @@ uint64_t do_inspect(falco_engine *engine,
 			throw sinsp_exception(inspector->getlasterr().c_str());
 		}

+		if (duration_start == 0)
+		{
+			duration_start = ev->get_ts();
+		} else if(duration_to_tot_ns > 0)
+		{
+			if(ev->get_ts() - duration_start >= duration_to_tot_ns)
+			{
+				break;
+			}
+		}
+
 		if(!inspector->is_debug_enabled() &&
 			ev->get_category() & EC_INTERNAL)
 		{
@@ -227,6 +248,7 @@ int falco_init(int argc, char **argv)
 	string* mesos_api = 0;
 	string output_format = "";
 	bool replace_container_info = false;
+	int duration_to_tot = 0;

 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -250,6 +272,7 @@ int falco_init(int argc, char **argv)
 		{"option", required_argument, 0, 'o'},
 		{"print", required_argument, 0, 'p' },
 		{"pidfile", required_argument, 0, 'P' },
+		{"version", no_argument, 0, 0 },
 		{"writefile", required_argument, 0, 'w' },

 		{0, 0, 0, 0}
@@ -259,12 +282,15 @@ int falco_init(int argc, char **argv)
 	{
 		set<string> disabled_rule_patterns;
 		string pattern;
+		string all_rules = ".*";
+		set<string> disabled_rule_tags;
+		set<string> enabled_rule_tags;

 		//
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:k:K:Ll:m:o:P:p:r:s:vw:",
+                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:vw:",
                                        long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -305,6 +331,13 @@ int falco_init(int argc, char **argv)
 			case 'm':
 				mesos_api = new string(optarg);
 				break;
+			case 'M':
+				duration_to_tot = atoi(optarg);
+				if(duration_to_tot <= 0)
+				{
+					throw sinsp_exception(string("invalid duration") + optarg);
+				}
+				break;
 			case 'o':
 				cmdline_options.push_back(optarg);
 				break;
@@ -339,6 +372,12 @@ int falco_init(int argc, char **argv)
 			case 's':
 				stats_filename = optarg;
 				break;
+			case 'T':
+				disabled_rule_tags.insert(optarg);
+				break;
+			case 't':
+				enabled_rule_tags.insert(optarg);
+				break;
 			case 'v':
 				verbose = true;
 				break;
@@ -354,6 +393,13 @@ int falco_init(int argc, char **argv)

 		}

+		if(string(long_options[long_index].name) == "version")
+		{
+			printf("falco version %s\n", FALCO_VERSION);
+			return EXIT_SUCCESS;
+		}
+
+
 		inspector = new sinsp();
 		engine = new falco_engine();
 		engine->set_inspector(inspector);
@@ -421,12 +467,40 @@ int falco_init(int argc, char **argv)
 			falco_logger::log(LOG_INFO, "Parsed rules from file " + filename + "\n");
 		}

+		// You can't both disable and enable rules
+		if((disabled_rule_patterns.size() + disabled_rule_tags.size() > 0) &&
+		    enabled_rule_tags.size() > 0) {
+			throw std::invalid_argument("You can not specify both disabled (-D/-T) and enabled (-t) rules");
+		}
+
 		for (auto pattern : disabled_rule_patterns)
 		{
 			falco_logger::log(LOG_INFO, "Disabling rules matching pattern: " + pattern + "\n");
 			engine->enable_rule(pattern, false);
 		}

+		if(disabled_rule_tags.size() > 0)
+		{
+			for(auto tag : disabled_rule_tags)
+			{
+				falco_logger::log(LOG_INFO, "Disabling rules with tag: " + tag + "\n");
+			}
+			engine->enable_rule_by_tag(disabled_rule_tags, false);
+		}
+
+		if(enabled_rule_tags.size() > 0)
+		{
+
+			// Since we only want to enable specific
+			// rules, first disable all rules.
+			engine->enable_rule(all_rules, false);
+			for(auto tag : enabled_rule_tags)
+			{
+				falco_logger::log(LOG_INFO, "Enabling rules with tag: " + tag + "\n");
+			}
+			engine->enable_rule_by_tag(enabled_rule_tags, true);
+		}
+
 		outputs->init(config.m_json_output, config.m_notifications_rate, config.m_notifications_max_burst);

 		if(!all_events)
@@ -610,6 +684,7 @@ int falco_init(int argc, char **argv)
 		num_evts = do_inspect(engine,
 				      outputs,
 				      inspector,
+				      uint64_t(duration_to_tot*ONE_SECOND_IN_NS),
 				      stats_filename);

 		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;
--- a/userspace/falco/lua/output.lua
+++ b/userspace/falco/lua/output.lua
@@ -24,8 +24,6 @@ mod.levels = levels

 local outputs = {}

-local formatters = {}
-
 function mod.stdout(level, msg)
   print (msg)
 end
@@ -84,14 +82,8 @@ function output_event(event, rule, priority, format)
   end

   format = "*%evt.time: "..levels[level+1].." "..format
-   if formatters[rule] == nil then
-      formatter = formats.formatter(format)
-      formatters[rule] = formatter
-   else
-      formatter = formatters[rule]
-   end

-   msg = formats.format_event(event, rule, levels[level+1], formatter)
+   msg = formats.format_event(event, rule, levels[level+1], format)

   for index,o in ipairs(outputs) do
      o.output(level, msg, o.config)
@@ -99,11 +91,7 @@ function output_event(event, rule, priority, format)
 end

 function output_cleanup()
-   for rule, formatter in pairs(formatters) do
-      formats.free_formatter(formatter)
-   end
-
-   formatters = {}
+   formats.free_formatters()
 end

 function add_output(output_name, config)