Merge branch 'dev' into agent-master

Rule updates next (#293 )
* Let luajit spawn shells. * Start support for db mgmt programs Add support for db management programs that tend to spawn shells. Starting with two lists mysql_mgmt_binaries/postgres_mgmt_binaries which are combined into db_mgmt_binaries. db_mgmt_binaries is added to both shell spawning rules and the individual programs are removed. * Let apache beam spawn shells The program is "python pipeline.py" but it appears to be related to https://github.com/apache/beam/blob/master/sdks/python/apache_beam/pipeline.py. * Better support for dovecot Allow dovecot to setuid by adding to mail_binaries. Allow the program auth, when run by dovecot, to spawn shells. * Better support for plesk Create a list plesk_binaries and allow them to run shells. Also let them write to files below /etc/sw/keys. * Let strongswan spawn shells. Specifically the program starter. Using the full command line to be more specific. * Let proftpd modify files below /etc. * Let chef binaries write below /etc * Let mandb read sensitive files * Let specific phusion passenger binaries run shells The program is "my_init", which is fairly generic, so capture it by the full command line. * Make git-remote-http more permissive. * Let networkmanager modify /etc/resolv.conf specifically nm-dispatcher * Let hostid open network connections It might perform dns lookups as a part of resolving ip addresses. * Let uwsgi spawn shells * Add docker-runc-cur as a docker binary. truncated version of docker-runc-current. * Add rule for allowed containers New rule Launch Disallowed Container triggers when a container is started that does not match the macro allowed_containers. In the main falco rules file, this macro never matches, so it never triggers. However, in a second rules file the macro allowed_containers could be filled in with the specific images that match. * Also let foreman spawn shells Used by Red Hat Sattelite. * Let confluence run shells. Appears as java program, so look for the classpath. * Make allowed_containers macro more foolproof. In some cases, the container image might not be known/is NULL, so the comparison aganst "dummy-not-allowed-container-image" doesn't work. Replace this with proc.vpid=1, which is in the main rule Launch Disallowed Continer. Ensures it will only trigger when the allowed_containers macro is overridden. * Let tomcat spawn shells. It's java so you need to look at the classpath. * Let pip install software. * Add another yarn command line. * Let add-shell write to /etc/shells.tmp * Let more plesk binaries setuid. * Add imap-login as a mail binary. * Fix plesk writing keys macro Should be testing proc.name, not proc.cmdline. * Let screen read sensitive files. * Add more shell spawners. S99qualys-cloud is the init script, cfn-signal is cloudformation. * Exclude nologin from user mgmt programs. * Let programs run by locales.postins write to /etc It can run scripts like sed to modify files before writing the final file. * Let install4j java progs spawn shells. Again, searching by classpath. * Let some shell cmds be spawned outside containers We had a list known_container_shell_spawn_cmdlines that contained innocuous commandlines, but it only worked for containers. Split this list into container-specific and general commandlines, and add an exception for the general commandlines for the Run Shell Untrusted rule. * Add addl ruby-based passenger spawners Add a different way to identify ruby run by phusion passenger. * Allow bundle ruby cmds to be identififed by name In some cases, bundle runs ruby scripts by direct script name (foo.rb). Also allow that to spawn shells. * Let nginx spawn shells. * Skip setuid rules for containers. For now, entirely skip the setuid rule for containers. Will add back once I can find a way to check for unknown users. * Let PassengerWatchd run shells * Add additional foreman shells Let the direct parent also be scl when the ancestor is tfm-rake,tfm-ruby. * Add additional innocuous command lines. * Also let cron spawn shells in containers Seen when using things like phusion passenger. * Also let run-parts run cmp/cp for sensitive files Might be a case of a missing process but might also be legitimate. * Let erlexec spawn shells. * Add additional innocuous shell cmdlines. * Add suexec as a userexec binary. * Add imap/mailmng-core as mail binaries. Also split list across multiple lines. * Let perl spawn shells when run by cpanm * Let apache_control_ spawn shells * Let ics_start/stop running java spawn shells java is the direct parent, ics_start/stop are ancestors. * Let PassengerAgent setuid. It setuids to nobody. * Let multilog write below /etc if run by supervise * Let bwrap setuid A container setup utility. * Detect writes below /, /root New rule Write below root detects writes either directly below / or anywhere below /root. * Don't let shells directly open network connections In addition to system binaries, don't let shells directly open network connections. Bash has /dev/{tcp,udp} which allows direct connections. * Add additional sensitive mounts. Add additional sensitive mounts, including the docker socket, /, anywhere below /root, or anywhere below /etc. * Let pki-realm write below /etc/pki/realms Appears to be an ansible script. * Let sgdisk write below dev * Let debconf-show read sensitive files. * Additional case for build-related scripts. * Add additional mail binaries. * Let ruby running discourse spawn shells. * Let beam.smp and paster run shells * Temporarily undo shells opening net conns update At some customers, at container create time events are being lost, and for that reason programs spawned by the shell that perform network connections are being misattributed to the shell. * Make the actual sensitive files a list. Make the actual sensitive files used by the sensitive files macro a list so it can be easily extended. * Print mounts in Launch Sensitive Mount Container Add the full list of mounts to the output of Launch Sensitive Mount Container, so it's easy to see which sensitive mount was used. * Add container.image to container-related rules. Helps in diagnosis. * Add sw-engine-kv as a plesk binary. * Allow sa-update to read sensitive files SpamAssassin updater. * Add additional shell spawners. * Allow sumologic secureFiles to run user mgmt progs See https://help.sumologic.com/Send-Data/Installed-Collectors/05Reference-Information-for-Collector-Installation/08Enhanced-File-System-Security-for-Installed-Collectors. * Only consider full mounts of /etc as sensitive A legitimate case is k8s mounting /etc/kubernetes/ssl, which was matching /etc*. The glob matcher we have isn't a full regex so you can't exclude strings, only characters. * Let htpasswd write below /etc Part of nginx * Let pam-auth-update read sensitive files * Let hawkular-metric spawn shells. * Generalize jenkins scripts spawning shells Generalize jenkins_script_sh to jenkins_scripts and add additional cases. * Let php run by assemble spawn shells Better than globally letting php spawn shells. * Add additional setuid binaries. * Add additional package mgmt prog rhsmcertd-worke(r), red hat subscription manager * Add additional yarn cmdlines. * Let dmeventd write below etc. device mapper event daemon. * Let rhsmcertd-worke(r) spawn shells. * Let node spawn bitnami-related shells. * Add user allowed sensitive mounts New macro user_sensitive_mount_containers allows a second rules file to specify containers/images that can perform sensitive mounts. * Add start-stop-daemon as setuid program It has -g/-u args to change gid/uid. Also move some other single setuid programs to the list known_setuid_binaries. * Add additional shell spawners/cmdlines. * Let python running localstack spawn shells. * Add additional chef binaries. * Let fluentd spawn shells. * Don't consider unix_chkpwd to be a user mgmt prog It only checks passwords. * Get setuid for NULL user in container working Reorganize the unknown_user_in_container macro to get it working again in containers. Previously, it was being skipped entirely due to a problem with handling of unknown users, which get returned as NULL. The new macro is known_user_in_container, which tests the user.name against "N/A". It happens that if user.name is NULL, the comparison fails, so it has the same effect as if the string "N/A" were being returned. Any valid user name won't match the string "N/A", so known users will cause the macro to return true. The setuid rule needs an additional check for not container, so add that. * Add exceptions for Write below root Add lists of files/directories that are acceptable to write.
2026-03-20 03:32:09 +00:00 · 2017-10-27 14:31:48 +02:00 · 2017-10-25 14:39:53 -07:00 · 2017-10-19 14:38:57 -07:00 · 2017-10-19 11:03:45 +02:00 · 2017-10-18 13:34:19 -07:00
38 changed files with 1231 additions and 177 deletions
--- a/.travis.yml
+++ b/.travis.yml
@@ -26,13 +26,6 @@ script:
    - set -e
    - export CC="gcc-4.8"
    - export CXX="g++-4.8"
-    - wget https://s3.amazonaws.com/download.draios.com/dependencies/cmake-3.3.2.tar.gz
-    - tar -xzf cmake-3.3.2.tar.gz
-    - cd cmake-3.3.2
-    - ./bootstrap --prefix=/usr
-    - make
-    - sudo make install
-    - cd ..
    - mkdir build
    - cd build
    - cmake .. -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DDRAIOS_DEBUG_FLAGS="-D_DEBUG -DNDEBUG"
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,9 +2,58 @@

 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).

+## v0.8.1
+
+Released 2017-10-10
+
+### Bug Fixes
+
+* Fix packaging to specify correct built-in config file [[#288](https://github.com/draios/falco/pull/288)]
+
+## v0.8.0
+
+Released 2017-10-10
+
+**Important**: the location for falco's configuration file has moved from `/etc/falco.yaml` to `/etc/falco/falco.yaml`. The default rules file has moved from `/etc/falco_rules.yaml` to `/etc/falco/falco_rules.yaml`. In addition, 0.8.0 has added a _local_ ruls file to `/etc/falco/falco_rules.local.yaml`. See [the documentation](https://github.com/draios/falco/wiki/Falco-Default-and-Local-Rules-Files) for more details.
+
+### Major Changes
+
+* Add the ability to append one list to another list by setting an `append: true` attribute. [[#264](https://github.com/draios/falco/pull/264)]
+* Add the ability to append one macro/rule to another list by setting an `append: true` attribute. [[#277](https://github.com/draios/falco/pull/277)]
+* Ensure that falco rules/config files are preserved across package upgrades/removes if modified. [[#278](https://github.com/draios/falco/pull/278)]
+* Add the notion of a "local" rules file that should contain modifications to the default falco rules file. [[#278](https://github.com/draios/falco/pull/278)]
+* When using json output, separately include the individual templated fields in the json object. [[#282](https://github.com/draios/falco/pull/282)]
+* Add the ability to keep a file/program pipe handle open across rule notifications. [[#283](https://github.com/draios/falco/pull/283)]
+* New argument `-V` validates rules file and immediately exits. [[#286](https://github.com/draios/falco/pull/286)]
+
+### Minor Changes
+
+* Minor updates to falco example programs [[#248](https://github.com/draios/falco/pull/248)] [[#275](https://github.com/draios/falco/pull/275)]
+* Also validate macros at rule parse time. [[#257](https://github.com/draios/falco/pull/257)]
+* Minor README typo fixes [[#276](https://github.com/draios/falco/pull/276)]
+* Add a government CLA (contributor license agreement). [[#263](https://github.com/draios/falco/pull/263)]
+* Add ability to only run rules with a priority >= some threshold [[#281](https://github.com/draios/falco/pull/281)]
+* Add ability to make output channels unbuffered [[#285](https://github.com/draios/falco/pull/285)]
+
+### Bug Fixes
+
+* Fix installation of falco on OSX [[#252](https://github.com/draios/falco/pull/252)]
+* Fix a bug that caused the trailing whitespace of a quoted string to be accidentally removed [[#254](https://github.com/draios/falco/pull/254)]
+* When multiple sets of kernel headers are installed, find the one for the running kernel [[#260](https://github.com/draios/falco/pull/260)]
+* Allow pathnames in rule/macro conditions to contain '.' characters [[#262](https://github.com/draios/falco/pull/262)]
+* Fix a bug where a list named "foo" would be substituted even if it were a substring of a longer word like "my_foo" [[#258](https://github.com/draios/falco/pull/258)]
+* Remove extra trailing newlines from rule output strings [[#265](https://github.com/draios/falco/pull/265)]
+* Improve build pathnames to avoid relative paths when possible [[#284](https://github.com/draios/falco/pull/284)]
+
+### Rule Changes
+
+* Significant changes to default ruleset to address FPs. These changes resulted from hundreds of hours of use in actual customer environments. [[#247](https://github.com/draios/falco/pull/247)] [[#259](https://github.com/draios/falco/pull/259)]
+* Add official gitlab EE docker image to list of known shell spawning images. Thanks @dkerwin! [[#270](https://github.com/draios/falco/pull/270)]
+* Add keepalived to list of shell spawning binaries. Thanks @dkerwin! [[#269](https://github.com/draios/falco/pull/269)]
+
 ## v0.7.0

-Released 2016-05-30
+Released 2017-05-30

 ### Major Changes

@@ -27,7 +76,7 @@ None.

 ## v0.6.1

-Released 2016-05-15
+Released 2017-05-15

 ### Major Changes

@@ -47,7 +96,7 @@ None

 ## v0.6.0

-Released 2016-03-29
+Released 2017-03-29

 ### Major Changes

--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -7,7 +7,7 @@ if(NOT DEFINED FALCO_VERSION)
 endif()

 if(NOT DEFINED FALCO_ETC_DIR)
-	set(FALCO_ETC_DIR "/etc")
+	set(FALCO_ETC_DIR "/etc/falco")
 endif()

 if(NOT CMAKE_BUILD_TYPE)
@@ -78,7 +78,7 @@ else()
        set(ZLIB_INCLUDE "${ZLIB_SRC}")
        set(ZLIB_LIB "${ZLIB_SRC}/libz.a")
        ExternalProject_Add(zlib
-		URL "http://download.draios.com/dependencies/zlib-1.2.8.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/zlib-1.2.8.tar.gz"
 		URL_MD5 "44d667c142d7cda120332623eab69f40"
 		CONFIGURE_COMMAND "./configure"
 		BUILD_COMMAND ${CMD_MAKE}
@@ -104,7 +104,7 @@ else()
        set(JQ_INCLUDE "${JQ_SRC}")
        set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
        ExternalProject_Add(jq
-                URL "http://download.draios.com/dependencies/jq-1.5.tar.gz"
+                URL "https://s3.amazonaws.com/download.draios.com/dependencies/jq-1.5.tar.gz"
                URL_MD5 "0933532b086bd8b6a41c1b162b1731f9"
                CONFIGURE_COMMAND ./configure --disable-maintainer-mode --enable-all-static --disable-dependency-tracking
                BUILD_COMMAND ${CMD_MAKE} LDFLAGS=-all-static
@@ -134,7 +134,7 @@ else()
 	set(CURSES_LIBRARIES "${CURSES_BUNDLE_DIR}/lib/libncurses.a")
 	message(STATUS "Using bundled ncurses in '${CURSES_BUNDLE_DIR}'")
 	ExternalProject_Add(ncurses
-		URL "http://download.draios.com/dependencies/ncurses-6.0-20150725.tgz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/ncurses-6.0-20150725.tgz"
 		URL_MD5 "32b8913312e738d707ae68da439ca1f4"
 		CONFIGURE_COMMAND ./configure --without-cxx --without-cxx-binding --without-ada --without-manpages --without-progs --without-tests --with-terminfo-dirs=/etc/terminfo:/lib/terminfo:/usr/share/terminfo
 		BUILD_COMMAND ${CMD_MAKE}
@@ -161,7 +161,7 @@ else()
 	set(B64_INCLUDE "${B64_SRC}/include")
 	set(B64_LIB "${B64_SRC}/src/libb64.a")
 	ExternalProject_Add(b64
-		URL "http://download.draios.com/dependencies/libb64-1.2.src.zip"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/libb64-1.2.src.zip"
 		URL_MD5 "a609809408327117e2c643bed91b76c5"
 		CONFIGURE_COMMAND ""
 		BUILD_COMMAND ${CMD_MAKE}
@@ -215,7 +215,7 @@ else()
 	message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")

 	ExternalProject_Add(openssl
-		URL "http://download.draios.com/dependencies/openssl-1.0.2j.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/openssl-1.0.2j.tar.gz"
 		URL_MD5 "96322138f0b69e61b7212bc53d5e912b"
 		CONFIGURE_COMMAND ./config shared --prefix=${OPENSSL_INSTALL_DIR}
 		BUILD_COMMAND ${CMD_MAKE}
@@ -246,9 +246,9 @@ else()

 	ExternalProject_Add(curl
 		DEPENDS openssl
-		URL "http://download.draios.com/dependencies/curl-7.52.1.tar.bz2"
-		URL_MD5 "dd014df06ff1d12e173de86873f9f77a"
-		CONFIGURE_COMMAND ./configure ${CURL_SSL_OPTION} --disable-shared --enable-optimize --disable-curldebug --disable-rt --enable-http --disable-ftp --disable-file --disable-ldap --disable-ldaps --disable-rtsp --disable-telnet --disable-tftp --disable-pop3 --disable-imap --disable-smb --disable-smtp --disable-gopher --disable-sspi --disable-ntlm-wb --disable-tls-srp --without-winssl --without-darwinssl --without-polarssl --without-cyassl --without-nss --without-axtls --without-ca-path --without-ca-bundle --without-libmetalink --without-librtmp --without-winidn --without-libidn --without-nghttp2 --without-libssh2
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/curl-7.56.0.tar.bz2"
+		URL_MD5 "e0caf257103e0c77cee5be7e9ac66ca4"
+		CONFIGURE_COMMAND ./configure ${CURL_SSL_OPTION} --disable-shared --enable-optimize --disable-curldebug --disable-rt --enable-http --disable-ftp --disable-file --disable-ldap --disable-ldaps --disable-rtsp --disable-telnet --disable-tftp --disable-pop3 --disable-imap --disable-smb --disable-smtp --disable-gopher --disable-sspi --disable-ntlm-wb --disable-tls-srp --without-winssl --without-darwinssl --without-polarssl --without-cyassl --without-nss --without-axtls --without-ca-path --without-ca-bundle --without-libmetalink --without-librtmp --without-winidn --without-libidn --without-nghttp2 --without-libssh2 --disable-threaded-resolver
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
@@ -280,7 +280,7 @@ else()
 	set(LUAJIT_INCLUDE "${LUAJIT_SRC}")
 	set(LUAJIT_LIB "${LUAJIT_SRC}/libluajit.a")
 	ExternalProject_Add(luajit
-		URL "http://download.draios.com/dependencies/LuaJIT-2.0.3.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/LuaJIT-2.0.3.tar.gz"
 		URL_MD5 "f14e9104be513913810cd59c8c658dc0"
 		CONFIGURE_COMMAND ""
 		BUILD_COMMAND ${CMD_MAKE}
@@ -303,13 +303,14 @@ if(NOT USE_BUNDLED_LPEG)
 else()
 	set(LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
 	set(LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
+	message(STATUS "Using bundled lpeg in '${LPEG_SRC}'")
 	set(LPEG_DEPENDENCIES "")
 	if(USE_BUNDLED_LUAJIT)
 		list(APPEND LPEG_DEPENDENCIES "luajit")
 	endif()
 	ExternalProject_Add(lpeg
 		DEPENDS ${LPEG_DEPENDENCIES}
-		URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
 		URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
 		BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
 		BUILD_IN_SOURCE 1
@@ -342,8 +343,9 @@ else()

 	set(LIBYAML_SRC "${PROJECT_BINARY_DIR}/libyaml-prefix/src/libyaml/src")
 	set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
+	message(STATUS "Using bundled libyaml in '${LIBYAML_SRC}'")
 	ExternalProject_Add(libyaml
-		URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
 		URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
 		BUILD_COMMAND ${CMD_MAKE}
 		BUILD_IN_SOURCE 1
@@ -369,6 +371,7 @@ if(NOT USE_BUNDLED_LYAML)
 else()
 	set(LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
 	set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
+	message(STATUS "Using bundled lyaml in '${LYAML_SRC}'")
 	set(LYAML_DEPENDENCIES "")
 	if(USE_BUNDLED_LUAJIT)
 		list(APPEND LYAML_DEPENDENCIES "luajit")
@@ -378,7 +381,7 @@ else()
 	endif()
 	ExternalProject_Add(lyaml
 		DEPENDS ${LYAML_DEPENDENCIES}
-		URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
+		URL "https://s3.amazonaws.com/download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
                URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
                BUILD_COMMAND ${CMD_MAKE}
                BUILD_IN_SOURCE 1
@@ -389,6 +392,8 @@ endif()
 install(FILES falco.yaml
 	DESTINATION "${FALCO_ETC_DIR}")

+add_subdirectory(rules)
+
 if(CMAKE_SYSTEM_NAME MATCHES "Linux")
 	add_subdirectory("${SYSDIG_DIR}/driver" "${PROJECT_BINARY_DIR}/driver")
 endif()
@@ -396,8 +401,9 @@ add_subdirectory("${SYSDIG_DIR}/userspace/libscap" "${PROJECT_BINARY_DIR}/usersp
 add_subdirectory("${SYSDIG_DIR}/userspace/libsinsp" "${PROJECT_BINARY_DIR}/userspace/libsinsp")

 set(FALCO_SINSP_LIBRARY sinsp)
-set(FALCO_SHARE_DIR ${CMAKE_INSTALL_PREFIX}/share/falco)
-set(FALCO_BIN_DIR ${CMAKE_INSTALL_PREFIX}/bin)
+set(FALCO_SHARE_DIR share/falco)
+set(FALCO_ABSOLUTE_SHARE_DIR "${CMAKE_INSTALL_PREFIX}/${FALCO_SHARE_DIR}")
+set(FALCO_BIN_DIR bin)
 add_subdirectory(scripts)
 add_subdirectory(userspace/engine)
 add_subdirectory(userspace/falco)
@@ -419,7 +425,7 @@ set(CPACK_DEBIAN_PACKAGE_MAINTAINER "Sysdig <support@sysdig.com>")
 set(CPACK_DEBIAN_PACKAGE_SECTION "utils")
 set(CPACK_DEBIAN_PACKAGE_HOMEPAGE "http://www.sysdig.org")
 set(CPACK_DEBIAN_PACKAGE_DEPENDS "dkms (>= 2.1.0.0)")
-set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/scripts/debian/postinst;${CMAKE_BINARY_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm")
+set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/scripts/debian/postinst;${CMAKE_BINARY_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm;${PROJECT_SOURCE_DIR}/cpack/debian/conffiles")

 set(CPACK_RPM_PACKAGE_LICENSE "GPLv2")
 set(CPACK_RPM_PACKAGE_URL "http://www.sysdig.org")
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@

 #### Latest release

-**v0.7.0**
+**v0.8.1**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)

 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -33,10 +33,9 @@ Documentation

 Join the Community
 ---
-* Contact the [official mailing list](https://groups.google.com/forum/#!forum/falco) for support and to talk with other users.
 * Follow us on [Twitter](https://twitter.com/sysdig) for general falco and sysdig news.
 * This is our [blog](https://sysdig.com/blog/), where you can find the latest [falco](https://sysdig.com/blog/tag/falco/) posts.
-* Join our [Public Slack](https://sysdig.slack.com) channel for sysdig and falco announcements and discussions.
+* Join our [Public Slack](https://slack.sysdig.com) channel for sysdig and falco announcements and discussions.

 License Terms
 ---
@@ -61,15 +60,27 @@ As always, we are grateful for your past and present contributions to falco.

 **Individual contributions**: Individuals who wish to make contributions must review the [Individual Contributor License Agreement](./cla/falco_contributor_agreement.txt) and indicate agreement by adding the following line to every GIT commit message:

+```
 falco-CLA-1.0-signed-off-by: Joe Smith <joe.smith@email.com>
+```

 Use your real name; pseudonyms or anonymous contributions are not allowed.

 **Corporate contributions**: Employees of corporations, members of LLCs or LLPs, or others acting on behalf of a contributing entity, must review the [Corporate Contributor License Agreement](./cla/falco_corp_contributor_agreement.txt), must be an authorized representative of the contributing entity, and indicate agreement to it on behalf of the contributing entity by adding the following lines to every GIT commit message:

 ```
- falco-CLA-1.0-contributing-entity: Full Legal Name of Entity
- falco-CLA-1.0-signed-off-by: Joe Smith <joe.smith@email.com>
+falco-CLA-1.0-contributing-entity: Full Legal Name of Entity
+falco-CLA-1.0-signed-off-by: Joe Smith <joe.smith@email.com>
+```
+
+Use a real name of a natural person who is an authorized representative of the contributing entity; pseudonyms or anonymous contributions are not allowed.
+
+**Government contributions**: Employees or officers of the United States Government, must review the [Government Contributor License Agreement](https://github.com/draios/falco/blob/dev/cla/falco_govt_contributor_agreement.txt), must be an authorized representative of the contributing entity, and indicate agreement to it on behalf of the contributing entity by adding the following lines to every GIT commit message:
+
+```
+falco-CLA-1.0-contributing-govt-entity: Full Legal Name of Entity
+falco-CLA-1.0-signed-off-by: Joe Smith <joe.smith@email.com>
+This file is a work of authorship of an employee or officer of the United States Government and is not subject to copyright in the United States under 17 USC 105.
 ```

 Use a real name of a natural person who is an authorized representative of the contributing entity; pseudonyms or anonymous contributions are not allowed.
--- a/cla/falco_govt_contributor_agreement.txt
+++ b/cla/falco_govt_contributor_agreement.txt
@@ -0,0 +1,33 @@
+DRAIOS, INC. <20> OPEN SOURCE CONTRIBUTION AGREEMENT FOR UNITED STATES GOVERNMENT CONTRIBUTING ENTITIES (<28>Agreement<6E>)
+
+Draios, Inc. (<28>Draios<6F> or <20>Sysdig<69>) welcomes the work of others on our open source software projects.  To contribute code back to our repositories, we require a contributing entity that is a United States Government agency to complete, and agree to, the Government Contributor Agreement (GCA) set forth here, by and through a designated authorized representative. This agreement clarifies the ability for us to use and incorporate the contributions of a government contributing entity in our projects and products. After agreeing to these terms, a contributing entity may contribute to our projects. To indicate the agreement of the contributing entity, an authorized representative shall follow the procedure set forth below under TO AGREE, after reading this Agreement. A <20>contributing entity<74> means any agency or unit of the United States government. We provide a separate CLA for individual contributors. 
+
+You accept and agree to the following terms and conditions for Your present and future Contributions that are submitted to Draios/Sysdig.
+
+1. Definitions.  "You" (or "Your") shall mean the contributing entity that has authored or otherwise has the right to contribute the Contribution, and that is making this Agreement with Draios/Sysdig. "Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to Draios/Sysdig for inclusion in, or documentation of, any of the products owned or managed by Draios/Sysdig (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to Draios/Sysdig or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, Draios/Sysdig for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."
+
+2. Contributions Not Subject to Copyright. Each Contribution is a work authored by the United States Government or an employee or officer thereof and is not subject to copyright under 17 U.S.C. 105.
+
+3. Grant of Patent License. Subject to the terms and conditions of this Agreement, You hereby grant to Draios/Sysdig and to recipients of software distributed by Draios/Sysdig a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims that You have the right to license and that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity other than Draios/Sysdig institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.
+
+4. You represent to Draios/Sysdig that You own or have the right to contribute Your Contributions to Draios/Sysdig, and that You are legally entitled to grant the license set forth above.
+
+5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which You are personally aware and which are associated with any part of Your Contributions. You represent that Your sign-off indicating assent to this Agreement includes the real name of a natural person who is an authorized representative of You, and not a pseudonym, and that You are not attempting or making an anonymous Contribution.
+
+6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions to Draios/Sysdig on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
+
+7. If You wish to submit work that is not Your original creation, You may submit it to Draios/Sysdig separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which You are aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]".
+
+8. You agree to notify Draios/Sysdig of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect.
+
+9. You understand and agree that this project and Your Contribution are public and that a record of the contribution, including all personal information that You submit with it, including the sign-off of Your authorized representative, may be stored by Draios/Sysdig indefinitely and may be redistributed to others. You understand and agree that Draios/Sysdig has no obligation to use any Contribution in any Draios/Sysdig project or product, and Draios/Sysdig may decline to accept Your Contributions or Draios/Sysdig may remove Your Contributions from Draios/Sysdig projects or products at any time without notice. You understand and agree that Draios/Sysdig is not and will not pay You any form of compensation, in currency, equity or otherwise, in exchange for Your Contributions or for Your assent to this Agreement. You understand and agree that You are independent of Draios/Sysdig and You are not, by entering into this Agreement or providing Your Contributions, becoming employed, hired as an independent contractor, or forming any other relationship with Draios/Sysdig relating to employment, compensation or ownership or involving any fiduciary obligation.
+
+TO AGREE:
+Add the following lines to every GIT commit message:
+
+falco-CLA-1.0-contributing-govt-entity: Full Legal Name of Entity
+falco-CLA-1.0-signed-off-by: Joe Smith joe.smith@email.com
+This file is a work of authorship of an employee or officer of the United States Government and is not subject to copyright in the United States under 17 USC 105.
+
+Use a real name of a natural person who is an authorized representative of the contributing entity; pseudonyms or anonymous contributions are not allowed.
+
--- a/cpack/debian/conffiles
+++ b/cpack/debian/conffiles
@@ -0,0 +1,3 @@
+/etc/falco/falco.yaml
+/etc/falco/falco_rules.yaml
+/etc/falco/falco_rules.local.yaml
--- a/examples/nodejs-bad-rest-api/demo.yml
+++ b/examples/nodejs-bad-rest-api/demo.yml
@@ -5,7 +5,7 @@ express_server:
  working_dir: /usr/src/app
  command: bash -c "npm install && node server.js"
  ports:
-    - "8080:8080"
+    - "8181:8181"
  volumes:
        - ${PWD}:/usr/src/app

--- a/examples/nodejs-bad-rest-api/server.js
+++ b/examples/nodejs-bad-rest-api/server.js
@@ -2,13 +2,13 @@ var express    = require('express');        // call express
 var app        = express();                 // define our app using express
 var child_process = require('child_process');

-var port = process.env.PORT || 8080;        // set our port
+var port = process.env.PORT || 8181;        // set our port

 // ROUTES FOR OUR API
 // =============================================================================
 var router = express.Router();              // get an instance of the express Router

-// test route to make sure everything is working (accessed at GET http://localhost:8080/api)
+// test route to make sure everything is working (accessed at GET http://localhost:8181/api)
 router.get('/', function(req, res) {
    res.json({ message: 'API available'});
 });
--- a/falco.yaml
+++ b/falco.yaml
@@ -1,5 +1,15 @@
-# File containing Falco rules, loaded at startup.
-rules_file: /etc/falco_rules.yaml
+# File(s) containing Falco rules, loaded at startup.
+#
+# falco_rules.yaml ships with the falco package and is overridden with
+# every new software version. falco_rules.local.yaml is only created
+# if it doesn't exist. If you want to customize the set of rules, add
+# your customizations to falco_rules.local.yaml.
+#
+# The files will be read in the order presented here, so make sure if
+# you have overrides they appear in later files.
+rules_file:
+ - /etc/falco/falco_rules.yaml
+ - /etc/falco/falco_rules.local.yaml

 # Whether to output events in json or text
 json_output: false
@@ -15,6 +25,16 @@ log_syslog: true
 # "alert", "critical", "error", "warning", "notice", "info", "debug".
 log_level: info

+# Minimum rule priority level to load and run. All rules having a
+# priority more severe than this level will be loaded/run.  Can be one
+# of "emergency", "alert", "critical", "error", "warning", "notice",
+# "info", "debug".
+priority: debug
+
+# Whether or not output to any of the output channels below is
+# buffered. Defaults to true
+buffered_outputs: true
+
 # A throttling mechanism implemented as a token bucket limits the
 # rate of falco notifications. This throttling is controlled by the following configuration
 # options:
@@ -37,8 +57,13 @@ outputs:
 syslog_output:
  enabled: true

+# If keep_alive is set to true, the file will be opened once and
+# continuously written to, with each output message on its own
+# line. If keep_alive is set to false, the file will be re-opened
+# for each output message.
 file_output:
  enabled: false
+  keep_alive: false
  filename: ./events.txt

 stdout_output:
@@ -49,7 +74,15 @@ stdout_output:
 #         program: "jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/XXX"
 #   - logging (alternate method than syslog):
 #         program: logger -t falco-test
+#   - send over a network connection:
+#         program: nc host.example.com 80
+
+# If keep_alive is set to true, the program will be started once and
+# continuously written to, with each output message on its own
+# line. If keep_alive is set to false, the program will be re-spawned
+# for each output message.

 program_output:
  enabled: false
+  keep_alive: false
  program: mail -s "Falco Notification" someone@example.com
--- a/rules/CMakeLists.txt
+++ b/rules/CMakeLists.txt
@@ -1,9 +1,10 @@
 if(NOT DEFINED FALCO_ETC_DIR)
-  set(FALCO_ETC_DIR "/etc")
+  set(FALCO_ETC_DIR "/etc/falco")
 endif()

 if(NOT DEFINED FALCO_RULES_DEST_FILENAME)
  set(FALCO_RULES_DEST_FILENAME "falco_rules.yaml")
+  set(FALCO_LOCAL_RULES_DEST_FILENAME "falco_rules.local.yaml")
 endif()

 if(DEFINED FALCO_COMPONENT)
@@ -11,9 +12,18 @@ install(FILES falco_rules.yaml
        COMPONENT "${FALCO_COMPONENT}"
 	DESTINATION "${FALCO_ETC_DIR}"
 	RENAME "${FALCO_RULES_DEST_FILENAME}")
+
+install(FILES falco_rules.local.yaml
+        COMPONENT "${FALCO_COMPONENT}"
+	DESTINATION "${FALCO_ETC_DIR}"
+	RENAME "${FALCO_LOCAL_RULES_DEST_FILENAME}")
 else()
 install(FILES falco_rules.yaml
-        DESTINATION "${FALCO_ETC_DIR}"
-        RENAME "${FALCO_RULES_DEST_FILENAME}")
+	DESTINATION "${FALCO_ETC_DIR}"
+	RENAME "${FALCO_RULES_DEST_FILENAME}")
+
+install(FILES falco_rules.local.yaml
+	DESTINATION "${FALCO_ETC_DIR}"
+	RENAME "${FALCO_LOCAL_RULES_DEST_FILENAME}")
 endif()

--- a/rules/falco_rules.local.yaml
+++ b/rules/falco_rules.local.yaml
@@ -0,0 +1,13 @@
+####################
+# Your custom rules!
+####################
+
+# Add new rules, like this one
+# - rule: The program "sudo" is run in a container
+#   desc: An event will trigger every time you run sudo in a container
+#   condition: evt.type = execve and evt.dir=< and container.id != host and proc.name = sudo
+#   output: "Sudo run in container (user=%user.name %container.info parent=%proc.pname cmdline=%proc.cmdline)"
+#   priority: ERROR
+#   tags: [users, container]
+
+# Or override/append to any rule, macro, or list from the Default Rules
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -56,6 +56,10 @@
 - macro: etc_dir
  condition: fd.name startswith /etc

+# This detects writes immediately below / or any write anywhere below /root
+- macro: root_dir
+  condition: (fd.directory=/ or fd.name startswith /root)
+
 - macro: ubuntu_so_dirs
  condition: >
    fd.name startswith /lib/x86_64-linux-gnu or
@@ -74,8 +78,11 @@
 - list: shell_binaries
  items: [bash, csh, ksh, sh, tcsh, zsh, dash]

+- list: shell_mgmt_binaries
+  items: [add-shell, remove-shell]
+
 - macro: shell_procs
-  condition: proc.name in (shell_binaries)
+  condition: (proc.name in (shell_binaries))

 - list: coreutils_binaries
  items: [
@@ -120,7 +127,7 @@
  items: [setup-backend, dragent, sdchecks]

 - list: docker_binaries
-  items: [docker, dockerd, exe, docker-compose, docker-entrypoi]
+  items: [docker, dockerd, exe, docker-compose, docker-entrypoi, docker-runc-cur]

 - list: k8s_binaries
  items: [hyperkube, skydns, kube2sky, exechealthz]
@@ -131,7 +138,19 @@
 # Utility/etc programs known to run on mesos slaves. Truncation
 # intentional.
 - list: mesos_slave_binaries
-  items: [mesos-health-ch, mesos-docker-ex, mesos-agent, mesos-logrotate, mesos-fetcher]
+  items: [mesos-health-ch, mesos-docker-ex, mesos-agent, mesos-slave, mesos-logrotate, mesos-fetcher, mesos-executor, 3dt]
+
+- list: phusion_passenger_binaries
+  items: [PassengerAgent, PassengerWatchd]
+
+# A bit longer to avoid the fairly generic my_init.
+- macro: parent_phusion_passenger_my_init
+  condition: >
+    (proc.pcmdline="my_init -u /sbin/my_init " or
+     proc.pcmdline="my_init -u /sbin/my_init")
+
+- list: chef_binaries
+  items: [chef-client]

 - list: http_server_binaries
  items: [nginx, httpd, httpd-foregroun, lighttpd]
@@ -139,6 +158,15 @@
 - list: db_server_binaries
  items: [mysqld]

+- list: mysql_mgmt_binaries
+  items: [mysql_install_d, mysql_ssl_rsa_s]
+
+- list: postgres_mgmt_binaries
+  items: [pg_dumpall, pg_ctl]
+
+- list: db_mgmt_binaries
+  items: [mysql_mgmt_binaries, postgres_mgmt_binaries]
+
 - list: gitlab_binaries
  items: [gitlab-shell, gitlab-mon, gitlab-runner-b, git]

@@ -148,13 +176,13 @@
 # The explicit quotes are needed to avoid the - characters being
 # interpreted by the filter expression.
 - list: rpm_binaries
-  items: [dnf, rpm, rpmkey, yum, '"75-system-updat"']
+  items: [dnf, rpm, rpmkey, yum, '"75-system-updat"', rhsmcertd-worke]

 - macro: rpm_procs
  condition: proc.name in (rpm_binaries)

 - list: deb_binaries
-  items: [dpkg, dpkg-preconfigu, apt, apt-get, aptitude,
+  items: [dpkg, dpkg-preconfigu, dpkg-reconfigur, apt, apt-get, aptitude,
    frontend, preinst, add-apt-reposit, apt-auto-remova, apt-key,
    apt-listchanges, unattended-upgr
    ]
@@ -162,7 +190,7 @@
 # The truncated dpkg-preconfigu is intentional, process names are
 # truncated at the sysdig level.
 - list: package_mgmt_binaries
-  items: [rpm_binaries, deb_binaries, update-alternat]
+  items: [rpm_binaries, deb_binaries, update-alternat, gem, pip]

 - macro: package_mgmt_procs
  condition: proc.name in (package_mgmt_binaries)
@@ -176,13 +204,20 @@
 # A canonical set of processes that run other programs with different
 # privileges or as a different user.
 - list: userexec_binaries
-  items: [sudo, su]
+  items: [sudo, su, suexec]
+
+- list: known_setuid_binaries
+  items: [
+    sshd, dbus-daemon-lau, ping, ping6, critical-stack-, pmmcli,
+    filemng, PassengerAgent, bwrap, osdetect, nginxmng, sw-engine-fpm,
+    start-stop-daem
+    ]

 - list: user_mgmt_binaries
  items: [login_binaries, passwd_binaries, shadowutils_binaries]

 - list: dev_creation_binaries
-  items: [blkid, rename_device]
+  items: [blkid, rename_device, update_engine, sgdisk]

 - list: aide_wrapper_binaries
  items: [aide.wrapper, update-aide.con]
@@ -190,25 +225,53 @@
 - list: hids_binaries
  items: [aide]

+- list: vpn_binaries
+  items: [openvpn]
+
+- list: nomachine_binaries
+  items: [nxexec, nxnode.bin, nxserver.bin, nxclient.bin]
+
+- list: x2go_binaries
+  items: [x2gosuspend-age, x2goagent]
+
+- list: xray_rabbitmq_binaries
+  items: ['"1_scheduler"', '"2_scheduler"', '"3_scheduler"', '"4_scheduler"']
+
 - list: nids_binaries
  items: [bro, broctl]

 - list: monitoring_binaries
-  items: [icinga2, nrpe, npcd, check_sar_perf.]
+  items: [icinga2, nrpe, npcd, check_sar_perf., qualys-cloud-ag, S99qualys-cloud]

 - macro: system_procs
  condition: proc.name in (coreutils_binaries, user_mgmt_binaries)

 - list: mail_binaries
-  items: [sendmail, sendmail-msp, postfix, procmail, exim4, pickup, showq]
+  items: [
+    sendmail, sendmail-msp, postfix, procmail, exim4,
+    pickup, showq, mailq, dovecot, imap-login, imap,
+    mailmng-core, pop3-login, dovecot-lda
+    ]
+
+- list: sendmail_config_binaries
+  items: [
+    update_conf, parse_mc, makemap_hash, newaliases, update_mk, update_tlsm4,
+    update_db, update_mc, ssmtp.postinst, mailq
+    ]

 - list: make_binaries
  items: [make, gmake, cmake]

+- list: keepalived_binaries
+  items: [keepalived]
+
+- list: sensitive_file_names
+  items: [/etc/shadow, /etc/sudoers, /etc/pam.conf]
+
 - macro: sensitive_files
  condition: >
    fd.name startswith /etc and
-    (fd.name in (/etc/shadow, /etc/sudoers, /etc/pam.conf)
+    (fd.name in (sensitive_file_names)
     or fd.directory in (/etc/sudoers.d, /etc/pam.d))

 # Indicates that the process is new. Currently detected using time
@@ -261,6 +324,17 @@
 - list: cron_binaries
  items: [anacron, cron, crond]

+# https://github.com/liske/needrestart
+- list: needrestart_binaries
+  items: [needrestart, 10-dpkg, 20-rpm, 30-pacman]
+
+# Possible scripts run by sshkit
+- list: sshkit_script_binaries
+  items: [10_etc_sudoers., 10_passwd_group]
+
+- list: plesk_binaries
+  items: [sw-engine, sw-engine-fpm, sw-engine-kv, filemng, f2bmng]
+
 # System users that should never log into a system. Consider adding your own
 # service users (e.g. 'apache' or 'mysqld') here.
 - macro: system_users
@@ -278,12 +352,21 @@
 - macro: ansible_running_python
  condition: (proc.name in (python, pypy) and proc.cmdline contains ansible)

+- macro: parent_beam_running_python
+  condition: proc.pcmdline="python pipeline.py -c conf.json"
+
+- macro: parent_strongswan_running_starter
+  condition: proc.pcmdline="starter --daemon charon"
+
 - macro: python_running_denyhosts
  condition: >
    (proc.name=python and
    (proc.cmdline contains /usr/sbin/denyhosts or
     proc.cmdline contains /usr/local/bin/denyhosts.py))

+- macro: parent_python_running_localstack
+  condition: (proc.pcmdline startswith "python bin/localstack")
+
 - macro: parent_python_running_denyhosts
  condition: >
    (proc.pname=python and
@@ -292,8 +375,8 @@

 - macro: parent_python_running_sdchecks
  condition: >
-    (proc.name in (python, python2.7) and
-    (proc.cmdline contains /opt/draios/bin/sdchecks))
+    (proc.pname in (python, python2.7) and
+    (proc.pcmdline contains /opt/draios/bin/sdchecks))

 - macro: parent_bro_running_python
  condition: (proc.pname=python and proc.cmdline contains /usr/share/broctl)
@@ -303,6 +386,113 @@
    (proc.pname=java and proc.pcmdline contains jenkins.war
    or proc.pcmdline contains /tmp/slave.jar)

+- macro: parent_cpanm_running_perl
+  condition: (proc.pname=perl and proc.aname[2]=cpanm)
+
+- macro: ics_running_java
+  condition: (proc.pname=java and proc.aname[3] in (ics_start.sh,ics_stop.sh))
+
+- macro: jenkins_scripts
+  condition: (proc.pcmdline startswith "script.sh -xe /var/jenkins_home" or
+              proc.cmdline="bash /usr/local/bin/jenkins-slave")
+
+- macro: parent_java_running_echo
+  condition: (proc.pname=java and proc.cmdline startswith "sh -c echo")
+
+- macro: parent_java_running_sbt
+  condition: (proc.pname=java and proc.pcmdline contains sbt-launch.jar)
+
+# The crxlsx is a bit different than the other build-like things, but
+# close enough to add here rather than create a separate macro.
+- macro: parent_scripting_running_builds
+  condition: >
+    (proc.pname in (php,php5-fpm,php-fpm7.1,python,ruby,ruby2.3,node) and (
+       proc.cmdline startswith "sh -c git" or
+       proc.cmdline startswith "sh -c date" or
+       proc.cmdline startswith "sh -c /usr/bin/g++" or
+       proc.cmdline startswith "sh -c /usr/bin/gcc" or
+       proc.cmdline startswith "sh -c gcc" or
+       proc.cmdline startswith "sh -c if type gcc" or
+       proc.cmdline startswith "sh -c cd '/var/www/edi/';LC_ALL=en_US.UTF-8 git" or
+       proc.cmdline startswith "sh -c /var/www/edi/bin/sftp.sh" or
+       proc.cmdline startswith "sh -c /usr/src/app/crxlsx/bin/linux/crxlsx" or
+       proc.pcmdline startswith "node /opt/nodejs/bin/yarn" or
+       proc.pcmdline startswith "node /root/.config/yarn" or
+       proc.pcmdline startswith "node /opt/yarn/bin/yarn.js"))
+
+- macro: parent_node_running_npm
+  condition: proc.pcmdline startswith "node /usr/local/bin/npm"
+
+- macro: parent_nginx_running_serf
+  condition: (proc.pname=nginx and proc.cmdline startswith "sh -c serf")
+
+- macro: parent_Xvfb_running_xkbcomp
+  condition: (proc.pname=Xvfb and proc.cmdline startswith 'sh -c "/usr/bin/xkbcomp"')
+
+- macro: mysql_image_running_healthcheck
+  condition: container.image=mysql and proc.cmdline="sh -c /healthcheck.sh"
+
+- macro: bundle_running_ruby
+  condition: >
+    ((proc.pname=ruby or proc.pname contains ".rb") and (
+     proc.aname[2]=bundle or
+     proc.aname[3]=bundle or
+     proc.aname[4]=bundle))
+
+- macro: assemble_running_php
+  condition: >
+    (proc.pname=php and (
+     proc.aname[2]=assemble or
+     proc.aname[3]=assemble or
+     proc.aname[4]=assemble))
+
+- macro: node_running_bitnami
+  condition: proc.pname=node and proc.cmdline startswith "sh -c /opt/bitnami"
+
+# Qualys seems to run a variety of shell subprocesses, at various
+# levels. This checks at a few levels without the cost of a full
+# proc.aname, which traverses the full parent heirarchy.
+- macro: run_by_qualys
+  condition: >
+    (proc.pname=qualys-cloud-ag or
+     proc.aname[2]=qualys-cloud-ag or
+     proc.aname[3]=qualys-cloud-ag or
+     proc.aname[4]=qualys-cloud-ag)
+
+- macro: run_by_sumologic_securefiles
+  condition: >
+    ((proc.cmdline="usermod -a -G sumologic_collector" or
+      proc.cmdline="groupadd sumologic_collector") and
+     (proc.pname=secureFiles.sh and proc.aname[2]=java))
+
+# Chef is similar.
+- macro: run_by_chef
+  condition: (proc.aname[2]=chef_command_wr or proc.aname[3]=chef_command_wr or
+              proc.aname[2]=chef-client or proc.aname[3]=chef-client)
+
+- macro: run_by_adclient
+  condition: (proc.aname[2]=adclient or proc.aname[3]=adclient or proc.aname[4]=adclient)
+
+- macro: run_by_centrify
+  condition: (proc.aname[2]=centrify or proc.aname[3]=centrify or proc.aname[4]=centrify)
+
+- macro: run_by_puppet
+  condition: (proc.aname[2]=puppet or proc.aname[3]=puppet)
+
+- macro: run_by_h2o
+  condition: (proc.pname=perl and proc.aname[2]=h2o)
+
+- macro: run_by_passenger_agent
+  condition: ((proc.pname=ruby and proc.aname[2]=PassengerAgent) or
+              proc.pcmdline startswith "ruby /usr/share/passenger/helper-scripts/rack-preloader.rb")
+
+# Also handles running semi-indirectly via scl
+- macro: run_by_foreman
+  condition: >
+    (user.name=foreman and
+     (proc.pname in (rake, ruby, scl) and proc.aname[5] in (tfm-rake,tfm-ruby)) or
+     (proc.pname=scl and proc.aname[2] in (tfm-rake,tfm-ruby)))
+
 # As a part of kernel upgrades, dpkg will spawn a perl script with the
 # name linux-image-N.N. This macro matches that.
 - macro: parent_linux_image_upgrade_script
@@ -311,6 +501,33 @@
 - macro: java_running_sdjagent
  condition: proc.name=java and proc.cmdline contains sdjagent.jar

+- macro: parent_java_running_confluence
+  condition: (proc.pname=java and proc.pcmdline contains "-classpath /opt/atlassian/confluence")
+
+- macro: parent_java_running_tomcat
+  condition: (proc.pname=java and proc.pcmdline contains "-classpath /usr/local/tomcat")
+
+- macro: parent_java_running_install4j
+  condition: (proc.pname=java and proc.pcmdline contains "-classpath i4jruntime.jar")
+
+- macro: parent_dovecot_running_auth
+  condition: (proc.pname=auth and proc.aname[2]=dovecot)
+
+- macro: parent_supervise_running_multilog
+  condition: (proc.name=multilog and proc.pname=supervise)
+
+- macro: parent_ruby_running_discourse
+  condition: (proc.pcmdline startswith "ruby /var/www/discourse/vendor/bundle/ruby")
+
+- macro: pki_realm_writing_realms
+  condition: (proc.cmdline startswith "bash /usr/local/lib/pki/pki-realm" and fd.name startswith /etc/pki/realms)
+
+- macro: htpasswd_writing_passwd
+  condition: (proc.name=htpasswd and fd.name=/etc/nginx/.htpasswd)
+
+- macro: dmeventd_writing_lvm_archive
+  condition: (proc.name=dmeventd and fd.name startswith /etc/lvm/archive/)
+
 ###############
 # General Rules
 ###############
@@ -324,25 +541,95 @@
  priority: ERROR
  tags: [filesystem]

+- list: safe_etc_dirs
+  items: [/etc/cassandra, /etc/ssl/certs/java, /etc/logstash, /etc/nginx/conf.d, /etc/container_environment, /etc/hrmconfig]
+
+- macro: fluentd_writing_conf_files
+  condition: (proc.name=start-fluentd and fd.name in (/etc/fluent/fluent.conf, /etc/td-agent/td-agent.conf))
+
+- macro: qualys_writing_conf_files
+  condition: (proc.name=qualys-cloud-ag and fd.name=/etc/qualys/cloud-agent/qagent-log.conf)
+
+- macro: git_writing_nssdb
+  condition: (proc.name=git-remote-http and fd.directory=/etc/pki/nssdb)
+
+- macro: plesk_writing_keys
+  condition: (proc.name in (plesk_binaries) and fd.name startswith /etc/sw/keys)
+
+- macro: networkmanager_writing_resolv_conf
+  condition: proc.aname[2]=nm-dispatcher and fd.name=/etc/resolv.conf
+
+- macro: add_shell_writing_shells_tmp
+  condition: (proc.name=add-shell and fd.name=/etc/shells.tmp)
+
+# Add conditions to this macro (probably in a separate file,
+# overwriting this macro) to allow for specific combinations of
+# programs writing below specific directories below
+# /etc. fluentd_writing_conf_files is a good example to follow, as it
+# specifies both the program doing the writing as well as the specific
+# files it is allowed to modify.
+#
+# In this file, it just takes one of the programs in the base macro
+# and repeats it.
+
+- macro: user_known_write_etc_conditions
+  condition: proc.name=confd
+
 - macro: write_etc_common
  condition: >
    etc_dir and evt.dir = < and open_write
    and not proc.name in (passwd_binaries, shadowutils_binaries, sysdigcloud_binaries,
                          package_mgmt_binaries, ssl_mgmt_binaries, dhcp_binaries,
-                          dev_creation_binaries,
+                          dev_creation_binaries, shell_mgmt_binaries,
+                          sendmail_config_binaries,
+                          sshkit_script_binaries,
                          ldconfig.real, ldconfig, confd, gpg, insserv,
                          apparmor_parser, update-mime, tzdata.config, tzdata.postinst,
-                          systemd-machine, debconf-show, rollerd, bind9.postinst, sv,
-                          gen_resolvconf.)
-    and not proc.pname in (sysdigcloud_binaries)
-    and not fd.directory in (/etc/cassandra, /etc/ssl/certs/java)
+                          systemd, systemd-machine, systemd-sysuser,
+                          debconf-show, rollerd, bind9.postinst, sv,
+                          gen_resolvconf., update-ca-certi, certbot, runsv,
+                          qualys-cloud-ag, locales.postins, nomachine_binaries,
+                          adclient, certutil, crlutil)
+    and not proc.pname in (sysdigcloud_binaries, sendmail_config_binaries, hddtemp.postins, sshkit_script_binaries, locales.postins)
+    and not fd.name pmatch (safe_etc_dirs)
+    and not fd.name in (/etc/container_environment.sh, /etc/container_environment.json, /etc/motd, /etc/motd.svc)
    and not ansible_running_python
    and not python_running_denyhosts
+    and not fluentd_writing_conf_files
+    and not user_known_write_etc_conditions
+    and not run_by_centrify
+    and not run_by_adclient
+    and not qualys_writing_conf_files
+    and not git_writing_nssdb
+    and not plesk_writing_keys
+    and not networkmanager_writing_resolv_conf
+    and not run_by_chef
+    and not add_shell_writing_shells_tmp
+    and not parent_supervise_running_multilog
+    and not pki_realm_writing_realms
+    and not htpasswd_writing_passwd
+    and not dmeventd_writing_lvm_archive

 - rule: Write below etc
  desc: an attempt to write to any file below /etc, not in a pipe installer session
  condition: write_etc_common and not proc.sname=fbash
-  output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
+  output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline parent=%proc.pname file=%fd.name name=%proc.name gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4])"
+  priority: ERROR
+  tags: [filesystem]
+
+- list: known_root_files
+  items: [/root/.monit.state]
+
+- list: known_root_directories
+  items: [/root/.oracle_jre_usage]
+
+- rule: Write below root
+  desc: an attempt to write to any file directly below / or /root
+  condition: >
+    root_dir and evt.dir = < and open_write
+    and not fd.name in (known_root_files)
+    and not fd.directory in (known_root_directories)
+  output: "File below / or /root opened for writing (user=%user.name command=%proc.cmdline parent=%proc.pname file=%fd.name name=%proc.name)"
  priority: ERROR
  tags: [filesystem]

@@ -357,7 +644,7 @@
  tags: [filesystem]

 - macro: cmp_cp_by_passwd
-  condition: proc.name in (cmp, cp) and proc.pname=passwd
+  condition: proc.name in (cmp, cp) and proc.pname in (passwd, run-parts)

 - rule: Read sensitive file trusted after startup
  desc: >
@@ -367,12 +654,29 @@
  condition: sensitive_files and open_read and server_procs and not proc_is_new and proc.name!="sshd"
  output: >
    Sensitive file opened for reading by trusted program after startup (user=%user.name
-    command=%proc.cmdline file=%fd.name)
+    command=%proc.cmdline parent=%proc.pname file=%fd.name parent=%proc.pname gparent=%proc.aname[2])
  priority: WARNING
  tags: [filesystem]

 - list: read_sensitive_file_binaries
-  items: [iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, sshd, vsftpd, systemd]
+  items: [
+    iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, sshd,
+    vsftpd, systemd, mysql_install_d, psql, screen, debconf-show, sa-update,
+    pam-auth-update
+    ]
+
+# Add conditions to this macro (probably in a separate file,
+# overwriting this macro) to allow for specific combinations of
+# programs accessing sensitive files.
+# fluentd_writing_conf_files is a good example to follow, as it
+# specifies both the program doing the writing as well as the specific
+# files it is allowed to modify.
+#
+# In this file, it just takes one of the macros in the base rule
+# and repeats it.
+
+- macro: user_read_sensitive_file_conditions
+  condition: cmp_cp_by_passwd

 - rule: Read sensitive file untrusted
  desc: >
@@ -381,13 +685,18 @@
  condition: >
    sensitive_files and open_read
    and not proc.name in (user_mgmt_binaries, userexec_binaries, package_mgmt_binaries,
-     cron_binaries, read_sensitive_file_binaries, shell_binaries, hids_binaries)
+     cron_binaries, read_sensitive_file_binaries, shell_binaries, hids_binaries,
+     vpn_binaries, sendmail_config_binaries, nomachine_binaries, sshkit_script_binaries,
+     in.proftpd, mandb)
    and not cmp_cp_by_passwd
    and not ansible_running_python
    and not proc.cmdline contains /usr/bin/mandb
+    and not run_by_qualys
+    and not run_by_chef
+    and not user_read_sensitive_file_conditions
  output: >
    Sensitive file opened for reading by non-trusted program (user=%user.name name=%proc.name
-    command=%proc.cmdline file=%fd.name)
+    command=%proc.cmdline file=%fd.name parent=%proc.pname gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4])
  priority: WARNING
  tags: [filesystem]

@@ -467,13 +776,18 @@
 - list: known_shell_spawn_binaries
  items: [
    sshd, sudo, su, tmux, screen, emacs, systemd, login, flock, fbash,
-    nginx, monit, supervisord, dragent, aws, initdb, docker-compose,
+    nginx, monit, supervisord, dragent, aws, awslogs, initdb, docker-compose,
    configure, awk, falco, fail2ban-server, fleetctl,
    logrotate, ansible, less, adduser, pycompile, py3compile,
    pyclean, py3clean, pip, pip2, ansible-playboo, man-db,
    init, pluto, mkinitramfs, unattended-upgr, watch, sysdig,
-    landscape-sysin, nessusd, PM2, syslog-summary, erl_child_setup,
-    npm, cloud-init, toybox, ceph
+    landscape-sysin, nessusd, PM2, syslog-summary, erl_child_setup, erlexec,
+    npm, cloud-init, toybox, ceph, hhvm, certbot,
+    serf, a2enmod, runsv, supervisord, varnishd, authconfig, tini,
+    timeout, updatedb.findut, adclient, systemd-udevd,
+    luajit, uwsgi, cfn-signal, apache_control_, beam.smp, paster, postfix-local,
+    nginx_control, mailmng-service, web_statistic_e, statistics_coll, install-info,
+    hawkular-metric, rhsmcertd-worke, parted, amuled, fluentd
    ]

 - rule: Run shell untrusted
@@ -484,16 +798,43 @@
    and proc.pname exists
    and not proc.pname in (cron_binaries, shell_binaries, make_binaries, known_shell_spawn_binaries, docker_binaries,
                           k8s_binaries, package_mgmt_binaries, aide_wrapper_binaries, nids_binaries,
-                           monitoring_binaries, gitlab_binaries, mesos_slave_binaries)
+                           monitoring_binaries, gitlab_binaries, mesos_slave_binaries,
+                           keepalived_binaries,
+                           needrestart_binaries, phusion_passenger_binaries, chef_binaries, nomachine_binaries,
+                           x2go_binaries, db_mgmt_binaries, plesk_binaries)
    and not parent_ansible_running_python
    and not parent_bro_running_python
    and not parent_python_running_denyhosts
    and not parent_python_running_sdchecks
    and not parent_linux_image_upgrade_script
    and not parent_java_running_jenkins
+    and not proc.cmdline in (known_shell_spawn_cmdlines)
+    and not jenkins_scripts
+    and not parent_java_running_echo
+    and not parent_scripting_running_builds
+    and not parent_Xvfb_running_xkbcomp
+    and not parent_nginx_running_serf
+    and not parent_node_running_npm
+    and not parent_java_running_sbt
+    and not parent_beam_running_python
+    and not parent_strongswan_running_starter
+    and not run_by_chef
+    and not run_by_puppet
+    and not run_by_adclient
+    and not run_by_centrify
+    and not parent_dovecot_running_auth
+    and not run_by_foreman
+    and not parent_java_running_tomcat
+    and not parent_java_running_install4j
+    and not parent_cpanm_running_perl
+    and not parent_ruby_running_discourse
+    and not assemble_running_php
+    and not node_running_bitnami
+    and not parent_python_running_localstack
  output: >
    Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname
-    cmdline=%proc.cmdline pcmdline=%proc.pcmdline)
+    cmdline=%proc.cmdline pcmdline=%proc.pcmdline gparent=%proc.aname[2] ggparent=%proc.aname[3]
+    gggparent=%proc.aname[4] ggggparent=%proc.aname[5])
  priority: DEBUG
  tags: [host, shell]

@@ -505,34 +846,92 @@
              container.image startswith sysdig/sysdig or
              container.image startswith gcr.io/google_containers/hyperkube or
              container.image startswith quay.io/coreos/flannel or
-              container.image startswith gcr.io/google_containers/kube-proxy)
+              container.image startswith gcr.io/google_containers/kube-proxy or
+              container.image startswith calico/node)
+
+# Add conditions to this macro (probably in a separate file,
+# overwriting this macro) to specify additional containers that are
+# allowed to perform sensitive mounts.
+#
+# In this file, it just takes one of the images in trusted_containers
+# and repeats it.
+- macro: user_sensitive_mount_containers
+  condition: (container.image startswith sysdig/agent)

 # These containers are ones that are known to spawn lots of
 # shells. Generally, they are for systems where the container is used
 # as a packaging mechanism more than for a dedicated microservice.
 - macro: shell_spawning_containers
  condition: (container.image startswith jenkins or
-              container.image startswith gitlab/gitlab-ce)
+              container.image startswith gitlab/gitlab-ce or
+              container.image startswith gitlab/gitlab-ee)

 - rule: Launch Privileged Container
  desc: Detect the initial process started in a privileged container. Exceptions are made for known trusted images.
  condition: evt.type=execve and proc.vpid=1 and container and container.privileged=true and not trusted_containers
-  output: Privileged container started (user=%user.name command=%proc.cmdline %container.info)
+  output: Privileged container started (user=%user.name command=%proc.cmdline %container.info image=%container.image)
  priority: INFO
  tags: [container, cis]

+# For now, only considering a full mount of /etc as
+# sensitive. Ideally, this would also consider all subdirectories
+# below /etc as well, but the globbing mechanism used by sysdig
+# doesn't allow exclusions of a full pattern, only single characters.
 - macro: sensitive_mount
-  condition: (container.mount.dest[/proc*] != "N/A")
+  condition: (container.mount.dest[/proc*] != "N/A" or
+              container.mount.dest[/var/run/docker.sock] != "N/A" or
+              container.mount.dest[/] != "N/A" or
+              container.mount.dest[/etc] != "N/A" or
+              container.mount.dest[/root*] != "N/A")
+
+# The steps libcontainer performs to set up the root program for a container are:
+# - clone + exec self to a program runc:[0:PARENT]
+# - clone a program runc:[1:CHILD] which sets up all the namespaces
+# - clone a second program runc:[2:INIT] + exec to the root program.
+#   The parent of runc:[2:INIT] is runc:0:PARENT]
+# As soon as 1:CHILD is created, 0:PARENT exits, so there's a race
+#   where at the time 2:INIT execs the root program, 0:PARENT might have
+#   already exited, or might still be around. So we handle both.
+# We also let runc:[1:CHILD] count as the parent process, which can occur
+# when we lose events and lose track of state.
+
+- macro: container_entrypoint
+  condition: (not proc.pname exists or proc.pname in (runc:[0:PARENT], runc:[1:CHILD], docker-runc))

 - rule: Launch Sensitive Mount Container
  desc: >
    Detect the initial process started by a container that has a mount from a sensitive host directory
    (i.e. /proc). Exceptions are made for known trusted images.
-  condition: evt.type=execve and proc.vpid=1 and container and sensitive_mount and not trusted_containers
-  output: Container with sensitive mount started (user=%user.name command=%proc.cmdline %container.info)
+  condition: >
+    evt.type=execve and proc.vpid=1 and container
+    and sensitive_mount
+    and not trusted_containers
+    and not user_sensitive_mount_containers
+  output: Container with sensitive mount started (user=%user.name command=%proc.cmdline %container.info image=%container.image mounts=%container.mounts)
  priority: INFO
  tags: [container, cis]

+# In a local/user rules file, you could override this macro to
+# explicitly enumerate the container images that you want to run in
+# your environment. In this main falco rules file, there isn't any way
+# to know all the containers that can run, so any container is
+# alllowed, by using a filter that is guaranteed to evaluate to true
+# (the same proc.vpid=1 that's in the Launch Disallowed Container
+# rule). In the overridden macro, the condition would look something
+# like (container.image startswith vendor/container-1 or
+# container.image startswith vendor/container-2 or ...)
+
+- macro: allowed_containers
+  condition: (proc.vpid=1)
+
+- rule: Launch Disallowed Container
+  desc: >
+    Detect the initial process started by a container that is not in a list of allowed containers.
+  condition: evt.type=execve and proc.vpid=1 and container and not allowed_containers
+  output: Container started and not in allowed list (user=%user.name command=%proc.cmdline %container.info image=%container.image)
+  priority: WARNING
+  tags: [container]
+
 # Anything run interactively by root
 # - condition: evt.type != switch and user.name = root and proc.name != sshd and interactive
 #  output: "Interactive root (%user.name %proc.name %evt.dir %evt.type %evt.args %fd.name)"
@@ -560,43 +959,131 @@
 # work with, and the container name is autogenerated, so there isn't
 # any stable aspect of the software to work with. In this case, we
 # fall back to allowing certain command lines.
+
+- list: known_shell_spawn_cmdlines
+  items: [
+    '"sh -c uname -p 2> /dev/null"',
+    '"sh -c uname -s 2>&1"',
+    '"sh -c uname -r 2>&1"',
+    '"sh -c uname -v 2>&1"',
+    '"sh -c uname -a 2>&1"',
+    '"sh -c ruby -v 2>&1"',
+    '"sh -c getconf CLK_TCK"',
+    '"sh -c getconf PAGESIZE"',
+    '"sh -c LC_ALL=C LANG=C /sbin/ldconfig -p 2>/dev/null"',
+    '"sh -c /sbin/ldconfig -p 2>/dev/null"',
+    '"sh -c stty -a 2>/dev/null"',
+    '"sh -c node index.js"',
+    '"sh -c node index"',
+    '"sh -c node ./src/start.js"',
+    '"sh -c node app.js"',
+    '"sh -c node -e \"require(''nan'')\")"',
+    '"sh -c node $NODE_DEBUG_OPTION index.js "',
+    '"sh -c crontab -l 2"',
+    '"sh -c lsb_release -a"',
+    '"sh -c whoami"',
+    '"sh -c node_modules/.bin/bower-installer"',
+    '"sh -c /bin/hostname -f 2> /dev/null"',
+    '"sh -c locale -a"',
+    '"sh -c  -t -i"'
+    ]
+
 - list: known_container_shell_spawn_cmdlines
  items: [
+    known_shell_spawn_cmdlines,
    '"bash -c curl -f localhost:$API_PORT/admin/healthcheck"',
    '"sh -c curl http://localhost:6060/debug/vars>/dev/null "',
    '"sh -c curl http://localhost:6060/debug/vars>/dev/null"',
    '"sh -c  curl http://localhost:6060/debug/vars>/dev/null"',
    '"sh -c  curl http://localhost:6060/debug/vars>/dev/null "',
    '"sh -c  pgrep java && exit 0 || exit 1 "',
-    '"sh -c uname -p 2> /dev/null"',
    '"sh -c  echo healthy "',
    '"sh -c  echo alive "'
    ]

+# This list allows for easy additions to the set of commands allowed
+# to run shells in containers without having to without having to copy
+# and override the entire run shell in container macro. Once
+# https://github.com/draios/falco/issues/255 is fixed this will be a
+# bit easier, as someone could append of any of the existing lists.
+- list: user_known_container_shell_spawn_binaries
+  items: []
+
+# This macro allows for easy additions to the set of commands allowed
+# to run shells in containers without having to override the entire
+# rule. Its default value is an expression that always is false, which
+# becomes true when the "not ..." in the rule is applied.
+- macro: user_shell_container_exclusions
+  condition: (evt.num=0)
+
+# Temporarily adding as an example
+- macro: node_running_edi_dynamodb
+  condition: >
+    (proc.pname=node and (proc.pcmdline contains /var/www/edi/process.js or
+                          proc.pcmdline contains "sh -c /var/www/edi/bin/sftp.sh"))
+
 - rule: Run shell in container
  desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
  condition: >
    spawned_process and container
    and shell_procs
-    and proc.pname exists
-    and not proc.pname in (shell_binaries, make_binaries, docker_binaries, k8s_binaries,
+    and not container_entrypoint
+    and not proc.pname in (shell_binaries, make_binaries, docker_binaries, k8s_binaries, package_mgmt_binaries,
                           lxd_binaries, mesos_slave_binaries, aide_wrapper_binaries, nids_binaries,
-                           monitoring_binaries, gitlab_binaries, initdb, pg_ctl, awk, falco, cron,
-                           erl_child_setup, ceph, PM2)
+                           cron_binaries,
+                           user_known_container_shell_spawn_binaries,
+                           needrestart_binaries,
+                           phusion_passenger_binaries,
+                           chef_binaries,
+                           nomachine_binaries,
+                           x2go_binaries,
+                           xray_rabbitmq_binaries,
+                           db_mgmt_binaries,
+                           plesk_binaries,
+                           monitoring_binaries, gitlab_binaries, initdb, awk, falco, cron,
+                           erl_child_setup, erlexec, ceph, PM2, pycompile, py3compile, hhvm, npm, serf,
+                           runsv, supervisord, varnishd, crond, logrotate, timeout, tini,
+                           xrdb, xfce4-session, weave, logdna-agent, bundle, configure, luajit, nginx,
+                           beam.smp, paster, postfix-local, hawkular-metric, fluentd)
    and not trusted_containers
    and not shell_spawning_containers
+    and not parent_java_running_echo
+    and not parent_scripting_running_builds
+    and not parent_Xvfb_running_xkbcomp
+    and not mysql_image_running_healthcheck
+    and not parent_nginx_running_serf
    and not proc.cmdline in (known_container_shell_spawn_cmdlines)
+    and not parent_node_running_npm
+    and not user_shell_container_exclusions
+    and not node_running_edi_dynamodb
+    and not run_by_h2o
+    and not run_by_passenger_agent
+    and not parent_java_running_jenkins
+    and not parent_beam_running_python
+    and not jenkins_scripts
+    and not bundle_running_ruby
+    and not parent_dovecot_running_auth
+    and not parent_strongswan_running_starter
+    and not parent_phusion_passenger_my_init
+    and not parent_java_running_confluence
+    and not parent_java_running_tomcat
+    and not parent_java_running_install4j
+    and not ics_running_java
+    and not parent_ruby_running_discourse
+    and not assemble_running_php
+    and not node_running_bitnami
+    and not parent_python_running_localstack
  output: >
-    Shell spawned in a container other than entrypoint (user=%user.name %container.info
-    shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)
-  priority: NOTICE
+    Shell spawned in a container other than entrypoint (user=%user.name %container.info image=%container.image
+    shell=%proc.name pcmdline=%proc.pcmdline cmdline=%proc.cmdline parent=%proc.pname gparent=%proc.aname[2] ggparent=%proc.aname[3])
+  priority: DEBUG
  tags: [container, shell]

 # sockfamily ip is to exclude certain processes (like 'groups') that communicate on unix-domain sockets
 # systemd can listen on ports to launch things like sshd on demand
 - rule: System procs network activity
  desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
-  condition: (fd.sockfamily = ip and system_procs) and (inbound or outbound) and not proc.name=systemd
+  condition: (fd.sockfamily = ip and system_procs) and (inbound or outbound) and not proc.name in (systemd, hostid)
  output: >
    Known system binary sent/received network traffic
    (user=%user.name command=%proc.cmdline connection=%fd.name)
@@ -614,7 +1101,19 @@

 - macro: somebody_becoming_themself
  condition: ((user.name=nobody and evt.arg.uid=nobody) or
-              (user.name=www-data and evt.arg.uid=www-data))
+              (user.name=www-data and evt.arg.uid=www-data) or
+              (user.name=_apt and evt.arg.uid=_apt) or
+              (user.name=postfix and evt.arg.uid=postfix) or
+              (user.name=pki-agent and evt.arg.uid=pki-agent) or
+              (user.name=pki-acme and evt.arg.uid=pki-acme) or
+              (user.name=nfsnobody and evt.arg.uid=nfsnobody))
+
+# In containers, the user name might be for a uid that exists in the
+# container but not on the host. (See
+# https://github.com/draios/sysdig/issues/954). So in that case, allow
+# a setuid.
+- macro: known_user_in_container
+  condition: (container and user.name != "N/A")

 # sshd, mail programs attempt to setuid to root even when running as non-root. Excluding here to avoid meaningless FPs
 - rule: Non sudo setuid
@@ -622,10 +1121,12 @@
    an attempt to change users by calling setuid. sudo/su are excluded. users "root" and "nobody"
    suing to itself are also excluded, as setuid calls typically involve dropping privileges.
  condition: >
-    evt.type=setuid and evt.dir=> and
-    not user.name=root and not somebody_becoming_themself
-    and not proc.name in (userexec_binaries, mail_binaries, docker_binaries,
-    sshd, dbus-daemon-lau, ping, ping6, critical-stack-)
+    evt.type=setuid and evt.dir=>
+    and (known_user_in_container or not container)
+    and not user.name=root and not somebody_becoming_themself
+    and not proc.name in (known_setuid_binaries, userexec_binaries, mail_binaries, docker_binaries,
+                          nomachine_binaries)
+    and not java_running_sdjagent
  output: >
    Unexpected setuid call by non-sudo, non-root program (user=%user.name parent=%proc.pname
    command=%proc.cmdline uid=%evt.arg.uid)
@@ -637,13 +1138,19 @@
    activity by any programs that can manage users, passwords, or permissions. sudo and su are excluded.
    Activity in containers is also excluded--some containers create custom users on top
    of a base linux distribution at startup.
+    Some innocuous commandlines that don't actually change anything are excluded.
  condition: >
    spawned_process and proc.name in (user_mgmt_binaries) and
-    not proc.name in (su, sudo) and not container and
-    not proc.pname in (cron_binaries, systemd, run-parts)
+    not proc.name in (su, sudo, lastlog, nologin, unix_chkpwd) and not container and
+    not proc.pname in (cron_binaries, systemd, run-parts) and
+    not proc.cmdline startswith "passwd -S" and
+    not proc.cmdline startswith "useradd -D" and
+    not proc.cmdline startswith "systemd --version" and
+    not run_by_qualys and
+    not run_by_sumologic_securefiles
  output: >
    User management binary command run outside of container
-    (user=%user.name command=%proc.cmdline parent=%proc.pname)
+    (user=%user.name command=%proc.cmdline parent=%proc.pname gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4])
  priority: NOTICE
  tags: [host, users]

--- a/test/falco_test.py
+++ b/test/falco_test.py
@@ -30,6 +30,7 @@ class FalcoTest(Test):
            self.trace_file = os.path.join(self.basedir, self.trace_file)

        self.json_output = self.params.get('json_output', '*', default=False)
+        self.priority = self.params.get('priority', '*', default='debug')
        self.rules_file = self.params.get('rules_file', '*', default=os.path.join(self.basedir, '../rules/falco_rules.yaml'))

        if not isinstance(self.rules_file, list):
@@ -295,7 +296,7 @@ class FalcoTest(Test):
            res = process.run("docker rm falco-test")

        elif self.package.endswith(".deb"):
-            cmdline = "dpkg -r falco"
+            cmdline = "dpkg --purge falco"
            self.log.debug("Uninstalling debian package via \"{}\"".format(cmdline))
            res = process.run(cmdline, timeout=120, sudo=True)

@@ -347,8 +348,8 @@ class FalcoTest(Test):
            trace_arg = "-e {}".format(self.trace_file)

        # Run falco
-        cmd = '{} {} {} -c {} {} -o json_output={} -v'.format(
-            self.falco_binary_path, self.rules_args, self.disabled_args, self.conf_file, trace_arg, self.json_output)
+        cmd = '{} {} {} -c {} {} -o json_output={} -o priority={} -v'.format(
+            self.falco_binary_path, self.rules_args, self.disabled_args, self.conf_file, trace_arg, self.json_output, self.priority)

        for tag in self.disable_tags:
            cmd += ' -T {}'.format(tag)
--- a/test/falco_tests.yaml
+++ b/test/falco_tests.yaml
@@ -129,6 +129,21 @@ trace_files: !mux
      - rules/double_rule.yaml
    trace_file: trace_files/cat_write.scap

+  multiple_rules_suppress_info:
+    detect: True
+    detect_level:
+      - WARNING
+      - ERROR
+    priority: WARNING
+    detect_counts:
+      - open_from_cat: 8
+      - exec_from_cat: 1
+      - access_from_cat: 0
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/double_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
  multiple_rules_overriding:
    detect: False
    rules_file:
@@ -599,3 +614,44 @@ trace_files: !mux
    rules_file:
      - rules/list_append_false.yaml
    trace_file: trace_files/cat_write.scap
+
+  macro_append_failure:
+    exit_status: 1
+    stderr_contains: "Macro my_macro has 'append' key but no macro by that name already exists. Exiting"
+    rules_file:
+      - rules/macro_append_failure.yaml
+    trace_file: trace_files/cat_write.scap
+
+  macro_append:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/macro_append.yaml
+    trace_file: trace_files/cat_write.scap
+
+  macro_append_false:
+    detect: False
+    rules_file:
+      - rules/macro_append_false.yaml
+    trace_file: trace_files/cat_write.scap
+
+  rule_append_failure:
+    exit_status: 1
+    stderr_contains: "Rule my_rule has 'append' key but no rule by that name already exists. Exiting"
+    rules_file:
+      - rules/rule_append_failure.yaml
+    trace_file: trace_files/cat_write.scap
+
+  rule_append:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/rule_append.yaml
+    trace_file: trace_files/cat_write.scap
+
+  rule_append_false:
+    detect: False
+    rules_file:
+      - rules/rule_append_false.yaml
+    trace_file: trace_files/cat_write.scap
+
--- a/test/falco_traces.yaml.in
+++ b/test/falco_traces.yaml.in
@@ -149,7 +149,7 @@ traces: !mux
  shell-in-container:
    trace_file: traces-positive/shell-in-container.scap
    detect: True
-    detect_level: NOTICE
+    detect_level: DEBUG
    detect_counts:
      - "Run shell in container": 1

--- a/test/rules/macro_append.yaml
+++ b/test/rules/macro_append.yaml
@@ -0,0 +1,12 @@
+- macro: my_macro
+  condition: proc.name=not-cat
+
+- macro: my_macro
+  append: true
+  condition: or proc.name=cat
+
+- rule: Open From Cat
+  desc: A process named cat does an open
+  condition: evt.type=open and my_macro
+  output: "An open was seen (command=%proc.cmdline)"
+  priority: WARNING
--- a/test/rules/macro_append_failure.yaml
+++ b/test/rules/macro_append_failure.yaml
@@ -0,0 +1,3 @@
+- macro: my_macro
+  condition: proc.name=not-cat
+  append: true
--- a/test/rules/macro_append_false.yaml
+++ b/test/rules/macro_append_false.yaml
@@ -0,0 +1,12 @@
+- macro: my_macro
+  condition: proc.name=cat
+
+- macro: my_macro
+  append: false
+  condition: proc.name=not-cat
+
+- rule: Open From Cat
+  desc: A process named cat does an open
+  condition: evt.type=open and my_macro
+  output: "An open was seen (command=%proc.cmdline)"
+  priority: WARNING
--- a/test/rules/rule_append.yaml
+++ b/test/rules/rule_append.yaml
@@ -0,0 +1,9 @@
+- rule: my_rule
+  desc: A process named cat does an open
+  condition: evt.type=open and fd.name=not-a-real-file
+  output: "An open of /dev/null was seen (command=%proc.cmdline)"
+  priority: WARNING
+
+- rule: my_rule
+  append: true
+  condition: or fd.name=/dev/null
--- a/test/rules/rule_append_failure.yaml
+++ b/test/rules/rule_append_failure.yaml
@@ -0,0 +1,3 @@
+- rule: my_rule
+  condition: evt.type=open
+  append: true
--- a/test/rules/rule_append_false.yaml
+++ b/test/rules/rule_append_false.yaml
@@ -0,0 +1,9 @@
+- rule: my_rule
+  desc: A process named cat does an open
+  condition: evt.type=open and fd.name=/dev/null
+  output: "An open of /dev/null was seen (command=%proc.cmdline)"
+  priority: WARNING
+
+- rule: my_rule
+  append: true
+  condition: and fd.name=not-a-real-file
--- a/userspace/engine/CMakeLists.txt
+++ b/userspace/engine/CMakeLists.txt
@@ -27,5 +27,3 @@ install(DIRECTORY lua
 	DESTINATION "${FALCO_SHARE_DIR}"
 	FILES_MATCHING PATTERN *.lua)
 endif()
-
-add_subdirectory("${PROJECT_SOURCE_DIR}/../falco/rules" "${PROJECT_BINARY_DIR}/rules")
--- a/userspace/engine/config_falco_engine.h.in
+++ b/userspace/engine/config_falco_engine.h.in
@@ -18,5 +18,5 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #pragma once

-#define FALCO_ENGINE_LUA_DIR "${FALCO_SHARE_DIR}/lua/"
+#define FALCO_ENGINE_LUA_DIR "${FALCO_ABSOLUTE_SHARE_DIR}/lua/"
 #define FALCO_ENGINE_SOURCE_LUA_DIR "${PROJECT_SOURCE_DIR}/../falco/userspace/engine/lua/"
--- a/userspace/engine/falco_common.cpp
+++ b/userspace/engine/falco_common.cpp
@@ -21,6 +21,16 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "config_falco_engine.h"
 #include "falco_common.h"

+std::vector<std::string> falco_common::priority_names = {
+	"Emergency",
+	"Alert",
+	"Critical",
+	"Error",
+	"Warning",
+	"Notice",
+	"Informational",
+	"Debug"};
+
 falco_common::falco_common()
 {
 	m_ls = lua_open();
--- a/userspace/engine/falco_common.h
+++ b/userspace/engine/falco_common.h
@@ -74,6 +74,22 @@ public:

 	void set_inspector(sinsp *inspector);

+        // Priority levels, as a vector of strings
+	static std::vector<std::string> priority_names;
+
+	// Same as numbers/indices into the above vector
+	enum priority_type
+	{
+		PRIORITY_EMERGENCY = 0,
+		PRIORITY_ALERT = 1,
+		PRIORITY_CRITICAL = 2,
+		PRIORITY_ERROR = 3,
+		PRIORITY_WARNING = 4,
+		PRIORITY_NOTICE = 5,
+		PRIORITY_INFORMATIONAL = 6,
+		PRIORITY_DEBUG = 7
+	};
+
 protected:
 	lua_State *m_ls;

--- a/userspace/engine/falco_engine.cpp
+++ b/userspace/engine/falco_engine.cpp
@@ -41,6 +41,7 @@ using namespace std;

 falco_engine::falco_engine(bool seed_rng)
 	: m_rules(NULL), m_next_ruleset_id(0),
+	  m_min_priority(falco_common::PRIORITY_DEBUG),
 	  m_sampling_ratio(1), m_sampling_multiplier(0),
 	  m_replace_container_info(false)
 {
@@ -89,7 +90,7 @@ void falco_engine::load_rules(const string &rules_content, bool verbose, bool al
 	bool json_output = false;
 	falco_formats::init(m_inspector, m_ls, json_output);

-	m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info);
+	m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info, m_min_priority);
 }

 void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events)
@@ -134,6 +135,11 @@ void falco_engine::enable_rule_by_tag(const set<string> &tags, bool enabled)
 	enable_rule_by_tag(tags, enabled, m_default_ruleset);
 }

+void falco_engine::set_min_priority(falco_common::priority_type priority)
+{
+	m_min_priority = priority;
+}
+
 uint16_t falco_engine::find_ruleset_id(const std::string &ruleset)
 {
 	auto it = m_known_rulesets.lower_bound(ruleset);
@@ -148,6 +154,13 @@ uint16_t falco_engine::find_ruleset_id(const std::string &ruleset)
 	return it->second;
 }

+void falco_engine::evttypes_for_ruleset(std::vector<bool> &evttypes, const std::string &ruleset)
+{
+	uint16_t ruleset_id = find_ruleset_id(ruleset);
+
+	return m_evttype_filter->evttypes_for_ruleset(evttypes, ruleset_id);
+}
+
 unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev, uint16_t ruleset_id)
 {
 	if(should_drop_evt())
@@ -178,7 +191,7 @@ unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev,
 		res->evt = ev;
 		const char *p =  lua_tostring(m_ls, -3);
 		res->rule = p;
-		res->priority = lua_tostring(m_ls, -2);
+		res->priority_num = (falco_common::priority_type) lua_tonumber(m_ls, -2);
 		res->format = lua_tostring(m_ls, -1);
 		lua_pop(m_ls, 3);
 	}
--- a/userspace/engine/falco_engine.h
+++ b/userspace/engine/falco_engine.h
@@ -67,10 +67,13 @@ public:
 	// Wrapper that assumes the default ruleset
 	void enable_rule_by_tag(const std::set<std::string> &tags, bool enabled);

+	// Only load rules having this priority or more severe.
+	void set_min_priority(falco_common::priority_type priority);
+
 	struct rule_result {
 		sinsp_evt *evt;
 		std::string rule;
-		std::string priority;
+		falco_common::priority_type priority_num;
 		std::string format;
 	};

@@ -82,6 +85,12 @@ public:
 	//
 	uint16_t find_ruleset_id(const std::string &ruleset);

+	//
+	// Given a ruleset, fill in a bitset containing the event
+	// types for which this ruleset can run.
+	//
+	void evttypes_for_ruleset(std::vector<bool> &evttypes, const std::string &ruleset);
+
 	//
 	// Given an event, check it against the set of rules in the
 	// engine and if a matching rule is found, return details on
@@ -158,6 +167,7 @@ private:
 	uint16_t m_next_ruleset_id;
 	std::map<string, uint16_t> m_known_rulesets;
 	std::unique_ptr<sinsp_evttype_filter> m_evttype_filter;
+	falco_common::priority_type m_min_priority;

 	//
 	// Here's how the sampling ratio and multiplier influence
--- a/userspace/engine/formats.cpp
+++ b/userspace/engine/formats.cpp
@@ -92,6 +92,7 @@ int falco_formats::free_formatters(lua_State *ls)
 int falco_formats::format_event (lua_State *ls)
 {
 	string line;
+	string json_line;

 	if (!lua_isstring(ls, -1) ||
 	    !lua_isstring(ls, -2) ||
@@ -109,6 +110,20 @@ int falco_formats::format_event (lua_State *ls)

 	try {
 		s_formatters->tostring(evt, sformat, &line);
+
+		if(s_json_output)
+		{
+			s_inspector->set_buffer_format(sinsp_evt::PF_JSON);
+			s_formatters->tostring(evt, sformat, &json_line);
+
+			// The formatted string might have a leading newline. If it does, remove it.
+			if (json_line[0] == '\n')
+			{
+				json_line.erase(0, 1);
+			}
+
+			s_inspector->set_buffer_format(sinsp_evt::PF_NORMAL);
+		}
 	}
 	catch (sinsp_exception& e)
 	{
@@ -117,13 +132,15 @@ int falco_formats::format_event (lua_State *ls)
 		lua_error(ls);
 	}

-	// For JSON output, the formatter returned just the output
-	// string containing the format text and values. Use this to
-	// build a more detailed object containing the event time,
-	// rule, severity, full output, and fields.
+	// For JSON output, the formatter returned a json-as-text
+	// object containing all the fields in the original format
+	// message as well as the event time in ns. Use this to build
+	// a more detailed object containing the event time, rule,
+	// severity, full output, and fields.
 	if (s_json_output) {
 		Json::Value event;
 		Json::FastWriter writer;
+		string full_line;

 		// Convert the time-as-nanoseconds to a more json-friendly ISO8601.
 		time_t evttime = evt->get_ts()/1000000000;
@@ -138,16 +155,26 @@ int falco_formats::format_event (lua_State *ls)
 		event["time"] = iso8601evttime;
 		event["rule"] = rule;
 		event["priority"] = level;
+		// This is the filled-in output line.
 		event["output"] = line;

-		line = writer.write(event);
+		full_line = writer.write(event);

 		// Json::FastWriter may add a trailing newline. If it
 		// does, remove it.
-		if (line[line.length()-1] == '\n')
+		if (full_line[full_line.length()-1] == '\n')
 		{
-			line.resize(line.length()-1);
+			full_line.resize(full_line.length()-1);
 		}
+
+		// Cheat-graft the output from the formatter into this
+		// string. Avoids an unnecessary json parse just to
+		// merge the formatted fields at the object level.
+		full_line.pop_back();
+		full_line.append(", \"output_fields\": ");
+		full_line.append(json_line);
+		full_line.append("}");
+		line = full_line;
 	}

 	lua_pushstring(ls, line.c_str());
--- a/userspace/engine/lua/rule_loader.lua
+++ b/userspace/engine/lua/rule_loader.lua
@@ -58,6 +58,17 @@ function map(f, arr)
   return res
 end

+priorities = {"Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"}
+
+local function priority_num_for(s)
+   s = string.lower(s)
+   for i,v in ipairs(priorities) do
+      if (string.find(string.lower(v), "^"..s)) then
+	 return i - 1 -- (numbers start at 0, lua indices start at 1)
+      end
+   end
+   error("Invalid priority level: "..s)
+end

 --[[
   Take a filter AST and set it up in the libsinsp runtime, using the filter API.
@@ -171,7 +182,7 @@ function table.tostring( tbl )
 end


-function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replace_container_info)
+function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replace_container_info, min_priority)

   compiler.set_verbose(verbose)
   compiler.set_all_events(all_events)
@@ -208,7 +219,23 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	    end
 	 end

-	 state.macros_by_name[v['macro']] = v
+	 -- Possibly append to the condition field of an existing macro
+	 append = false
+
+	 if v['append'] then
+	    append = v['append']
+	 end
+
+	 if append then
+	    if state.macros_by_name[v['macro']] == nil then
+	       error ("Macro " ..v['macro'].. " has 'append' key but no macro by that name already exists")
+	    end
+
+	    state.macros_by_name[v['macro']]['condition'] = state.macros_by_name[v['macro']]['condition'] .. " " .. v['condition']
+
+	 else
+	    state.macros_by_name[v['macro']] = v
+	 end

      elseif (v['list']) then

@@ -247,25 +274,54 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	    error ("Missing name in rule")
 	 end

-	 for i, field in ipairs({'condition', 'output', 'desc', 'priority'}) do
-	    if (v[field] == nil) then
-	       error ("Missing "..field.." in rule with name "..v['rule'])
+	 -- Possibly append to the condition field of an existing rule
+	 append = false
+
+	 if v['append'] then
+	    append = v['append']
+	 end
+
+	 if append then
+
+	    -- For append rules, all you need is the condition
+	    for i, field in ipairs({'condition'}) do
+	       if (v[field] == nil) then
+		  error ("Missing "..field.." in rule with name "..v['rule'])
+	       end
+	    end
+
+	    if state.rules_by_name[v['rule']] == nil then
+	       error ("Rule " ..v['rule'].. " has 'append' key but no rule by that name already exists")
+	    end
+
+	    state.rules_by_name[v['rule']]['condition'] = state.rules_by_name[v['rule']]['condition'] .. " " .. v['condition']
+
+	 else
+
+	    for i, field in ipairs({'condition', 'output', 'desc', 'priority'}) do
+	       if (v[field] == nil) then
+		  error ("Missing "..field.." in rule with name "..v['rule'])
+	       end
+	    end
+
+	    -- Convert the priority-as-string to a priority-as-number now
+	    v['priority_num'] = priority_num_for(v['priority'])
+
+	    if v['priority_num'] <= min_priority then
+	       -- Note that we can overwrite rules, but the rules are still
+	       -- loaded in the order in which they first appeared,
+	       -- potentially across multiple files.
+	       if state.rules_by_name[v['rule']] == nil then
+		  state.ordered_rule_names[#state.ordered_rule_names+1] = v['rule']
+	       end
+
+	       -- The output field might be a folded-style, which adds a
+	       -- newline to the end. Remove any trailing newlines.
+	       v['output'] = compiler.trim(v['output'])
+
+	       state.rules_by_name[v['rule']] = v
 	    end
 	 end
-
-	 -- Note that we can overwrite rules, but the rules are still
-	 -- loaded in the order in which they first appeared,
-	 -- potentially across multiple files.
-	 if state.rules_by_name[v['rule']] == nil then
-	    state.ordered_rule_names[#state.ordered_rule_names+1] = v['rule']
-	 end
-
-	 -- The output field might be a folded-style, which adds a
-	 -- newline to the end. Remove any trailing newlines.
-	 v['output'] = compiler.trim(v['output'])
-
-	 state.rules_by_name[v['rule']] = v
-
      else
 	 error ("Unknown rule object: "..table.tostring(v))
      end
@@ -464,7 +520,7 @@ function on_event(evt_, rule_id)
   -- Prefix output with '*' so formatting is permissive
   output = "*"..rule.output

-   return rule.rule, rule.priority, output
+   return rule.rule, rule.priority_num, output
 end

 function print_stats()
--- a/userspace/engine/rules.cpp
+++ b/userspace/engine/rules.cpp
@@ -145,7 +145,8 @@ void falco_rules::enable_rule(string &rule, bool enabled)

 void falco_rules::load_rules(const string &rules_content,
 			     bool verbose, bool all_events,
-			     string &extra, bool replace_container_info)
+			     string &extra, bool replace_container_info,
+			     falco_common::priority_type min_priority)
 {
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
@@ -221,7 +222,8 @@ void falco_rules::load_rules(const string &rules_content,
 		lua_pushboolean(m_ls, (all_events ? 1 : 0));
 		lua_pushstring(m_ls, extra.c_str());
 		lua_pushboolean(m_ls, (replace_container_info ? 1 : 0));
-		if(lua_pcall(m_ls, 6, 0, 0) != 0)
+		lua_pushnumber(m_ls, min_priority);
+		if(lua_pcall(m_ls, 7, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules:" + string(lerr);
--- a/userspace/engine/rules.h
+++ b/userspace/engine/rules.h
@@ -24,6 +24,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #include "lua_parser.h"

+#include "falco_common.h"
+
 class falco_engine;

 class falco_rules
@@ -32,7 +34,8 @@ class falco_rules
 	falco_rules(sinsp* inspector, falco_engine *engine, lua_State *ls);
 	~falco_rules();
 	void load_rules(const string &rules_content, bool verbose, bool all_events,
-			std::string &extra, bool replace_container_info);
+			std::string &extra, bool replace_container_info,
+			falco_common::priority_type min_priority);
 	void describe_rule(string *rule);

 	static void init(lua_State *ls);
--- a/userspace/falco/config_falco.h.in
+++ b/userspace/falco/config_falco.h.in
@@ -5,7 +5,7 @@
 #define FALCO_LUA_DIR "${CMAKE_INSTALL_PREFIX}/${FALCO_SHARE_DIR}/lua/"
 #define FALCO_SOURCE_DIR "${PROJECT_SOURCE_DIR}"
 #define FALCO_SOURCE_CONF_FILE "${PROJECT_SOURCE_DIR}/falco.yaml"
-#define FALCO_INSTALL_CONF_FILE "/etc/falco.yaml"
+#define FALCO_INSTALL_CONF_FILE "/etc/falco/falco.yaml"
 #define FALCO_SOURCE_LUA_DIR "${PROJECT_SOURCE_DIR}/userspace/falco/lua/"

 #define PROBE_NAME "${PROBE_NAME}"
--- a/userspace/falco/configuration.cpp
+++ b/userspace/falco/configuration.cpp
@@ -22,7 +22,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 using namespace std;

 falco_configuration::falco_configuration()
-	: m_config(NULL)
+	: m_buffered_outputs(true),
+	  m_config(NULL)
 {
 }

@@ -51,20 +52,37 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio

 	init_cmdline_options(cmdline_options);

-	m_rules_filenames.push_back(m_config->get_scalar<string>("rules_file", "/etc/falco_rules.yaml"));
+	list<string> rules_files;
+
+	m_config->get_sequence<list<string>>(rules_files, string("rules_file"));
+
+	for(auto &file : rules_files)
+	{
+		// Here, we only include files that exist
+		struct stat buffer;
+		if(stat(file.c_str(), &buffer) == 0)
+		{
+			m_rules_filenames.push_back(file);
+		}
+	}
+
 	m_json_output = m_config->get_scalar<bool>("json_output", false);

 	falco_outputs::output_config file_output;
 	file_output.name = "file";
 	if (m_config->get_scalar<bool>("file_output", "enabled", false))
 	{
-		string filename;
+		string filename, keep_alive;
 		filename = m_config->get_scalar<string>("file_output", "filename", "");
 		if (filename == string(""))
 		{
 			throw invalid_argument("Error reading config file (" + m_config_file + "): file output enabled but no filename in configuration block");
 		}
 		file_output.options["filename"] = filename;
+
+		keep_alive = m_config->get_scalar<string>("file_output", "keep_alive", "");
+		file_output.options["keep_alive"] = keep_alive;
+
 		m_outputs.push_back(file_output);
 	}

@@ -86,13 +104,17 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 	program_output.name = "program";
 	if (m_config->get_scalar<bool>("program_output", "enabled", false))
 	{
-		string program;
+		string program, keep_alive;
 		program = m_config->get_scalar<string>("program_output", "program", "");
 		if (program == string(""))
 		{
 			throw sinsp_exception("Error reading config file (" + m_config_file + "): program output enabled but no program in configuration block");
 		}
 		program_output.options["program"] = program;
+
+		keep_alive = m_config->get_scalar<string>("program_output", "keep_alive", "");
+		program_output.options["keep_alive"] = keep_alive;
+
 		m_outputs.push_back(program_output);
 	}

@@ -108,6 +130,21 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 	m_notifications_rate = m_config->get_scalar<uint32_t>("outputs", "rate", 1);
 	m_notifications_max_burst = m_config->get_scalar<uint32_t>("outputs", "max_burst", 1000);

+	string priority = m_config->get_scalar<string>("priority", "debug");
+	vector<string>::iterator it;
+
+	auto comp = [priority] (string &s) {
+		return (strcasecmp(s.c_str(), priority.c_str()) == 0);
+	};
+
+	if((it = std::find_if(falco_common::priority_names.begin(), falco_common::priority_names.end(), comp)) == falco_common::priority_names.end())
+	{
+		throw invalid_argument("Unknown priority \"" + priority + "\"--must be one of emergency, alert, critical, error, warning, notice, informational, debug");
+	}
+	m_min_priority = (falco_common::priority_type) (it - falco_common::priority_names.begin());
+
+	m_buffered_outputs = m_config->get_scalar<bool>("buffered_outputs", true);
+
 	falco_logger::log_stderr = m_config->get_scalar<bool>("log_stderr", false);
 	falco_logger::log_syslog = m_config->get_scalar<bool>("log_syslog", true);
 }
--- a/userspace/falco/configuration.h
+++ b/userspace/falco/configuration.h
@@ -18,6 +18,9 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.

 #pragma once

+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
 #include <yaml-cpp/yaml.h>
 #include <string>
 #include <vector>
@@ -127,6 +130,27 @@ public:
 		}
 	}

+	// called with the last variadic arg (where the sequence is expected to be found)
+	template <typename T>
+	void get_sequence(T& ret, const std::string& name)
+	{
+		YAML::Node child_node = m_root[name];
+		if(child_node.IsDefined())
+		{
+			if(child_node.IsSequence())
+			{
+				for(const YAML::Node& item : child_node)
+				{
+					ret.insert(ret.end(), item.as<typename T::value_type>());
+				}
+			}
+			else if(child_node.IsScalar())
+			{
+				ret.insert(ret.end(), child_node.as<typename T::value_type>());
+			}
+		}
+	}
+
 private:
 	YAML::Node m_root;
 };
@@ -146,6 +170,10 @@ class falco_configuration
 	std::vector<falco_outputs::output_config> m_outputs;
 	uint32_t m_notifications_rate;
 	uint32_t m_notifications_max_burst;
+
+	falco_common::priority_type m_min_priority;
+
+	bool m_buffered_outputs;
 private:
 	void init_cmdline_options(std::list<std::string> &cmdline_options);

--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -107,6 +107,11 @@ static void usage()
 	   "                               Can not be specified with -t.\n"
 	   " -t <tag>                      Only run those rules with a tag=<tag>. Can be specified multiple times.\n"
 	   "                               Can not be specified with -T/-D.\n"
+	   " -U,--unbuffered               Turn off output buffering to configured outputs. This causes every\n"
+	   "                               single line emitted by falco to be flushed, which generates higher CPU\n"
+	   "                               usage but is useful when piping those outputs into another process\n"
+	   "                               or into a script.\n"
+	   " -V,--validate <rules_file>    Read the contents of the specified rules file and exit\n"
 	   " -v                            Verbose output.\n"
           " --version                     Print version number.\n"
 	   "\n"
@@ -212,7 +217,7 @@ uint64_t do_inspect(falco_engine *engine,
 		unique_ptr<falco_engine::rule_result> res = engine->process_event(ev);
 		if(res)
 		{
-			outputs->handle_event(res->evt, res->rule, res->priority, res->format);
+			outputs->handle_event(res->evt, res->rule, res->priority_num, res->format);
 		}

 		num_evts++;
@@ -240,6 +245,7 @@ int falco_init(int argc, char **argv)
 	string pidfilename = "/var/run/falco.pid";
 	bool describe_all_rules = false;
 	string describe_rule = "";
+	string validate_rules_file = "";
 	string stats_filename = "";
 	bool verbose = false;
 	bool all_events = false;
@@ -256,6 +262,8 @@ int falco_init(int argc, char **argv)
 	int file_limit = 0;
 	unsigned long event_limit = 0L;
 	bool compress = false;
+	bool buffered_outputs = true;
+	bool buffered_cmdline = false;

 	// Used for stats
 	uint64_t num_evts;
@@ -272,7 +280,9 @@ int falco_init(int argc, char **argv)
 		{"option", required_argument, 0, 'o'},
 		{"print", required_argument, 0, 'p' },
 		{"pidfile", required_argument, 0, 'P' },
+		{"unbuffered", no_argument, 0, 'U' },
 		{"version", no_argument, 0, 0 },
+		{"validate", required_argument, 0, 0 },
 		{"writefile", required_argument, 0, 'w' },

 		{0, 0, 0, 0}
@@ -290,7 +300,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:vw:",
+                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
                                        long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -378,9 +388,16 @@ int falco_init(int argc, char **argv)
 			case 't':
 				enabled_rule_tags.insert(optarg);
 				break;
+			case 'U':
+				buffered_outputs = false;
+				buffered_cmdline = true;
+				break;
 			case 'v':
 				verbose = true;
 				break;
+			case 'V':
+				validate_rules_file = optarg;
+				break;
 			case 'w':
 				outfile = optarg;
 				break;
@@ -443,6 +460,14 @@ int falco_init(int argc, char **argv)
 			}
 		}

+		if(validate_rules_file != "")
+		{
+			falco_logger::log(LOG_INFO, "Validating rules file: " + validate_rules_file + "...\n");
+			engine->load_rules_file(validate_rules_file, verbose, all_events);
+			falco_logger::log(LOG_INFO, "Ok\n");
+			goto exit;
+		}
+
 		falco_configuration config;
 		if (conf_filename.size())
 		{
@@ -461,6 +486,18 @@ int falco_init(int argc, char **argv)
 			config.m_rules_filenames = rules_filenames;
 		}

+		engine->set_min_priority(config.m_min_priority);
+
+		if(buffered_cmdline)
+		{
+			config.m_buffered_outputs = buffered_outputs;
+		}
+
+		if(config.m_rules_filenames.size() == 0)
+		{
+			throw std::invalid_argument("You must specify at least one rules file via -r or a rules_file entry in falco.yaml");
+		}
+
 		for (auto filename : config.m_rules_filenames)
 		{
 			engine->load_rules_file(filename, verbose, all_events);
@@ -501,7 +538,9 @@ int falco_init(int argc, char **argv)
 			engine->enable_rule_by_tag(enabled_rule_tags, true);
 		}

-		outputs->init(config.m_json_output, config.m_notifications_rate, config.m_notifications_max_burst);
+		outputs->init(config.m_json_output,
+			      config.m_notifications_rate, config.m_notifications_max_burst,
+			      config.m_buffered_outputs);

 		if(!all_events)
 		{
--- a/userspace/falco/falco_outputs.cpp
+++ b/userspace/falco/falco_outputs.cpp
@@ -27,7 +27,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 using namespace std;

 falco_outputs::falco_outputs()
-	: m_initialized(false)
+	: m_initialized(false),
+	  m_buffered(true)
 {

 }
@@ -51,7 +52,7 @@ falco_outputs::~falco_outputs()
 	}
 }

-void falco_outputs::init(bool json_output, uint32_t rate, uint32_t max_burst)
+void falco_outputs::init(bool json_output, uint32_t rate, uint32_t max_burst, bool buffered)
 {
 	// The engine must have been given an inspector by now.
 	if(! m_inspector)
@@ -70,12 +71,14 @@ void falco_outputs::init(bool json_output, uint32_t rate, uint32_t max_burst)

 	m_notifications_tb.init(rate, max_burst);

+	m_buffered = buffered;
+
 	m_initialized = true;
 }

 void falco_outputs::add_output(output_config oc)
 {
-	uint8_t nargs = 1;
+	uint8_t nargs = 2;
 	lua_getglobal(m_ls, m_lua_add_output.c_str());

 	if(!lua_isfunction(m_ls, -1))
@@ -83,11 +86,12 @@ void falco_outputs::add_output(output_config oc)
 		throw falco_exception("No function " + m_lua_add_output + " found. ");
 	}
 	lua_pushstring(m_ls, oc.name.c_str());
+	lua_pushnumber(m_ls, (m_buffered ? 1 : 0));

 	// If we have options, build up a lua table containing them
 	if (oc.options.size())
 	{
-		nargs = 2;
+		nargs = 3;
 		lua_createtable(m_ls, 0, oc.options.size());

 		for (auto it = oc.options.cbegin(); it != oc.options.cend(); ++it)
@@ -105,7 +109,7 @@ void falco_outputs::add_output(output_config oc)

 }

-void falco_outputs::handle_event(sinsp_evt *ev, string &rule, string &priority, string &format)
+void falco_outputs::handle_event(sinsp_evt *ev, string &rule, falco_common::priority_type priority, string &format)
 {
 	if(!m_notifications_tb.claim())
 	{
@@ -119,10 +123,11 @@ void falco_outputs::handle_event(sinsp_evt *ev, string &rule, string &priority,
 	{
 		lua_pushlightuserdata(m_ls, ev);
 		lua_pushstring(m_ls, rule.c_str());
-		lua_pushstring(m_ls, priority.c_str());
+		lua_pushstring(m_ls, falco_common::priority_names[priority].c_str());
+		lua_pushnumber(m_ls, priority);
 		lua_pushstring(m_ls, format.c_str());

-		if(lua_pcall(m_ls, 4, 0, 0) != 0)
+		if(lua_pcall(m_ls, 5, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error invoking function output: " + string(lerr);
--- a/userspace/falco/falco_outputs.h
+++ b/userspace/falco/falco_outputs.h
@@ -41,7 +41,7 @@ public:
 		std::map<std::string, std::string> options;
 	};

-	void init(bool json_output, uint32_t rate, uint32_t max_burst);
+	void init(bool json_output, uint32_t rate, uint32_t max_burst, bool buffered);

 	void add_output(output_config oc);

@@ -49,7 +49,7 @@ public:
 	// ev is an event that has matched some rule. Pass the event
 	// to all configured outputs.
 	//
-	void handle_event(sinsp_evt *ev, std::string &rule, std::string &priority, std::string &format);
+	void handle_event(sinsp_evt *ev, std::string &rule, falco_common::priority_type priority, std::string &format);

 private:
 	bool m_initialized;
@@ -57,6 +57,8 @@ private:
 	// Rate limits notifications
 	token_bucket m_notifications_tb;

+	bool m_buffered;
+
 	std::string m_lua_add_output = "add_output";
 	std::string m_lua_output_event = "output_event";
 	std::string m_lua_output_cleanup = "output_cleanup";
--- a/userspace/falco/lua/output.lua
+++ b/userspace/falco/lua/output.lua
@@ -18,22 +18,25 @@

 local mod = {}

-levels = {"Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"}
-
-mod.levels = levels
-
 local outputs = {}

-function mod.stdout(level, msg)
+function mod.stdout(priority, priority_num, buffered, msg)
+   if buffered == 0 then
+      io.stdout:setvbuf 'no'
+   end
   print (msg)
 end

+function mod.stdout_cleanup()
+   io.stdout:flush()
+end
+
 function mod.file_validate(options)
   if (not type(options.filename) == 'string') then
      error("File output needs to be configured with a valid filename")
   end

-   file, err = io.open(options.filename, "a+")
+   local file, err = io.open(options.filename, "a+")
   if file == nil then
      error("Error with file output: "..err)
   end
@@ -41,60 +44,100 @@ function mod.file_validate(options)

 end

-function mod.file(level, msg, options)
-   file = io.open(options.filename, "a+")
+function mod.file(priority, priority_num, buffered, msg, options)
+   if options.keep_alive == "true" then
+      if file == nil then
+	 file = io.open(options.filename, "a+")
+	 if buffered == 0 then
+	    file:setvbuf 'no'
+	 end
+      end
+   else
+      file = io.open(options.filename, "a+")
+   end
+
   file:write(msg, "\n")
-   file:close()
+
+   if options.keep_alive == nil or
+      options.keep_alive ~= "true" then
+	 file:close()
+	 file = nil
+   end
 end

-function mod.syslog(level, msg, options)
-   falco.syslog(level, msg)
+function mod.file_cleanup()
+   if file ~= nil then
+      file:flush()
+      file:close()
+      file = nil
+   end
 end

-function mod.program(level, msg, options)
+function mod.syslog(priority, priority_num, buffered, msg, options)
+   falco.syslog(priority_num, msg)
+end
+
+function mod.syslog_cleanup()
+end
+
+function mod.program(priority, priority_num, buffered, msg, options)
   -- XXX Ideally we'd check that the program ran
   -- successfully. However, the luajit we're using returns true even
   -- when the shell can't run the program.

-   file = io.popen(options.program, "w")
+   -- Note: options are all strings
+   if options.keep_alive == "true" then
+      if file == nil then
+	 file = io.popen(options.program, "w")
+	 if buffered == 0 then
+	    file:setvbuf 'no'
+	 end
+      end
+   else
+      file = io.popen(options.program, "w")
+   end

   file:write(msg, "\n")
-   file:close()
-end

-local function level_of(s)
-   s = string.lower(s)
-   for i,v in ipairs(levels) do
-      if (string.find(string.lower(v), "^"..s)) then
-	 return i - 1 -- (syslog levels start at 0, lua indices start at 1)
-      end
+   if options.keep_alive == nil or
+      options.keep_alive ~= "true" then
+	 file:close()
+	 file = nil
   end
-   error("Invalid severity level: "..s)
 end

-function output_event(event, rule, priority, format)
-   local level = level_of(priority)
+function mod.program_cleanup()
+   if file ~= nil then
+      file:flush()
+      file:close()
+      file = nil
+   end
+end

+function output_event(event, rule, priority, priority_num, format)
   -- If format starts with a *, remove it, as we're adding our own
   -- prefix here.
   if format:sub(1,1) == "*" then
      format = format:sub(2)
   end

-   format = "*%evt.time: "..levels[level+1].." "..format
+   format = "*%evt.time: "..priority.." "..format

-   msg = formats.format_event(event, rule, levels[level+1], format)
+   msg = formats.format_event(event, rule, priority, format)

   for index,o in ipairs(outputs) do
-      o.output(level, msg, o.config)
+      o.output(priority, priority_num, o.buffered, msg, o.config)
   end
 end

 function output_cleanup()
   formats.free_formatters()
+   for index,o in ipairs(outputs) do
+      o.cleanup()
+   end
 end

-function add_output(output_name, config)
+function add_output(output_name, buffered, config)
   if not (type(mod[output_name]) == 'function') then
      error("rule_loader.add_output(): invalid output_name: "..output_name)
   end
@@ -105,7 +148,9 @@ function add_output(output_name, config)
     mod[output_name.."_validate"](config)
   end

-   table.insert(outputs, {output = mod[output_name], config=config})
+   table.insert(outputs, {output = mod[output_name],
+			  cleanup = mod[output_name.."_cleanup"],
+			  buffered=buffered, config=config})
 end

 return mod