Merge branch 'dev' into agent-master

Don't allow apache2 to spawn shells in containers

.travis.yml

@@ -2,6 +2,9 @@ language: c
 env:
     - BUILD_TYPE=Debug
     - BUILD_TYPE=Release
+sudo: required
+services:
+    - docker
 before_install:
     - sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
     - sudo apt-get update
@@ -9,12 +12,12 @@ install:
     - sudo apt-get --force-yes install g++-4.8
     - sudo apt-get install rpm linux-headers-$(uname -r)
     - git clone https://github.com/draios/sysdig.git ../sysdig
-    - sudo apt-get install -y python-pip libvirt-dev jq
+    - sudo apt-get install -y python-pip libvirt-dev jq dkms
     - cd ..
     - curl -Lo avocado-36.0-tar.gz https://github.com/avocado-framework/avocado/archive/36.0lts.tar.gz
     - tar -zxvf avocado-36.0-tar.gz
     - cd avocado-36.0lts
-    - sudo pip install -r requirements-travis.txt
+    - sudo -H pip install -r requirements.txt
     - sudo python setup.py install
     - cd ../falco
 before_script:
@@ -35,7 +38,10 @@ script:
     - cmake .. -DCMAKE_BUILD_TYPE=$BUILD_TYPE -DDRAIOS_DEBUG_FLAGS="-D_DEBUG -DNDEBUG"
     - make VERBOSE=1
     - make package
-    - cd ..
+    - cp falco*.deb ../docker/local
+    - cd ../docker/local
+    - docker build -t sysdig/falco:test .
+    - cd ../..
     - sudo test/run_regression_tests.sh $TRAVIS_BRANCH
 notifications:
   webhooks:

CHANGELOG.md

@@ -2,6 +2,44 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
+## v0.6.0
+
+Released 2016-03-29
+
+### Major Changes
+
+* Add the notion of tagged falco rules. Full documentation for this feature is available on the [wiki](https://github.com/draios/falco/wiki/Falco-Rules#rule-tags). [[#58](https://github.com/draios/falco/issues/58)] [[#59](https://github.com/draios/falco/issues/59)] [[#60](https://github.com/draios/falco/issues/60)] [[#206](https://github.com/draios/falco/pull/206)]
+* Falco now has its own dedicated kernel module. Previously, it would depend on sysdig being installed and would use sysdig's `sysdig-probe` kernel module. This ensures you can upgrade sysdig and falco without kernel driver compatibility problems. More details on the kernel module and its installation are on the [wiki](https://github.com/draios/falco/wiki/Falco-Kernel-Module). [[#215](https://github.com/draios/falco/issues/215)] [[#223](https://github.com/draios/falco/issues/223)] [[#224](https://github.com/draios/falco/pull/224)]
+* When providing multiple rules files by specifying `-r' multiple times, make sure that you can override rules/lists/macros. Previously, a list/macro/rule specified in an earlier file could not be overridden in a later file. [[#176](https://github.com/draios/falco/issues/176)] [[#177](https://github.com/draios/falco/pull/177)]
+* Add example k8s yaml files that show how to run falco as a k8s DaemonSet, and how to run falco-event-generator as a deployment running on one node. [[#222](https://github.com/draios/falco/pull/222)] [[#225](https://github.com/draios/falco/issues/225)] [[#226](https://github.com/draios/falco/pull/226)]
+* Update third party libraries to address security vulnerabilities. [[#182](https://github.com/draios/falco/pull/182)]
+* Falco can now be built on OSX. Like sysdig, on OSX it is limited to reading existing trace files. [[#210](https://github.com/draios/falco/pull/210)]
+
+### Minor Changes
+* Several changes to [falco-event-generator](https://github.com/draios/falco/wiki/Generating-Sample-Events) to improve usability. [[#205](https://github.com/draios/falco/pull/205)]
+* Switch to a formatter cache provided by sysdig code instead of using our own. [[#212](https://github.com/draios/falco/pull/212)]
+* Add automated tests that use locally-built docker images. [[#188](https://github.com/draios/falco/issues/188)]
+
+### Bug Fixes
+
+* Make sure output strings are not truncated when a given %field expression has a NULL value. [[#180](https://github.com/draios/falco/issues/180)] [[#181](https://github.com/draios/falco/pull/181)]
+* Allow ASSERTs when running travisci tests. [[#199](https://github.com/draios/falco/pull/199)]
+* Fix make dependencies for lyaml. [[#204](https://github.com/draios/falco/pull/204)] [[#130](https://github.com/draios/falco/issues/130)]
+* (This was a change in sysdig, but affected falco). Prevent hangs when traversing malformed parent thread state. [[#208](https://github.com/draios/falco/issues/208)]
+
+### Rule Changes
+
+* Add confd as a program that can write files below /etc and fleetctl as a program that can spawn shells. [[#175](https://github.com/draios/falco/pull/175)]
+* Add [exechealthz](https://github.com/kubernetes/contrib/tree/master/exec-healthz), a k8s liveness checking utility, to the list of shell spawners. [[#190](https://github.com/draios/falco/pull/190)]
+* Eliminate FPs related to weekly ubuntu cron jobs. [[#192](https://github.com/draios/falco/pull/192)]
+* Allow shells spawned by ansible, and eliminate FPs when managing machines via ansible. [[#193](https://github.com/draios/falco/pull/193)] [[#196](https://github.com/draios/falco/pull/196)] [[#202](https://github.com/draios/falco/pull/202)]
+* Eliminate FPs related to use of other security products. Thanks to @juju4 for the useful rule updates. [[#200](https://github.com/draios/falco/pull/200)]
+* Add additional possible locations for denyhosts, add [PM2](http://pm2.keymetrics.io/) as a shell spawner. [[#202](https://github.com/draios/falco/pull/202)]
+* Add flanneld as a privileged container, improve grouping for the "x running y" macros, allow denyhosts to spawn shells. [[#207](https://github.com/draios/falco/pull/207)]
+* Handle systemd changing its name to "(systemd)", add sv (part of [runit](http://smarden.org/runit/)) as a program that can write below /etc, allow writing to all `/dev/tty*` files. [[#209](https://github.com/draios/falco/pull/209)]
+* Add erl_child_setup as a shell spawner. Thanks to @dkerwin for the useful rule updates. [[#218](https://github.com/draios/falco/pull/218)]  [[#221](https://github.com/draios/falco/pull/221)]
+* Add support for gitlab omnibus containers/pods. Thanks to @dkerwin for the useful rule updates. [[#220](https://github.com/draios/falco/pull/220)]
+
 ## v0.5.0
 
 Released 2016-12-22

CMakeLists.txt

@@ -41,8 +41,8 @@ endif()
 
 set(PACKAGE_NAME "falco")
 set(PROBE_VERSION "${FALCO_VERSION}")
-set(PROBE_NAME "sysdig-probe")
-set(PROBE_DEVICE_NAME "sysdig")
+set(PROBE_NAME "falco-probe")
+set(PROBE_DEVICE_NAME "falco")
 set(CMAKE_INSTALL_PREFIX /usr)
 
 set(CMD_MAKE make)
@@ -415,12 +415,12 @@ set(CPACK_GENERATOR DEB RPM TGZ)
 set(CPACK_DEBIAN_PACKAGE_MAINTAINER "Sysdig <support@sysdig.com>")
 set(CPACK_DEBIAN_PACKAGE_SECTION "utils")
 set(CPACK_DEBIAN_PACKAGE_HOMEPAGE "http://www.sysdig.org")
-set(CPACK_DEBIAN_PACKAGE_DEPENDS "sysdig")
-set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${PROJECT_SOURCE_DIR}/scripts/debian/postinst;${PROJECT_SOURCE_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm")
+set(CPACK_DEBIAN_PACKAGE_DEPENDS "dkms (>= 2.1.0.0)")
+set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/scripts/debian/postinst;${CMAKE_BINARY_DIR}/scripts/debian/prerm;${PROJECT_SOURCE_DIR}/scripts/debian/postrm")
 
 set(CPACK_RPM_PACKAGE_LICENSE "GPLv2")
 set(CPACK_RPM_PACKAGE_URL "http://www.sysdig.org")
-set(CPACK_RPM_PACKAGE_REQUIRES "sysdig")
+set(CPACK_RPM_PACKAGE_REQUIRES "dkms, gcc, make, kernel-devel, perl")
 set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/postinstall")
 set(CPACK_RPM_PRE_UNINSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/preuninstall")
 set(CPACK_RPM_POST_UNINSTALL_SCRIPT_FILE "${PROJECT_SOURCE_DIR}/scripts/rpm/postuninstall")

README.md

@@ -1,8 +1,8 @@
 # Sysdig Falco
 
-####Latest release
+#### Latest release
 
-**v0.5.0**
+**v0.6.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -44,7 +44,7 @@ Falco is licensed to you under the [GPL 2.0](./COPYING) open source license.
 
 Contributor License Agreements
 ---
-###Background
+### Background
  As we did for sysdig, we are formalizing the way that we accept contributions of code from the contributing community. We must now ask that contributions to falco be provided subject to the terms and conditions of a [Contributor License Agreement (CLA)](./cla). The CLA comes in two forms, applicable to contributions by individuals, or by legal entities such as corporations and their employees. We recognize that entering into a CLA with us involves real consideration on your part, and we’ve tried to make this process as clear and simple as possible.
 
  We’ve modeled our CLA off of industry standards, such as [the CLA used by Kubernetes](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md). Note that this agreement is not a transfer of copyright ownership, this simply is a license agreement for contributions, intended to clarify the intellectual property license granted with contributions from any person or entity. It is for your protection as a contributor as well as the protection of falco; it does not change your rights to use your own contributions for any other purpose.
@@ -57,7 +57,7 @@ Contributor License Agreements
 
 As always, we are grateful for your past and present contributions to falco.
 
-###What do I need to do in order to contribute code?
+### What do I need to do in order to contribute code?
 
 **Individual contributions**: Individuals who wish to make contributions must review the [Individual Contributor License Agreement](./cla/falco_contributor_agreement.txt) and indicate agreement by adding the following line to every GIT commit message:
 

docker/dev/docker-entrypoint.sh

@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
         ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
     done
 
-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi
 
 exec "$@"

docker/local/Dockerfile

@@ -25,7 +25,7 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
 	gcc \
 	gcc-5 \
 	gcc-4.9 \
-	sysdig && rm -rf /var/lib/apt/lists/*
+	dkms && rm -rf /var/lib/apt/lists/*
 
 # Since our base Debian image ships with GCC 5.0 which breaks older kernels, revert the
 # default to gcc-4.9. Also, since some customers use some very old distributions whose kernel

docker/local/docker-entrypoint.sh

@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
         ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
     done
 
-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi
 
 exec "$@"

docker/stable/docker-entrypoint.sh

@@ -11,7 +11,7 @@ if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
         ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
     done
 
-    /usr/bin/sysdig-probe-loader
+    /usr/bin/falco-probe-loader
 fi
 
 exec "$@"

examples/k8s-using-daemonset/README.md

@@ -1,4 +1,4 @@
-=Example K8s Services for Falco=
+# Example K8s Services for Falco
 
 The yaml file in this directory installs the following:
  - Open Source Falco, as a DaemonSet. Falco is configured to communicate with the K8s API server via its service account, and changes its output to be K8s-friendly. It also sends to a slack webhook for the `#demo-falco-alerts` channel on our [public slack](https://sysdig.slack.com/messages/demo-falco-alerts/).

examples/k8s-using-daemonset/falco-daemonset.yaml

@@ -18,7 +18,7 @@ spec:
           image: sysdig/falco:latest
           securityContext:
             privileged: true
-          command: [ "/usr/bin/falco", "-K", "/var/run/secrets/kubernetes.io/serviceaccount/token", "-k", "https://kubernetes", "-pk", "-o", "json_output=true", "-o", "program_output.enabled=true", "-o",  "program_output.program=jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/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"]
+          args: [ "/usr/bin/falco", "-K", "/var/run/secrets/kubernetes.io/serviceaccount/token", "-k", "https://kubernetes", "-pk", "-o", "json_output=true", "-o", "program_output.enabled=true", "-o",  "program_output.program=jq '{text: .output}' | curl -d @- -X POST https://hooks.slack.com/services/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"]
           volumeMounts:
             - mountPath: /host/var/run/docker.sock
               name: docker-socket

rules/falco_rules.yaml

@@ -433,7 +433,7 @@
     and shell_procs
     and proc.pname exists
     and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, lxd_binaries, aide_wrapper_binaries, nids_binaries,
-                           monitoring_binaries, gitlab_binaries, initdb, pg_ctl, awk, apache2, falco, cron, erl_child_setup)
+                           monitoring_binaries, gitlab_binaries, initdb, pg_ctl, awk, falco, cron, erl_child_setup)
     and not trusted_containers
   output: "Shell spawned in a container other than entrypoint (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
   priority: WARNING

scripts/CMakeLists.txt

@@ -1,5 +1,12 @@
+configure_file(debian/postinst.in debian/postinst)
+configure_file(debian/prerm.in debian/prerm)
+
 file(COPY "${PROJECT_SOURCE_DIR}/scripts/debian/falco"
 	DESTINATION "${PROJECT_BINARY_DIR}/scripts/debian")
 
 file(COPY "${PROJECT_SOURCE_DIR}/scripts/rpm/falco"
 	DESTINATION "${PROJECT_BINARY_DIR}/scripts/rpm")
+
+install(PROGRAMS ${SYSDIG_DIR}/scripts/sysdig-probe-loader
+	DESTINATION bin
+	RENAME falco-probe-loader)

scripts/debian/postinst

@@ -1,9 +0,0 @@
-#!/bin/sh
-set -e
-
-NAME=falco
-
-if [ -x "/etc/init.d/$NAME" ]; then
-        update-rc.d $NAME defaults >/dev/null
-fi
-

scripts/debian/postinst.in

@@ -0,0 +1,32 @@
+#!/bin/sh
+set -e
+
+DKMS_PACKAGE_NAME="@PACKAGE_NAME@"
+DKMS_VERSION="@PROBE_VERSION@"
+NAME="@PACKAGE_NAME@"
+
+postinst_found=0
+
+case "$1" in
+	configure)
+		for DKMS_POSTINST in /usr/lib/dkms/common.postinst /usr/share/$DKMS_PACKAGE_NAME/postinst; do
+			if [ -f $DKMS_POSTINST ]; then
+				$DKMS_POSTINST $DKMS_PACKAGE_NAME $DKMS_VERSION /usr/share/$DKMS_PACKAGE_NAME "" $2
+				postinst_found=1
+				break
+			fi
+		done
+		if [ "$postinst_found" -eq 0 ]; then
+			echo "ERROR: DKMS version is too old and $DKMS_PACKAGE_NAME was not"
+			echo "built with legacy DKMS support."
+			echo "You must either rebuild $DKMS_PACKAGE_NAME with legacy postinst"
+			echo "support or upgrade DKMS to a more current version."
+			exit 1
+		fi
+	;;
+esac
+
+if [ -x "/etc/init.d/$NAME" ]; then
+        update-rc.d $NAME defaults >/dev/null
+fi
+

scripts/debian/prerm

@@ -1,13 +0,0 @@
-#!/bin/sh
-set -e
-
-NAME=falco
-
-if [ -x "/etc/init.d/$NAME" ]; then
-		if [ -x "`which invoke-rc.d 2>/dev/null`" ]; then
-				invoke-rc.d $NAME stop || exit $?
-		else
-				/etc/init.d/$NAME stop || exit $?
-		fi
-fi
-

scripts/debian/prerm.in

@@ -0,0 +1,23 @@
+#!/bin/sh
+set -e
+
+NAME="@PACKAGE_NAME@"
+
+if [ -x "/etc/init.d/$NAME" ]; then
+		if [ -x "`which invoke-rc.d 2>/dev/null`" ]; then
+				invoke-rc.d $NAME stop || exit $?
+		else
+				/etc/init.d/$NAME stop || exit $?
+		fi
+fi
+
+DKMS_PACKAGE_NAME="@PACKAGE_NAME@"
+DKMS_VERSION="@PROBE_VERSION@"
+
+case "$1" in
+	remove|upgrade|deconfigure)
+		if [  "$(dkms status -m $DKMS_PACKAGE_NAME -v $DKMS_VERSION)" ]; then
+			dkms remove -m $DKMS_PACKAGE_NAME -v $DKMS_VERSION --all
+		fi
+	;;
+esac

scripts/rpm/postinstall

@@ -1 +1,15 @@
+dkms add -m falco -v %{version} --rpm_safe_upgrade
+if [ `uname -r | grep -c "BOOT"` -eq 0 ] && [ -e /lib/modules/`uname -r`/build/include ]; then
+	dkms build -m falco -v %{version}
+	dkms install --force -m falco -v %{version}
+elif [ `uname -r | grep -c "BOOT"` -gt 0 ]; then
+	echo -e ""
+	echo -e "Module build for the currently running kernel was skipped since you"
+	echo -e "are running a BOOT variant of the kernel."
+else
+	echo -e ""
+	echo -e "Module build for the currently running kernel was skipped since the"
+	echo -e "kernel source for this kernel does not seem to be installed."
+fi
+
 /sbin/chkconfig --add falco

scripts/rpm/preuninstall

@@ -2,3 +2,5 @@ if [ $1 = 0 ]; then
 	/sbin/service falco stop > /dev/null 2>&1
 	/sbin/chkconfig --del falco
 fi
+
+dkms remove -m falco -v %{version} --all --rpm_safe_upgrade

test/falco_test.py

@@ -4,6 +4,9 @@ import os
 import re
 import json
 import sets
+import glob
+import shutil
+import subprocess
 
 from avocado import Test
 from avocado.utils import process
@@ -21,9 +24,9 @@ class FalcoTest(Test):
         self.stderr_contains = self.params.get('stderr_contains', '*', default='')
         self.exit_status = self.params.get('exit_status', '*', default=0)
         self.should_detect = self.params.get('detect', '*', default=False)
-        self.trace_file = self.params.get('trace_file', '*')
+        self.trace_file = self.params.get('trace_file', '*', default='')
 
-        if not os.path.isabs(self.trace_file):
+        if self.trace_file and not os.path.isabs(self.trace_file):
             self.trace_file = os.path.join(self.basedir, self.trace_file)
 
         self.json_output = self.params.get('json_output', '*', default=False)
@@ -43,6 +46,8 @@ class FalcoTest(Test):
         if not os.path.isabs(self.conf_file):
             self.conf_file = os.path.join(self.basedir, self.conf_file)
 
+        self.run_duration = self.params.get('run_duration', '*', default='')
+
         self.disabled_rules = self.params.get('disabled_rules', '*', default='')
 
         if self.disabled_rules == '':
@@ -89,15 +94,23 @@ class FalcoTest(Test):
             if not isinstance(self.detect_level, list):
                 self.detect_level = [self.detect_level]
 
-        # Doing this in 2 steps instead of simply using
-        # module_is_loaded to avoid logging lsmod output to the log.
-        lsmod_output = process.system_output("lsmod", verbose=False)
+        self.package = self.params.get('package', '*', default='None')
 
-        if linux_modules.parse_lsmod_for_module(lsmod_output, 'sysdig_probe') == {}:
-            self.log.debug("Loading sysdig kernel module")
-            process.run('sudo insmod {}/driver/sysdig-probe.ko'.format(self.falcodir))
+        if self.package == 'None':
+            # Doing this in 2 steps instead of simply using
+            # module_is_loaded to avoid logging lsmod output to the log.
+            lsmod_output = process.system_output("lsmod", verbose=False)
 
-        self.str_variant = self.trace_file
+            if linux_modules.parse_lsmod_for_module(lsmod_output, 'falco_probe') == {}:
+                self.log.debug("Loading falco kernel module")
+                process.run('insmod {}/driver/falco-probe.ko'.format(self.falcodir), sudo=True)
+
+        self.addl_docker_run_args = self.params.get('addl_docker_run_args', '*', default='')
+
+        self.copy_local_driver = self.params.get('copy_local_driver', '*', default=False)
+
+        # Used by possibly_copy_local_driver as well as docker run
+        self.module_dir = os.path.expanduser("~/.sysdig")
 
         self.outputs = self.params.get('outputs', '*', default='')
 
@@ -111,6 +124,10 @@ class FalcoTest(Test):
                     output['file'] = item2[0]
                     output['line'] = item2[1]
                     outputs.append(output)
+                    filedir = os.path.dirname(output['file'])
+                    # Create the parent directory for the trace file if it doesn't exist.
+                    if not os.path.isdir(filedir):
+                        os.makedirs(filedir)
             self.outputs = outputs
 
         self.disable_tags = self.params.get('disable_tags', '*', default='')
@@ -123,6 +140,10 @@ class FalcoTest(Test):
         if self.run_tags == '':
             self.run_tags=[]
 
+    def tearDown(self):
+        if self.package != 'None':
+            self.uninstall_package()
+
     def check_rules_warnings(self, res):
 
         found_warning = sets.Set()
@@ -231,12 +252,103 @@ class FalcoTest(Test):
                         if not attr in obj:
                             self.fail("Falco JSON object {} does not contain property \"{}\"".format(line, attr))
 
+    def install_package(self):
+
+        if self.package.startswith("docker:"):
+
+            image = self.package.split(":", 1)[1]
+            # Remove an existing falco-test container first. Note we don't check the output--docker rm
+            # doesn't have an -i equivalent.
+            res = process.run("docker rm falco-test", ignore_status=True)
+            rules_dir = os.path.abspath(os.path.join(self.basedir, "./rules"))
+            conf_dir = os.path.abspath(os.path.join(self.basedir, "../"))
+            traces_dir = os.path.abspath(os.path.join(self.basedir, "./trace_files"))
+            self.falco_binary_path = "docker run -i -t --name falco-test --privileged " \
+                                     "-v {}:/host/rules -v {}:/host/conf -v {}:/host/traces " \
+                                     "-v /var/run/docker.sock:/host/var/run/docker.sock " \
+                                     "-v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro " \
+                                     "-v /lib/modules:/host/lib/modules:ro -v {}:/root/.sysdig:ro -v " \
+                                     "/usr:/host/usr:ro {} {} falco".format(
+                                         rules_dir, conf_dir, traces_dir,
+                                         self.module_dir, self.addl_docker_run_args, image)
+
+        elif self.package.endswith(".deb"):
+            self.falco_binary_path = '/usr/bin/falco';
+
+            package_glob = "{}/{}".format(self.falcodir, self.package)
+
+            matches = glob.glob(package_glob)
+
+            if len(matches) != 1:
+                self.fail("Package path {} did not match exactly 1 file. Instead it matched: {}", package_glob, ",".join(matches))
+
+            package_path = matches[0]
+
+            cmdline = "dpkg -i {}".format(package_path)
+            self.log.debug("Installing debian package via \"{}\"".format(cmdline))
+            res = process.run(cmdline, timeout=120, sudo=True)
+
+    def uninstall_package(self):
+
+        if self.package.startswith("docker:"):
+            # Remove the falco-test image. Here we *do* check the return value
+            res = process.run("docker rm falco-test")
+
+        elif self.package.endswith(".deb"):
+            cmdline = "dpkg -r falco"
+            self.log.debug("Uninstalling debian package via \"{}\"".format(cmdline))
+            res = process.run(cmdline, timeout=120, sudo=True)
+
+    def possibly_copy_driver(self):
+        # Remove the contents of ~/.sysdig regardless of
+        # copy_local_driver.
+        self.log.debug("Checking for module dir {}".format(self.module_dir))
+        if os.path.isdir(self.module_dir):
+            self.log.info("Removing files below directory {}".format(self.module_dir))
+            for rmfile in glob.glob(self.module_dir + "/*"):
+                self.log.debug("Removing file {}".format(rmfile))
+                os.remove(rmfile)
+
+        if self.copy_local_driver:
+            verstr = subprocess.check_output([self.falco_binary_path, "--version"]).rstrip()
+            self.log.info("verstr {}".format(verstr))
+            falco_version = verstr.split(" ")[2]
+            self.log.info("falco_version {}".format(falco_version))
+            arch = subprocess.check_output(["uname", "-m"]).rstrip()
+            self.log.info("arch {}".format(arch))
+            kernel_release = subprocess.check_output(["uname", "-r"]).rstrip()
+            self.log.info("kernel release {}".format(kernel_release))
+
+            # sysdig-probe-loader has a more comprehensive set of ways to
+            # find the config hash. We only look at /boot/config-<kernel release>
+            md5_output = subprocess.check_output(["md5sum", "/boot/config-{}".format(kernel_release)]).rstrip()
+            config_hash = md5_output.split(" ")[0]
+
+            probe_filename = "falco-probe-{}-{}-{}-{}.ko".format(falco_version, arch, kernel_release, config_hash)
+            driver_path = os.path.join(self.falcodir, "driver", "falco-probe.ko")
+            module_path = os.path.join(self.module_dir, probe_filename)
+            self.log.debug("Copying {} to {}".format(driver_path, module_path))
+            shutil.copyfile(driver_path, module_path)
+
     def test(self):
         self.log.info("Trace file %s", self.trace_file)
 
-        # Run the provided trace file though falco
-        cmd = '{}/userspace/falco/falco {} {} -c {} -e {} -o json_output={} -v'.format(
-            self.falcodir, self.rules_args, self.disabled_args, self.conf_file, self.trace_file, self.json_output)
+        self.falco_binary_path = '{}/userspace/falco/falco'.format(self.falcodir)
+
+        self.possibly_copy_driver()
+
+        if self.package != 'None':
+            # This sets falco_binary_path as a side-effect.
+            self.install_package()
+
+        trace_arg = self.trace_file
+
+        if self.trace_file:
+            trace_arg = "-e {}".format(self.trace_file)
+
+        # Run falco
+        cmd = '{} {} {} -c {} {} -o json_output={} -v'.format(
+            self.falco_binary_path, self.rules_args, self.disabled_args, self.conf_file, trace_arg, self.json_output)
 
         for tag in self.disable_tags:
             cmd += ' -T {}'.format(tag)
@@ -244,6 +356,9 @@ class FalcoTest(Test):
         for tag in self.run_tags:
             cmd += ' -t {}'.format(tag)
 
+        if self.run_duration:
+            cmd += ' -M {}'.format(self.run_duration)
+
         self.falco_proc = process.SubProcess(cmd)
 
         res = self.falco_proc.run(timeout=180, sig=9)

test/falco_tests.yaml.in

@@ -1,4 +1,38 @@
 trace_files: !mux
+
+  docker_package:
+    package: docker:sysdig/falco:test
+    detect: True
+    detect_level: WARNING
+    rules_file: /host/rules/rule_names_with_spaces.yaml
+    trace_file: /host/traces/cat_write.scap
+    conf_file: /host/conf/falco.yaml
+
+  # This uses a volume mount to overwrite and prevent /usr/sbin/dkms
+  # from being run. As a result, it will force falco-probe-loader to
+  # fall back to loading the driver from ~/.sysdig. Setting
+  # copy_local_driver to True copied the driver to ~/.sysdig, so it
+  # will be available. In this case, we're running live for 5 seconds
+  # just to see if falco can load the driver.
+
+  docker_package_local_driver:
+    package: docker:sysdig/falco:test
+    addl_docker_run_args: -v /dev/null:/usr/sbin/dkms
+    copy_local_driver: True
+    detect: False
+    detect_level: WARNING
+    rules_file: /host/rules/tagged_rules.yaml
+    conf_file: /host/conf/falco.yaml
+    run_duration: 5
+
+  debian_package:
+    package: falco*.deb
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/rule_names_with_spaces.yaml
+    trace_file: trace_files/cat_write.scap
+
   builtin_rules_no_warnings:
     detect: False
     trace_file: trace_files/empty.scap

userspace/falco/falco.cpp

@@ -53,6 +53,7 @@ static void signal_callback(int signal)
 static void usage()
 {
     printf(
+	   "falco version " FALCO_VERSION "\n"
 	   "Usage: falco [options]\n\n"
 	   "Options:\n"
 	   " -h, --help                    Print this page\n"
@@ -86,6 +87,7 @@ static void usage()
 	   "                               Marathon url is optional and defaults to Mesos address, port 8080.\n"
 	   "                               The API servers can also be specified via the environment variable\n"
 	   "                               FALCO_MESOS_API.\n"
+	   " -M <num_seconds>              Stop collecting after <num_seconds> reached.\n"
 	   " -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.\n"
 	   "                               <key> can be a two-part <key>.<subkey>\n"
 	   " -p <output_format>, --print=<output_format>\n"
@@ -106,6 +108,7 @@ static void usage()
 	   " -t <tag>                      Only run those rules with a tag=<tag>. Can be specified multiple times.\n"
 	   "                               Can not be specified with -T/-D.\n"
 	   " -v                            Verbose output.\n"
+           " --version                     Print version number.\n"
 	   "\n"
     );
 }
@@ -133,12 +136,14 @@ std::list<string> cmdline_options;
 uint64_t do_inspect(falco_engine *engine,
 		    falco_outputs *outputs,
 		    sinsp* inspector,
+		    uint64_t duration_to_tot_ns,
 		    string &stats_filename)
 {
 	uint64_t num_evts = 0;
 	int32_t res;
 	sinsp_evt* ev;
 	StatsFileWriter writer;
+	uint64_t duration_start = 0;
 
 	if (stats_filename != "")
 	{
@@ -182,6 +187,17 @@ uint64_t do_inspect(falco_engine *engine,
 			throw sinsp_exception(inspector->getlasterr().c_str());
 		}
 
+		if (duration_start == 0)
+		{
+			duration_start = ev->get_ts();
+		} else if(duration_to_tot_ns > 0)
+		{
+			if(ev->get_ts() - duration_start >= duration_to_tot_ns)
+			{
+				break;
+			}
+		}
+
 		if(!inspector->is_debug_enabled() &&
 			ev->get_category() & EC_INTERNAL)
 		{
@@ -232,6 +248,7 @@ int falco_init(int argc, char **argv)
 	string* mesos_api = 0;
 	string output_format = "";
 	bool replace_container_info = false;
+	int duration_to_tot = 0;
 
 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -255,6 +272,7 @@ int falco_init(int argc, char **argv)
 		{"option", required_argument, 0, 'o'},
 		{"print", required_argument, 0, 'p' },
 		{"pidfile", required_argument, 0, 'P' },
+		{"version", no_argument, 0, 0 },
 		{"writefile", required_argument, 0, 'w' },
 
 		{0, 0, 0, 0}
@@ -272,7 +290,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:k:K:Ll:m:o:P:p:r:s:T:t:vw:",
+                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:vw:",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -313,6 +331,13 @@ int falco_init(int argc, char **argv)
 			case 'm':
 				mesos_api = new string(optarg);
 				break;
+			case 'M':
+				duration_to_tot = atoi(optarg);
+				if(duration_to_tot <= 0)
+				{
+					throw sinsp_exception(string("invalid duration") + optarg);
+				}
+				break;
 			case 'o':
 				cmdline_options.push_back(optarg);
 				break;
@@ -368,6 +393,13 @@ int falco_init(int argc, char **argv)
 
 		}
 
+		if(string(long_options[long_index].name) == "version")
+		{
+			printf("falco version %s\n", FALCO_VERSION);
+			return EXIT_SUCCESS;
+		}
+
+
 		inspector = new sinsp();
 		engine = new falco_engine();
 		engine->set_inspector(inspector);
@@ -652,6 +684,7 @@ int falco_init(int argc, char **argv)
 		num_evts = do_inspect(engine,
 				      outputs,
 				      inspector,
+				      uint64_t(duration_to_tot*ONE_SECOND_IN_NS),
 				      stats_filename);
 
 		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;