Merge remote-tracking branch 'origin/dev'

Changelog and README.

.github/stale.yaml

@@ -0,0 +1,18 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 60
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 7
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - cncf
+  - roadmap
+  - enhancement
+# Label to use when marking an issue as stale
+staleLabel: wontfix
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you
+  for your contributions.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: false

.gitignore

@@ -18,3 +18,4 @@ docker/event-generator/mysqld
 docker/event-generator/httpd
 docker/event-generator/sha1sum
 docker/event-generator/vipw
+.vscode/*

.travis.yml

@@ -48,7 +48,7 @@ script:
     - make package
     - cp falco*.deb ../docker/local
     - cd ../docker/local
-    - docker build -t sysdig/falco:test .
+    - docker build -t falcosecurity/falco:test .
     - cd ../..
     - sudo test/run_regression_tests.sh $TRAVIS_BRANCH
 notifications:

CHANGELOG.md

@@ -2,6 +2,88 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
+## v0.14.0
+
+Released 2019-02-06
+
+## Major Changes
+
+* Rules versioning support: The falco engine and executable now have an *engine version* that represents the fields they support. Similarly, rules files have an optional *required_engine_version: NNN* object that names the minimum engine version required to read that rules file. Any time the engine adds new fields, event sources, etc, the engine version will be incremented, and any time a rules file starts using new fields, event sources, etc, the required engine version will be incremented. [[#492](https://github.com/falcosecurity/falco/pull/492)]
+
+* Allow SSL for K8s audit endpoint/embedded webserver [[#471](https://github.com/falcosecurity/falco/pull/471)]
+
+* Add stale issues bot that automatically flags old github issues as stale after 60 days of inactivity and closes issues after 67 days of inactivity. [[#500](https://github.com/falcosecurity/falco/pull/500)]
+
+* Support bundle: When run with `--support`, falco will print a json object containing necessary information like falco version, command line, operating system information, and falco rules files contents. This could be useful when reporting issues. [[#517](https://github.com/falcosecurity/falco/pull/517)]
+
+## Minor Changes
+
+* Support new third-party library dependencies from open source sysdig. [[#498](https://github.com/falcosecurity/falco/pull/498)]
+
+* Add CII best practices badge. [[#499](https://github.com/falcosecurity/falco/pull/499)]
+
+* Fix kernel module builds when running on centos as a container by installing gcc 5 by hand instead of directly from debian/unstable. [[#501](https://github.com/falcosecurity/falco/pull/501)]
+
+* Mount `/etc` when running as a container, which allows container to build kernel module/ebpf program on COS/Minikube. [[#475](https://github.com/falcosecurity/falco/pull/475)]
+
+* Improved way to specify the source of generic event objects [[#480](https://github.com/falcosecurity/falco/pull/480)]
+
+* Readability/clarity improvements to K8s Audit/K8s Daemonset READMEs. [[#503](https://github.com/falcosecurity/falco/pull/503)]
+
+* Add additional RBAC permissions to track deployments/daemonsets/replicasets. [[#514](https://github.com/falcosecurity/falco/pull/514)]
+
+## Bug Fixes
+
+* Fix formatting of nodejs examples README [[#502](https://github.com/falcosecurity/falco/pull/502)]
+
+## Rule Changes
+
+* Remove FPs for `Launch Sensitive Mount Container` rule [[#509](https://github.com/falcosecurity/falco/pull/509/files)]
+
+* Update Container rules/macros to use the more reliable `container.image.{repository,tag}` that always return the repository/tag of an image instead of `container.image`, which may not for some docker daemon versions. [[#513](https://github.com/falcosecurity/falco/pull/513)]
+
+## v0.13.1
+
+Released 2019-01-16
+
+## Major Changes
+
+
+## Minor Changes
+
+* Unbuffer outputs by default. This helps make output readable when used in environments like K8s. [[#494](https://github.com/falcosecurity/falco/pull/494)]
+
+* Improved documentation for running Falco within K8s and getting K8s Audit Logging to work with Minikube and Falco as a Daemonset within K8s. [[#496](https://github.com/falcosecurity/falco/pull/496)]
+
+* Fix AWS Permissions for Kubernetes Response Engine [[#465](https://github.com/falcosecurity/falco/pull/465)]
+
+* Tighten compilation flags to include `-Wextra` and `-Werror` [[#479](https://github.com/falcosecurity/falco/pull/479)]
+
+* Add `k8s.ns.name` to outputs when `-pk` argument is used [[#472](https://github.com/falcosecurity/falco/pull/472)]
+
+* Remove kubernetes-response-engine from system:masters [[#488](https://github.com/falcosecurity/falco/pull/488)]
+
+## Bug Fixes
+
+* Ensure `-pc`/`-pk` only apply to syscall rules and not k8s_audit rules [[#495](https://github.com/falcosecurity/falco/pull/495)]
+
+* Fix a potential crash that could occur when using the falco engine and rulesets [[#468](https://github.com/falcosecurity/falco/pull/468)]
+
+* Fix a regression where format output options were mistakenly removed [[#485](https://github.com/falcosecurity/falco/pull/485)]
+
+## Rule Changes
+
+* Fix FPs related to calico and writing files below etc [[#481](https://github.com/falcosecurity/falco/pull/481)]
+
+* Fix FPs related to `apt-config`/`apt-cache`, `apk` [[#490](https://github.com/falcosecurity/falco/pull/490)]
+
+* New rules `Launch Package Management Process in Container`, `Netcat Remote Code Execution in Container`, `Lauch Suspicious Network Tool in Container` look for host-level network tools like `netcat`, package management tools like `apt-get`, or network tool binaries being run in a container. [[#490](https://github.com/falcosecurity/falco/pull/490)]
+
+* Fix the `inbound` and `outbound` macros so they work with sendto/recvfrom/sendmsg/recvmsg. [[#470](https://github.com/falcosecurity/falco/pull/470)]
+
+* Fix FPs related to prometheus/openshift writing config below /etc. [[#470](https://github.com/falcosecurity/falco/pull/470)]
+
+
 ## v0.13.0
 
 Released 2018-11-09

CMakeLists.txt

@@ -19,6 +19,8 @@ cmake_minimum_required(VERSION 2.8.2)
 
 project(falco)
 
+option(BUILD_WARNINGS_AS_ERRORS "Enable building with -Wextra -Werror flags")
+
 if(NOT DEFINED FALCO_VERSION)
 	set(FALCO_VERSION "0.1.1dev")
 endif()
@@ -35,8 +37,15 @@ if(NOT DRAIOS_DEBUG_FLAGS)
 	set(DRAIOS_DEBUG_FLAGS "-D_DEBUG")
 endif()
 
-set(CMAKE_C_FLAGS "-Wall -ggdb ${DRAIOS_FEATURE_FLAGS}")
-set(CMAKE_CXX_FLAGS "-Wall -ggdb --std=c++0x ${DRAIOS_FEATURE_FLAGS}")
+set(CMAKE_COMMON_FLAGS "-Wall -ggdb ${DRAIOS_FEATURE_FLAGS}")
+
+if(BUILD_WARNINGS_AS_ERRORS)
+	set(CMAKE_SUPPRESSED_WARNINGS "-Wno-unused-parameter -Wno-missing-field-initializers -Wno-sign-compare -Wno-type-limits -Wno-implicit-fallthrough -Wno-format-truncation")
+	set(CMAKE_COMMON_FLAGS "${CMAKE_COMMON_FLAGS} -Wextra -Werror ${CMAKE_SUPPRESSED_WARNINGS}")
+endif()
+
+set(CMAKE_C_FLAGS "${CMAKE_COMMON_FLAGS}")
+set(CMAKE_CXX_FLAGS "--std=c++0x ${CMAKE_COMMON_FLAGS}")
 
 set(CMAKE_C_FLAGS_DEBUG "${DRAIOS_DEBUG_FLAGS}")
 set(CMAKE_CXX_FLAGS_DEBUG "${DRAIOS_DEBUG_FLAGS}")
@@ -496,6 +505,92 @@ else()
 		INSTALL_COMMAND ${CMD_MAKE} install-lib install-headers PREFIX=${CIVETWEB_SRC}/install WITH_CPP=1)
 endif()
 
+option(USE_BUNDLED_CARES "Enable building of the bundled c-ares" ${USE_BUNDLED_DEPS})
+if(NOT USE_BUNDLED_CARES)
+	find_path(CARES_INCLUDE NAMES cares/ares.h)
+	find_library(CARES_LIB NAMES libcares.a)
+	if(CARES_INCLUDE AND CARES_LIB)
+		message(STATUS "Found c-ares: include: ${CARES_INCLUDE}, lib: ${CARES_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system c-ares")
+	endif()
+else()
+	set(CARES_SRC "${PROJECT_BINARY_DIR}/c-ares-prefix/src/c-ares")
+	message(STATUS "Using bundled c-ares in '${CARES_SRC}'")
+	set(CARES_INCLUDE "${CARES_SRC}/target/include")
+	set(CARES_LIB "${CARES_SRC}/target/lib/libcares.a")
+	ExternalProject_Add(c-ares
+		URL "https://download.sysdig.com/dependencies/c-ares-1.13.0.tar.gz"
+		URL_MD5 "d2e010b43537794d8bedfb562ae6bba2"
+		CONFIGURE_COMMAND ./configure --prefix=${CARES_SRC}/target
+		BUILD_COMMAND ${CMD_MAKE}
+		BUILD_IN_SOURCE 1
+		BUILD_BYPRODUCTS ${CARES_INCLUDE} ${CARES_LIB}
+		INSTALL_COMMAND ${CMD_MAKE} install)
+endif()
+
+option(USE_BUNDLED_PROTOBUF "Enable building of the bundled protobuf" ${USE_BUNDLED_DEPS})
+if(NOT USE_BUNDLED_PROTOBUF)
+	find_program(PROTOC NAMES protoc)
+	find_path(PROTOBUF_INCLUDE NAMES google/protobuf/message.h)
+	find_library(PROTOBUF_LIB NAMES libprotobuf.a)
+	if(PROTOC AND PROTOBUF_INCLUDE AND PROTOBUF_LIB)
+		message(STATUS "Found protobuf: compiler: ${PROTOC}, include: ${PROTOBUF_INCLUDE}, lib: ${PROTOBUF_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system protobuf")
+	endif()
+else()
+	set(PROTOBUF_SRC "${PROJECT_BINARY_DIR}/protobuf-prefix/src/protobuf")
+	message(STATUS "Using bundled protobuf in '${PROTOBUF_SRC}'")
+	set(PROTOC "${PROTOBUF_SRC}/target/bin/protoc")
+	set(PROTOBUF_INCLUDE "${PROTOBUF_SRC}/target/include")
+	set(PROTOBUF_LIB "${PROTOBUF_SRC}/target/lib/libprotobuf.a")
+	ExternalProject_Add(protobuf
+		DEPENDS openssl zlib
+		URL "https://github.com/google/protobuf/releases/download/v3.5.0/protobuf-cpp-3.5.0.tar.gz"
+		URL_MD5 "e4ba8284a407712168593e79e6555eb2"
+		# TODO what if using system zlib?
+		CONFIGURE_COMMAND /usr/bin/env CPPFLAGS=-I${ZLIB_INCLUDE} LDFLAGS=-L${ZLIB_SRC} ./configure --with-zlib --prefix=${PROTOBUF_SRC}/target
+		BUILD_COMMAND ${CMD_MAKE}
+		BUILD_IN_SOURCE 1
+		BUILD_BYPRODUCTS ${PROTOC} ${PROTOBUF_INCLUDE} ${PROTOBUF_LIB}
+		# TODO s390x support
+		INSTALL_COMMAND make install)
+endif()
+
+option(USE_BUNDLED_GRPC "Enable building of the bundled grpc" ${USE_BUNDLED_DEPS})
+if(NOT USE_BUNDLED_GRPC)
+	find_path(GRPC_INCLUDE grpc++/impl/codegen/rpc_method.h)
+	find_library(GRPC_LIB NAMES libgrpc_unsecure.a)
+	find_library(GRPCPP_LIB NAMES libgrpc++_unsecure.a)
+	if(GRPC_INCLUDE AND GRPC_LIB AND GRPCPP_LIB)
+		message(STATUS "Found grpc: include: ${GRPC_INCLUDE}, C lib: ${GRPC_LIB}, C++ lib: ${GRPC_PP_LIB}")
+	else()
+		message(FATAL_ERROR "Couldn't find system grpc")
+	endif()
+else()
+	set(GRPC_SRC "${PROJECT_BINARY_DIR}/grpc-prefix/src/grpc")
+	message(STATUS "Using bundled grpc in '${GRPC_SRC}'")
+	set(GRPC_INCLUDE "${GRPC_SRC}/include")
+	set(GRPC_LIB "${GRPC_SRC}/libs/opt/libgrpc_unsecure.a")
+	set(GRPCPP_LIB "${GRPC_SRC}/libs/opt/libgrpc++_unsecure.a")
+
+	get_filename_component(PROTOC_DIR ${PROTOC} DIRECTORY)
+
+	ExternalProject_Add(grpc
+		DEPENDS protobuf zlib c-ares
+		URL "http://download.draios.com/dependencies/grpc-1.8.1.tar.gz"
+		URL_MD5 "2fc42c182a0ed1b48ad77397f76bb3bc"
+		CONFIGURE_COMMAND ""
+		# TODO what if using system openssl, protobuf or cares?
+		BUILD_COMMAND HAS_SYSTEM_ZLIB=false LDFLAGS=-static PATH=${PROTOC_DIR}:$ENV{PATH} PKG_CONFIG_PATH=${OPENSSL_BUNDLE_DIR}:${PROTOBUF_SRC}:${CARES_SRC} make grpc_cpp_plugin static_cxx static_c
+		BUILD_IN_SOURCE 1
+		BUILD_BYPRODUCTS ${GRPC_LIB} ${GRPCPP_LIB}
+		# TODO s390x support
+		# TODO what if using system zlib
+		PATCH_COMMAND rm -rf third_party/zlib && ln -s ${ZLIB_SRC} third_party/zlib && wget https://download.sysdig.com/dependencies/grpc-1.1.4-Makefile.patch && patch < grpc-1.1.4-Makefile.patch
+		INSTALL_COMMAND "")
+endif()
 
 
 install(FILES falco.yaml

README.md

@@ -2,11 +2,13 @@
 
 #### Latest release
 
-**v0.13.0**
+**v0.14.0**
 Read the [change log](https://github.com/falcosecurity/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/falcosecurity/falco.svg?branch=dev)](https://travis-ci.org/falcosecurity/falco)<br />
-Master Branch: [![Build Status](https://travis-ci.org/falcosecurity/falco.svg?branch=master)](https://travis-ci.org/falcosecurity/falco)
+Master Branch: [![Build Status](https://travis-ci.org/falcosecurity/falco.svg?branch=master)](https://travis-ci.org/falcosecurity/falco)<br />
+CII Best Practices: [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2317/badge)](https://bestpractices.coreinfrastructure.org/projects/2317)
+
 
 ## Overview
 Falco is a behavioral activity monitor designed to detect anomalous activity in your applications. Powered by [sysdig’s](https://github.com/draios/sysdig) system call capture infrastructure, Falco lets you continuously monitor and detect container, application, host, and network activity... all in one place, from one source of data, with one set of rules.

docker/dev/Dockerfile

@@ -24,7 +24,6 @@ RUN apt-get update \
 	dkms \
 	gnupg2 \
 	gcc \
-	gcc-5 \
 	gcc-6 \
 	gdb \
 	jq \
@@ -35,6 +34,21 @@ RUN apt-get update \
 	xz-utils \
  && rm -rf /var/lib/apt/lists/*
 
+# gcc 5 is no longer included in debian unstable, but we need it to
+# build centos kernels, which are 3.x based and explicitly want a gcc
+# version 3, 4, or 5 compiler. So grab copies we've saved from debian
+# snapshots with the prefix https://snapshot.debian.org/archive/debian/20190122T000000Z.
+
+RUN curl -o cpp-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/cpp-5_5.5.0-12_amd64.deb \
+ && curl -o gcc-5-base_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5-base_5.5.0-12_amd64.deb \
+ && curl -o gcc-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5_5.5.0-12_amd64.deb \
+ && curl -o libasan2_5.5.0-12_amd64.deb	https://s3.amazonaws.com/download.draios.com/dependencies/libasan2_5.5.0-12_amd64.deb \
+ && curl -o libgcc-5-dev_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libgcc-5-dev_5.5.0-12_amd64.deb \
+ && curl -o libisl15_0.18-4_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libisl15_0.18-4_amd64.deb \
+ && curl -o libmpx0_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libmpx0_5.5.0-12_amd64.deb \
+ && dpkg -i cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb \
+ && rm -f cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb
+
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.
 RUN rm -rf /usr/bin/gcc && ln -s /usr/bin/gcc-5 /usr/bin/gcc

docker/local/Dockerfile

@@ -24,7 +24,6 @@ RUN apt-get update \
 	dkms \
 	gnupg2 \
 	gcc \
-	gcc-5 \
 	gcc-6 \
 	jq \
 	libc6-dev \
@@ -34,6 +33,21 @@ RUN apt-get update \
 	xz-utils \
  && rm -rf /var/lib/apt/lists/*
 
+# gcc 5 is no longer included in debian unstable, but we need it to
+# build centos kernels, which are 3.x based and explicitly want a gcc
+# version 3, 4, or 5 compiler. So grab copies we've saved from debian
+# snapshots with the prefix https://snapshot.debian.org/archive/debian/20190122T000000Z.
+
+RUN curl -o cpp-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/cpp-5_5.5.0-12_amd64.deb \
+ && curl -o gcc-5-base_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5-base_5.5.0-12_amd64.deb \
+ && curl -o gcc-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5_5.5.0-12_amd64.deb \
+ && curl -o libasan2_5.5.0-12_amd64.deb	https://s3.amazonaws.com/download.draios.com/dependencies/libasan2_5.5.0-12_amd64.deb \
+ && curl -o libgcc-5-dev_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libgcc-5-dev_5.5.0-12_amd64.deb \
+ && curl -o libisl15_0.18-4_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libisl15_0.18-4_amd64.deb \
+ && curl -o libmpx0_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libmpx0_5.5.0-12_amd64.deb \
+ && dpkg -i cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb \
+ && rm -f cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb
+
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.
 RUN rm -rf /usr/bin/gcc && ln -s /usr/bin/gcc-5 /usr/bin/gcc

docker/stable/Dockerfile

@@ -24,7 +24,6 @@ RUN apt-get update \
 	dkms \
 	gnupg2 \
 	gcc \
-	gcc-5 \
 	gcc-6 \
 	jq \
 	libc6-dev \
@@ -34,6 +33,21 @@ RUN apt-get update \
 	xz-utils \
  && rm -rf /var/lib/apt/lists/*
 
+# gcc 5 is no longer included in debian unstable, but we need it to
+# build centos kernels, which are 3.x based and explicitly want a gcc
+# version 3, 4, or 5 compiler. So grab copies we've saved from debian
+# snapshots with the prefix https://snapshot.debian.org/archive/debian/20190122T000000Z.
+
+RUN curl -o cpp-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/cpp-5_5.5.0-12_amd64.deb \
+ && curl -o gcc-5-base_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5-base_5.5.0-12_amd64.deb \
+ && curl -o gcc-5_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/gcc-5_5.5.0-12_amd64.deb \
+ && curl -o libasan2_5.5.0-12_amd64.deb	https://s3.amazonaws.com/download.draios.com/dependencies/libasan2_5.5.0-12_amd64.deb \
+ && curl -o libgcc-5-dev_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libgcc-5-dev_5.5.0-12_amd64.deb \
+ && curl -o libisl15_0.18-4_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libisl15_0.18-4_amd64.deb \
+ && curl -o libmpx0_5.5.0-12_amd64.deb https://s3.amazonaws.com/download.draios.com/dependencies/libmpx0_5.5.0-12_amd64.deb \
+ && dpkg -i cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb \
+ && rm -f cpp-5_5.5.0-12_amd64.deb gcc-5-base_5.5.0-12_amd64.deb gcc-5_5.5.0-12_amd64.deb libasan2_5.5.0-12_amd64.deb libgcc-5-dev_5.5.0-12_amd64.deb libisl15_0.18-4_amd64.deb libmpx0_5.5.0-12_amd64.deb
+
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.
 RUN rm -rf /usr/bin/gcc && ln -s /usr/bin/gcc-5 /usr/bin/gcc

examples/k8s_audit_config/README.md

@@ -3,21 +3,36 @@
 The files in this directory can be used to configure k8s audit logging. The relevant files are:
 
 * [audit-policy.yaml](./audit-policy.yaml): The k8s audit log configuration we used to create the rules in [k8s_audit_rules.yaml](../../rules/k8s_audit_rules.yaml). You may find it useful as a reference when creating your own K8s Audit Log configuration.
-* [webhook-config.yaml](./webhook-config.yaml): A webhook configuration that sends audit events to localhost, port 8765. You may find it useful as a starting point when deciding how to route audit events to the embedded webserver within falco.
+* [webhook-config.yaml.in](./webhook-config.yaml.in): A (templated) webhook configuration that sends audit events to an ip associated with the falco service, port 8765. It is templated in that the *actual* ip is defined in an environment variable `FALCO_SERVICE_CLUSTERIP`, which can be plugged in using a program like `envsubst`. You may find it useful as a starting point when deciding how to route audit events to the embedded webserver within falco.
 
-This file is only needed when using Minikube, which doesn't currently
+These files are only needed when using Minikube, which doesn't currently
 have the ability to provide an audit config/webhook config directly
 from the minikube commandline. See [this issue](https://github.com/kubernetes/minikube/issues/2741) for more details.
 
 * [apiserver-config.patch.sh](./apiserver-config.patch.sh): A script that changes the configuration file `/etc/kubernetes/manifests/kube-apiserver.yaml` to add necessary config options and mounts for the kube-apiserver container that runs within the minikube vm.
 
-A way to use these files with minikube to enable audit logging would be to run the following commands, from this directory:
+A way to use these files with minikube to run falco and enable audit logging would be the following:
+
+#### Start Minikube with Audit Logging Enabled
+
+Run the following to start minikube with Audit Logging Enabled:
 
 ```
 minikube start --kubernetes-version v1.11.0 --mount --mount-string $PWD:/tmp/k8s_audit_config --feature-gates AdvancedAuditing=true
-ssh -i $(minikube ssh-key) docker@$(minikube ip) sudo bash /tmp/k8s_audit_config/apiserver-config.patch.sh
-ssh -i $(minikube ssh-key) -R 8765:localhost:8765 docker@$(minikube ip)
 ```
 
-K8s audit events will then be sent to localhost on the host (not minikube vm) machine, port 8765.
+#### Create a Falco DaemonSet and Supporting Accounts/Services
+
+Follow the [K8s Using Daemonset](../../integrations/k8s-using-daemonset/README.md) instructions to create a falco service account, service, configmap, and daemonset.
+
+#### Configure Audit Logging with a Policy and Webhook
+
+Run the following commands to fill in the template file with the ClusterIP ip address you created with the `falco-service` service above, and configure audit logging to use a policy and webhook that directs the right events to the falco daemonset. Although services like `falco-service.default.svc.cluster.local` can not be resolved from the kube-apiserver container within the minikube vm (they're run as pods but not *really* a part of the cluster), the ClusterIPs associated with those services are routable.
+
+```
+FALCO_SERVICE_CLUSTERIP=$(kubectl get service falco-service -o=jsonpath={.spec.clusterIP}) envsubst < webhook-config.yaml.in > webhook-config.yaml
+minikube ssh sudo bash /tmp/k8s_audit_config/apiserver-config.patch.sh
+```
+
+K8s audit events will then be routed to the falco daemonset within the cluster, which you can observe via `kubectl logs -f $(kubectl get pods -l app=falco-example -o jsonpath={.items[0].metadata.name})`.
 

examples/k8s_audit_config/webhook-config.yaml.in

@@ -3,7 +3,7 @@ kind: Config
 clusters:
 - name: falco
   cluster:
-    server: http://127.0.0.1:8765/k8s_audit
+    server: http://$FALCO_SERVICE_CLUSTERIP:8765/k8s_audit
 contexts:
 - context:
     cluster: falco

examples/nodejs-bad-rest-api/README.md

@@ -1,4 +1,4 @@
-#Demo of falco with bash exec via poorly designed REST API.
+# Demo of falco with bash exec via poorly designed REST API.
 
 ## Introduction
 

falco.yaml

@@ -60,8 +60,8 @@ log_level: info
 priority: debug
 
 # Whether or not output to any of the output channels below is
-# buffered. Defaults to true
-buffered_outputs: true
+# buffered. Defaults to false
+buffered_outputs: false
 
 # A throttling mechanism implemented as a token bucket limits the
 # rate of falco notifications. This throttling is controlled by the following configuration
@@ -104,11 +104,20 @@ stdout_output:
 # Falco contains an embedded webserver that can be used to accept K8s
 # Audit Events. These config options control the behavior of that
 # webserver. (By default, the webserver is disabled).
-#  enabled: false
+#
+# The ssl_certificate is a combination SSL Certificate and corresponding
+# key contained in a single file. You can generate a key/cert as follows:
+#
+# $ openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem
+# $ cat certificate.pem key.pem > falco.pem
+# $ sudo cp falco.pem /etc/falco/falco.pem
+
 webserver:
   enabled: true
   listen_port: 8765
   k8s_audit_endpoint: /k8s_audit
+  ssl_enabled: false
+  ssl_certificate: /etc/falco/falco.pem
 
 # Possible additional things you might want to do with program output:
 #   - send to a slack webhook:

integrations/k8s-using-daemonset/README.md

@@ -4,7 +4,7 @@ This directory gives you the required YAML files to stand up Sysdig Falco on Kub
 
 The two options are provided to deploy a Daemon Set:
 - `k8s-with-rbac` - This directory provides a definition to deploy a Daemon Set on Kubernetes with RBAC enabled.
-- `k8s-without-rbac` - This directory provides a definition to deploy a Daemon Set on Kubernetes without RBAC enabled. 
+- `k8s-without-rbac` - This directory provides a definition to deploy a Daemon Set on Kubernetes without RBAC enabled. **This method is deprecated in favor of RBAC-based installs, and won't be updated going forward.**
 
 Also provided:
 - `falco-event-generator-deployment.yaml` - A Kubernetes Deployment to generate sample events. This is useful for testing, but note it will generate a large number of events.
@@ -21,11 +21,20 @@ clusterrolebinding "falco-cluster-role-binding" created
 k8s-using-daemonset$
 ```
 
-The Daemon Set also relies on a Kubernetes ConfigMap to store the Falco configuration and make the configuration available to the Falco Pods. This allows you to manage custom configuration without rebuilding and redeploying the underlying Pods. In order to create the ConfigMap you'll need to first need to copy the required configuration from their location in this GitHub repo to the `k8s-with-rbac/falco-config/` directory. Any modification of the configuration should be performed on these copies rather than the original files.
+We also create a service that allows other services to reach the embedded webserver in falco, which listens on https port 8765:
+
+```
+k8s-using-daemonset$ kubectl create -f k8s-with-rbac/falco-service.yaml
+service/falco-service created
+k8s-using-daemonset$
+```
+
+The Daemon Set also relies on a Kubernetes ConfigMap to store the Falco configuration and make the configuration available to the Falco Pods. This allows you to manage custom configuration without rebuilding and redeploying the underlying Pods. In order to create the ConfigMap you'll need to first need to copy the required configuration from their location in this GitHub repo to the `k8s-with-rbac/falco-config/` directory (please note that you will need to create the /falco-config directory). Any modification of the configuration should be performed on these copies rather than the original files.
 
 ```
 k8s-using-daemonset$ cp ../../falco.yaml k8s-with-rbac/falco-config/
 k8s-using-daemonset$ cp ../../rules/falco_rules.* k8s-with-rbac/falco-config/
+k8s-using-daemonset$ cp ../../rules/k8s_audit_rules.yaml k8s-with-rbac/falco-config/
 ```
 
 If you want to send Falco alerts to a Slack channel, you'll want to modify the `falco.yaml` file to point to your Slack webhook. For more information on getting a webhook URL for your Slack team, refer to the [Slack documentation](https://api.slack.com/incoming-webhooks). Add the below to the bottom of the `falco.yaml` config file you just copied to enable Slack messages.
@@ -54,7 +63,7 @@ k8s-using-daemonset$
 ```
 
 
-## Deploying to Kubernetes without RBAC enabled
+## Deploying to Kubernetes without RBAC enabled (**Deprecated**)
 
 If you are running Kubernetes with Legacy Authorization enabled, you can use `kubectl` to deploy the Daemon Set provided in the `k8s-without-rbac` directory. The example provides the ability to post messages to a Slack channel via a webhook. For more information on getting a webhook URL for your Slack team, refer to the [Slack documentation](https://api.slack.com/incoming-webhooks). Modify the [`args`](https://github.com/draios/falco/blob/dev/examples/k8s-using-daemonset/falco-daemonset.yaml#L21) passed to the Falco container to point to the appropriate URL for your webhook.
 

integrations/k8s-using-daemonset/k8s-with-rbac/falco-account.yaml

@@ -2,14 +2,20 @@ apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: falco-account
+  labels:
+    app: falco-example
+    role: security
 ---
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1beta1
 metadata:
   name: falco-cluster-role
+  labels:
+    app: falco-example
+    role: security
 rules:
   - apiGroups: ["extensions",""]
-    resources: ["nodes","namespaces","pods","replicationcontrollers","services","events","configmaps"]
+    resources: ["nodes","namespaces","pods","replicationcontrollers","replicasets","services","daemonsets","deployments","events","configmaps"]
     verbs: ["get","list","watch"]
   - nonResourceURLs: ["/healthz", "/healthz/*"]
     verbs: ["get"]
@@ -19,6 +25,9 @@ apiVersion: rbac.authorization.k8s.io/v1beta1
 metadata:
   name: falco-cluster-role-binding
   namespace: default
+  labels:
+    app: falco-example
+    role: security
 subjects:
   - kind: ServiceAccount
     name: falco-account

integrations/k8s-using-daemonset/k8s-with-rbac/falco-daemonset-configmap.yaml

@@ -1,16 +1,15 @@
 apiVersion: extensions/v1beta1
 kind: DaemonSet
 metadata:
-  name: falco
+  name: falco-daemonset
   labels:
-    name: falco-daemonset
-    app: demo
+    app: falco-example
+    role: security
 spec:
   template:
     metadata:
       labels:
-        name: falco
-        app: demo
+        app: falco-example
         role: security
     spec:
       serviceAccount: falco-account
@@ -19,6 +18,13 @@ spec:
           image: falcosecurity/falco:latest
           securityContext:
             privileged: true
+# Uncomment the 3 lines below to enable eBPF support for Falco.
+# This allows Falco to run on Google COS.
+# Leave blank for the default probe location, or set to the path
+# of a precompiled probe.
+#          env:
+#          - name: SYSDIG_BPF_PROBE
+#            value: ""
           args: [ "/usr/bin/falco", "-K", "/var/run/secrets/kubernetes.io/serviceaccount/token", "-k", "https://kubernetes.default", "-pk"]
           volumeMounts:
             - mountPath: /host/var/run/docker.sock
@@ -37,6 +43,9 @@ spec:
             - mountPath: /host/usr
               name: usr-fs
               readOnly: true
+            - mountPath: /host/etc/
+              name: etc-fs
+              readOnly: true
             - mountPath: /etc/falco
               name: falco-config
       volumes:
@@ -58,6 +67,9 @@ spec:
         - name: usr-fs
           hostPath:
             path: /usr
+        - name: etc-fs
+          hostPath:
+            path: /etc
         - name: falco-config
           configMap:
             name: falco-config

integrations/k8s-using-daemonset/k8s-with-rbac/falco-service.yaml

@@ -0,0 +1,13 @@
+kind: Service
+apiVersion: v1
+metadata:
+  name: falco-service
+  labels:
+    app: falco-example
+    role: security
+spec:
+  selector:
+    app: falco-example
+  ports:
+  - protocol: TCP
+    port: 8765

integrations/kubernetes-response-engine/deployment/aws/.gitignore

@@ -1,4 +1,4 @@
 .terraform/*
 .terraform.*
 terraform.*
-*.yaml
+aws-auth-patch.yml

integrations/kubernetes-response-engine/deployment/aws/Makefile

@@ -1,11 +1,17 @@
-all: create configure
+deploy: rbac create configure
+
+rbac:
+	kubectl apply -f cluster-role.yaml
+	kubectl apply -f cluster-role-binding.yaml
 
 create:
-	terraform apply
+	terraform apply -auto-approve
 
 configure:
 	kubectl get -n kube-system configmap/aws-auth -o yaml | awk "/mapRoles: \|/{print;print \"$(shell terraform output patch_for_aws_auth)\";next}1" > aws-auth-patch.yml
 	kubectl -n kube-system replace -f aws-auth-patch.yml
 
 clean:
-	terraform destroy
+	terraform destroy -force
+	kubectl delete -f cluster-role-binding.yaml
+	kubectl delete -f cluster-role.yaml

integrations/kubernetes-response-engine/deployment/aws/cluster-role-binding.yaml

@@ -0,0 +1,12 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: kubernetes-response-engine-cluster-role-binding
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: kubernetes-response-engine-cluster-role
+subjects:
+- kind: User
+  apiGroup: rbac.authorization.k8s.io
+  name: kubernetes-response-engine

integrations/kubernetes-response-engine/deployment/aws/cluster-role.yaml

@@ -0,0 +1,25 @@
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: kubernetes-response-engine-cluster-role
+rules:
+  - apiGroups:
+    - ""
+    resources:
+      - pods
+    verbs:
+      - delete
+      - list
+      - patch
+  - apiGroups:
+    - ""
+    resources:
+      - nodes
+    verbs:
+      - patch
+  - apiGroups:
+      - batch
+    resources:
+      - jobs
+    verbs:
+      - create

integrations/kubernetes-response-engine/deployment/aws/lambda.tf

@@ -1,3 +1,7 @@
+resource "aws_iam_user" "kubernetes-response-engine-user" {
+  name = "kubernetes_response_engine"
+}
+
 resource "aws_iam_role" "iam-for-lambda" {
   name = "iam_for_lambda"
 
@@ -9,7 +13,7 @@ resource "aws_iam_role" "iam-for-lambda" {
       "Action": "sts:AssumeRole",
       "Principal": {
         "Service": "lambda.amazonaws.com",
-        "AWS": "${var.iam-user-arn}"
+        "AWS": "${aws_iam_user.kubernetes-response-engine-user.arn}"
       },
       "Effect": "Allow",
       "Sid": ""

integrations/kubernetes-response-engine/deployment/aws/outputs.tf

@@ -1,9 +1,7 @@
 locals {
   patch_for_aws_auth = <<CONFIGMAPAWSAUTH
     - rolearn: ${aws_iam_role.iam-for-lambda.arn}\n
-     username: kubernetes-admin\n
-     groups:\n
-       - system:masters
+     username: kubernetes-response-engine
 CONFIGMAPAWSAUTH
 }
 

integrations/kubernetes-response-engine/deployment/aws/variables.tf

@@ -1,3 +0,0 @@
-variable "iam-user-arn" {
-  type    = "string"
-}

integrations/kubernetes-response-engine/deployment/cncf/Makefile

@@ -1,7 +1,6 @@
 deploy:
 	kubectl apply -f nats/
 	kubectl apply -f kubeless/
-	kubectl apply -f network-policy.yaml
 	kubectl apply -f .
 
 clean:

integrations/kubernetes-response-engine/deployment/cncf/cluster-role-binding.yaml

@@ -1,11 +1,11 @@
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRoleBinding
 metadata:
-  name: sysdig-kubeless
+  name: kubernetes-response-engine-cluster-role-binding
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: cluster-admin
+  name: kubernetes-response-engine-cluster-role
 subjects:
 - kind: ServiceAccount
   name: default

integrations/kubernetes-response-engine/deployment/cncf/cluster-role.yaml

@@ -0,0 +1,25 @@
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: kubernetes-response-engine-cluster-role
+rules:
+  - apiGroups:
+    - ""
+    resources:
+      - pods
+    verbs:
+      - delete
+      - list
+      - patch
+  - apiGroups:
+    - ""
+    resources:
+      - nodes
+    verbs:
+      - patch
+  - apiGroups:
+      - batch
+    resources:
+      - jobs
+    verbs:
+      - create

integrations/kubernetes-response-engine/playbooks/deploy_playbook_aws

@@ -16,7 +16,7 @@ You must pass the playbook and at least one topic to subscribe.
 
 Example:
 
-deploy_playbook -p slack -t "falco.error.*" -e SLACK_WEBHOOK_URL=http://foobar.com/... -k sysdig_eks
+deploy_playbook -p slack -e SLACK_WEBHOOK_URL=http://foobar.com/... -k sysdig_eks
 EOF
   exit 1
 }
@@ -27,7 +27,7 @@ playbook=""
 environment=("KUBECONFIG=kubeconfig" "KUBERNETES_LOAD_KUBE_CONFIG=1")
 eks_cluster="${EKS_CLUSTER}"
 
-while getopts "r:e:t:" arg; do
+while getopts "p:e:k:" arg; do
   case $arg in
     p)
       playbook="${OPTARG}"

rules/application_rules.yaml

@@ -16,6 +16,8 @@
 # limitations under the License.
 #
 
+- required_engine_version: 2
+
 ################################################################
 # By default all application-related rules are disabled for
 # performance reasons. Depending on the application(s) you use,

rules/falco_rules.yaml

@@ -16,6 +16,16 @@
 # limitations under the License.
 #
 
+# See xxx for details on falco engine and rules versioning. Currently,
+# this specific rules file is compatible with engine version 0
+# (e.g. falco releases <= 0.13.1), so we'll keep the
+# required_engine_version lines commented out, so maintain
+# compatibility with older falco releases. With the first incompatible
+# change to this rules file, we'll uncomment this line and set it to
+# the falco engine version in use at the time.
+#
+#- required_engine_version: 2
+
 # Currently disabled as read/write are ignored syscalls. The nearly
 # similar open_write/open_read check for files being opened for
 # reading/writing.
@@ -178,17 +188,23 @@
 - list: deb_binaries
   items: [dpkg, dpkg-preconfigu, dpkg-reconfigur, dpkg-divert, apt, apt-get, aptitude,
     frontend, preinst, add-apt-reposit, apt-auto-remova, apt-key,
-    apt-listchanges, unattended-upgr, apt-add-reposit
+    apt-listchanges, unattended-upgr, apt-add-reposit, apt-config, apt-cache
     ]
 
 # The truncated dpkg-preconfigu is intentional, process names are
 # truncated at the sysdig level.
 - list: package_mgmt_binaries
-  items: [rpm_binaries, deb_binaries, update-alternat, gem, pip, pip3, sane-utils.post, alternatives, chef-client]
+  items: [rpm_binaries, deb_binaries, update-alternat, gem, pip, pip3, sane-utils.post, alternatives, chef-client, apk]
 
 - macro: package_mgmt_procs
   condition: proc.name in (package_mgmt_binaries)
 
+- macro: package_mgmt_ancestor_procs
+  condition: proc.pname in (package_mgmt_binaries) or
+             proc.aname[2] in (package_mgmt_binaries) or
+             proc.aname[3] in (package_mgmt_binaries) or
+             proc.aname[4] in (package_mgmt_binaries)
+
 - macro: coreos_write_ssh_dir
   condition: (proc.name=update-ssh-keys and fd.name startswith /home/core/.ssh)
 
@@ -262,14 +278,18 @@
 # Network
 - macro: inbound
   condition: >
-    (((evt.type in (accept,listen) and evt.dir=<)) or
+    (((evt.type in (accept,listen) and evt.dir=<) or
+      (evt.type in (recvfrom,recvmsg) and evt.dir=< and
+       fd.l4proto != tcp and fd.connected=false and fd.name_changed=true)) and
      (fd.typechar = 4 or fd.typechar = 6) and
      (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
      (evt.rawres >= 0 or evt.res = EINPROGRESS))
 
 - macro: outbound
   condition: >
-    (((evt.type = connect and evt.dir=<)) or
+    (((evt.type = connect and evt.dir=<) or
+      (evt.type in (sendto,sendmsg) and evt.dir=< and
+       fd.l4proto != tcp and fd.connected=false and fd.name_changed=true)) and
      (fd.typechar = 4 or fd.typechar = 6) and
      (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
      (evt.rawres >= 0 or evt.res = EINPROGRESS))
@@ -580,7 +600,7 @@
   condition: (proc.pname=run-openldap.sh and fd.name startswith /etc/openldap)
 
 - macro: ucpagent_writing_conf
-  condition: (proc.name=apiserver and container.image startswith docker/ucp-agent and fd.name=/etc/authorization_config.cfg)
+  condition: (proc.name=apiserver and container.image.repository=docker/ucp-agent and fd.name=/etc/authorization_config.cfg)
 
 - macro: iscsi_writing_conf
   condition: (proc.name=iscsiadm and fd.name startswith /etc/iscsi)
@@ -628,7 +648,7 @@
   condition: (veritas_progs and (fd.name startswith /etc/vx or fd.name startswith /etc/opt/VRTS or fd.name startswith /etc/vom))
 
 - macro: nginx_writing_conf
-  condition: (proc.name=nginx and fd.name startswith /etc/nginx)
+  condition: (proc.name in (nginx,nginx-ingress-c) and fd.name startswith /etc/nginx)
 
 - macro: nginx_writing_certs
   condition: >
@@ -802,7 +822,7 @@
   condition: (proc.name=rabbitmq-server and fd.directory=/etc/rabbitmq)
 
 - macro: rook_writing_conf
-  condition: (proc.name=toolbox.sh and container.image startswith rook/toolbox
+  condition: (proc.name=toolbox.sh and container.image.repository=rook/toolbox
               and fd.directory=/etc/ceph)
 
 - macro: httpd_writing_conf_logs
@@ -839,7 +859,17 @@
   condition: (proc.aname[2] in (dpkg-reconfigur, dpkg-preconfigu))
 
 - macro: ufw_writing_conf
-  condition: proc.name=ufw and fd.directory=/etc/ufw
+  condition: (proc.name=ufw and fd.directory=/etc/ufw)
+
+- macro: calico_writing_conf
+  condition: >
+    (proc.name = calico-node and fd.name startswith /etc/calico)
+
+- macro: prometheus_conf_writing_conf
+  condition: (proc.name=prometheus-conf and fd.directory=/etc/prometheus/config_out)
+
+- macro: openshift_writing_conf
+  condition: (proc.name=oc and fd.name=/etc/origin/node/node.kubeconfig)
 
 # Add conditions to this macro (probably in a separate file,
 # overwriting this macro) to allow for specific combinations of
@@ -943,6 +973,9 @@
     and not iscsi_writing_conf
     and not istio_writing_conf
     and not ufw_writing_conf
+    and not calico_writing_conf
+    and not prometheus_conf_writing_conf
+    and not openshift_writing_conf
 
 - rule: Write below etc
   desc: an attempt to write to any file below /etc
@@ -1153,7 +1186,7 @@
     as a part of creating a container) by calling setns.
   condition: >
     evt.type = setns
-    and not proc.name in (docker_binaries, k8s_binaries, lxd_binaries, sysdigcloud_binaries, sysdig, nsenter)
+    and not proc.name in (docker_binaries, k8s_binaries, lxd_binaries, sysdigcloud_binaries, sysdig, nsenter, calico)
     and not proc.name in (user_known_change_thread_namespace_binaries)
     and not proc.name startswith "runc:"
     and not proc.pname in (sysdigcloud_binaries)
@@ -1314,50 +1347,43 @@
 
 - macro: allowed_openshift_registry_root
   condition: >
-    (container.image startswith openshift3/ or
-     container.image startswith registry.access.redhat.com/openshift3/)
+    (container.image.repository startswith openshift3/ or
+     container.image.repository startswith registry.access.redhat.com/openshift3/)
 
 # Source: https://docs.openshift.com/enterprise/3.2/install_config/install/disconnected_install.html
 - macro: openshift_image
   condition: >
     (allowed_openshift_registry_root and
-      (container.image contains logging-deployment or
-       container.image contains logging-elasticsearch or
-       container.image contains logging-kibana or
-       container.image contains logging-fluentd or
-       container.image contains logging-auth-proxy or
-       container.image contains metrics-deployer or
-       container.image contains metrics-hawkular-metrics or
-       container.image contains metrics-cassandra or
-       container.image contains metrics-heapster or
-       container.image contains ose-haproxy-router or
-       container.image contains ose-deployer or
-       container.image contains ose-sti-builder or
-       container.image contains ose-docker-builder or
-       container.image contains ose-pod or
-       container.image contains ose-docker-registry or
-       container.image contains image-inspector))
+      (container.image.repository contains logging-deployment or
+       container.image.repository contains logging-elasticsearch or
+       container.image.repository contains logging-kibana or
+       container.image.repository contains logging-fluentd or
+       container.image.repository contains logging-auth-proxy or
+       container.image.repository contains metrics-deployer or
+       container.image.repository contains metrics-hawkular-metrics or
+       container.image.repository contains metrics-cassandra or
+       container.image.repository contains metrics-heapster or
+       container.image.repository contains ose-haproxy-router or
+       container.image.repository contains ose-deployer or
+       container.image.repository contains ose-sti-builder or
+       container.image.repository contains ose-docker-builder or
+       container.image.repository contains ose-pod or
+       container.image.repository contains ose-docker-registry or
+       container.image.repository contains image-inspector))
+
+- list: trusted_images
+  items: [
+    sysdig/agent, sysdig/falco, sysdig/sysdig, gcr.io/google_containers/hyperkube,
+    quay.io/coreos/flannel, gcr.io/google_containers/kube-proxy, calico/node,
+    rook/toolbox, cloudnativelabs/kube-router, consul, mesosphere/mesos-slave,
+    datadog/docker-dd-agent, datadog/agent, docker/ucp-agent, gliderlabs/logspout
+    ]
 
 - macro: trusted_containers
-  condition: (container.image startswith sysdig/agent or
-              (container.image startswith sysdig/falco and
-               not container.image startswith sysdig/falco-event-generator) or
-              container.image startswith quay.io/sysdig or
-              container.image startswith sysdig/sysdig or
-              container.image startswith gcr.io/google_containers/hyperkube or
-              container.image startswith quay.io/coreos/flannel or
-              container.image startswith gcr.io/google_containers/kube-proxy or
-              container.image startswith calico/node or
-              container.image startswith rook/toolbox or
-              openshift_image or
-              container.image startswith cloudnativelabs/kube-router or
-              container.image startswith "consul:" or
-              container.image startswith mesosphere/mesos-slave or
-              container.image startswith istio/proxy_ or
-              container.image startswith datadog/docker-dd-agent or
-              container.image startswith datadog/agent or
-              container.image startswith docker/ucp-agent or
-              container.image startswith gliderlabs/logspout)
+  condition: (openshift_image or
+              container.image.repository in (trusted_images) or
+              container.image.repository startswith istio/proxy_ or
+              container.image.repository startswith quay.io/sysdig)
 
 # Add conditions to this macro (probably in a separate file,
 # overwriting this macro) to specify additional containers that are
@@ -1366,7 +1392,7 @@
 # In this file, it just takes one of the images in trusted_containers
 # and repeats it.
 - macro: user_trusted_containers
-  condition: (container.image startswith sysdig/agent)
+  condition: (container.image.repository=sysdig/agent)
 
 # Add conditions to this macro (probably in a separate file,
 # overwriting this macro) to specify additional containers that are
@@ -1375,7 +1401,7 @@
 # In this file, it just takes one of the images in trusted_containers
 # and repeats it.
 - macro: user_sensitive_mount_containers
-  condition: (container.image startswith sysdig/agent)
+  condition: (container.image.repository=sysdig/agent)
 
 - rule: Launch Privileged Container
   desc: Detect the initial process started in a privileged container. Exceptions are made for known trusted images.
@@ -1384,7 +1410,7 @@
     and container.privileged=true
     and not trusted_containers
     and not user_trusted_containers
-  output: Privileged container started (user=%user.name command=%proc.cmdline %container.info image=%container.image)
+  output: Privileged container started (user=%user.name command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag)
   priority: INFO
   tags: [container, cis]
 
@@ -1395,7 +1421,8 @@
 - macro: sensitive_mount
   condition: (container.mount.dest[/proc*] != "N/A" or
               container.mount.dest[/var/run/docker.sock] != "N/A" or
-              container.mount.dest[/var/lib/kubelet*] != "N/A" or
+              container.mount.dest[/var/lib/kubelet] != "N/A" or
+              container.mount.dest[/var/lib/kubelet/pki] != "N/A" or
               container.mount.dest[/] != "N/A" or
               container.mount.dest[/etc] != "N/A" or
               container.mount.dest[/root*] != "N/A")
@@ -1423,7 +1450,7 @@
     and sensitive_mount
     and not trusted_containers
     and not user_sensitive_mount_containers
-  output: Container with sensitive mount started (user=%user.name command=%proc.cmdline %container.info image=%container.image mounts=%container.mounts)
+  output: Container with sensitive mount started (user=%user.name command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag mounts=%container.mounts)
   priority: INFO
   tags: [container, cis]
 
@@ -1434,8 +1461,8 @@
 # alllowed, by using a filter that is guaranteed to evaluate to true
 # (the same proc.vpid=1 that's in the Launch Disallowed Container
 # rule). In the overridden macro, the condition would look something
-# like (container.image startswith vendor/container-1 or
-# container.image startswith vendor/container-2 or ...)
+# like (container.image.repository = vendor/container-1 or
+# container.image.repository = vendor/container-2 or ...)
 
 - macro: allowed_containers
   condition: (proc.vpid=1)
@@ -1444,7 +1471,7 @@
   desc: >
     Detect the initial process started by a container that is not in a list of allowed containers.
   condition: evt.type=execve and proc.vpid=1 and container and not allowed_containers
-  output: Container started and not in allowed list (user=%user.name command=%proc.cmdline %container.info image=%container.image)
+  output: Container started and not in allowed list (user=%user.name command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag)
   priority: WARNING
   tags: [container]
 
@@ -1686,8 +1713,8 @@
 # any way to know all the containers that should have access, so any
 # container is alllowed, by repeating the "container" macro. In the
 # overridden macro, the condition would look something like
-# (container.image startswith vendor/container-1 or container.image
-# startswith vendor/container-2 or ...)
+# (container.image.repository = vendor/container-1 or
+# container.image.repository = vendor/container-2 or ...)
 - macro: ec2_metadata_containers
   condition: container
 
@@ -1697,7 +1724,7 @@
 - rule: Contact EC2 Instance Metadata Service From Container
   desc: Detect attempts to contact the EC2 Instance Metadata Service from a container
   condition: outbound and fd.sip="169.254.169.254" and container and not ec2_metadata_containers
-  output: Outbound connection to EC2 instance metadata service (command=%proc.cmdline connection=%fd.name %container.info image=%container.image)
+  output: Outbound connection to EC2 instance metadata service (command=%proc.cmdline connection=%fd.name %container.info image=%container.image.repository:%container.image.tag)
   priority: NOTICE
   tags: [network, aws, container]
 
@@ -1713,16 +1740,14 @@
 # within a container.
 - macro: k8s_containers
   condition: >
-    (container.image startswith gcr.io/google_containers/hyperkube-amd64 or
-     container.image startswith gcr.io/google_containers/kube2sky or
-     container.image startswith sysdig/agent or
-     container.image startswith sysdig/falco or
-     container.image startswith sysdig/sysdig)
+    (container.image.repository in (gcr.io/google_containers/hyperkube-amd64,
+     gcr.io/google_containers/kube2sky, sysdig/agent, sysdig/falco,
+     sysdig/sysdig))
 
 - rule: Contact K8S API Server From Container
   desc: Detect attempts to contact the K8S API Server from a container
   condition: outbound and k8s_api_server and container and not k8s_containers
-  output: Unexpected connection to K8s API Server from container (command=%proc.cmdline %container.info image=%container.image connection=%fd.name)
+  output: Unexpected connection to K8s API Server from container (command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag connection=%fd.name)
   priority: NOTICE
   tags: [network, k8s, container]
 
@@ -1742,6 +1767,46 @@
   priority: NOTICE
   tags: [network, k8s, container]
 
+- list: network_tool_binaries
+  items: [nc, ncat, nmap]
+
+- macro: network_tool_procs
+  condition: proc.name in (network_tool_binaries)
+
+# Container is supposed to be immutable. Package management should be done in building the image.
+- rule: Launch Package Management Process in Container
+  desc: Package management process ran inside container
+  condition: >
+    spawned_process and container and user.name != "_apt" and package_mgmt_procs and not package_mgmt_ancestor_procs
+  output: >
+    Package management process launched in container (user=%user.name
+    command=%proc.cmdline container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag)
+  priority: ERROR
+  tags: [process]
+
+- rule: Netcat Remote Code Execution in Container
+  desc: Netcat Program runs inside container that allows remote code execution
+  condition: >
+    spawned_process and container and
+    ((proc.name = "nc" and (proc.args contains "-e" or proc.args contains "-c")) or
+     (proc.name = "ncat" and (proc.args contains "--sh-exec" or proc.args contains "--exec"))
+    )
+  output: >
+    Netcat runs inside container that allows remote code execution (user=%user.name
+    command=%proc.cmdline container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag)
+  priority: WARNING
+  tags: [network, process]
+
+- rule: Lauch Suspicious Network Tool in Container
+  desc: Detect network tools launched inside container
+  condition: >
+    spawned_process and container and network_tool_procs
+  output: >
+    Network tool launched in container (user=%user.name
+    command=%proc.cmdline container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag)
+  priority: NOTICE
+  tags: [network, process]
+
 # Application rules have moved to application_rules.yaml. Please look
 # there if you want to enable them by adding to
 # falco_rules.local.yaml.

rules/k8s_audit_rules.yaml

@@ -1,3 +1,22 @@
+#
+# Copyright (C) 2016-2018 Draios Inc dba Sysdig.
+#
+# This file is part of falco.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+- required_engine_version: 2
+
 # Generally only consider audit events once the response has completed
 - list: k8s_audit_stages
   items: ["ResponseComplete"]

test/falco_tests.yaml

@@ -18,7 +18,7 @@
 trace_files: !mux
 
   docker_package:
-    package: docker:sysdig/falco:test
+    package: docker:falcosecurity/falco:test
     detect: True
     detect_level: WARNING
     rules_file: /host/rules/rule_names_with_spaces.yaml
@@ -33,7 +33,7 @@ trace_files: !mux
   # just to see if falco can load the driver.
 
   docker_package_local_driver:
-    package: docker:sysdig/falco:test
+    package: docker:falcosecurity/falco:test
     addl_docker_run_args: -v /dev/null:/usr/sbin/dkms
     copy_local_driver: True
     detect: False
@@ -764,3 +764,10 @@ trace_files: !mux
     rules_file:
       - rules/skip_unknown_unspec.yaml
     trace_file: trace_files/cat_write.scap
+
+  engine_version_mismatch:
+    exit_status: 1
+    stderr_contains: Rules require engine version 9999999, but engine version is
+    rules_file:
+      - rules/engine_version_mismatch.yaml
+    trace_file: trace_files/cat_write.scap

test/rules/engine_version_mismatch.yaml

@@ -0,0 +1,34 @@
+#
+# Copyright (C) 2016-2018 Draios Inc dba Sysdig.
+#
+# This file is part of falco.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+- required_engine_version: 9999999
+
+- list: cat_binaries
+  items: [cat]
+
+- list: cat_capable_binaries
+  items: [cat_binaries]
+
+- macro: is_cat
+  condition: proc.name in (cat_capable_binaries)
+
+- rule: open_from_cat
+  desc: A process named cat does an open
+  condition: evt.type=open and is_cat
+  output: "An open was seen (command=%proc.cmdline)"
+  priority: WARNING

test/rules/single_rule.yaml

@@ -15,6 +15,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+
+- required_engine_version: 2
+
 - list: cat_binaries
   items: [cat]
 

userspace/engine/falco_engine.cpp

@@ -23,6 +23,7 @@ limitations under the License.
 #include <fstream>
 
 #include "falco_engine.h"
+#include "falco_engine_version.h"
 #include "config_falco_engine.h"
 
 #include "formats.h"
@@ -76,7 +77,79 @@ falco_engine::~falco_engine()
 	}
 }
 
+uint32_t falco_engine::engine_version()
+{
+	return (uint32_t) FALCO_ENGINE_VERSION;
+}
+
+#define DESCRIPTION_TEXT_START 16
+
+#define CONSOLE_LINE_LEN 79
+
+void falco_engine::list_fields(bool names_only)
+{
+	for(auto &chk_field : json_factory().get_fields())
+	{
+		if(!names_only)
+		{
+			printf("\n----------------------\n");
+			printf("Field Class: %s (%s)\n\n", chk_field.name.c_str(), chk_field.desc.c_str());
+		}
+
+		for(auto &field : chk_field.fields)
+		{
+			uint32_t l, m;
+
+			printf("%s", field.name.c_str());
+
+			if(names_only)
+			{
+				printf("\n");
+				continue;
+			}
+			uint32_t namelen = field.name.size();
+
+			if(namelen >= DESCRIPTION_TEXT_START)
+			{
+				printf("\n");
+				namelen = 0;
+			}
+
+			for(l = 0; l < DESCRIPTION_TEXT_START - namelen; l++)
+			{
+				printf(" ");
+			}
+
+			size_t desclen = field.desc.size();
+
+			for(l = 0; l < desclen; l++)
+			{
+				if(l % (CONSOLE_LINE_LEN - DESCRIPTION_TEXT_START) == 0 && l != 0)
+				{
+					printf("\n");
+
+					for(m = 0; m < DESCRIPTION_TEXT_START; m++)
+					{
+						printf(" ");
+					}
+				}
+
+				printf("%c", field.desc.at(l));
+			}
+
+			printf("\n");
+		}
+	}
+}
+
 void falco_engine::load_rules(const string &rules_content, bool verbose, bool all_events)
+{
+	uint64_t dummy;
+
+	return load_rules(rules_content, verbose, all_events, dummy);
+}
+
+void falco_engine::load_rules(const string &rules_content, bool verbose, bool all_events, uint64_t &required_engine_version)
 {
 	// The engine must have been given an inspector by now.
 	if(! m_inspector)
@@ -105,10 +178,17 @@ void falco_engine::load_rules(const string &rules_content, bool verbose, bool al
 	bool json_include_output_property = false;
 	falco_formats::init(m_inspector, this, m_ls, json_output, json_include_output_property);
 
-	m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info, m_min_priority);
+	m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info, m_min_priority, required_engine_version);
 }
 
 void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events)
+{
+	uint64_t dummy;
+
+	return load_rules_file(rules_filename, verbose, all_events, dummy);
+}
+
+void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events, uint64_t &required_engine_version)
 {
 	ifstream is;
 
@@ -123,7 +203,7 @@ void falco_engine::load_rules_file(const string &rules_filename, bool verbose, b
 	string rules_content((istreambuf_iterator<char>(is)),
 			     istreambuf_iterator<char>());
 
-	load_rules(rules_content, verbose, all_events);
+	load_rules(rules_content, verbose, all_events, required_engine_version);
 }
 
 void falco_engine::enable_rule(const string &pattern, bool enabled, const string &ruleset)

userspace/engine/falco_engine.h

@@ -53,12 +53,28 @@ public:
 	falco_engine(bool seed_rng=true, const std::string& alternate_lua_dir=FALCO_ENGINE_SOURCE_LUA_DIR);
 	virtual ~falco_engine();
 
+	// A given engine has a version which identifies the fields
+	// and rules file format it supports. This version will change
+	// any time the code that handles rules files, expression
+	// fields, etc, changes.
+	static uint32_t engine_version();
+
+	// Print to stdout (using printf) a description of each field supported by this engine.
+	void list_fields(bool names_only=false);
+
 	//
 	// Load rules either directly or from a filename.
 	//
 	void load_rules_file(const std::string &rules_filename, bool verbose, bool all_events);
 	void load_rules(const std::string &rules_content, bool verbose, bool all_events);
 
+	//
+	// Identical to above, but also returns the required engine version for the file/content.
+	// (If no required engine version is specified, returns 0).
+	//
+	void load_rules_file(const std::string &rules_filename, bool verbose, bool all_events, uint64_t &required_engine_version);
+	void load_rules(const std::string &rules_content, bool verbose, bool all_events, uint64_t &required_engine_version);
+
 	//
 	// Enable/Disable any rules matching the provided pattern
 	// (regex). When provided, enable/disable these rules in the

userspace/engine/falco_engine_version.h

@@ -0,0 +1,27 @@
+/*
+Copyright (C) 2016-2018 Draios Inc dba Sysdig.
+
+This file is part of falco.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+*/
+
+// The version of rules/filter fields/etc supported by this falco
+// engine.
+#define FALCO_ENGINE_VERSION (2)
+
+// This is the result of running "falco --list -N | sha256sum" and
+// represents the fields supported by this version of falco. It's used
+// at build time to detect a changed set of fields.
+#define FALCO_FIELDS_CHECKSUM "32a91c003ab34f198dcb4c3100fbfb22bf402ad36549f193afa43d73f1f2eba3"

userspace/engine/formats.cpp

@@ -152,7 +152,27 @@ int falco_formats::format_event (lua_State *ls)
 
 			if(s_json_output)
 			{
-				s_inspector->set_buffer_format(sinsp_evt::PF_JSON);
+				switch(s_inspector->get_buffer_format())
+				{
+					case sinsp_evt::PF_NORMAL:
+						s_inspector->set_buffer_format(sinsp_evt::PF_JSON);
+						break;
+					case sinsp_evt::PF_EOLS:
+						s_inspector->set_buffer_format(sinsp_evt::PF_JSONEOLS);
+						break;
+					case sinsp_evt::PF_HEX:
+						s_inspector->set_buffer_format(sinsp_evt::PF_JSONHEX);
+						break;
+					case sinsp_evt::PF_HEXASCII:
+						s_inspector->set_buffer_format(sinsp_evt::PF_JSONHEXASCII);
+						break;
+					case sinsp_evt::PF_BASE64:
+						s_inspector->set_buffer_format(sinsp_evt::PF_JSONBASE64);
+						break;
+					default:
+						// do nothing
+						break;
+				}
 				s_formatters->tostring((sinsp_evt *) evt, sformat, &json_line);
 
 				// The formatted string might have a leading newline. If it does, remove it.
@@ -160,8 +180,6 @@ int falco_formats::format_event (lua_State *ls)
 				{
 					json_line.erase(0, 1);
 				}
-
-				s_inspector->set_buffer_format(sinsp_evt::PF_NORMAL);
 			}
 		}
 		catch (sinsp_exception& e)

userspace/engine/json_evt.h

@@ -42,6 +42,17 @@ public:
 
 	uint64_t get_ts();
 
+	inline uint16_t get_source()
+	{
+		return ESRC_K8S_AUDIT;
+	}
+
+	inline uint16_t get_type()
+	{
+		// All k8s audit events have the single tag "1". - see falco_engine::process_k8s_audit_event
+		return 1;
+	}
+
 protected:
 	nlohmann::json m_jevt;
 
@@ -275,10 +286,11 @@ public:
 	std::string tostring(json_event *ev);
 	std::string tojson(json_event *ev);
 
+	void resolve_tokens(json_event *ev, std::list<std::pair<std::string,std::string>> &resolved);
+
 private:
 	void parse_format();
 
-	void resolve_tokens(json_event *ev, std::list<std::pair<std::string,std::string>> &resolved);
 
 	// A format token is either a combination of a filtercheck
 	// name (ka.value) and filtercheck object as key, or an empty

userspace/engine/lua/rule_loader.lua

@@ -195,10 +195,11 @@ function load_rules(sinsp_lua_parser,
 		    min_priority)
 
    local rules = yaml.load(rules_content)
+   local required_engine_version = 0
 
    if rules == nil then
       -- An empty rules file is acceptable
-      return
+      return required_engine_version
    end
 
    if type(rules) ~= "table" then
@@ -215,7 +216,13 @@ function load_rules(sinsp_lua_parser,
 	 error ("Unexpected element of type " ..type(v)..". Each element should be a yaml associative array.")
       end
 
-      if (v['macro']) then
+      if (v['required_engine_version']) then
+	 required_engine_version = v['required_engine_version']
+	 if falco_rules.engine_version(rules_mgr) < v['required_engine_version'] then
+	    error("Rules require engine version "..v['required_engine_version']..", but engine version is "..falco_rules.engine_version(rules_mgr))
+	 end
+
+      elseif (v['macro']) then
 
 	 if v['source'] == nil then
 	    v['source'] = "syscall"
@@ -493,24 +500,26 @@ function load_rules(sinsp_lua_parser,
 	 -- If the format string contains %container.info, replace it
 	 -- with extra. Otherwise, add extra onto the end of the format
 	 -- string.
-	 if string.find(v['output'], "%container.info", nil, true) ~= nil then
+	 if v['source'] == "syscall" then
+	    if string.find(v['output'], "%container.info", nil, true) ~= nil then
 
-	    -- There may not be any extra, or we're not supposed
-	    -- to replace it, in which case we use the generic
-	    -- "%container.name (id=%container.id)"
-	    if replace_container_info == false then
-	       v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
+	       -- There may not be any extra, or we're not supposed
+	       -- to replace it, in which case we use the generic
+	       -- "%container.name (id=%container.id)"
+	       if replace_container_info == false then
+		  v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
+		  if extra ~= "" then
+		     v['output'] = v['output'].." "..extra
+		  end
+	       else
+		  safe_extra = string.gsub(extra, "%%", "%%%%")
+		  v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
+	       end
+	    else
+	       -- Just add the extra to the end
 	       if extra ~= "" then
 		  v['output'] = v['output'].." "..extra
 	       end
-	    else
-	       safe_extra = string.gsub(extra, "%%", "%%%%")
-	       v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
-	    end
-	 else
-	    -- Just add the extra to the end
-	    if extra ~= "" then
-	       v['output'] = v['output'].." "..extra
 	    end
 	 end
 
@@ -542,6 +551,8 @@ function load_rules(sinsp_lua_parser,
    end
 
    io.flush()
+
+   return required_engine_version
 end
 
 local rule_fmt = "%-50s %s"

userspace/engine/rules.cpp

@@ -33,6 +33,7 @@ const static struct luaL_reg ll_falco_rules [] =
 	{"add_filter", &falco_rules::add_filter},
 	{"add_k8s_audit_filter", &falco_rules::add_k8s_audit_filter},
 	{"enable_rule", &falco_rules::enable_rule},
+	{"engine_version", &falco_rules::engine_version},
 	{NULL,NULL}
 };
 
@@ -204,10 +205,26 @@ void falco_rules::enable_rule(string &rule, bool enabled)
 	m_engine->enable_rule(rule, enabled);
 }
 
+int falco_rules::engine_version(lua_State *ls)
+{
+	if (! lua_islightuserdata(ls, -1))
+	{
+		lua_pushstring(ls, "Invalid arguments passed to engine_version()");
+		lua_error(ls);
+	}
+
+	falco_rules *rules = (falco_rules *) lua_topointer(ls, -1);
+
+	lua_pushnumber(ls, rules->m_engine->engine_version());
+
+	return 1;
+}
+
 void falco_rules::load_rules(const string &rules_content,
 			     bool verbose, bool all_events,
 			     string &extra, bool replace_container_info,
-			     falco_common::priority_type min_priority)
+			     falco_common::priority_type min_priority,
+			     uint64_t &required_engine_version)
 {
 	lua_getglobal(m_ls, m_lua_load_rules.c_str());
 	if(lua_isfunction(m_ls, -1))
@@ -382,12 +399,15 @@ void falco_rules::load_rules(const string &rules_content,
 		lua_pushstring(m_ls, extra.c_str());
 		lua_pushboolean(m_ls, (replace_container_info ? 1 : 0));
 		lua_pushnumber(m_ls, min_priority);
-		if(lua_pcall(m_ls, 9, 0, 0) != 0)
+		if(lua_pcall(m_ls, 9, 1, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
 			string err = "Error loading rules: " + string(lerr);
 			throw falco_exception(err);
 		}
+
+		required_engine_version = lua_tonumber(m_ls, -1);
+		lua_pop(m_ls, 1);
 	} else {
 		throw falco_exception("No function " + m_lua_load_rules + " found in lua rule module");
 	}

userspace/engine/rules.h

@@ -41,7 +41,8 @@ class falco_rules
 	~falco_rules();
 	void load_rules(const string &rules_content, bool verbose, bool all_events,
 			std::string &extra, bool replace_container_info,
-			falco_common::priority_type min_priority);
+			falco_common::priority_type min_priority,
+			uint64_t &required_engine_version);
 	void describe_rule(string *rule);
 
 	static void init(lua_State *ls);
@@ -49,6 +50,7 @@ class falco_rules
 	static int add_filter(lua_State *ls);
 	static int add_k8s_audit_filter(lua_State *ls);
 	static int enable_rule(lua_State *ls);
+	static int engine_version(lua_State *ls);
 
  private:
 	void clear_filters();

userspace/engine/ruleset.cpp

@@ -234,6 +234,11 @@ bool falco_ruleset::run(gen_event *evt, uint32_t etag, uint16_t ruleset)
 
 void falco_ruleset::event_tags_for_ruleset(vector<bool> &evttypes, uint16_t ruleset)
 {
+	if(m_rulesets.size() < (size_t) ruleset + 1)
+	{
+		return;
+	}
+
 	return m_rulesets[ruleset]->event_tags_for_ruleset(evttypes);
 }
 
@@ -314,7 +319,7 @@ void falco_sinsp_ruleset::evttypes_for_ruleset(vector<bool> &evttypes, uint16_t
 	{
 		uint32_t etag = evttype_to_event_tag(etype);
 
-		if(event_tags[etag])
+		if(etag < event_tags.size() && event_tags[etag])
 		{
 			evttypes[etype] = true;
 		}
@@ -333,7 +338,7 @@ void falco_sinsp_ruleset::syscalls_for_ruleset(vector<bool> &syscalls, uint16_t
 	{
 		uint32_t etag = evttype_to_event_tag(syscallid);
 
-		if(event_tags[etag])
+		if(etag < event_tags.size() && event_tags[etag])
 		{
 			syscalls[syscallid] = true;
 		}

userspace/falco/CMakeLists.txt

@@ -48,6 +48,17 @@ target_link_libraries(falco
 
 configure_file(config_falco.h.in config_falco.h)
 
+add_custom_command(TARGET falco
+	COMMAND bash ${CMAKE_CURRENT_SOURCE_DIR}/verify_engine_fields.sh ${CMAKE_SOURCE_DIR}
+	WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+	COMMENT "Comparing engine fields checksum in falco_engine.h to actual fields"
+	)
+
+# add_custom_target(verify_engine_fields
+# 	DEPENDS verify_engine_fields.sh falco_engine.h)
+
+# add_dependencies(verify_engine_fields falco)
+
 install(TARGETS falco DESTINATION ${FALCO_BIN_DIR})
 install(DIRECTORY lua
 	DESTINATION ${FALCO_SHARE_DIR}

userspace/falco/configuration.cpp

@@ -30,10 +30,11 @@ limitations under the License.
 using namespace std;
 
 falco_configuration::falco_configuration()
-	: m_buffered_outputs(true),
+	: m_buffered_outputs(false),
 	  m_webserver_enabled(false),
 	  m_webserver_listen_port(8765),
 	  m_webserver_k8s_audit_endpoint("/k8s_audit"),
+	  m_webserver_ssl_enabled(false),
 	  m_config(NULL)
 {
 }
@@ -155,14 +156,16 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 	}
 	m_min_priority = (falco_common::priority_type) (it - falco_common::priority_names.begin());
 
-	m_buffered_outputs = m_config->get_scalar<bool>("buffered_outputs", true);
+	m_buffered_outputs = m_config->get_scalar<bool>("buffered_outputs", false);
 
 	falco_logger::log_stderr = m_config->get_scalar<bool>("log_stderr", false);
 	falco_logger::log_syslog = m_config->get_scalar<bool>("log_syslog", true);
 
 	m_webserver_enabled = m_config->get_scalar<bool>("webserver", "enabled", false);
 	m_webserver_listen_port = m_config->get_scalar<uint32_t>("webserver", "listen_port", 8765);
-	m_webserver_k8s_audit_endpoint = m_config->get_scalar<string>("websever", "k8s_audit_endpoint", "/k8s_audit");
+	m_webserver_k8s_audit_endpoint = m_config->get_scalar<string>("webserver", "k8s_audit_endpoint", "/k8s_audit");
+	m_webserver_ssl_enabled = m_config->get_scalar<bool>("webserver", "ssl_enabled", false);
+	m_webserver_ssl_certificate = m_config->get_scalar<string>("webserver", "ssl_certificate","/etc/falco/falco.pem");
 }
 
 void falco_configuration::read_rules_file_directory(const string &path, list<string> &rules_filenames)

userspace/falco/configuration.h

@@ -182,6 +182,8 @@ class falco_configuration
 	bool m_webserver_enabled;
 	uint32_t m_webserver_listen_port;
 	std::string m_webserver_k8s_audit_endpoint;
+	bool m_webserver_ssl_enabled;
+	std::string m_webserver_ssl_certificate;
 
  private:
 	void init_cmdline_options(std::list<std::string> &cmdline_options);

userspace/falco/falco.cpp

@@ -28,6 +28,7 @@ limitations under the License.
 #include <string>
 #include <signal.h>
 #include <fcntl.h>
+#include <sys/utsname.h>
 #include <sys/stat.h>
 #include <unistd.h>
 #include <getopt.h>
@@ -114,6 +115,7 @@ static void usage()
 	   "                               The API servers can also be specified via the environment variable\n"
 	   "                               FALCO_MESOS_API.\n"
 	   " -M <num_seconds>              Stop collecting after <num_seconds> reached.\n"
+	   " -N                            When used with --list, only print field names.\n"
 	   " -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.\n"
 	   "                               <key> can be a two-part <key>.<subkey>\n"
 	   " -p <output_format>, --print=<output_format>\n"
@@ -134,6 +136,8 @@ static void usage()
 	   "                  		   Capture the first <len> bytes of each I/O buffer.\n"
 	   "                   		   By default, the first 80 bytes are captured. Use this\n"
 	   "                   		   option with caution, it can generate huge trace files.\n"
+	   " --support                     Print support information including version, rules files used, etc.\n"
+	   "                               and exit.\n"
 	   " -T <tag>                      Disable any rules with a tag=<tag>. Can be specified multiple times.\n"
 	   "                               Can not be specified with -t.\n"
 	   " -t <tag>                      Only run those rules with a tag=<tag>. Can be specified multiple times.\n"
@@ -196,6 +200,15 @@ void read_k8s_audit_trace_file(falco_engine *engine,
 	}
 }
 
+static std::string read_file(std::string filename)
+{
+	std::ifstream t(filename);
+	std::string str((std::istreambuf_iterator<char>(t)),
+			std::istreambuf_iterator<char>());
+
+	return str;
+}
+
 //
 // Event processing loop
 //
@@ -335,59 +348,7 @@ static void print_all_ignored_events(sinsp *inspector)
 	printf("\n");
 }
 
-// Must match the value in the zsh tab completion
-#define DESCRIPTION_TEXT_START 16
-
-#define CONSOLE_LINE_LEN 79
-
-static void list_falco_fields(falco_engine *engine)
-{
-	for(auto &chk_field : engine->json_factory().get_fields())
-	{
-		printf("\n----------------------\n");
-		printf("Field Class: %s (%s)\n\n", chk_field.name.c_str(), chk_field.desc.c_str());
-
-		for(auto &field : chk_field.fields)
-		{
-			uint32_t l, m;
-
-			printf("%s", field.name.c_str());
-			uint32_t namelen = field.name.size();
-
-			if(namelen >= DESCRIPTION_TEXT_START)
-			{
-				printf("\n");
-				namelen = 0;
-			}
-
-			for(l = 0; l < DESCRIPTION_TEXT_START - namelen; l++)
-			{
-				printf(" ");
-			}
-
-			size_t desclen = field.desc.size();
-
-			for(l = 0; l < desclen; l++)
-			{
-				if(l % (CONSOLE_LINE_LEN - DESCRIPTION_TEXT_START) == 0 && l != 0)
-				{
-					printf("\n");
-
-					for(m = 0; m < DESCRIPTION_TEXT_START; m++)
-					{
-						printf(" ");
-					}
-				}
-
-				printf("%c", field.desc.at(l));
-			}
-
-			printf("\n");
-		}
-	}
-}
-
-static void list_source_fields(falco_engine *engine, bool verbose, std::string &source)
+static void list_source_fields(falco_engine *engine, bool verbose, bool names_only, std::string &source)
 {
 	if(source.size() > 0 &&
 	   !(source == "syscall" || source == "k8s_audit"))
@@ -396,11 +357,11 @@ static void list_source_fields(falco_engine *engine, bool verbose, std::string &
 	}
 	if(source == "" || source == "syscall")
 	{
-		list_fields(verbose, false);
+		list_fields(verbose, false, names_only);
 	}
 	if(source == "" || source == "k8s_audit")
 	{
-		list_falco_fields(engine);
+		engine->list_fields(names_only);
 	}
 }
 
@@ -428,6 +389,7 @@ int falco_init(int argc, char **argv)
 	list<string> validate_rules_filenames;
 	string stats_filename = "";
 	bool verbose = false;
+	bool names_only = false;
 	bool all_events = false;
 	string* k8s_api = 0;
 	string* k8s_api_cert = 0;
@@ -439,6 +401,7 @@ int falco_init(int argc, char **argv)
 	bool print_ignored_events = false;
 	bool list_flds = false;
 	string list_flds_source = "";
+	bool print_support = false;
 
 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -448,6 +411,7 @@ int falco_init(int argc, char **argv)
 	bool compress = false;
 	bool buffered_outputs = true;
 	bool buffered_cmdline = false;
+	std::map<string,uint64_t> required_engine_versions;
 
 	// Used for stats
 	double duration;
@@ -458,6 +422,7 @@ int falco_init(int argc, char **argv)
 	static struct option long_options[] =
 	{
 		{"help", no_argument, 0, 'h' },
+		{"print-base64", no_argument, 0, 'b'},
 		{"daemon", no_argument, 0, 'd' },
 		{"k8s-api", required_argument, 0, 'k'},
 		{"k8s-api-cert", required_argument, 0, 'K' },
@@ -467,6 +432,7 @@ int falco_init(int argc, char **argv)
 		{"print", required_argument, 0, 'p' },
 		{"pidfile", required_argument, 0, 'P' },
 		{"snaplen", required_argument, 0, 'S' },
+		{"support", no_argument, 0},
 		{"unbuffered", no_argument, 0, 'U' },
 		{"version", no_argument, 0, 0 },
 		{"validate", required_argument, 0, 'V' },
@@ -488,7 +454,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:ik:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
+                                        "hc:AbdD:e:F:ik:K:Ll:m:M:No:P:p:r:S:s:T:t:UvV:w:",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -545,6 +511,9 @@ int falco_init(int argc, char **argv)
 					throw sinsp_exception(string("invalid duration") + optarg);
 				}
 				break;
+			case 'N':
+				names_only = true;
+				break;
 			case 'o':
 				cmdline_options.push_back(optarg);
 				break;
@@ -559,7 +528,7 @@ int falco_init(int argc, char **argv)
 				}
 				else if(string(optarg) == "k" || string(optarg) == "kubernetes")
 				{
-					output_format = "k8s.pod=%k8s.pod.name container=%container.id";
+					output_format = "k8s.ns=%k8s.ns.name k8s.pod=%k8s.pod.name container=%container.id";
 					replace_container_info = true;
 				}
 				else if(string(optarg) == "m" || string(optarg) == "mesos")
@@ -619,6 +588,10 @@ int falco_init(int argc, char **argv)
 						list_flds_source = optarg;
 					}
 				}
+				else if (string(long_options[long_index].name) == "support")
+				{
+					print_support = true;
+				}
 				break;
 
 			default:
@@ -651,7 +624,7 @@ int falco_init(int argc, char **argv)
 
 		if(list_flds)
 		{
-			list_source_fields(engine, verbose, list_flds_source);
+			list_source_fields(engine, verbose, names_only, list_flds_source);
 			return EXIT_SUCCESS;
 		}
 
@@ -747,7 +720,10 @@ int falco_init(int argc, char **argv)
 		for (auto filename : config.m_rules_filenames)
 		{
 			falco_logger::log(LOG_INFO, "Loading rules from file " + filename + ":\n");
-			engine->load_rules_file(filename, verbose, all_events);
+			uint64_t required_engine_version;
+
+			engine->load_rules_file(filename, verbose, all_events, required_engine_version);
+			required_engine_versions[filename] = required_engine_version;
 		}
 
 		// You can't both disable and enable rules
@@ -784,6 +760,49 @@ int falco_init(int argc, char **argv)
 			engine->enable_rule_by_tag(enabled_rule_tags, true);
 		}
 
+		if(print_support)
+		{
+			nlohmann::json support;
+			struct utsname sysinfo;
+			std::string cmdline;
+
+			if(uname(&sysinfo) != 0)
+			{
+				throw std::runtime_error(string("Could not uname() to find system info: %s\n") + strerror(errno));
+			}
+
+			for(char **arg = argv; *arg; arg++)
+			{
+				if(cmdline.size() > 0)
+				{
+					cmdline += " ";
+				}
+				cmdline += *arg;
+			}
+
+			support["version"] = FALCO_VERSION;
+			support["system_info"]["sysname"] = sysinfo.sysname;
+			support["system_info"]["nodename"] = sysinfo.nodename;
+			support["system_info"]["release"] = sysinfo.release;
+			support["system_info"]["version"] = sysinfo.version;
+			support["system_info"]["machine"] = sysinfo.machine;
+			support["cmdline"] = cmdline;
+			support["config"] = read_file(conf_filename);
+			support["rules_files"] = nlohmann::json::array();
+			for(auto filename : config.m_rules_filenames)
+			{
+				nlohmann::json finfo;
+				finfo["name"] = filename;
+				nlohmann::json variant;
+				variant["required_engine_version"] = required_engine_versions[filename];
+				variant["content"] = read_file(filename);
+				finfo["variants"].push_back(variant);
+				support["rules_files"].push_back(finfo);
+			}
+			printf("%s\n", support.dump().c_str());
+			goto exit;
+		}
+
 		outputs->init(config.m_json_output,
 			      config.m_json_include_output_property,
 			      config.m_notifications_rate, config.m_notifications_max_burst,
@@ -1023,7 +1042,8 @@ int falco_init(int argc, char **argv)
 
 		if(trace_filename.empty() && config.m_webserver_enabled)
 		{
-			falco_logger::log(LOG_INFO, "Starting internal webserver, listening on port " + to_string(config.m_webserver_listen_port) + "\n");
+			std::string ssl_option = (config.m_webserver_ssl_enabled ? " (SSL)" : "");
+			falco_logger::log(LOG_INFO, "Starting internal webserver, listening on port " + to_string(config.m_webserver_listen_port) + ssl_option + "\n");
 			webserver.init(&config, engine, outputs);
 			webserver.start();
 		}

userspace/falco/falco_outputs.cpp

@@ -37,19 +37,26 @@ falco_outputs::falco_outputs(falco_engine *engine)
 
 falco_outputs::~falco_outputs()
 {
+	// Note: The assert()s in this destructor were previously places where
+	//       exceptions were thrown.  C++11 doesn't allow destructors to
+	//       emit exceptions; if they're thrown, they'll trigger a call
+	//       to 'terminate()'.  To maintain similar behavior, the exceptions
+	//       were replace with calls to 'assert()'
 	if(m_initialized)
 	{
 		lua_getglobal(m_ls, m_lua_output_cleanup.c_str());
 
 		if(!lua_isfunction(m_ls, -1))
 		{
-			throw falco_exception("No function " + m_lua_output_cleanup + " found. ");
+			falco_logger::log(LOG_ERR, std::string("No function ") + m_lua_output_cleanup + " found. ");
+			assert(nullptr == "Missing lua cleanup function in ~falco_outputs");
 		}
 
 		if(lua_pcall(m_ls, 0, 0, 0) != 0)
 		{
 			const char* lerr = lua_tostring(m_ls, -1);
-			throw falco_exception(string(lerr));
+			falco_logger::log(LOG_ERR, std::string("lua_pcall failed, err: ") + lerr);
+			assert(nullptr == "lua_pcall failed in ~falco_outputs");
 		}
 	}
 }

userspace/falco/verify_engine_fields.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+
+set -euo pipefail
+
+SOURCE_DIR=$1
+OPENSSL=../../openssl-prefix/src/openssl/target/bin/openssl
+
+if ! command -v ${OPENSSL} version > /dev/null 2>&1; then
+    echo "No openssl command at ${OPENSSL}"
+    exit 1
+fi
+
+NEW_CHECKSUM=$(./falco --list -N | ${OPENSSL} dgst -sha256 | awk '{print $2}')
+CUR_CHECKSUM=$(grep FALCO_FIELDS_CHECKSUM ${SOURCE_DIR}/userspace/engine/falco_engine_version.h | awk '{print $3}' | sed -e 's/"//g')
+
+
+if [ $NEW_CHECKSUM != $CUR_CHECKSUM ]; then
+    echo "Set of fields supported by falco/sysdig libraries has changed (new checksum $NEW_CHECKSUM != old checksum $CUR_CHECKSUM)."
+    echo "Update checksum and/or version in falco_engine_version.h."
+    exit 1
+fi
+
+exit 0

userspace/falco/webserver.cpp

@@ -189,10 +189,20 @@ void falco_webserver::start()
 	}
 
 	std::vector<std::string> cpp_options = {
-		"listening_ports", to_string(m_config->m_webserver_listen_port),
 		"num_threads", to_string(1)
 	};
 
+	if (m_config->m_webserver_ssl_enabled)
+	{
+		cpp_options.push_back("listening_ports");
+		cpp_options.push_back(to_string(m_config->m_webserver_listen_port) + "s");
+		cpp_options.push_back("ssl_certificate");
+		cpp_options.push_back(m_config->m_webserver_ssl_certificate);
+	} else {
+		cpp_options.push_back("listening_ports");
+		cpp_options.push_back(to_string(m_config->m_webserver_listen_port));
+	}
+
 	try {
 		m_server = make_unique<CivetServer>(cpp_options);
 	}