Compare commits

..

17 Commits

Author SHA1 Message Date
Luca Marturana
212fd9353e Push formatter on lua stack only if does not throw exceptions 2016-12-02 16:13:37 +01:00
Mark Stemm
28558959f3 Merge pull request #153 from djcross/dnf
Adding DNF as non-alerting for RPM and package management
2016-12-01 17:51:58 -08:00
Daniel Cross
a8662c60da Adding DNF as non-alerting for RPM and package management
falco-CLA-1.0-signed-off-by: Daniel Cross <daniel.cross@rea-group.com>
2016-12-02 11:52:08 +11:00
Mark Stemm
b3c691e920 Prevent rule_result from leaking on error.
Change falco_engine::process_event to return a unique_ptr that wraps the
rule result, so it won't be leaked if this method throws an exception.

This means that callers don't need to create their own.
2016-12-01 13:53:34 -06:00
Mark Stemm
ded3ee5bed Add unit test for rule with invalid output.
Add the ability to check falco's return code with exit_status and to
generally match stderr with stderr_contains in a test.

Use those to create a test that has an invalid output expression using
%not_a_real_field. It expects falco to exit with 1 and the output to
contain a message about the invalid output.
2016-12-01 13:53:34 -06:00
Mark Stemm
064b39f2be Validate rule outputs when loading rules.
Validate rule outputs when loading rules by attempting to create a
formatter based on the rule's output field. If there's an error, it will
propagate up through load_rules and cause falco to exit rather than
discover the problem only when trying to format the event and the rule's
output field.

This required moving formats.{cpp,h} into the falco engine directory
from the falco general directory. Note that these functions are loaded
twice in the two lua states used by falco (engine and outputs).

There's also a couple of minor cleanups:

 - falco_formats had a private instance variable that was unused, remove
   it.
 - rename the package for the falco_formats functions to formats instead
   of falco so it's more standalone.
 - don't throw a c++ exception in falco_formats::formatter. Instead
   generate a lua error, which is handled more cleanly.
 - free_formatter doesn't return any values, so set the return value of
   the function to 0.
2016-12-01 13:53:34 -06:00
Mark Stemm
2961eb4d21 Move container.info handling to falco engine.
container.info handling used to be handled by the the falco_outputs
object. However, this caused problems for applications that only used
the falco engine, doing their own output formatting for matching events.

Fix this by moving output formatting into the falco engine itself. The
part that replaces %container.info/adds extra formatting to the end of a
rule's output now happens while loading the rule.
2016-12-01 13:53:34 -06:00
Mark Stemm
704eb57e3c Allow run_performance_tests to run test_mm.
Make necessary changes to allow run_performance_tests to invoke the
'test_mm' program we use internally.

Also add ability to run with a build directory separate from the source
directory and to specify an alternate rules file.

Finally, set up the kubernetes demo using sudo, a result of recent changes.
2016-11-28 14:54:14 -08:00
Mark Stemm
9ca8ed96b9 Improve error messages when loading rules.
Related to the changes in https://github.com/draios/agent/pull/267,
improve error messages when trying to load sets of rules with errors:

 - Check that yaml parsing of rules_content actually resulted in
   something.
 - Return an error for rules that have an empty name.
 - Return an error for yaml objects that aren't a rule/macro/list.
 - When compiling, don't print an error message, simply return one,
   including a wrapper "can not compile ..." string.
2016-11-28 12:53:36 -06:00
Mark Stemm
8b18315c1e Fully specify FALCO_SHARE_DIR.
Instead of having FALCO_SHARE_DIR be a relative path, fully specify it
by prepending CMAKE_INSTALL_PREFIX in the top level CMakeLists.txt and
don't prepend CMAKE_INSTALL_PREFIX in config_falco_engine.h.in. This
makes it consistent with its use in the agent.
2016-11-10 16:51:51 -06:00
Mark Stemm
f95a0ead62 Honor USE_BUNDLED_DEPS option for third-party libs
Honor a USE_BUNDLED_DEPS option for third-party libraries which can be
applied globally. There are also USE_BUNDLED_XXX options that can be
used individually for each library.

Verified that this works by first building with USE_BUNDLED_DEPS=ON (the
default), installing external packages ncurses-dev libssl-dev
libcurl4-openssl-dev so CMake's find_package could use them, modifying
the CMakeLists.txt to add "PATHS ${PROJECT_BINARY_DIR}/..." options to
each find_path()/find_library() command to point to the previously
installed third party libraries. It found them as expected.

The sysdig fix in https://github.com/draios/sysdig/pull/672 forced this
change, but it does also happen to fix a falco feature request
https://github.com/draios/falco/issues/144.
2016-11-10 15:43:25 -06:00
Carl Sverre
b1ad9e644e Added envvar SYSDIG_SKIP_LOAD to Dockerfile to skip kernel module manipulation
This helps when running on a system which has the module loaded, but getting
access to the module file is hard for some reason.  Since I know that the right
version of the module is loaded I just want falco to connect.

I tested this with this run command:

docker run -e SYSDIG_SKIP_LOAD=1 -it -v /dev:/host/dev -v /proc:/host/proc --privileged falco

And it successfully connected to Sysdig and started printing out warnings for my
system.

falco-CLA-1.0-signed-off-by: Carl Sverre accounts@carlsverre.com
2016-11-02 12:45:21 -05:00
Mark Stemm
8a2924ad72 Updating for 0.4.0.
CHANGELOG for release notes, README to update version.
2016-10-25 09:53:54 -07:00
Mark Stemm
f98ec60c88 Rule fixes for dragent.
Make sure falco doesn't detect the things draios-agent does as
suspicious. It's possible that you might run open source falco alongside
sysdig cloud.

App checks spawned by sysdig cloud binaries might also change namespace,
so also allow children of sysdigcloud binaries to call setns.
2016-10-24 15:14:09 -07:00
Mark Stemm
0211a94f60 Add stats on events processed/dropped.
Collect stats on the number of events processed and dropped. When run
with -v, print these stats. This duplicates syddig behavior and can be
useful when dianosing problems related to dropped events throwing off
internal state tracking.
2016-10-24 11:02:52 -07:00
Mark Stemm
e0e640c67f Add ability to write trace files.
Bring over functionality from sysdig to write trace files. This is easy
as all of the code to actually write the files is in the inspector. This
just handles the -w option and arguments.

This can be useful to write a trace file in parallel with live event
monitoring so you can reproduce it later.
2016-10-24 10:15:10 -07:00
Mark Stemm
faef5621dd Add k8s binaries as trusted programs
Add a new list k8s_binaries and allow those binaries to do things like
setns/spawn shells. It's not the case that all of these binaries
actually do these things, but keeping it as a single list makes
management easier.
2016-10-21 15:55:10 -07:00
26 changed files with 574 additions and 189 deletions

View File

@@ -2,6 +2,51 @@
This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
## v0.4.0
Released 2016-10-25
As falco depends heavily on sysdig, many changes here were actually made to sysdig and pulled in as a part of the build process. Issues/PRs starting with `sysdig/#XXX` are sysdig changes.
### Major Changes
* Improved visibility into containers:
** New filter `container.privileged` to match containers running in privileged mode [[sysdig/#655](https://github.com/draios/sysdig/pull/655)] [[sysdig/#658](https://github.com/draios/sysdig/pull/658)]
** New rules utilizing privileged state [[#121](https://github.com/draios/falco/pull/121)]
** New filters `container.mount*` to match container mount points [[sysdig/#655](https://github.com/draios/sysdig/pull/655)]
** New rules utilizing container mount points [[#120](https://github.com/draios/falco/pull/120)]
** New filter `container.image.id` to match container image id [[sysdig/#661](https://github.com/draios/sysdig/pull/661)]
* Improved visibility into orchestration environments:
** New k8s.deployment.* and k8s.rs.* filters to support latest kubernetes features [[sysdg/#dbf9b5c](https://github.com/draios/sysdig/commit/dbf9b5c893d49f945c59684b4effe5700d730973)]
** Rule changes to avoid FPs when monitoring k8s environments [[#138](https://github.com/draios/falco/pull/138)]
** Add new options `-pc`/`-pk`/`-pm`/`-k`/`-m` analogous to sysdig command line options. These options pull metadata information from k8s/mesos servers and adjust default falco notification outputs to contain container/orchestration information when applicable. [[#131](https://github.com/draios/falco/pull/131)] [[#134](https://github.com/draios/falco/pull/134)]
* Improved ability to work with file pathnames:
** Added `glob` operator for strings, works as classic shell glob path matcher [[sysdig/#653](https://github.com/draios/sysdig/pull/653)]
** Added `pmatch` operator to efficiently test a subject pathname against a set of target pathnames, to see if the subject is a prefix of any target [[sysdig/#660](https://github.com/draios/sysdig/pull/660)] [[#125](https://github.com/draios/falco/pull/125)]
### Minor Changes
* Add an event generator program that simulates suspicious activity that can be detected by falco. This is also available as a docker image [[sysdig/falco-event-generator](https://hub.docker.com/r/sysdig/falco-event-generator/)]. [[#113](https://github.com/draios/falco/pull/113)] [[#132](https://github.com/draios/falco/pull/132)]
* Changed rule names to be human readable [[#116](https://github.com/draios/falco/pull/116)]
* Add Copyright notice to all source files [[#126](https://github.com/draios/falco/pull/126)]
* Changes to docker images to make it easier to massage JSON output for webhooks [[#133](https://github.com/draios/falco/pull/133)]
* When run with `-v`, print statistics on the number of events processed and dropped [[#139](https://github.com/draios/falco/pull/139)]
* Add ability to write trace files with `-w`. This can be useful to write a trace file in parallel with live event monitoring so you can reproduce it later. [[#140](https://github.com/draios/falco/pull/140)]
* All rules can now take an optional `enabled` flag. With `enabled: false`, a rule will not be loaded or run against events. By default all rules are enabled [[#119](https://github.com/draios/falco/pull/119)]
### Bug Fixes
* Fixed rule FPs related to docker's `docker`/`dockerd` split in 1.12 [[#112](https://github.com/draios/falco/pull/112)]
* Fixed rule FPs related to sysdigcloud agent software [[#141](https://github.com/draios/falco/pull/141)]
* Minor changes to node.js example to avoid falco false positives [[#111](https://github.com/draios/falco/pull/111/)]
* Fixed regression that broke configurable outputs [[#117](https://github.com/draios/falco/pull/117)]. This was not broken in 0.3.0, just between 0.3.0 and 0.4.0.
* Fixed a lua stack leak that could cause problems when matching millions of events against a large set of rules [[#123](https://github.com/draios/falco/pull/123)]
* Update docker files to reflect changes to `debian:unstable` docker image [[#124](https://github.com/draios/falco/pull/124)]
* Fixed logic for detecting config files to ensure config files in `/etc/falco.yaml` are properly detected [[#135](https://github.com/draios/falco/pull/135)] [[#136](https://github.com/draios/falco/pull/136)]
* Don't alert on falco spawning a shell for program output notifications [[#137](https://github.com/draios/falco/pull/137)]
## v0.3.0
Released 2016-08-05

View File

@@ -47,99 +47,193 @@ set(SYSDIG_DIR "${PROJECT_SOURCE_DIR}/../sysdig")
include(ExternalProject)
set(ZLIB_SRC "${PROJECT_BINARY_DIR}/zlib-prefix/src/zlib")
message(STATUS "Using bundled zlib in '${ZLIB_SRC}'")
set(ZLIB_INCLUDE "${ZLIB_SRC}")
set(ZLIB_LIB "${ZLIB_SRC}/libz.a")
ExternalProject_Add(zlib
option(USE_BUNDLED_DEPS "Enable bundled dependencies instead of using the system ones" ON)
#
# zlib
option(USE_BUNDLED_ZLIB "Enable building of the bundled zlib" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_ZLIB)
find_path(ZLIB_INCLUDE zlib.h PATH_SUFFIXES zlib)
find_library(ZLIB_LIB NAMES z)
if(ZLIB_INCLUDE AND ZLIB_LIB)
message(STATUS "Found zlib: include: ${ZLIB_INCLUDE}, lib: ${ZLIB_LIB}")
else()
message(FATAL_ERROR "Couldn't find system zlib")
endif()
else()
set(ZLIB_SRC "${PROJECT_BINARY_DIR}/zlib-prefix/src/zlib")
message(STATUS "Using bundled zlib in '${ZLIB_SRC}'")
set(ZLIB_INCLUDE "${ZLIB_SRC}")
set(ZLIB_LIB "${ZLIB_SRC}/libz.a")
ExternalProject_Add(zlib
URL "http://download.draios.com/dependencies/zlib-1.2.8.tar.gz"
URL_MD5 "44d667c142d7cda120332623eab69f40"
CONFIGURE_COMMAND "./configure"
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
set(JQ_SRC "${PROJECT_BINARY_DIR}/jq-prefix/src/jq")
message(STATUS "Using bundled jq in '${JQ_SRC}'")
set(JQ_INCLUDE "${JQ_SRC}")
set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
ExternalProject_Add(jq
#
# jq
#
option(USE_BUNDLED_JQ "Enable building of the bundled jq" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_JQ)
find_path(JQ_INCLUDE jq.h PATH_SUFFIXES jq)
find_library(JQ_LIB NAMES jq)
if(JQ_INCLUDE AND JQ_LIB)
message(STATUS "Found jq: include: ${JQ_INCLUDE}, lib: ${JQ_LIB}")
else()
message(FATAL_ERROR "Couldn't find system jq")
endif()
else()
set(JQ_SRC "${PROJECT_BINARY_DIR}/jq-prefix/src/jq")
message(STATUS "Using bundled jq in '${JQ_SRC}'")
set(JQ_INCLUDE "${JQ_SRC}")
set(JQ_LIB "${JQ_SRC}/.libs/libjq.a")
ExternalProject_Add(jq
URL "http://download.draios.com/dependencies/jq-1.5.tar.gz"
URL_MD5 "0933532b086bd8b6a41c1b162b1731f9"
CONFIGURE_COMMAND ./configure --disable-maintainer-mode --enable-all-static --disable-dependency-tracking
BUILD_COMMAND ${CMD_MAKE} LDFLAGS=-all-static
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
set(JSONCPP_SRC "${SYSDIG_DIR}/userspace/libsinsp/third-party/jsoncpp")
set(JSONCPP_INCLUDE "${JSONCPP_SRC}")
set(JSONCPP_LIB_SRC "${JSONCPP_SRC}/jsoncpp.cpp")
#
# curses
#
# we pull this in because libsinsp won't build without it
set(CURSES_BUNDLE_DIR "${PROJECT_BINARY_DIR}/ncurses-prefix/src/ncurses")
set(CURSES_INCLUDE_DIR "${CURSES_BUNDLE_DIR}/include/")
set(CURSES_LIBRARIES "${CURSES_BUNDLE_DIR}/lib/libncurses.a")
message(STATUS "Using bundled ncurses in '${CURSES_BUNDLE_DIR}'")
ExternalProject_Add(ncurses
option(USE_BUNDLED_NCURSES "Enable building of the bundled ncurses" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_NCURSES)
set(CURSES_NEED_NCURSES TRUE)
find_package(Curses REQUIRED)
message(STATUS "Found ncurses: include: ${CURSES_INCLUDE_DIR}, lib: ${CURSES_LIBRARIES}")
else()
set(CURSES_BUNDLE_DIR "${PROJECT_BINARY_DIR}/ncurses-prefix/src/ncurses")
set(CURSES_INCLUDE_DIR "${CURSES_BUNDLE_DIR}/include/")
set(CURSES_LIBRARIES "${CURSES_BUNDLE_DIR}/lib/libncurses.a")
message(STATUS "Using bundled ncurses in '${CURSES_BUNDLE_DIR}'")
ExternalProject_Add(ncurses
URL "http://download.draios.com/dependencies/ncurses-6.0-20150725.tgz"
URL_MD5 "32b8913312e738d707ae68da439ca1f4"
CONFIGURE_COMMAND ./configure --without-cxx --without-cxx-binding --without-ada --without-manpages --without-progs --without-tests --with-terminfo-dirs=/etc/terminfo:/lib/terminfo:/usr/share/terminfo
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
#
# libb64
#
option(USE_BUNDLED_B64 "Enable building of the bundled b64" ${USE_BUNDLED_DEPS})
set(B64_SRC "${PROJECT_BINARY_DIR}/b64-prefix/src/b64")
message(STATUS "Using bundled b64 in '${B64_SRC}'")
set(B64_INCLUDE "${B64_SRC}/include")
set(B64_LIB "${B64_SRC}/src/libb64.a")
ExternalProject_Add(b64
if(NOT USE_BUNDLED_B64)
find_path(B64_INCLUDE NAMES b64/encode.h)
find_library(B64_LIB NAMES b64)
if(B64_INCLUDE AND B64_LIB)
message(STATUS "Found b64: include: ${B64_INCLUDE}, lib: ${B64_LIB}")
else()
message(FATAL_ERROR "Couldn't find system b64")
endif()
else()
set(B64_SRC "${PROJECT_BINARY_DIR}/b64-prefix/src/b64")
message(STATUS "Using bundled b64 in '${B64_SRC}'")
set(B64_INCLUDE "${B64_SRC}/include")
set(B64_LIB "${B64_SRC}/src/libb64.a")
ExternalProject_Add(b64
URL "http://download.draios.com/dependencies/libb64-1.2.src.zip"
URL_MD5 "a609809408327117e2c643bed91b76c5"
CONFIGURE_COMMAND ""
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
#
# yamlcpp
#
option(USE_BUNDLED_YAMLCPP "Enable building of the bundled yamlcpp" ${USE_BUNDLED_DEPS})
set(YAMLCPP_SRC "${PROJECT_BINARY_DIR}/yamlcpp-prefix/src/yamlcpp")
message(STATUS "Using bundled yaml-cpp in '${YAMLCPP_SRC}'")
set(YAMLCPP_LIB "${YAMLCPP_SRC}/libyaml-cpp.a")
set(YAMLCPP_INCLUDE_DIR "${YAMLCPP_SRC}/include")
# Once the next version of yaml-cpp is released (first version not requiring
# boost), we can switch to that and no longer pull from github.
ExternalProject_Add(yamlcpp
if(NOT USE_BUNDLED_YAMLCPP)
find_path(YAMLCPP_INCLUDE_DIR NAMES yaml-cpp/yaml.h)
find_library(YAMLCPP_LIB NAMES yaml-cpp)
if(YAMLCPP_INCLUDE_DIR AND YAMLCPP_LIB)
message(STATUS "Found yamlcpp: include: ${YAMLCPP_INCLUDE_DIR}, lib: ${YAMLCPP_LIB}")
else()
message(FATAL_ERROR "Couldn't find system yamlcpp")
endif()
else()
set(YAMLCPP_SRC "${PROJECT_BINARY_DIR}/yamlcpp-prefix/src/yamlcpp")
message(STATUS "Using bundled yaml-cpp in '${YAMLCPP_SRC}'")
set(YAMLCPP_LIB "${YAMLCPP_SRC}/libyaml-cpp.a")
set(YAMLCPP_INCLUDE_DIR "${YAMLCPP_SRC}/include")
# Once the next version of yaml-cpp is released (first version not requiring
# boost), we can switch to that and no longer pull from github.
ExternalProject_Add(yamlcpp
GIT_REPOSITORY "https://github.com/jbeder/yaml-cpp.git"
GIT_TAG "7d2873ce9f2202ea21b6a8c5ecbc9fe38032c229"
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
set(OPENSSL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl")
set(OPENSSL_INSTALL_DIR "${OPENSSL_BUNDLE_DIR}/target")
set(OPENSSL_INCLUDE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl/include")
set(OPENSSL_LIBRARY_SSL "${OPENSSL_INSTALL_DIR}/lib/libssl.a")
set(OPENSSL_LIBRARY_CRYPTO "${OPENSSL_INSTALL_DIR}/lib/libcrypto.a")
#
# OpenSSL
#
option(USE_BUNDLED_OPENSSL "Enable building of the bundled OpenSSL" ${USE_BUNDLED_DEPS})
message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")
if(NOT USE_BUNDLED_OPENSSL)
find_package(OpenSSL REQUIRED)
message(STATUS "Found OpenSSL: include: ${OPENSSL_INCLUDE_DIR}, lib: ${OPENSSL_LIBRARIES}")
else()
ExternalProject_Add(openssl
set(OPENSSL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl")
set(OPENSSL_INSTALL_DIR "${OPENSSL_BUNDLE_DIR}/target")
set(OPENSSL_INCLUDE_DIR "${PROJECT_BINARY_DIR}/openssl-prefix/src/openssl/include")
set(OPENSSL_LIBRARY_SSL "${OPENSSL_INSTALL_DIR}/lib/libssl.a")
set(OPENSSL_LIBRARY_CRYPTO "${OPENSSL_INSTALL_DIR}/lib/libcrypto.a")
message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")
ExternalProject_Add(openssl
URL "http://download.draios.com/dependencies/openssl-1.0.2d.tar.gz"
URL_MD5 "38dd619b2e77cbac69b99f52a053d25a"
CONFIGURE_COMMAND ./config shared --prefix=${OPENSSL_INSTALL_DIR}
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND ${CMD_MAKE} install)
endif()
set(CURL_SSL_OPTION "--with-ssl=${OPENSSL_INSTALL_DIR}")
#
# libcurl
#
option(USE_BUNDLED_CURL "Enable building of the bundled curl" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_CURL)
find_package(CURL REQUIRED)
message(STATUS "Found CURL: include: ${CURL_INCLUDE_DIR}, lib: ${CURL_LIBRARIES}")
else()
set(CURL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/curl-prefix/src/curl")
set(CURL_INCLUDE_DIR "${CURL_BUNDLE_DIR}/include/")
set(CURL_LIBRARIES "${CURL_BUNDLE_DIR}/lib/.libs/libcurl.a")
set(CURL_BUNDLE_DIR "${PROJECT_BINARY_DIR}/curl-prefix/src/curl")
set(CURL_INCLUDE_DIR "${CURL_BUNDLE_DIR}/include/")
set(CURL_LIBRARIES "${CURL_BUNDLE_DIR}/lib/.libs/libcurl.a")
message(STATUS "Using bundled curl in '${CURL_BUNDLE_DIR}'")
message(STATUS "Using SSL for curl in '${CURL_SSL_OPTION}'")
if(NOT USE_BUNDLED_OPENSSL)
set(CURL_SSL_OPTION "--with-ssl")
else()
set(CURL_SSL_OPTION "--with-ssl=${OPENSSL_INSTALL_DIR}")
message(STATUS "Using bundled curl in '${CURL_BUNDLE_DIR}'")
message(STATUS "Using SSL for curl in '${CURL_SSL_OPTION}'")
endif()
ExternalProject_Add(curl
ExternalProject_Add(curl
DEPENDS openssl
URL "http://download.draios.com/dependencies/curl-7.45.0.tar.bz2"
URL_MD5 "62c1a352b28558f25ba6209214beadc8"
@@ -147,50 +241,120 @@ ExternalProject_Add(curl
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
set(LUAJIT_SRC "${PROJECT_BINARY_DIR}/luajit-prefix/src/luajit/src")
message(STATUS "Using bundled LuaJIT in '${LUAJIT_SRC}'")
set(LUAJIT_INCLUDE "${LUAJIT_SRC}")
set(LUAJIT_LIB "${LUAJIT_SRC}/libluajit.a")
ExternalProject_Add(luajit
#
# LuaJIT
#
option(USE_BUNDLED_LUAJIT "Enable building of the bundled LuaJIT" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_LUAJIT)
find_path(LUAJIT_INCLUDE luajit.h PATH_SUFFIXES luajit-2.0 luajit)
find_library(LUAJIT_LIB NAMES luajit luajit-5.1)
if(LUAJIT_INCLUDE AND LUAJIT_LIB)
message(STATUS "Found LuaJIT: include: ${LUAJIT_INCLUDE}, lib: ${LUAJIT_LIB}")
else()
# alternatively try stock Lua
find_package(Lua51)
set(LUAJIT_LIB ${LUA_LIBRARY})
set(LUAJIT_INCLUDE ${LUA_INCLUDE_DIR})
if(NOT ${LUA51_FOUND})
message(FATAL_ERROR "Couldn't find system LuaJIT or Lua")
endif()
endif()
else()
set(LUAJIT_SRC "${PROJECT_BINARY_DIR}/luajit-prefix/src/luajit/src")
message(STATUS "Using bundled LuaJIT in '${LUAJIT_SRC}'")
set(LUAJIT_INCLUDE "${LUAJIT_SRC}")
set(LUAJIT_LIB "${LUAJIT_SRC}/libluajit.a")
ExternalProject_Add(luajit
URL "http://download.draios.com/dependencies/LuaJIT-2.0.3.tar.gz"
URL_MD5 "f14e9104be513913810cd59c8c658dc0"
CONFIGURE_COMMAND ""
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
INSTALL_COMMAND "")
endif()
set (LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
set (LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
ExternalProject_Add(lpeg
DEPENDS luajit
URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ""
INSTALL_COMMAND "")
#
# Lpeg
#
option(USE_BUNDLED_LPEG "Enable building of the bundled lpeg" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_LPEG)
find_library(LPEG_LIB NAMES lpeg.a)
if(LPEG_LIB)
message(STATUS "Found lpeg: lib: ${LPEG_LIB}")
else()
message(FATAL_ERROR "Couldn't find system lpeg")
endif()
else()
set(LPEG_SRC "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg")
set(LPEG_LIB "${PROJECT_BINARY_DIR}/lpeg-prefix/src/lpeg/build/lpeg.a")
ExternalProject_Add(lpeg
DEPENDS luajit
URL "http://s3.amazonaws.com/download.draios.com/dependencies/lpeg-1.0.0.tar.gz"
URL_MD5 "0aec64ccd13996202ad0c099e2877ece"
BUILD_COMMAND LUA_INCLUDE=${LUAJIT_INCLUDE} "${PROJECT_SOURCE_DIR}/scripts/build-lpeg.sh" "${LPEG_SRC}/build"
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ""
INSTALL_COMMAND "")
endif()
set (LIBYAML_SRC "${PROJECT_BINARY_DIR}/libyaml-prefix/src/libyaml/src")
set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
ExternalProject_Add(libyaml
URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ./bootstrap && ./configure
INSTALL_COMMAND "")
#
# Libyaml
#
option(USE_BUNDLED_LIBYAML "Enable building of the bundled libyaml" ${USE_BUNDLED_DEPS})
set (LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
ExternalProject_Add(lyaml
URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ./configure --enable-static LIBS=-L../../../libyaml-prefix/src/libyaml/src/.libs CFLAGS=-I../../../libyaml-prefix/src/libyaml/include CPPFLAGS=-I../../../libyaml-prefix/src/libyaml/include LUA_INCLUDE=-I../../../luajit-prefix/src/luajit/src LUA=../../../luajit-prefix/src/luajit/src/luajit
INSTALL_COMMAND sh -c "cp -R ${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/lib/* ${PROJECT_SOURCE_DIR}/userspace/engine/lua")
if(NOT USE_BUNDLED_LIBYAML)
# Note: to distinguish libyaml.a and yaml.a we specify a full
# file name here, so you'll have to arrange for static
# libraries being available.
find_library(LIBYAML_LIB NAMES libyaml.a)
if(LIBYAML_LIB)
message(STATUS "Found libyaml: lib: ${LIBYAML_LIB}")
else()
message(FATAL_ERROR "Couldn't find system libyaml")
endif()
else()
set(LIBYAML_SRC "${PROJECT_BINARY_DIR}/libyaml-prefix/src/libyaml/src")
set(LIBYAML_LIB "${LIBYAML_SRC}/.libs/libyaml.a")
ExternalProject_Add(libyaml
URL "http://download.draios.com/dependencies/libyaml-0.1.4.tar.gz"
URL_MD5 "4a4bced818da0b9ae7fc8ebc690792a7"
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ./bootstrap && ./configure
INSTALL_COMMAND "")
endif()
#
# lyaml
#
option(USE_BUNDLED_LYAML "Enable building of the bundled lyaml" ${USE_BUNDLED_DEPS})
if(NOT USE_BUNDLED_LYAML)
# Note: to distinguish libyaml.a and yaml.a we specify a full
# file name here, so you'll have to arrange for static
# libraries being available.
find_library(LYAML_LIB NAMES yaml.a)
if(LYAML_LIB)
message(STATUS "Found lyaml: lib: ${LYAML_LIB}")
else()
message(FATAL_ERROR "Couldn't find system lyaml")
endif()
else()
set(LYAML_SRC "${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/ext/yaml")
set(LYAML_LIB "${LYAML_SRC}/.libs/yaml.a")
ExternalProject_Add(lyaml
URL "http://download.draios.com/dependencies/lyaml-release-v6.0.tar.gz"
URL_MD5 "dc3494689a0dce7cf44e7a99c72b1f30"
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ./configure --enable-static LIBS=-L../../../libyaml-prefix/src/libyaml/src/.libs CFLAGS=-I../../../libyaml-prefix/src/libyaml/include CPPFLAGS=-I../../../libyaml-prefix/src/libyaml/include LUA_INCLUDE=-I../../../luajit-prefix/src/luajit/src LUA=../../../luajit-prefix/src/luajit/src/luajit
INSTALL_COMMAND sh -c "cp -R ${PROJECT_BINARY_DIR}/lyaml-prefix/src/lyaml/lib/* ${PROJECT_SOURCE_DIR}/userspace/engine/lua")
endif()
install(FILES falco.yaml
DESTINATION "${FALCO_ETC_DIR}")
@@ -201,7 +365,7 @@ add_subdirectory("${SYSDIG_DIR}/userspace/libsinsp" "${PROJECT_BINARY_DIR}/users
add_subdirectory(scripts)
set(FALCO_SINSP_LIBRARY sinsp)
set(FALCO_SHARE_DIR share/falco)
set(FALCO_SHARE_DIR ${CMAKE_INSTALL_PREFIX}/share/falco)
add_subdirectory(userspace/engine)
add_subdirectory(userspace/falco)

View File

@@ -2,7 +2,7 @@
####Latest release
**v0.3.0**
**v0.4.0**
Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />
@@ -16,6 +16,7 @@ Sysdig Falco is a behavioral activity monitor designed to detect anomalous activ
Falco can detect and alert on any behavior that involves making Linux system calls. Thanks to Sysdig's core decoding and state tracking functionality, falco alerts can be triggered by the use of specific system calls, their arguments, and by properties of the calling process. For example, you can easily detect things like:
- A shell is run inside a container
- A container is running in privileged mode, or is mounting a sensitive path like `/proc` from the host.
- A server process spawns a child process of an unexpected type
- Unexpected read of a sensitive file (like `/etc/shadow`)
- A non-device file is written to `/dev`

View File

@@ -1,13 +1,17 @@
#!/bin/bash
#set -e
echo "* Setting up /usr/src links from host"
# Set the SYSDIG_SKIP_LOAD variable to skip loading the sysdig kernel module
for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
do
ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
done
if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
echo "* Setting up /usr/src links from host"
/usr/bin/sysdig-probe-loader
for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
do
ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
done
/usr/bin/sysdig-probe-loader
fi
exec "$@"

View File

@@ -1,13 +1,17 @@
#!/bin/bash
#set -e
echo "* Setting up /usr/src links from host"
# Set the SYSDIG_SKIP_LOAD variable to skip loading the sysdig kernel module
for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
do
ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
done
if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
echo "* Setting up /usr/src links from host"
/usr/bin/sysdig-probe-loader
for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
do
ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
done
/usr/bin/sysdig-probe-loader
fi
exec "$@"

View File

@@ -96,11 +96,14 @@
]
- list: sysdigcloud_binaries
items: [setup-backend, dragent]
items: [setup-backend, dragent, sdchecks]
- list: docker_binaries
items: [docker, dockerd, exe]
- list: k8s_binaries
items: [hyperkube, skydns, kube2sky]
- list: http_server_binaries
items: [nginx, httpd, httpd-foregroun, lighttpd]
@@ -113,7 +116,7 @@
# The truncated dpkg-preconfigu is intentional, process names are
# truncated at the sysdig level.
- list: package_mgmt_binaries
items: [dpkg, dpkg-preconfigu, rpm, rpmkey, yum, frontend]
items: [dpkg, dpkg-preconfigu, dnf, rpm, rpmkey, yum, frontend]
- macro: package_mgmt_procs
condition: proc.name in (package_mgmt_binaries)
@@ -233,7 +236,7 @@
# Only let rpm-related programs write to the rpm database
- rule: Write below rpm database
desc: an attempt to write to the rpm database by any non-rpm related program
condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (rpm,rpmkey,yum)
condition: fd.name startswith /var/lib/rpm and open_write and not proc.name in (dnf,rpm,rpmkey,yum)
output: "Rpm database opened for writing by a non-rpm program (command=%proc.cmdline file=%fd.name)"
priority: WARNING
@@ -273,13 +276,13 @@
- rule: Change thread namespace
desc: an attempt to change a program/thread\'s namespace (commonly done as a part of creating a container) by calling setns.
condition: evt.type = setns and not proc.name in (docker_binaries, sysdig, dragent, nsenter)
output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline %container.info)"
condition: evt.type = setns and not proc.name in (docker_binaries, k8s_binaries, sysdigcloud_binaries, sysdig, nsenter) and not proc.pname in (sysdigcloud_binaries)
output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline parent=%proc.pname %container.info)"
priority: WARNING
- rule: Run shell untrusted
desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries.
condition: spawned_process and not container and shell_procs and proc.pname exists and not proc.pname in (cron_binaries, shell_binaries, sshd, sudo, docker_binaries, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent, aws, initdb, docker-compose, make, configure, awk, falco)
condition: spawned_process and not container and shell_procs and proc.pname exists and not proc.pname in (cron_binaries, shell_binaries, sshd, sudo, docker_binaries, k8s_binaries, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent, aws, initdb, docker-compose, make, configure, awk, falco)
output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
priority: WARNING
@@ -314,7 +317,7 @@
- rule: Run shell in container
desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, initdb, pg_ctl, awk, apache2, falco)
condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, initdb, pg_ctl, awk, apache2, falco, cron)
output: "Shell spawned in a container other than entrypoint (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
priority: WARNING

View File

@@ -6,4 +6,4 @@ VARIANT=$3
RESULTS_FILE=$4
CPU_INTERVAL=$5
top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig|dragent)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"time\": \"%s\", \"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", strftime("%Y-%m-%d %H:%M:%S", systime(), 1), NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE
top -d $CPU_INTERVAL -b -p $SUBJ_PID | grep -E '(falco|sysdig|dragent|test_mm)' --line-buffered | awk -v benchmark=$BENCHMARK -v variant=$VARIANT '{printf("{\"time\": \"%s\", \"sample\": %d, \"benchmark\": \"%s\", \"variant\": \"%s\", \"cpu_usage\": %s},\n", strftime("%Y-%m-%d %H:%M:%S", systime(), 1), NR, benchmark, variant, $9); fflush();}' >> $RESULTS_FILE

View File

@@ -17,6 +17,8 @@ class FalcoTest(Test):
"""
self.falcodir = self.params.get('falcodir', '/', default=os.path.join(self.basedir, '../build'))
self.stderr_contains = self.params.get('stderr_contains', '*', default='')
self.exit_status = self.params.get('exit_status', '*', default=0)
self.should_detect = self.params.get('detect', '*', default=False)
self.trace_file = self.params.get('trace_file', '*')
@@ -197,9 +199,18 @@ class FalcoTest(Test):
res = self.falco_proc.run(timeout=180, sig=9)
if self.stderr_contains != '':
match = re.search(self.stderr_contains, res.stderr)
if match is None:
self.fail("Stderr of falco process did not contain content matching {}".format(self.stderr_contains))
if res.exit_status != self.exit_status:
self.error("Falco command \"{}\" exited with unexpected return value {} (!= {})".format(
cmd, res.exit_status, self.exit_status))
# No need to check any outputs if the falco process exited abnormally.
if res.exit_status != 0:
self.error("Falco command \"{}\" exited with non-zero return value {}".format(
cmd, res.exit_status))
return
self.check_rules_warnings(res)
if len(self.rules_events) > 0:

View File

@@ -95,6 +95,13 @@ trace_files: !mux
- rules/double_rule.yaml
trace_file: trace_files/cat_write.scap
invalid_rule_output:
exit_status: 1
stderr_contains: "Runtime error: Error loading rules:.* Invalid output format 'An open was seen %not_a_real_field': 'invalid formatting token not_a_real_field'. Exiting."
rules_file:
- rules/invalid_rule_output.yaml
trace_file: trace_files/cat_write.scap
disabled_rules:
detect: False
rules_file:

View File

@@ -0,0 +1,5 @@
- rule: rule_with_invalid_output
desc: A rule with an invalid output field
condition: evt.type=open
output: "An open was seen %not_a_real_field"
priority: WARNING

View File

@@ -28,7 +28,11 @@ function time_cmd() {
function run_falco_on() {
file="$1"
cmd="$ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false -e $file"
if [ -z $RULES_FILE ]; then
RULES_FILE=$SOURCE/rules/falco_rules.yaml
fi
cmd="$ROOT/userspace/falco/falco -c $SOURCE/falco.yaml -r $SOURCE/rules/falco_rules.yaml --option=stdout_output.enabled=false -e $file -A"
time_cmd "$cmd" "$file"
}
@@ -131,15 +135,26 @@ function start_monitor_cpu_usage() {
function start_subject_prog() {
echo " starting falco/sysdig/agent program"
# Do a blocking sudo command now just to ensure we have a password
sudo bash -c ""
if [[ $ROOT == *"falco"* ]]; then
sudo $ROOT/userspace/falco/falco -c $ROOT/../falco.yaml -r $ROOT/../rules/falco_rules.yaml --option=stdout_output.enabled=false > ./prog-output.txt 2>&1 &
if [[ $ROOT == *"multimatch"* ]]; then
echo " starting test_mm..."
if [ -z $RULES_FILE ]; then
RULES_FILE=$SOURCE/../output/rules.yaml
fi
sudo $ROOT/test_mm -s $SOURCE/search_order.yaml -r $RULES_FILE > ./prog-output.txt 2>&1 &
elif [[ $ROOT == *"falco"* ]]; then
echo " starting falco..."
if [ -z $RULES_FILE ]; then
RULES_FILE=$SOURCE/rules/falco_rules.yaml
fi
sudo $ROOT/userspace/falco/falco -c $SOURCE/falco.yaml -r $RULES_FILE --option=stdout_output.enabled=false > ./prog-output.txt -A 2>&1 &
elif [[ $ROOT == *"sysdig"* ]]; then
echo " starting sysdig..."
sudo $ROOT/userspace/sysdig/sysdig -N -z evt.type=none &
else
echo " starting agent..."
write_agent_config
pushd $ROOT/userspace/dragent
sudo ./dragent > ./prog-output.txt 2>&1 &
@@ -180,8 +195,8 @@ function run_juttle_examples() {
function run_kubernetes_demo() {
pushd $SCRIPTDIR/../../infrastructure/test-infrastructures/kubernetes-demo
bash run-local.sh
bash init.sh
sudo bash run-local.sh
sudo bash init.sh
sleep 600
docker stop $(docker ps -qa)
docker rm -fv $(docker ps -qa)
@@ -306,8 +321,10 @@ usage() {
echo " -h/--help: show this help"
echo " -v/--variant: a variant name to attach to this set of test results"
echo " -r/--root: root directory containing falco/sysdig binaries (i.e. where you ran 'cmake')"
echo " -s/--source: root directory containing falco/sysdig source code"
echo " -R/--results: append test results to this file"
echo " -o/--output: append program output to this file"
echo " -U/--rules: path to rules file (only applicable for falco/test_mm)"
echo " -t/--test: test to run. Argument has the following format:"
echo " trace:<trace>: read the specified trace file."
echo " trace:all means run all traces"
@@ -325,7 +342,7 @@ usage() {
echo " -F/--falco-agent: When running an agent, whether or not to enable falco"
}
OPTS=`getopt -o hv:r:R:o:t:T: --long help,variant:,root:,results:,output:,test:,tracedir:,agent-autodrop:,falco-agent: -n $0 -- "$@"`
OPTS=`getopt -o hv:r:s:R:o:U:t:T: --long help,variant:,root:,source:,results:,output:,rules:,test:,tracedir:,agent-autodrop:,falco-agent: -n $0 -- "$@"`
if [ $? != 0 ]; then
echo "Exiting" >&2
@@ -336,9 +353,11 @@ eval set -- "$OPTS"
VARIANT="falco"
ROOT=`dirname $0`/../build
SOURCE=$ROOT
SCRIPTDIR=`dirname $0`
RESULTS_FILE=`dirname $0`/results.json
OUTPUT_FILE=`dirname $0`/program-output.txt
RULES_FILE=
TEST=trace:all
TRACEDIR=/tmp/falco-perf-traces.$USER
CPU_INTERVAL=10
@@ -350,8 +369,10 @@ while true; do
-h | --help ) usage; exit 1;;
-v | --variant ) VARIANT="$2"; shift 2;;
-r | --root ) ROOT="$2"; shift 2;;
-s | --source ) SOURCE="$2"; shift 2;;
-R | --results ) RESULTS_FILE="$2"; shift 2;;
-o | --output ) OUTPUT_FILE="$2"; shift 2;;
-U | --rules ) RULES_FILE="$2"; shift 2;;
-t | --test ) TEST="$2"; shift 2;;
-T | --tracedir ) TRACEDIR="$2"; shift 2;;
-A | --agent-autodrop ) AGENT_AUTODROP="$2"; shift 2;;
@@ -372,6 +393,12 @@ fi
ROOT=`realpath $ROOT`
if [ -z $SOURCE ]; then
echo "A source directory containing falco/sysdig source code. Not continuing."
exit 1
fi
SOURCE=`realpath $SOURCE`
if [ -z $RESULTS_FILE ]; then
echo "An output file for test results must be provided. Not continuing."

View File

@@ -4,7 +4,7 @@ include_directories("${PROJECT_SOURCE_DIR}/../sysdig/userspace/libsinsp")
include_directories("${PROJECT_BINARY_DIR}/userspace/engine")
include_directories("${LUAJIT_INCLUDE}")
add_library(falco_engine STATIC rules.cpp falco_common.cpp falco_engine.cpp)
add_library(falco_engine STATIC rules.cpp falco_common.cpp falco_engine.cpp formats.cpp)
target_include_directories(falco_engine PUBLIC
"${LUAJIT_INCLUDE}")

View File

@@ -18,5 +18,5 @@ along with falco. If not, see <http://www.gnu.org/licenses/>.
#pragma once
#define FALCO_ENGINE_LUA_DIR "${CMAKE_INSTALL_PREFIX}/${FALCO_SHARE_DIR}/lua/"
#define FALCO_ENGINE_LUA_DIR "${FALCO_SHARE_DIR}/lua/"
#define FALCO_ENGINE_SOURCE_LUA_DIR "${PROJECT_SOURCE_DIR}/../falco/userspace/engine/lua/"

View File

@@ -24,6 +24,8 @@ along with falco. If not, see <http://www.gnu.org/licenses/>.
#include "falco_engine.h"
#include "config_falco_engine.h"
#include "formats.h"
extern "C" {
#include "lpeg.h"
#include "lyaml.h"
@@ -38,7 +40,8 @@ string lua_print_stats = "print_stats";
using namespace std;
falco_engine::falco_engine(bool seed_rng)
: m_rules(NULL), m_sampling_ratio(1), m_sampling_multiplier(0)
: m_rules(NULL), m_sampling_ratio(1), m_sampling_multiplier(0),
m_replace_container_info(false)
{
luaopen_lpeg(m_ls);
luaopen_yaml(m_ls);
@@ -72,7 +75,16 @@ void falco_engine::load_rules(const string &rules_content, bool verbose, bool al
{
m_rules = new falco_rules(m_inspector, this, m_ls);
}
m_rules->load_rules(rules_content, verbose, all_events);
// Note that falco_formats is added to both the lua state used
// by the falco engine as well as the separate lua state used
// by falco outputs. Within the engine, only
// formats.formatter is used, so we can unconditionally set
// json_output to false.
bool json_output = false;
falco_formats::init(m_inspector, m_ls, json_output);
m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info);
}
void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events)
@@ -98,20 +110,20 @@ void falco_engine::enable_rule(string &pattern, bool enabled)
m_evttype_filter.enable(pattern, enabled);
}
falco_engine::rule_result *falco_engine::process_event(sinsp_evt *ev)
unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
{
if(should_drop_evt())
{
return NULL;
return unique_ptr<struct rule_result>();
}
if(!m_evttype_filter.run(ev))
{
return NULL;
return unique_ptr<struct rule_result>();
}
struct rule_result *res = new rule_result();
unique_ptr<struct rule_result> res(new rule_result());
lua_getglobal(m_ls, lua_on_event.c_str());
@@ -184,6 +196,12 @@ void falco_engine::set_sampling_multiplier(double sampling_multiplier)
m_sampling_multiplier = sampling_multiplier;
}
void falco_engine::set_extra(string &extra, bool replace_container_info)
{
m_extra = extra;
m_replace_container_info = replace_container_info;
}
inline bool falco_engine::should_drop_evt()
{
if(m_sampling_multiplier == 0)

View File

@@ -63,7 +63,7 @@ public:
// the rule that matched. If no rule matched, returns NULL.
//
// the reutrned rule_result is allocated and must be delete()d.
rule_result *process_event(sinsp_evt *ev);
std::unique_ptr<rule_result> process_event(sinsp_evt *ev);
//
// Print details on the given rule. If rule is NULL, print
@@ -96,6 +96,16 @@ public:
//
void set_sampling_multiplier(double sampling_multiplier);
//
// You can optionally add "extra" formatting fields to the end
// of all output expressions. You can also choose to replace
// %container.info with the extra information or add it to the
// end of the expression. This is used in open source falco to
// add k8s/mesos/container information to outputs when
// available.
//
void set_extra(string &extra, bool replace_container_info);
private:
//
@@ -132,5 +142,8 @@ private:
double m_sampling_multiplier;
std::string m_lua_main_filename = "rule_loader.lua";
std::string m_extra;
bool m_replace_container_info;
};

View File

@@ -39,7 +39,7 @@ void falco_formats::init(sinsp* inspector, lua_State *ls, bool json_output)
s_inspector = inspector;
s_json_output = json_output;
luaL_openlib(ls, "falco", ll_falco, 0);
luaL_openlib(ls, "formats", ll_falco, 0);
}
int falco_formats::formatter(lua_State *ls)
@@ -49,14 +49,13 @@ int falco_formats::formatter(lua_State *ls)
try
{
formatter = new sinsp_evt_formatter(s_inspector, format);
lua_pushlightuserdata(ls, formatter);
}
catch(sinsp_exception& e)
{
throw falco_exception("Invalid output format '" + format + "'.\n");
luaL_error(ls, "Invalid output format '%s': '%s'", format.c_str(), e.what());
}
lua_pushlightuserdata(ls, formatter);
return 1;
}
@@ -64,14 +63,14 @@ int falco_formats::free_formatter(lua_State *ls)
{
if (!lua_islightuserdata(ls, -1))
{
throw falco_exception("Invalid argument passed to free_formatter");
luaL_error(ls, "Invalid argument passed to free_formatter");
}
sinsp_evt_formatter *formatter = (sinsp_evt_formatter *) lua_topointer(ls, 1);
delete(formatter);
return 1;
return 0;
}
int falco_formats::format_event (lua_State *ls)

View File

@@ -43,7 +43,4 @@ class falco_formats
static int format_event(lua_State *ls);
static sinsp* s_inspector;
private:
lua_State* m_ls;
};

View File

@@ -290,8 +290,8 @@ function compiler.compile_macro(line, list_defs)
local ast, error_msg = parser.parse_filter(line)
if (error_msg) then
print ("Compilation error when compiling \""..line.."\": ", error_msg)
error(error_msg)
msg = "Compilation error when compiling \""..line.."\": ".. error_msg
error(msg)
end
-- Traverse the ast looking for events/syscalls in the ignored
@@ -315,8 +315,8 @@ function compiler.compile_filter(name, source, macro_defs, list_defs)
local ast, error_msg = parser.parse_filter(source)
if (error_msg) then
print ("Compilation error when compiling \""..source.."\": ", error_msg)
error(error_msg)
msg = "Compilation error when compiling \""..source.."\": "..error_msg
error(msg)
end
-- Traverse the ast looking for events/syscalls in the ignored

View File

@@ -123,7 +123,46 @@ end
-- to a rule.
local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
function load_rules(rules_content, rules_mgr, verbose, all_events)
-- From http://lua-users.org/wiki/TableUtils
--
function table.val_to_str ( v )
if "string" == type( v ) then
v = string.gsub( v, "\n", "\\n" )
if string.match( string.gsub(v,"[^'\"]",""), '^"+$' ) then
return "'" .. v .. "'"
end
return '"' .. string.gsub(v,'"', '\\"' ) .. '"'
else
return "table" == type( v ) and table.tostring( v ) or
tostring( v )
end
end
function table.key_to_str ( k )
if "string" == type( k ) and string.match( k, "^[_%a][_%a%d]*$" ) then
return k
else
return "[" .. table.val_to_str( k ) .. "]"
end
end
function table.tostring( tbl )
local result, done = {}, {}
for k, v in ipairs( tbl ) do
table.insert( result, table.val_to_str( v ) )
done[ k ] = true
end
for k, v in pairs( tbl ) do
if not done[ k ] then
table.insert( result,
table.key_to_str( k ) .. "=" .. table.val_to_str( v ) )
end
end
return "{" .. table.concat( result, "," ) .. "}"
end
function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replace_container_info)
compiler.set_verbose(verbose)
compiler.set_all_events(all_events)
@@ -135,6 +174,10 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)
return
end
if type(rules) ~= "table" then
error("Rules content \""..rules_content.."\" is not yaml")
end
for i,v in ipairs(rules) do -- iterate over yaml list
if (not (type(v) == "table")) then
@@ -164,9 +207,9 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)
state.lists[v['list']] = items
else -- rule
elseif (v['rule']) then
if (v['rule'] == nil) then
if (v['rule'] == nil or type(v['rule']) == "table") then
error ("Missing name in rule")
end
@@ -214,9 +257,41 @@ function load_rules(rules_content, rules_mgr, verbose, all_events)
if (v['enabled'] == false) then
falco_rules.enable_rule(rules_mgr, v['rule'], 0)
end
-- If the format string contains %container.info, replace it
-- with extra. Otherwise, add extra onto the end of the format
-- string.
if string.find(v['output'], "%container.info", nil, true) ~= nil then
-- There may not be any extra, or we're not supposed
-- to replace it, in which case we use the generic
-- "%container.name (id=%container.id)"
if replace_container_info == false then
v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
else
safe_extra = string.gsub(extra, "%%", "%%%%")
v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
end
else
-- Just add the extra to the end
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
end
-- Ensure that the output field is properly formatted by
-- creating a formatter from it. Any error will be thrown
-- up to the top level.
formatter = formats.formatter(v['output'])
formats.free_formatter(formatter)
else
error ("Unexpected type in load_rule: "..filter_ast.type)
end
else
error ("Unknown rule object: "..table.tostring(v))
end
end

View File

@@ -108,7 +108,9 @@ void falco_rules::enable_rule(string &rule, bool enabled)
m_engine->enable_rule(rule, enabled);
}
void falco_rules::load_rules(const string &rules_content, bool verbose, bool all_events)
void falco_rules::load_rules(const string &rules_content,
bool verbose, bool all_events,
string &extra, bool replace_container_info)
{
lua_getglobal(m_ls, m_lua_load_rules.c_str());
if(lua_isfunction(m_ls, -1))
@@ -182,7 +184,9 @@ void falco_rules::load_rules(const string &rules_content, bool verbose, bool all
lua_pushlightuserdata(m_ls, this);
lua_pushboolean(m_ls, (verbose ? 1 : 0));
lua_pushboolean(m_ls, (all_events ? 1 : 0));
if(lua_pcall(m_ls, 4, 0, 0) != 0)
lua_pushstring(m_ls, extra.c_str());
lua_pushboolean(m_ls, (replace_container_info ? 1 : 0));
if(lua_pcall(m_ls, 6, 0, 0) != 0)
{
const char* lerr = lua_tostring(m_ls, -1);
string err = "Error loading rules:" + string(lerr);

View File

@@ -31,7 +31,8 @@ class falco_rules
public:
falco_rules(sinsp* inspector, falco_engine *engine, lua_State *ls);
~falco_rules();
void load_rules(const string &rules_content, bool verbose, bool all_events);
void load_rules(const string &rules_content, bool verbose, bool all_events,
std::string &extra, bool replace_container_info);
void describe_rule(string *rule);
static void init(lua_State *ls);

View File

@@ -9,7 +9,7 @@ include_directories("${CURL_INCLUDE_DIR}")
include_directories("${YAMLCPP_INCLUDE_DIR}")
include_directories("${DRAIOS_DEPENDENCIES_DIR}/yaml-${DRAIOS_YAML_VERSION}/target/include")
add_executable(falco configuration.cpp formats.cpp logger.cpp falco_outputs.cpp falco.cpp)
add_executable(falco configuration.cpp logger.cpp falco_outputs.cpp falco.cpp)
target_link_libraries(falco falco_engine sinsp)
target_link_libraries(falco

View File

@@ -122,10 +122,11 @@ std::list<string> cmdline_options;
//
// Event processing loop
//
void do_inspect(falco_engine *engine,
falco_outputs *outputs,
sinsp* inspector)
uint64_t do_inspect(falco_engine *engine,
falco_outputs *outputs,
sinsp* inspector)
{
uint64_t num_evts = 0;
int32_t res;
sinsp_evt* ev;
@@ -170,13 +171,16 @@ void do_inspect(falco_engine *engine,
// engine, which will match the event against the set
// of rules. If a match is found, pass the event to
// the outputs.
falco_engine::rule_result *res = engine->process_event(ev);
unique_ptr<falco_engine::rule_result> res = engine->process_event(ev);
if(res)
{
outputs->handle_event(res->evt, res->rule, res->priority, res->format);
delete(res);
}
num_evts++;
}
return num_evts;
}
//
@@ -192,6 +196,7 @@ int falco_init(int argc, char **argv)
int long_index = 0;
string scap_filename;
string conf_filename;
string outfile;
list<string> rules_filenames;
bool daemon = false;
string pidfilename = "/var/run/falco.pid";
@@ -205,6 +210,18 @@ int falco_init(int argc, char **argv)
string output_format = "";
bool replace_container_info = false;
// Used for writing trace files
int duration_seconds = 0;
int rollover_mb = 0;
int file_limit = 0;
unsigned long event_limit = 0L;
bool compress = false;
// Used for stats
uint64_t num_evts;
double duration;
scap_stats cstats;
static struct option long_options[] =
{
{"help", no_argument, 0, 'h' },
@@ -215,6 +232,7 @@ int falco_init(int argc, char **argv)
{"option", required_argument, 0, 'o'},
{"print", required_argument, 0, 'p' },
{"pidfile", required_argument, 0, 'P' },
{"writefile", required_argument, 0, 'w' },
{0, 0, 0, 0}
};
@@ -228,7 +246,7 @@ int falco_init(int argc, char **argv)
// Parse the args
//
while((op = getopt_long(argc, argv,
"hc:AdD:e:k:K:Ll:m:o:P:p:r:v",
"hc:AdD:e:k:K:Ll:m:o:P:p:r:vw:",
long_options, &long_index)) != -1)
{
switch(op)
@@ -303,6 +321,9 @@ int falco_init(int argc, char **argv)
case 'v':
verbose = true;
break;
case 'w':
outfile = optarg;
break;
case '?':
result = EXIT_FAILURE;
goto exit;
@@ -315,10 +336,10 @@ int falco_init(int argc, char **argv)
inspector = new sinsp();
engine = new falco_engine();
engine->set_inspector(inspector);
engine->set_extra(output_format, replace_container_info);
outputs = new falco_outputs();
outputs->set_inspector(inspector);
outputs->set_extra(output_format, replace_container_info);
// Some combinations of arguments are not allowed.
if (daemon && pidfilename == "") {
@@ -500,6 +521,14 @@ int falco_init(int argc, char **argv)
open("/dev/null", O_RDWR);
}
if(outfile != "")
{
inspector->setup_cycle_writer(outfile, rollover_mb, duration_seconds, file_limit, event_limit, compress);
inspector->autodump_next_file();
}
duration = ((double)clock()) / CLOCKS_PER_SEC;
//
// run k8s, if required
//
@@ -557,9 +586,25 @@ int falco_init(int argc, char **argv)
delete mesos_api;
mesos_api = 0;
do_inspect(engine,
outputs,
inspector);
num_evts = do_inspect(engine,
outputs,
inspector);
duration = ((double)clock()) / CLOCKS_PER_SEC - duration;
inspector->get_capture_stats(&cstats);
if(verbose)
{
fprintf(stderr, "Driver Events:%" PRIu64 "\nDriver Drops:%" PRIu64 "\n",
cstats.n_evts,
cstats.n_drops);
fprintf(stderr, "Elapsed time: %.3lf, Captured Events: %" PRIu64 ", %.2lf eps\n",
duration,
num_evts,
num_evts / duration);
}
inspector->close();

View File

@@ -27,7 +27,6 @@ along with falco. If not, see <http://www.gnu.org/licenses/>.
using namespace std;
falco_outputs::falco_outputs()
: m_replace_container_info(false)
{
}
@@ -47,17 +46,14 @@ void falco_outputs::init(bool json_output)
falco_common::init(m_lua_main_filename.c_str(), FALCO_SOURCE_LUA_DIR);
// Note that falco_formats is added to both the lua state used
// by the falco engine as well as the separate lua state used
// by falco outputs.
falco_formats::init(m_inspector, m_ls, json_output);
falco_logger::init(m_ls);
}
void falco_outputs::set_extra(string &extra, bool replace_container_info)
{
m_extra = extra;
m_replace_container_info = replace_container_info;
}
void falco_outputs::add_output(output_config oc)
{
uint8_t nargs = 1;
@@ -94,42 +90,12 @@ void falco_outputs::handle_event(sinsp_evt *ev, string &level, string &priority,
{
lua_getglobal(m_ls, m_lua_output_event.c_str());
// If the format string contains %container.info, replace it
// with extra. Otherwise, add extra onto the end of the format
// string.
string format_w_extra = format;
size_t pos;
if((pos = format_w_extra.find("%container.info")) != string::npos)
{
// There may not be any extra, or we're not supposed
// to replace it, in which case we use the generic
// "%container.name (id=%container.id)"
if(m_extra == "" || ! m_replace_container_info)
{
// 15 == strlen(%container.info)
format_w_extra.replace(pos, 15, "%container.name (id=%container.id)");
}
else
{
format_w_extra.replace(pos, 15, m_extra);
}
}
else
{
// Just add the extra to the end
if (m_extra != "")
{
format_w_extra += " " + m_extra;
}
}
if(lua_isfunction(m_ls, -1))
{
lua_pushlightuserdata(m_ls, ev);
lua_pushstring(m_ls, level.c_str());
lua_pushstring(m_ls, priority.c_str());
lua_pushstring(m_ls, format_w_extra.c_str());
lua_pushstring(m_ls, format.c_str());
if(lua_pcall(m_ls, 4, 0, 0) != 0)
{

View File

@@ -44,8 +44,6 @@ public:
void add_output(output_config oc);
void set_extra(string &extra, bool replace_container_info);
//
// ev is an event that has matched some rule. Pass the event
// to all configured outputs.
@@ -56,6 +54,4 @@ private:
std::string m_lua_add_output = "add_output";
std::string m_lua_output_event = "output_event";
std::string m_lua_main_filename = "output.lua";
std::string m_extra;
bool m_replace_container_info;
};

View File

@@ -75,14 +75,14 @@ end
function output_event(event, rule, priority, format)
local level = level_of(priority)
format = "*%evt.time: "..levels[level+1].." "..format
formatter = falco.formatter(format)
msg = falco.format_event(event, rule, levels[level+1], formatter)
formatter = formats.formatter(format)
msg = formats.format_event(event, rule, levels[level+1], formatter)
for index,o in ipairs(outputs) do
o.output(level, msg, o.config)
end
falco.free_formatter(formatter)
formats.free_formatter(formatter)
end
function add_output(output_name, config)