Compare commits

...

105 Commits

Author SHA1 Message Date
Luca Marturana
1e0ddba11a Merge branch 'dev' into agent-master 2017-01-25 18:08:35 +01:00
Mark Stemm
34e17cb951 Several changes to reduce FPs
Several changes to reduce spurious alerts when managing machines via
ansible:

 - Add ansible_running_python (that is, ansible-spawned python scripts)
   as scripts that can read sensitive files and write below
   /etc. Notably this is the user ansible module.
 - Also add comments to ansible_running_python suggesting users make it
   more strict by specifically naming the root directory for ansible
   scripts.
 - Add pypy as a python variant that can run ansible-related scripts.

Also other changes to reduce FPs:

 - add apt-add-reposit, apt-auto-remova (truncation intentional),
   apt-get, apt, apt-key as package management programs, and add package
   management binaries to the set of shell spawners. The overlapping
   binaries that were in known_shell_spawn_binaries were removed.
 - add passwd_binaries, gpg, insserv, apparmor_parser, update-mime,
   tzdata.{config,postinst}, systemd-machine, and debconf-show to
   the set of binaries that can write below /etc.
 - Add vsftpd as a program that can read sensitive files.
 - Add additional programs (incl. python support programs like pip,
   pycompile) as ones that can spawn shells.
 - Allow privileged containers to spawn shells.
 - Break out the set of files below /dev that are written to with O_CREAT
   into a separate list, and add /dev/random,urandom,console to the list.
 - Add python running denyhosts as a program that can write below /etc.
 - Also add binaries starting with linux-image- as ones that can spawn
   shells. These are perl scripts run as a part of installing
   linux-image-N.N packages.
2017-01-25 08:34:52 -08:00
Mark Stemm
bc83ac18a0 Allow shells spawned by ansible.
Changes to allow shells spawned by ansible. In general this is actually
pretty difficult--on the remote managed machine, ansible performs
actions simply by running python over ssh without any explicit ansible
helper or command line.

One (weak) hint is that the python scripts being run are usually under a
directory with ansible in the name. So use that as the basis for a macro
ansible_running_python. In turn, that macro is used as a negative
condition for the run shell untrusted rule.

This is a pretty fragile and easily exploited condition, so add a note
to the macro saying so.
2017-01-19 15:09:24 -08:00
Mark Stemm
10d0c8f982 Add a local dockerfile variant.
Add a local dockerfile variant that allows creating an image from a
local .deb package.
2017-01-17 10:24:38 -08:00
Mark Stemm
8f53bcbb05 Patch jq 1.5 with a fix for security vulns.
After downloading jq 1.5, apply the changes in
stedolan/jq@8eb1367
by downloading the commit as a patch and applying it. This fixes
CVE-2015-8863.
2017-01-17 10:24:38 -08:00
Mark Stemm
7286b50f4d Update libcurl to 7.52.1.
This fixes a set of ~10 security vulnerabilities.
2017-01-17 10:24:38 -08:00
Mark Stemm
4c60b7c1d2 Update openssl to 1.0.2j.
This fixes a set of ~25 security vulnerabilities.
2017-01-17 10:24:38 -08:00
Mark Stemm
85480f32d6 Avoid FPs resulting from ubuntu weekly cron jobs
Feedback from a falco user:

--
to more findings from last night:

logrotate cronjob (Debian default):

Shell spawned by untrusted binary (user=root shell=sh parent=logrotate cmdline=sh -c invoke-rc.d rsyslog rotate > /dev/null logrotate_script /var/log/syslog)

passwd cronjob (Debian default):

Sensitive file opened for reading by non-trusted program (user=root name=cmp command=cmp -s shadow.bak /etc/shadow file=/etc/shadow)
--

New macro cmp_cp_by_passwd allows cmp/cp to be run by passwd to examine
sensitive files. Add logrotate as a program that can spawn a shell.

Also do some cleanups, moving items to lists and splitting long
single-line conditions into multiple lines.
2017-01-17 09:12:53 -08:00
Luca Marturana
4139370df5 Merge branch 'agent-master' into dev
agent-master went out of sync, probably some rebase/forcepush happened
on dev. Used `git merge -s ours agent-master` here to put all the
commits of agent-master on dev and ignoring anything from agent-master.

So now we can merge from dev to agent-master with fast forward and no
conflicts
2017-01-17 10:58:08 +01:00
Luca Marturana
b6d1101cb6 Merge branch 'agent-master' into dev 2017-01-17 10:55:07 +01:00
Mark Stemm
43d53bb09e Add exechealthz as a k8s binary.
For customers who use
https://github.com/kubernetes/contrib/tree/master/exec-healthz to
perform liveness checking, exechealthz will spawn shells in a
container. Add it to the k8s_binaries list.
2017-01-12 10:32:13 -08:00
Luca Marturana
af3a708251 Improve comment 2017-01-04 18:05:46 +01:00
Mark Stemm
f4bb49f1f5 Add test for truncated outputs.
Add a test that specifically tests truncated outputs. A rule contains an
output field %fd.cport which has no value for an open event. Ensure that
the rule's output has <NA> for the cport and the remainder of the rule's
output is filled in.
2017-01-03 12:58:01 -08:00
Mark Stemm
362a6b7b9a Prefix outputs with * within the engine.
Prefix output strings with * so they are always permissive in the
engine.

In falco outputs, which adds its own prefix, remove any leading * before
adding the custom prefix.
2017-01-03 12:58:01 -08:00
Mark Stemm
77a5429cae Add cchh/sysdig as a trusted container.
Add cchh/sysdig as a trusted container. We'll probably remove this once
the next agent release occurs that has the fix
https://github.com/draios/falco/pull/177.

Also reformat to avoid long lines.
2016-12-30 12:28:02 -08:00
Mark Stemm
9ecdf30314 tests for overriding rules/macros/lists
New tests that test every possible override:

 - Overriding a rule with one that doesn't match
 - Overriding a macro to one that doesn't match
 - Overriding a top level list to a binary that doesn't match
 - Overriding an embedded list to one that doesn't match

In each case, the override results in no longer matching an open by the
program "cat".
2016-12-29 13:32:55 -08:00
Mark Stemm
7c419b6d6b Allow any macro/list/rule to be overridden
Allow any list/macro/rule to be overridden by a subsequent file. The
persistent state that lives across invocations of load_rules are the 3
arrays ordered_{list,macro,rule}_names, which have the
lists/macros/rules in the order in which they first appear, and tables
{rules,macros,lists}_by_name, which maps from a name to a yaml object.

With each call to load_rules, the set of loaded rules is reset and the
state of expanded lists, compiled macros, compiled rules, and rule
metadata are recreated from scratch, using the ordered_*_names arrays
and *_by_name tables. That way, any list/macro/rule can be redefined in
a subsequent file with new values.
2016-12-29 13:32:55 -08:00
Mark Stemm
767f2d5bb4 Add ability to clear loaded rules.
Add the ability to clear the set of loaded rules from lua. It simply
recreates the sinsp_evttype_filter instance m_evttype_filter, which is
now a unique_ptr.
2016-12-29 13:32:55 -08:00
Mark Stemm
3cbf641ded Add confd/fleetctl as acceptable programs.
Add confd as a program that can write files below /etc and fleetctl as a
program that can spawn shells.
2016-12-28 12:38:39 -08:00
Mark Stemm
1d0c9b1714 Merge pull request #169 from jcoetzee/systemd
Add systemd as a login binary
2016-12-16 11:43:07 -08:00
Mark Stemm
8aa9c21d11 Merge pull request #168 from jcoetzee/fail2ban
Add fail2ban-server as trusted binary
2016-12-16 09:38:57 -08:00
Jonathan Coetzee
64ecd157fd Add systemd as a login binary
SSH'ing into an Ubuntu 16.04 box triggers a bunch of "Sensitive file opened for reading by non-trusted program" errors caused by systemd

falco-CLA-1.0-signed-off-by: Jonathan Coetzee jon@thancoetzee.com
2016-12-16 11:27:43 +02:00
Jonathan Coetzee
2bad529d33 Add fail2ban-server as trusted binary
fail2ban spawns shells to modify iptables

falco-CLA-1.0-signed-off-by: Jonathan Coetzee <jon@thancoetzee.com>
2016-12-16 11:18:47 +02:00
Mark Stemm
09a9ab4f85 Merge pull request #164 from draios/revert-163-dev
Revert "Add fail2ban-server as spawn shell trusted binary"
2016-12-14 18:29:27 -08:00
Mark Stemm
39e9043ac7 Revert "Add fail2ban-server as spawn shell trusted binary" 2016-12-14 18:28:37 -08:00
Mark Stemm
f4abec4639 Merge pull request #163 from jcoetzee/dev
Add fail2ban-server as spawn shell trusted binary
2016-12-14 18:28:16 -08:00
Jonathan Coetzee
bed5ab4f0c Add fail2ban-server as spawn shell trusted binary
fail2ban spawns a shell to adjust iptables in order to ban/unban IP addresses.
2016-12-15 00:12:31 +02:00
Mark Stemm
4f645c49e1 Use sinsp utils version of get time.
sinsp_utils::get_current_time_ns() has the same purpose as
get_epoch_ns(), and now that we're including the token bucket in
falco_engine, it's easy to package the dependency. So use that function
instead.
2016-12-08 13:35:17 -08:00
Mark Stemm
54b30bc248 Add rate-limiting for notifications
Add token-bucket based rate limiting for falco notifications.

The token bucket is implemented in token_bucket.cpp (actually in the
engine directory, just to make it easier to include in other
programs). It maintains a current count of tokens (i.e. right to send a
notification). Its main method is claim(), which attemps to claim a
token and returns true if one was claimed successfully. It has a
configurable configurable max burst size and rate. The token bucket
gains "rate" tokens per second, up to a maximum of max_burst tokens.

These parameters are configurable in falco.yaml via the config
options (defaults shown):

outputs:
  rate: 1
  max_burst: 1000

In falco_outputs::handle_event(), try to claim a token, and if
unsuccessful log a debug message and return immediately.
2016-12-08 09:46:16 -08:00
Mark Stemm
b509c4f0c8 Fix misleading variable name.
The second argument to handle_event is actually a rule name, but the
variable was a misleading "level". Fix.
2016-12-08 09:46:16 -08:00
Mark Stemm
af8d6c9d10 Make google_containers/kube-proxy a trusted image.
Add google_containers/kube-proxy as a trusted image (can be run
privileged, can mount sensitive filesystems). While our k8s deployments
run kube-proxy via the hyperkube image, evidently it's sometimes run via
its own image.

This is one of the fixes for #156.

Also update the output message for this rule.
2016-12-08 09:41:47 -08:00
Mark Stemm
ef08478bb7 Add log levels.
Previously, log messages had levels, but it only influenced the level
argument passed to syslog(). Now, add the ability to control log level
from falco itself.

New falco.yaml argument "log_level" can be one of the strings
corresponding to the well-known syslog levels, which is converted to a
syslog-style level as integer.

In falco_logger::log(), skip messages below the specified level.
2016-12-07 16:47:44 -08:00
Mark Stemm
a616301bd9 Cache formatters.
Instead of creating a formatter for each event, cache them and create
them only when needed. A new function output_cleanup cleans up the
cached formatters, and is called in the destructor if init() was called.
2016-12-07 10:33:35 -08:00
Mark Stemm
8e2a3ef5c3 Modify plotting script to handle drop stats.
New argument --metric, which can be cpu|drops, controls whether to graph
cpu usage or event drop percentage. Titles/axis labels/etc. change
appropriately.
2016-12-06 11:01:38 -08:00
Mark Stemm
47bd6af69a Add ability to write "extra" stuff to stats file.
When run via scripts like run_performance_tests.sh, it's useful to
include extra info like the test being run and the specific program
variant to the stats file. So support that via the
environment. Environment keys starting with FALCO_STATS_EXTRA_XXX will
have the XXX and environment value added to the stats file.

It's undocumented as I doubt other programs will need this functionality
and it keeps the docs simpler.
2016-12-05 16:56:13 -08:00
Mark Stemm
d1d0dbdbde Add ability to write capture stats to a file.
With -s, periodically fetch capture stats from the inspector and write
them to the provided file.

Separate class StatsFileWriter handles the details. It does rely on a
timer + SIGALRM handler so you can only practically create a single
object, but it does keep the code/state separate.

The output format has a sample number, the set of current stats, a
delta with the difference from the prior sample, and the percentage of
events dropped during that sample.
2016-12-05 16:56:13 -08:00
Luca Marturana
212fd9353e Push formatter on lua stack only if does not throw exceptions 2016-12-02 16:13:37 +01:00
Mark Stemm
28558959f3 Merge pull request #153 from djcross/dnf
Adding DNF as non-alerting for RPM and package management
2016-12-01 17:51:58 -08:00
Daniel Cross
a8662c60da Adding DNF as non-alerting for RPM and package management
falco-CLA-1.0-signed-off-by: Daniel Cross <daniel.cross@rea-group.com>
2016-12-02 11:52:08 +11:00
Mark Stemm
b3c691e920 Prevent rule_result from leaking on error.
Change falco_engine::process_event to return a unique_ptr that wraps the
rule result, so it won't be leaked if this method throws an exception.

This means that callers don't need to create their own.
2016-12-01 13:53:34 -06:00
Mark Stemm
ded3ee5bed Add unit test for rule with invalid output.
Add the ability to check falco's return code with exit_status and to
generally match stderr with stderr_contains in a test.

Use those to create a test that has an invalid output expression using
%not_a_real_field. It expects falco to exit with 1 and the output to
contain a message about the invalid output.
2016-12-01 13:53:34 -06:00
Mark Stemm
064b39f2be Validate rule outputs when loading rules.
Validate rule outputs when loading rules by attempting to create a
formatter based on the rule's output field. If there's an error, it will
propagate up through load_rules and cause falco to exit rather than
discover the problem only when trying to format the event and the rule's
output field.

This required moving formats.{cpp,h} into the falco engine directory
from the falco general directory. Note that these functions are loaded
twice in the two lua states used by falco (engine and outputs).

There's also a couple of minor cleanups:

 - falco_formats had a private instance variable that was unused, remove
   it.
 - rename the package for the falco_formats functions to formats instead
   of falco so it's more standalone.
 - don't throw a c++ exception in falco_formats::formatter. Instead
   generate a lua error, which is handled more cleanly.
 - free_formatter doesn't return any values, so set the return value of
   the function to 0.
2016-12-01 13:53:34 -06:00
Mark Stemm
2961eb4d21 Move container.info handling to falco engine.
container.info handling used to be handled by the the falco_outputs
object. However, this caused problems for applications that only used
the falco engine, doing their own output formatting for matching events.

Fix this by moving output formatting into the falco engine itself. The
part that replaces %container.info/adds extra formatting to the end of a
rule's output now happens while loading the rule.
2016-12-01 13:53:34 -06:00
Mark Stemm
704eb57e3c Allow run_performance_tests to run test_mm.
Make necessary changes to allow run_performance_tests to invoke the
'test_mm' program we use internally.

Also add ability to run with a build directory separate from the source
directory and to specify an alternate rules file.

Finally, set up the kubernetes demo using sudo, a result of recent changes.
2016-11-28 14:54:14 -08:00
Mark Stemm
9ca8ed96b9 Improve error messages when loading rules.
Related to the changes in https://github.com/draios/agent/pull/267,
improve error messages when trying to load sets of rules with errors:

 - Check that yaml parsing of rules_content actually resulted in
   something.
 - Return an error for rules that have an empty name.
 - Return an error for yaml objects that aren't a rule/macro/list.
 - When compiling, don't print an error message, simply return one,
   including a wrapper "can not compile ..." string.
2016-11-28 12:53:36 -06:00
Mark Stemm
8b18315c1e Fully specify FALCO_SHARE_DIR.
Instead of having FALCO_SHARE_DIR be a relative path, fully specify it
by prepending CMAKE_INSTALL_PREFIX in the top level CMakeLists.txt and
don't prepend CMAKE_INSTALL_PREFIX in config_falco_engine.h.in. This
makes it consistent with its use in the agent.
2016-11-10 16:51:51 -06:00
Mark Stemm
f95a0ead62 Honor USE_BUNDLED_DEPS option for third-party libs
Honor a USE_BUNDLED_DEPS option for third-party libraries which can be
applied globally. There are also USE_BUNDLED_XXX options that can be
used individually for each library.

Verified that this works by first building with USE_BUNDLED_DEPS=ON (the
default), installing external packages ncurses-dev libssl-dev
libcurl4-openssl-dev so CMake's find_package could use them, modifying
the CMakeLists.txt to add "PATHS ${PROJECT_BINARY_DIR}/..." options to
each find_path()/find_library() command to point to the previously
installed third party libraries. It found them as expected.

The sysdig fix in https://github.com/draios/sysdig/pull/672 forced this
change, but it does also happen to fix a falco feature request
https://github.com/draios/falco/issues/144.
2016-11-10 15:43:25 -06:00
Carl Sverre
b1ad9e644e Added envvar SYSDIG_SKIP_LOAD to Dockerfile to skip kernel module manipulation
This helps when running on a system which has the module loaded, but getting
access to the module file is hard for some reason.  Since I know that the right
version of the module is loaded I just want falco to connect.

I tested this with this run command:

docker run -e SYSDIG_SKIP_LOAD=1 -it -v /dev:/host/dev -v /proc:/host/proc --privileged falco

And it successfully connected to Sysdig and started printing out warnings for my
system.

falco-CLA-1.0-signed-off-by: Carl Sverre accounts@carlsverre.com
2016-11-02 12:45:21 -05:00
Mark Stemm
8a2924ad72 Updating for 0.4.0.
CHANGELOG for release notes, README to update version.
2016-10-25 09:53:54 -07:00
Mark Stemm
f98ec60c88 Rule fixes for dragent.
Make sure falco doesn't detect the things draios-agent does as
suspicious. It's possible that you might run open source falco alongside
sysdig cloud.

App checks spawned by sysdig cloud binaries might also change namespace,
so also allow children of sysdigcloud binaries to call setns.
2016-10-24 15:14:09 -07:00
Mark Stemm
0211a94f60 Add stats on events processed/dropped.
Collect stats on the number of events processed and dropped. When run
with -v, print these stats. This duplicates syddig behavior and can be
useful when dianosing problems related to dropped events throwing off
internal state tracking.
2016-10-24 11:02:52 -07:00
Mark Stemm
e0e640c67f Add ability to write trace files.
Bring over functionality from sysdig to write trace files. This is easy
as all of the code to actually write the files is in the inspector. This
just handles the -w option and arguments.

This can be useful to write a trace file in parallel with live event
monitoring so you can reproduce it later.
2016-10-24 10:15:10 -07:00
Mark Stemm
faef5621dd Add k8s binaries as trusted programs
Add a new list k8s_binaries and allow those binaries to do things like
setns/spawn shells. It's not the case that all of these binaries
actually do these things, but keeping it as a single list makes
management easier.
2016-10-21 15:55:10 -07:00
Mark Stemm
e543fbf247 Allow falco to spawn shells in containers.
Falco is allowed to spawn shells in containers as a part of its program
output method.
2016-10-14 16:58:50 -07:00
Mark Stemm
f761ddff9f Fix logic for detecting conf files.
The logic for detecting if a file exists was backwards. It would treat a
file as existing if it could *not* be opened. Reverse that logic so it
works.

This fixes https://github.com/draios/falco/issues/135.
2016-10-14 14:03:33 -07:00
Mark Stemm
1f7c711a69 Merge pull request #134 from draios/add-k8s-mesos-support
Add k8s/mesos/container info to rule outputs.
2016-10-13 15:15:48 -07:00
Mark Stemm
880c39633d Add k8s/mesos/container info to rule outputs
Copy handling of -pk/-pm/-pc/-k/-m arguments from sysdig. All of the
relevant code was already in the inspector so that was easy.

The information from k8s/mesos/containers is used in two ways:

- In rule outputs, if the format string contains %container.info, that
  is replaced with the value from -pk/-pm/-pc, if one of those options
  was provided. If no option was provided, %container.info is replaced
  with a generic %container.name (id=%container.id) instead.

- If the format string does not contain %container.info, and one of
  -pk/-pm/-pc was provided, that is added to the end of the formatting
  string.

- If -p was specified with a general value (i.e. not
  kubernetes/mesos/container), the value is simply added to the end and
  any %container.info is replaced with the generic value.
2016-10-13 14:48:32 -07:00
Mark Stemm
3bb84f5498 Alphabetize command line options.
There are a lot of command line options now, so sort them alphabetically
in the usage and getopt handling to make them easier to find.

Also rename -p <pidfile> to -P <pidfile>, thinking ahead to the next
commit.
2016-10-13 14:47:00 -07:00
Mark Stemm
7e60b4b6c2 Merge pull request #133 from draios/add-jq-to-docker
Add jq to docker
2016-10-12 18:12:08 -07:00
Mark Stemm
1a78e45d7a Merge pull request #132 from draios/event-generator-env
Add exfiltration action, env-specified actions.
2016-10-12 18:11:40 -07:00
Mark Stemm
20440912b7 Add notes on how to post to slack webhooks.
Add comments for program_output that show how to post to a slack webhook
and an alernate logging method--came up in one of the github issues.
2016-10-12 17:08:28 -07:00
Mark Stemm
f6720d3993 Add jq to docker images.
Add jq to the docker image containing falco. jq is very handy for
transforming json, which comes into play if you want to post to
slack (or other) webhooks.
2016-10-12 17:05:07 -07:00
Mark Stemm
82903359cb Add exfiltration action, env-specified actions.
Add an exfiltration action that reads /etc/shadow and sends the contents
to a arbitrary ip address and port via a udp datagram.

Add the ability to specify actions via the environment instead of the
command line. If actions are specified via the environment, they replace
any actions specified on the command line.
2016-10-12 17:04:01 -07:00
Mark Stemm
144789475e Merge pull request #126 from draios/add-licenses
Add license comments to all source code.
2016-10-07 11:40:38 -07:00
Mark Stemm
644f017b2a Add license comments to all source code.
Add comment blocks to all source code w/ our gpl copyright notice.
2016-10-07 10:51:25 -07:00
Mark Stemm
5008003600 Merge pull request #125 from draios/add-pmatch
Add the new pmatch operator.
2016-10-03 11:20:34 -07:00
Mark Stemm
82597c9830 Merge pull request #124 from draios/fix-docker-gcc
Install gcc-4.9 from Debian Jessie repositories
2016-09-30 10:07:46 -07:00
Mark Stemm
4354043a44 Install gcc-4.9 from Debian Jessie repositories
As luca did for the agent, install gcc 4.9 from the debian jesse
repository, as it has been removed from unstable.
2016-09-30 09:39:01 -07:00
Mark Stemm
08d204dde9 Merge pull request #123 from draios/fix-stack-leak
Fix lua stack leak.
2016-09-23 16:02:01 -07:00
Mark Stemm
9a5e08d712 Fix lua stack leak.
Need to pop the results of process_event so the stack doesn't grow
without bound.
2016-09-23 15:34:32 -07:00
Mark Stemm
930b38b894 Add the new pmatch operator.
Make changes to the lua-specific rule parser/compiler to handle the
pmatch operator.
2016-09-22 14:57:43 -07:00
Mark Stemm
889b252a3f Merge pull request #121 from draios/improve-docker-rules
Reduce FPs related to Kubernetes.
2016-09-15 15:36:37 -05:00
Mark Stemm
164d5016ef Reduce FPs related to Kubernetes.
The new privileged falco rule was noisy when running kubernetes, which
can run privileged. Add it to the trusted_containers list.

Also eliminate a couple spurious warnings related to spawning shells in
containers.
2016-09-14 13:53:59 -07:00
Mark Stemm
6e9241a983 Merge pull request #120 from draios/addl-container-rules
Addl container rules
2016-09-12 15:01:51 -05:00
Mark Stemm
23e3e99162 New rules related to containers.
New rule 'File Open by Privileged Container' triggers when a container
that is running privileged opens a file.

New rule 'Sensitive Mount by Container' triggers when a container that
has a sensitive mount opens a file. Currently, a sensitive mount is a
mount of /proc.

This depends on https://github.com/draios/sysdig/pull/655.
2016-09-09 11:18:54 -07:00
Mark Stemm
f632fa62b0 Parser changes to support new sysdig features
Support "glob" as an operator and allow pathnames to be the index into
bracketed selectors of fields.
2016-09-09 11:18:30 -07:00
Mark Stemm
33b9ef5d50 Include condition in compilation errors.
When a macro/rule condition can't be compiled, include the condition in
the error message.
2016-09-08 16:15:10 -07:00
Mark Stemm
fbcddba06a Merge pull request #119 from draios/add-enabled-flag
Add enabled flag
2016-09-07 10:40:07 -05:00
Mark Stemm
5644919e70 Add test for enabled flag.
New test case disables a rule that would otherwise match.
2016-09-03 08:41:11 -07:00
Mark Stemm
f974922f84 Support enabled flag for rules.
If a rule has a enabled attribute, and if the value is false, call the
engine's enable_rule() method to disable the rule. Like add_filter,
there's a static method which takes the object as the first argument and
a non-static method that calls the engine.

This fixes #72.
2016-09-03 08:40:33 -07:00
Mark Stemm
08c3befe25 Merge pull request #117 from draios/fix-outputs
Fix outputs
2016-08-24 10:06:12 -07:00
Mark Stemm
ef52e627ec Add regression tests for configurable outputs.
- In the regression tests, make the config file configurable in the
   multiplex file via 'conf_file'.
 - A new multiplex file item 'outputs' containing a list of <filename>:
   <regex> tuples. For each item, the test reads the file and matches
   each line against the regex. A match must be found for the test to
   pass.
 - Add 2 new tests that test file output and program output. They write
   to files below /tmp/falco_outputs/ and the contents are checked to
   ensure that alerts are written.
2016-08-23 15:58:18 -07:00
Mark Stemm
23a9b6e1b0 Fix output methods that take configurations.
The falco engine changes broke the output methods that take
configuration (like the filename for file output, or the program for
program output). Fix that by properly passing the options argument to
each method's output function.
2016-08-23 14:15:52 -07:00
Mark Stemm
3ee1c0f602 Don't alert on falco program notifications.
Falco itself spawns a shell when using program notifications, so add
falco to the set of trusted programs. (Also add some other programs like
make, awk, configure, that are run while building).
2016-08-23 14:12:28 -07:00
Mark Stemm
ceee146f39 Merge pull request #116 from draios/rule-names-with-spaces
Make rule names human readable
2016-08-23 07:34:04 -07:00
Mark Stemm
ceedd772c7 Change rule names to be human readable.
Given the prior test, change all rule names to be human readable. This
is especially important for the agent integration as they are visible.
2016-08-23 06:19:24 -07:00
Mark Stemm
2731fd5ae1 Verifying rule names can have spaces.
Related to discussion on https://github.com/draios/agent/pull/160,
verifying we can have rule names with spaces.
2016-08-23 06:19:24 -07:00
Mark Stemm
e717e3e3e0 Merge pull request #114 from draios/configurable-rules-filename
Install falco rules with configurable filename.
2016-08-17 14:44:13 -07:00
Mark Stemm
34fcce7c26 Install falco rules with configurable filename.
New variable FALCO_RULES_DEST_FILENAME allows the rules file to be
installed with a different filename. Not set in the falco repo, but in
the agent repo it's installed as falco_rules.default.yaml.
2016-08-17 13:24:25 -07:00
Mark Stemm
822770a154 Merge pull request #113 from draios/add-event-simulator
Add event simulator
2016-08-12 15:05:39 -07:00
Mark Stemm
65f3725e76 Improve ruleset based on falco event-generator.
Improve ruleset after using with falco event_generator:

 - Instead of assuming all shells are bash, add a list shell_binaries
   and macro shell_procs, and replace references to bash with
   shell_procs. This revealed some other programs that can spawn shells.

 - Add "login" as an interactive command. systemd-login isn't in alpine
   linux, which is the linux distro used for the container.

 - Move read_sensitive_file_untrusted before
   read_sensitive_file_trusted_after_startup, so it can hit first.
2016-08-12 14:28:07 -07:00
Mark Stemm
6e1f23b9a5 Program/docker image that performs bad activities.
C++ program that performs bad activities related to the current falco
ruleset. There are configurable actions for almost all of the current
ruleset, via the --action argument.

By default runs in a loop forever. Can be overridden via --once.

Also add a Dockerfile that compiles event_generator.cpp within an alpine
linux image and copies it to /usr/local/bin. This image has been pushed
to docker hub as "sysdig/falco-event-generator:latest".

Add a Makefile that runs the right docker build command.
2016-08-12 14:27:57 -07:00
Mark Stemm
2aa8a5c114 Merge pull request #112 from draios/fix-addl-false-positives
Eliminate FPs.
2016-08-10 15:55:12 -07:00
Mark Stemm
39ae7680a7 Handle dbus-daemon-launch-helper.
It starts dbus-daemon. Process names are truncated, though, so use
dbus-daemon-lau.
2016-08-10 14:15:26 -07:00
Mark Stemm
12391ee508 Eliminate FPs.
Docker 1.12 split docker into docker and dockerd, so add dockerd as a
docker binary. Also be consistent about using docker_binares instead of
just references to docker.

Also add ldconfig as a program that can write to files below /etc.
2016-08-10 13:58:56 -07:00
Mark Stemm
dcaeebda77 Merge pull request #103 from draios/falco-engine
Falco engine
2016-08-10 10:50:09 -07:00
Mark Stemm
f1748060c5 Add tests for multiple files, disabled rules.
Add test that cover reading from multiple sets of rule files and
disabling rules. Specific changes:

 - Modify falco to allow multiple -r arguments to read from multiple
   files.
 - In the test multiplex file, add a disabled_rules attribute,
   containing a sequence of rules to disable. Result in -D arguments
   when running falco.
 - In the test multiplex file, 'rules_file' can be a sequence. It
   results in multiple -r arguments when running falco.
 - In the test multiplex file, 'detect_level' can be a squence of
   multiple severity levels. All levels will be checked for in the
   output.
 - Move all test rules files to a rules subdirectory and all trace files
   to a traces subdirectory.
 - Add a small trace file for a simple cat of /dev/null. Used by the
   new tests.
 - Add the following new tests:
     - Reading from multiple files, with the first file being
       empty. Ensure that the rules from the second file are properly
       loaded.
     - Reading from multiple files with the last being empty. Ensures
       that the empty file doesn't overwrite anything from the first
       file.
     - Reading from multiple files with varying severity levels for each
       rule. Ensures that both files are properly read.
     - Disabling rules from a rules file, both with full rule names
       and regexes. Will result in not detecting anything.
2016-08-10 10:20:04 -07:00
Mark Stemm
09405e4fad Add configurable event dropping for falco engine.
Add the ability to drop events at the falco engine level in a way that
can scale with the dropping that already occurs at the kernel/inspector
level.

New inline function should_drop_evt() controls whether or not events are
matched against the set of rules, and is controlled by two
values--sampling ratio and sampling multiplier.

Here's how the sampling ratio and multiplier influence whether or not an
event is dropped in should_drop_evt(). The intent is that
m_sampling_ratio is generally changing external to the engine e.g. in
the main inspector class based on how busy the inspector is. A sampling
ratio implies no dropping. Values > 1 imply increasing levels of
dropping. External to the engine, the sampling ratio results in events
being dropped at the kernel/inspector interface.  The sampling
multiplier is an amplification to the sampling factor in
m_sampling_ratio. If 0, no additional events are dropped other than
those that might be dropped by the kernel/inspector interface. If 1,
events that make it past the kernel module are subject to an additional
level of dropping at the falco engine, scaling with the sampling ratio
in m_sampling_ratio.

Unlike the dropping that occurs at the kernel level, where the events in
the first part of each second are dropped, this dropping is random.
2016-08-10 08:44:42 -07:00
Mark Stemm
b1857eff35 Move falco engine to its own library.
Move the c++ and lua code implementing falco engine/falco common to its
own directory userspace/engine. It's compiled as a static library
libfalco_engine.a, and has its own CMakeLists.txt so it can be included
by other projects.

The engine's CMakeLists.txt has a add_subdirectory for the falco rules
directory, so including the engine also builds the rules.

The variables you need to set to use the engine's CMakeLists.txt are:

- CMAKE_INSTALL_PREFIX: the root directory below which everything is
  installed.
- FALCO_ETC_DIR: where to install the rules file.
- FALCO_SHARE_DIR: where to install lua code, relative to the
- install/package root.
- LUAJIT_INCLUDE: where to find header files for lua.
- FALCO_SINSP_LIBRARY: the library containing sinsp code. It will be
- considered a dependency of the engine.
- LPEG_LIB/LYAML_LIB/LIBYAML_LIB: locations for third-party libraries.
- FALCO_COMPONENT: if set, will be included as a part of any install()
  commands.

Instead of specifying /usr/share/falco in config_falco_*.h.in, use
CMAKE_INSTALL_PREFIX and FALCO_SHARE_DIR.

The lua code for the engine has also moved, so the two lua source
directories (userspace/engine/lua and userspace/falco/lua) need to be
available separately via falco_common, so make it an argument to
falco_common::init.

As a part of making it easy to include in another project, also clean up
LPEG build/defs. Modify build-lpeg to add a PREFIX argument to allow for
object files/libraries being in an alternate location, and when building
lpeg, put object files in a build/ subdirectory.
2016-08-10 08:44:42 -07:00
Mark Stemm
fc9690b1d3 Create embeddable falco engine.
Create standalone classes falco_engine/falco_outputs that can be
embedded in other programs. falco_engine is responsible for matching
events against rules, and falco_output is responsible for formatting an
alert string given an event and writing the alert string to all
configured outputs.

falco_engine's main interfaces are:

 - load_rules/load_rules_file: Given a path to a rules file or a string
   containing a set of rules, load the rules. Also loads needed lua code.
 - process_event(): check the event against the set of rules and return
   the results of a match, if any.
 - describe_rule(): print details on a specific rule or all rules.
 - print_stats(): print stats on the rules that matched.
 - enable_rule(): enable/disable any rules matching a pattern. New falco
   command line option -D allows you to disable one or more rules on the
   command line.

falco_output's main interfaces are:
 - init(): load needed lua code.
 - add_output(): add an output channel for alert notifications.
 - handle_event(): given an event that matches one or more rules, format
   an alert message and send it to any output channels.

Each of falco_engine/falco_output maintains a separate lua state and
loads separate sets of lua files. The code to create and initialize the
lua state is in a base class falco_common.

falco_engine no longer logs anything. In the case of errors, it throws
exceptions. falco_logger is now only used as a logging mechanism for
falco itself and as an output method for alert messages. (This should
really probably be split, but it's ok for now).

falco_engine contains an sinsp_evttype_filter object containing the set
of eventtype filters. Instead of calling
m_inspector->add_evttype_filter() to add a filter created by the
compiler, call falco_engine::add_evttype_filter() instead. This means
that the inspector runs with a NULL filter and all events are returned
from do_inspect. This depends on
https://github.com/draios/sysdig/pull/633 which has a wrapper around a
set of eventtype filters.

Some additional changes along with creating these classes:

- Some cleanups of unnecessary header files, cmake include_directory()s,
  etc to only include necessary includes and only include them in header
  files when required.

- Try to avoid 'using namespace std' in header files, or assuming
  someone else has done that. Generally add 'using namespace std' to all
  source files.

- Instead of using sinsp_exception for all errors, define a
  falco_engine_exception class for exceptions coming from the falco
  engine and use it instead. For falco program code, switch to general
  exceptions under std::exception and catch + display an error for all
  exceptions, not just sinsp_exceptions.

- Remove fields.{cpp,h}. This was dead code.

- Start tracking counts of rules by priority string (i.e. what's in the
  falco rules file) as compared to priority level (i.e. roughtly
  corresponding to a syslog level). This keeps the rule processing and
  rule output halves separate. This led to some test changes. The regex
  used in the test is now case insensitive to be a bit more flexible.

- Now that https://github.com/draios/sysdig/pull/632 is merged, we can
  delete the rules object (and its lua_parser) safely.

- Move loading the initial lua script to the constructor. Otherwise,
  calling load_rules() twice re-loads the lua script and throws away any
  state like the mapping from rule index to rule.

- Allow an empty rules file.

Finally, fix most memory leaks found by valgrind:

 - falco_configuration wasn't deleting the allocated m_config yaml
   config.
 - several ifstreams were being created simply to test which falco
   config file to use.
 - In the lua output methods, an event formatter was being created using
   falco.formatter() but there was no corresponding free_formatter().

This depends on changes in https://github.com/draios/sysdig/pull/640.
2016-08-10 08:43:53 -07:00
Mark Stemm
03e6c1b3d9 Merge pull request #111 from draios/update-nodejs-example
Don't run the spawned program in a shell.
2016-08-09 11:00:07 -07:00
Mark Stemm
bf431cf222 Don't run the spawned program in a shell.
Instead, run it directly. This avoids false positives when running
non-bash commands and false negatives when trying to run a shell.
2016-08-09 10:32:40 -07:00
Mark Stemm
b57eb8659f Add ignores for test-related files.
Ignore results.json and similar names. Also ignore the file created when
running phoronix tests.
2016-08-09 08:18:42 -07:00
Mark Stemm
f82288f373 Merge pull request #110 from draios/fix-docker-build
Fix docker builds.
2016-08-05 18:16:59 -07:00
Mark Stemm
a769373bb8 Fix docker builds.
gnupg2 is missing on latest debian:unstable.
2016-08-05 17:51:54 -07:00
20 changed files with 418 additions and 115 deletions

View File

@@ -99,6 +99,7 @@ else()
CONFIGURE_COMMAND ./configure --disable-maintainer-mode --enable-all-static --disable-dependency-tracking
BUILD_COMMAND ${CMD_MAKE} LDFLAGS=-all-static
BUILD_IN_SOURCE 1
PATCH_COMMAND wget -O jq-1.5-fix-tokenadd.patch https://github.com/stedolan/jq/commit/8eb1367ca44e772963e704a700ef72ae2e12babd.patch && patch -i jq-1.5-fix-tokenadd.patch
INSTALL_COMMAND "")
endif()
@@ -204,8 +205,8 @@ else()
message(STATUS "Using bundled openssl in '${OPENSSL_BUNDLE_DIR}'")
ExternalProject_Add(openssl
URL "http://download.draios.com/dependencies/openssl-1.0.2d.tar.gz"
URL_MD5 "38dd619b2e77cbac69b99f52a053d25a"
URL "http://download.draios.com/dependencies/openssl-1.0.2j.tar.gz"
URL_MD5 "96322138f0b69e61b7212bc53d5e912b"
CONFIGURE_COMMAND ./config shared --prefix=${OPENSSL_INSTALL_DIR}
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1
@@ -235,8 +236,8 @@ else()
ExternalProject_Add(curl
DEPENDS openssl
URL "http://download.draios.com/dependencies/curl-7.45.0.tar.bz2"
URL_MD5 "62c1a352b28558f25ba6209214beadc8"
URL "http://download.draios.com/dependencies/curl-7.52.1.tar.bz2"
URL_MD5 "dd014df06ff1d12e173de86873f9f77a"
CONFIGURE_COMMAND ./configure ${CURL_SSL_OPTION} --disable-shared --enable-optimize --disable-curldebug --disable-rt --enable-http --disable-ftp --disable-file --disable-ldap --disable-ldaps --disable-rtsp --disable-telnet --disable-tftp --disable-pop3 --disable-imap --disable-smb --disable-smtp --disable-gopher --disable-sspi --disable-ntlm-wb --disable-tls-srp --without-winssl --without-darwinssl --without-polarssl --without-cyassl --without-nss --without-axtls --without-ca-path --without-ca-bundle --without-libmetalink --without-librtmp --without-winidn --without-libidn --without-nghttp2 --without-libssh2
BUILD_COMMAND ${CMD_MAKE}
BUILD_IN_SOURCE 1

View File

@@ -26,10 +26,10 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
gcc-5 \
gcc-4.9 && rm -rf /var/lib/apt/lists/*
# Terribly terrible hacks: since our base Debian image ships with GCC 5.0 which breaks older kernels,
# revert the default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
# makefile is hardcoded for gcc-4.6 or so (e.g. Debian Wheezy), we pretend to have gcc 4.6/4.7 by symlinking
# it to 4.9
# Since our base Debian image ships with GCC 5.0 which breaks older kernels, revert the
# default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
# makefile is hardcoded for gcc-4.6 or so (e.g. Debian Wheezy), we pretend to have gcc 4.6/4.7
# by symlinking it to 4.9
RUN rm -rf /usr/bin/gcc \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc \

50
docker/local/Dockerfile Normal file
View File

@@ -0,0 +1,50 @@
FROM debian:unstable
MAINTAINER Sysdig <support@sysdig.com>
ENV FALCO_VERSION 0.1.1dev
LABEL RUN="docker run -i -t -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro --name NAME IMAGE"
ENV SYSDIG_HOST_ROOT /host
ENV HOME /root
RUN cp /etc/skel/.bashrc /root && cp /etc/skel/.profile /root
ADD http://download.draios.com/apt-draios-priority /etc/apt/preferences.d/
RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources.list.d/jessie.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
bash-completion \
curl \
jq \
gnupg2 \
ca-certificates \
gcc \
gcc-5 \
gcc-4.9 \
sysdig && rm -rf /var/lib/apt/lists/*
# Since our base Debian image ships with GCC 5.0 which breaks older kernels, revert the
# default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
# makefile is hardcoded for gcc-4.6 or so (e.g. Debian Wheezy), we pretend to have gcc 4.6/4.7
# by symlinking it to 4.9
RUN rm -rf /usr/bin/gcc \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc-4.8 \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc-4.7 \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc-4.6
RUN ln -s $SYSDIG_HOST_ROOT/lib/modules /lib/modules
ADD falco-${FALCO_VERSION}-x86_64.deb /
RUN dpkg -i /falco-${FALCO_VERSION}-x86_64.deb
COPY ./docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["/usr/bin/falco"]

View File

@@ -0,0 +1,17 @@
#!/bin/bash
#set -e
# Set the SYSDIG_SKIP_LOAD variable to skip loading the sysdig kernel module
if [[ -z "${SYSDIG_SKIP_LOAD}" ]]; then
echo "* Setting up /usr/src links from host"
for i in $(ls $SYSDIG_HOST_ROOT/usr/src)
do
ln -s $SYSDIG_HOST_ROOT/usr/src/$i /usr/src/$i
done
/usr/bin/sysdig-probe-loader
fi
exec "$@"

View File

@@ -26,10 +26,10 @@ RUN echo "deb http://httpredir.debian.org/debian jessie main" > /etc/apt/sources
gcc-5 \
gcc-4.9 && rm -rf /var/lib/apt/lists/*
# Terribly terrible hacks: since our base Debian image ships with GCC 5.0 which breaks older kernels,
# revert the default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
# makefile is hardcoded for gcc-4.6 or so (e.g. Debian Wheezy), we pretend to have gcc 4.6/4.7 by symlinking
# it to 4.9
# Since our base Debian image ships with GCC 5.0 which breaks older kernels, revert the
# default to gcc-4.9. Also, since some customers use some very old distributions whose kernel
# makefile is hardcoded for gcc-4.6 or so (e.g. Debian Wheezy), we pretend to have gcc 4.6/4.7
# by symlinking it to 4.9
RUN rm -rf /usr/bin/gcc \
&& ln -s /usr/bin/gcc-4.9 /usr/bin/gcc \

View File

@@ -102,7 +102,7 @@
items: [docker, dockerd, exe]
- list: k8s_binaries
items: [hyperkube, skydns, kube2sky]
items: [hyperkube, skydns, kube2sky, exechealthz]
- list: http_server_binaries
items: [nginx, httpd, httpd-foregroun, lighttpd]
@@ -116,7 +116,11 @@
# The truncated dpkg-preconfigu is intentional, process names are
# truncated at the sysdig level.
- list: package_mgmt_binaries
items: [dpkg, dpkg-preconfigu, dnf, rpm, rpmkey, yum, frontend]
items: [
dpkg, dpkg-preconfigu, dnf, rpm, rpmkey, yum, frontend,
apt, apt-get, apt-add-reposit, apt-auto-remova, apt-key,
preinst
]
- macro: package_mgmt_procs
condition: proc.name in (package_mgmt_binaries)
@@ -190,6 +194,22 @@
- macro: system_users
condition: user.name in (bin, daemon, games, lp, mail, nobody, sshd, sync, uucp, www-data)
# SPECIAL NOTE: This macro eliminates false positives that result from
# running python scripts as a part of ansible. However, the condition
# that the command line contains "ansible" is very
# permissive. Ideally, you should change this macro to explicitly
# scope the python scripts to a specific directory (namely, your
# configured remote_tmp directory).
- macro: ansible_running_python
condition: proc.pname in (python, pypy) and proc.pcmdline contains ansible
- macro: python_running_denyhosts
condition: proc.pname=python and proc.pcmdline contains /usr/sbin/denyhosts
# As a part of kernel upgrades, dpkg will spawn a perl script with the
# name linux-image-N.N. This macro matches that.
- macro: parent_linux_image_upgrade_script
condition: proc.pname startswith linux-image-
###############
# General Rules
@@ -204,9 +224,15 @@
- macro: write_etc_common
condition: >
etc_dir and evt.dir = < and open_write
and not proc.name in (shadowutils_binaries, sysdigcloud_binaries, package_mgmt_binaries, ssl_mgmt_binaries, dhcp_binaries, ldconfig.real, ldconfig)
and not proc.name in (passwd_binaries, shadowutils_binaries, sysdigcloud_binaries,
package_mgmt_binaries, ssl_mgmt_binaries, dhcp_binaries,
ldconfig.real, ldconfig, confd, gpg, insserv,
apparmor_parser, update-mime, tzdata.config, tzdata.postinst,
systemd-machine, debconf-show)
and not proc.pname in (sysdigcloud_binaries)
and not fd.directory in (/etc/cassandra, /etc/ssl/certs/java)
and not ansible_running_python
and not python_running_denyhosts
- rule: Write below etc
desc: an attempt to write to any file below /etc, not in a pipe installer session
@@ -221,15 +247,26 @@
output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name) within pipe installer session"
priority: INFO
- macro: cmp_cp_by_passwd
condition: proc.name in (cmp, cp) and proc.pname=passwd
- rule: Read sensitive file trusted after startup
desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information) by a trusted program after startup. Trusted programs might read these files at startup to load initial state, but not afterwards.
condition: sensitive_files and open_read and server_procs and not proc_is_new and proc.name!="sshd"
output: "Sensitive file opened for reading by trusted program after startup (user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING
- list: read_sensitive_file_binaries
items: [iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, sshd, vsftpd]
- rule: Read sensitive file untrusted
desc: an attempt to read any sensitive file (e.g. files containing user/password/authentication information). Exceptions are made for known trusted programs.
condition: sensitive_files and open_read and not proc.name in (user_mgmt_binaries, userexec_binaries, package_mgmt_binaries, cron_binaries, iptables, ps, lsb_release, check-new-relea, dumpe2fs, accounts-daemon, shell_binaries, sshd) and not proc.cmdline contains /usr/bin/mandb
condition: >
sensitive_files and open_read
and not proc.name in (user_mgmt_binaries, userexec_binaries, package_mgmt_binaries, cron_binaries, read_sensitive_file_binaries, shell_binaries)
and not cmp_cp_by_passwd
and not ansible_running_python
and not proc.cmdline contains /usr/bin/mandb
output: "Sensitive file opened for reading by non-trusted program (user=%user.name name=%proc.name command=%proc.cmdline file=%fd.name)"
priority: WARNING
@@ -280,14 +317,36 @@
output: "Namespace change (setns) by unexpected program (user=%user.name command=%proc.cmdline parent=%proc.pname %container.info)"
priority: WARNING
- list: known_shell_spawn_binaries
items: [
sshd, sudo, su, tmux, screen, emacs, systemd, login, flock, fbash,
nginx, monit, supervisord, dragent, aws, initdb, docker-compose,
make, configure, awk, falco, fail2ban-server, fleetctl,
logrotate, ansible, less, adduser, pycompile, py3compile,
pyclean, py3clean, pip, pip2, ansible-playboo, man-db,
init, pluto, mkinitramfs, unattended-upgr, watch, sysdig,
landscape-sysin, nessusd
]
- rule: Run shell untrusted
desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries.
condition: spawned_process and not container and shell_procs and proc.pname exists and not proc.pname in (cron_binaries, shell_binaries, sshd, sudo, docker_binaries, k8s_binaries, su, tmux, screen, emacs, systemd, login, flock, fbash, nginx, monit, supervisord, dragent, aws, initdb, docker-compose, make, configure, awk, falco, fail2ban-server, apt-get, apt)
output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
condition: >
spawned_process and not container
and shell_procs
and proc.pname exists
and not proc.pname in (cron_binaries, shell_binaries, known_shell_spawn_binaries, docker_binaries, k8s_binaries, package_mgmt_binaries)
and not ansible_running_python
and not parent_linux_image_upgrade_script
output: "Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline pcmdline=%proc.pcmdline)"
priority: WARNING
- macro: trusted_containers
condition: (container.image startswith sysdig/agent or container.image startswith sysdig/falco or container.image startswith sysdig/sysdig or container.image startswith gcr.io/google_containers/hyperkube or container.image startswith gcr.io/google_containers/kube-proxy)
condition: (container.image startswith sysdig/agent or
container.image startswith sysdig/falco or
container.image startswith sysdig/sysdig or
container.image startswith gcr.io/google_containers/hyperkube or
container.image startswith gcr.io/google_containers/kube-proxy or
container.image startswith cchh/sysdig)
- rule: File Open by Privileged Container
desc: Any open by a privileged container. Exceptions are made for known trusted images.
@@ -317,7 +376,7 @@
- rule: Run shell in container
desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded.
condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, initdb, pg_ctl, awk, apache2, falco, cron)
condition: spawned_process and container and shell_procs and proc.pname exists and not proc.pname in (shell_binaries, docker_binaries, k8s_binaries, initdb, pg_ctl, awk, apache2, falco, cron) and not trusted_containers
output: "Shell spawned in a container other than entrypoint (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline)"
priority: WARNING
@@ -350,10 +409,13 @@
output: "User management binary command run outside of container (user=%user.name command=%proc.cmdline parent=%proc.pname)"
priority: WARNING
- list: allowed_dev_files
items: [/dev/null, /dev/stdin, /dev/stdout, /dev/stderr, /dev/tty, /dev/random, /dev/urandom, /dev/console]
# (we may need to add additional checks against false positives, see: https://bugs.launchpad.net/ubuntu/+source/rkhunter/+bug/86153)
- rule: Create files below dev
desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
condition: fd.directory = /dev and (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT)) and proc.name != blkid and not fd.name in (/dev/null,/dev/stdin,/dev/stdout,/dev/stderr,/dev/tty)
condition: fd.directory = /dev and (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT)) and proc.name != blkid and not fd.name in (allowed_dev_files)
output: "File created below /dev by untrusted program (user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING

View File

@@ -17,6 +17,7 @@ class FalcoTest(Test):
"""
self.falcodir = self.params.get('falcodir', '/', default=os.path.join(self.basedir, '../build'))
self.stdout_contains = self.params.get('stdout_contains', '*', default='')
self.stderr_contains = self.params.get('stderr_contains', '*', default='')
self.exit_status = self.params.get('exit_status', '*', default=0)
self.should_detect = self.params.get('detect', '*', default=False)
@@ -204,6 +205,11 @@ class FalcoTest(Test):
if match is None:
self.fail("Stderr of falco process did not contain content matching {}".format(self.stderr_contains))
if self.stdout_contains != '':
match = re.search(self.stdout_contains, res.stdout)
if match is None:
self.fail("Stdout of falco process '{}' did not contain content matching {}".format(res.stdout, self.stdout_contains))
if res.exit_status != self.exit_status:
self.error("Falco command \"{}\" exited with unexpected return value {} (!= {})".format(
cmd, res.exit_status, self.exit_status))

View File

@@ -95,6 +95,34 @@ trace_files: !mux
- rules/double_rule.yaml
trace_file: trace_files/cat_write.scap
multiple_rules_overriding:
detect: False
rules_file:
- rules/single_rule.yaml
- rules/override_rule.yaml
trace_file: trace_files/cat_write.scap
macro_overriding:
detect: False
rules_file:
- rules/single_rule.yaml
- rules/override_macro.yaml
trace_file: trace_files/cat_write.scap
list_overriding:
detect: False
rules_file:
- rules/single_rule.yaml
- rules/override_list.yaml
trace_file: trace_files/cat_write.scap
nested_list_overriding:
detect: False
rules_file:
- rules/single_rule.yaml
- rules/override_nested_list.yaml
trace_file: trace_files/cat_write.scap
invalid_rule_output:
exit_status: 1
stderr_contains: "Runtime error: Error loading rules:.* Invalid output format 'An open was seen %not_a_real_field': 'invalid formatting token not_a_real_field'. Exiting."
@@ -126,6 +154,14 @@ trace_files: !mux
- rules/single_rule_enabled_flag.yaml
trace_file: trace_files/cat_write.scap
null_output_field:
detect: True
detect_level: WARNING
rules_file:
- rules/null_output_field.yaml
trace_file: trace_files/cat_write.scap
stdout_contains: "Warning An open was seen .cport=<NA> command=cat /dev/null."
file_output:
detect: True
detect_level: WARNING

View File

@@ -0,0 +1,5 @@
- rule: open_from_cat
desc: A process named cat does an open
condition: evt.type=open and proc.name=cat
output: "An open was seen (cport=%fd.cport command=%proc.cmdline)"
priority: WARNING

View File

@@ -0,0 +1,2 @@
- list: cat_capable_binaries
items: [not-cat]

View File

@@ -0,0 +1,2 @@
- macro: is_cat
condition: proc.name in (not-cat)

View File

@@ -0,0 +1,2 @@
- list: cat_binaries
items: [not-cat]

View File

@@ -0,0 +1,5 @@
- rule: open_from_cat
desc: A process named cat does an open
condition: evt.type=open and proc.name=not-cat
output: "An open was seen (command=%proc.cmdline)"
priority: WARNING

View File

@@ -1,5 +1,11 @@
- list: cat_binaries
items: [cat]
- list: cat_capable_binaries
items: [cat_binaries]
- macro: is_cat
condition: proc.name=cat
condition: proc.name in (cat_capable_binaries)
- rule: open_from_cat
desc: A process named cat does an open

View File

@@ -49,6 +49,8 @@ falco_engine::falco_engine(bool seed_rng)
falco_common::init(m_lua_main_filename.c_str(), FALCO_ENGINE_SOURCE_LUA_DIR);
falco_rules::init(m_ls);
m_evttype_filter.reset(new sinsp_evttype_filter());
if(seed_rng)
{
srandom((unsigned) getpid());
@@ -107,7 +109,7 @@ void falco_engine::load_rules_file(const string &rules_filename, bool verbose, b
void falco_engine::enable_rule(string &pattern, bool enabled)
{
m_evttype_filter.enable(pattern, enabled);
m_evttype_filter->enable(pattern, enabled);
}
unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
@@ -118,7 +120,7 @@ unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev)
return unique_ptr<struct rule_result>();
}
if(!m_evttype_filter.run(ev))
if(!m_evttype_filter->run(ev))
{
return unique_ptr<struct rule_result>();
}
@@ -183,7 +185,12 @@ void falco_engine::add_evttype_filter(string &rule,
list<uint32_t> &evttypes,
sinsp_filter* filter)
{
m_evttype_filter.add(rule, evttypes, filter);
m_evttype_filter->add(rule, evttypes, filter);
}
void falco_engine::clear_filters()
{
m_evttype_filter.reset(new sinsp_evttype_filter());
}
void falco_engine::set_sampling_ratio(uint32_t sampling_ratio)

View File

@@ -19,6 +19,7 @@ along with falco. If not, see <http://www.gnu.org/licenses/>.
#pragma once
#include <string>
#include <memory>
#include "sinsp.h"
#include "filter.h"
@@ -84,6 +85,9 @@ public:
list<uint32_t> &evttypes,
sinsp_filter* filter);
// Clear all existing filters.
void clear_filters();
//
// Set the sampling ratio, which can affect which events are
// matched against the set of rules.
@@ -116,7 +120,7 @@ private:
inline bool should_drop_evt();
falco_rules *m_rules;
sinsp_evttype_filter m_evttype_filter;
std::unique_ptr<sinsp_evttype_filter> m_evttype_filter;
//
// Here's how the sampling ratio and multiplier influence

View File

@@ -121,7 +121,16 @@ end
-- object. The by_name index is used for things like describing rules,
-- and the by_idx index is used to map the relational node index back
-- to a rule.
local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, n_rules=0, rules_by_idx={}}
local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, macros_by_name={}, lists_by_name={},
n_rules=0, rules_by_idx={}, ordered_rule_names={}, ordered_macro_names={}, ordered_list_names={}}
local function reset_rules(rules_mgr)
falco_rules.clear_filters(rules_mgr)
state.n_rules = 0
state.rules_by_idx = {}
state.macros = {}
state.lists = {}
end
-- From http://lua-users.org/wiki/TableUtils
--
@@ -178,34 +187,42 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
error("Rules content \""..rules_content.."\" is not yaml")
end
for i,v in ipairs(rules) do -- iterate over yaml list
-- Iterate over yaml list. In this pass, all we're doing is
-- populating the set of rules, macros, and lists. We're not
-- expanding/compiling anything yet. All that will happen in a
-- second pass
for i,v in ipairs(rules) do
if (not (type(v) == "table")) then
error ("Unexpected element of type " ..type(v)..". Each element should be a yaml associative array.")
end
if (v['macro']) then
local ast = compiler.compile_macro(v['condition'], state.lists)
state.macros[v['macro']] = ast.filter.value
if state.macros_by_name[v['macro']] == nil then
state.ordered_macro_names[#state.ordered_macro_names+1] = v['macro']
end
elseif (v['list']) then
-- list items are represented in yaml as a native list, so no
-- parsing necessary
local items = {}
-- List items may be references to other lists, so go through
-- the items and expand any references to the items in the list
for i, item in ipairs(v['items']) do
if (state.lists[item] == nil) then
items[#items+1] = item
else
for i, exp_item in ipairs(state.lists[item]) do
items[#items+1] = exp_item
end
for i, field in ipairs({'condition'}) do
if (v[field] == nil) then
error ("Missing "..field.." in macro with name "..v['macro'])
end
end
state.lists[v['list']] = items
state.macros_by_name[v['macro']] = v
elseif (v['list']) then
if state.lists_by_name[v['list']] == nil then
state.ordered_list_names[#state.ordered_list_names+1] = v['list']
end
for i, field in ipairs({'items'}) do
if (v[field] == nil) then
error ("Missing "..field.." in list with name "..v['list'])
end
end
state.lists_by_name[v['list']] = v
elseif (v['rule']) then
@@ -219,82 +236,132 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
end
end
-- Note that we can overwrite rules, but the rules are still
-- loaded in the order in which they first appeared,
-- potentially across multiple files.
if state.rules_by_name[v['rule']] == nil then
state.ordered_rule_names[#state.ordered_rule_names+1] = v['rule']
end
state.rules_by_name[v['rule']] = v
local filter_ast, evttypes = compiler.compile_filter(v['rule'], v['condition'],
state.macros, state.lists)
if (filter_ast.type == "Rule") then
state.n_rules = state.n_rules + 1
state.rules_by_idx[state.n_rules] = v
-- Store the index of this formatter in each relational expression that
-- this rule contains.
-- This index will eventually be stamped in events passing this rule, and
-- we'll use it later to determine which output to display when we get an
-- event.
mark_relational_nodes(filter_ast.filter.value, state.n_rules)
install_filter(filter_ast.filter.value)
-- Pass the filter and event types back up
falco_rules.add_filter(rules_mgr, v['rule'], evttypes)
-- Rule ASTs are merged together into one big AST, with "OR" between each
-- rule.
if (state.filter_ast == nil) then
state.filter_ast = filter_ast.filter.value
else
state.filter_ast = { type = "BinaryBoolOp", operator = "or", left = state.filter_ast, right = filter_ast.filter.value }
end
-- Enable/disable the rule
if (v['enabled'] == nil) then
v['enabled'] = true
end
if (v['enabled'] == false) then
falco_rules.enable_rule(rules_mgr, v['rule'], 0)
end
-- If the format string contains %container.info, replace it
-- with extra. Otherwise, add extra onto the end of the format
-- string.
if string.find(v['output'], "%container.info", nil, true) ~= nil then
-- There may not be any extra, or we're not supposed
-- to replace it, in which case we use the generic
-- "%container.name (id=%container.id)"
if replace_container_info == false then
v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
else
safe_extra = string.gsub(extra, "%%", "%%%%")
v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
end
else
-- Just add the extra to the end
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
end
-- Ensure that the output field is properly formatted by
-- creating a formatter from it. Any error will be thrown
-- up to the top level.
formatter = formats.formatter(v['output'])
formats.free_formatter(formatter)
else
error ("Unexpected type in load_rule: "..filter_ast.type)
end
else
error ("Unknown rule object: "..table.tostring(v))
end
end
-- We've now loaded all the rules, macros, and list. Now
-- compile/expand the rules, macros, and lists. We use
-- ordered_rule_{lists,macros,names} to compile them in the order
-- in which they appeared in the file(s).
reset_rules(rules_mgr)
for i, name in ipairs(state.ordered_list_names) do
local v = state.lists_by_name[name]
-- list items are represented in yaml as a native list, so no
-- parsing necessary
local items = {}
-- List items may be references to other lists, so go through
-- the items and expand any references to the items in the list
for i, item in ipairs(v['items']) do
if (state.lists[item] == nil) then
items[#items+1] = item
else
for i, exp_item in ipairs(state.lists[item]) do
items[#items+1] = exp_item
end
end
end
state.lists[v['list']] = items
end
for i, name in ipairs(state.ordered_macro_names) do
local v = state.macros_by_name[name]
local ast = compiler.compile_macro(v['condition'], state.lists)
state.macros[v['macro']] = ast.filter.value
end
for i, name in ipairs(state.ordered_rule_names) do
local v = state.rules_by_name[name]
local filter_ast, evttypes = compiler.compile_filter(v['rule'], v['condition'],
state.macros, state.lists)
if (filter_ast.type == "Rule") then
state.n_rules = state.n_rules + 1
state.rules_by_idx[state.n_rules] = v
-- Store the index of this formatter in each relational expression that
-- this rule contains.
-- This index will eventually be stamped in events passing this rule, and
-- we'll use it later to determine which output to display when we get an
-- event.
mark_relational_nodes(filter_ast.filter.value, state.n_rules)
install_filter(filter_ast.filter.value)
-- Pass the filter and event types back up
falco_rules.add_filter(rules_mgr, v['rule'], evttypes)
-- Rule ASTs are merged together into one big AST, with "OR" between each
-- rule.
if (state.filter_ast == nil) then
state.filter_ast = filter_ast.filter.value
else
state.filter_ast = { type = "BinaryBoolOp", operator = "or", left = state.filter_ast, right = filter_ast.filter.value }
end
-- Enable/disable the rule
if (v['enabled'] == nil) then
v['enabled'] = true
end
if (v['enabled'] == false) then
falco_rules.enable_rule(rules_mgr, v['rule'], 0)
end
-- If the format string contains %container.info, replace it
-- with extra. Otherwise, add extra onto the end of the format
-- string.
if string.find(v['output'], "%container.info", nil, true) ~= nil then
-- There may not be any extra, or we're not supposed
-- to replace it, in which case we use the generic
-- "%container.name (id=%container.id)"
if replace_container_info == false then
v['output'] = string.gsub(v['output'], "%%container.info", "%%container.name (id=%%container.id)")
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
else
safe_extra = string.gsub(extra, "%%", "%%%%")
v['output'] = string.gsub(v['output'], "%%container.info", safe_extra)
end
else
-- Just add the extra to the end
if extra ~= "" then
v['output'] = v['output'].." "..extra
end
end
-- Ensure that the output field is properly formatted by
-- creating a formatter from it. Any error will be thrown
-- up to the top level.
formatter = formats.formatter(v['output'])
formats.free_formatter(formatter)
else
error ("Unexpected type in load_rule: "..filter_ast.type)
end
end
io.flush()
end
@@ -369,7 +436,10 @@ function on_event(evt_, rule_id)
rule_output_counts.by_name[rule.rule] = rule_output_counts.by_name[rule.rule] + 1
end
return rule.rule, rule.priority, rule.output
-- Prefix output with '*' so formatting is permissive
output = "*"..rule.output
return rule.rule, rule.priority, output
end
function print_stats()

View File

@@ -28,6 +28,7 @@ extern "C" {
#include "falco_engine.h"
const static struct luaL_reg ll_falco_rules [] =
{
{"clear_filters", &falco_rules::clear_filters},
{"add_filter", &falco_rules::add_filter},
{"enable_rule", &falco_rules::enable_rule},
{NULL,NULL}
@@ -44,6 +45,24 @@ void falco_rules::init(lua_State *ls)
luaL_openlib(ls, "falco_rules", ll_falco_rules, 0);
}
int falco_rules::clear_filters(lua_State *ls)
{
if (! lua_islightuserdata(ls, -1))
{
throw falco_exception("Invalid arguments passed to clear_filters()\n");
}
falco_rules *rules = (falco_rules *) lua_topointer(ls, -1);
rules->clear_filters();
return 0;
}
void falco_rules::clear_filters()
{
m_engine->clear_filters();
}
int falco_rules::add_filter(lua_State *ls)
{
if (! lua_islightuserdata(ls, -3) ||

View File

@@ -36,10 +36,12 @@ class falco_rules
void describe_rule(string *rule);
static void init(lua_State *ls);
static int clear_filters(lua_State *ls);
static int add_filter(lua_State *ls);
static int enable_rule(lua_State *ls);
private:
void clear_filters();
void add_filter(string &rule, list<uint32_t> &evttypes);
void enable_rule(string &rule, bool enabled);

View File

@@ -76,6 +76,13 @@ end
function output_event(event, rule, priority, format)
local level = level_of(priority)
-- If format starts with a *, remove it, as we're adding our own
-- prefix here.
if format:sub(1,1) == "*" then
format = format:sub(2)
end
format = "*%evt.time: "..levels[level+1].." "..format
if formatters[rule] == nil then
formatter = formats.formatter(format)