Make changes to rules to improve performance and reduce FPs:
- Rely on https://github.com/draios/sysdig/pull/610 that allows
specifying an open/openat for reading/writing without having to search
through all the flags individually.
- For a two-item list (open, openat), and thinking ahead to
https://github.com/draios/sysdig/pull/624, check the event type
individually instead of as a set membership test, which is a bit
faster.
- Switch to consistently using evt.type instead of syscall.type.
- Move positive tests like etc_dir, bin_dir, sensitive_files,
proc.sname, etc., which are most likely to not succeed, to the
beginning of rules, so they have a greater chance to cause the rest of
the rule to be skipped, which saves time.
- Using exim as a mail program--exim also can suid to root.
- add a new macro for ssl management binaries and allow them to write
below /etc and read sensitive files.
- add a new macro for dhcp client binaries and allow them to write below
/etc.
- Add exe (docker-related program) as a program that can set a namespace
using setns.
- Don't count /dev/tty as an important file under /dev.
https://github.com/draios/sysdig/pull/623 adds support for a startswith
operator to allow for string prefix matching. Modify the parser to
recognize that operator, and use that operator for rules that really
want to check the beginning of a pathname, directory, etc. to make them
faster and avoid FPs.
Once sysdig adds support for handling "in (...)" filter expressions as
set membership tests, it will be advantageous to combine lists of items
together into a single list so they can all be checked in a single set
membership test.
This commit adds support for a new yaml item type "list" containing a
field "name" and field "items" containing a list of items. These are
represented as a yaml list, which allows yaml to handle some of the
initial parsing with the list items maintained natively in lua.
Allow lists to contain list references by expanding any references to
the items in the list, before storing the list items in
state.lists.
When parsing macro or rule conditions, replace all references to a list
name with the list items as a comma separated string.
Modify the falco rules to switch to lists whenever possible. The
new convention is to use the suffix _binaries for lists of program names
and _procs for macros that define a filter expression using the list.
Simple docker-compose environment that starts a simple express server
with a poorly-designed /api/exec/<cmd> endpoint that executes arbitrary
commands, and uses falco to detect running bash.
Adding docker-compose based example of man-in-the-middle attack against
installation scripts and how it can be detected using sysdig falco.
The docker-compose environment starts a good web server, compromised
nginx installation, evil web server, and a copy of sysdig falco. The
README walks through the process of compromising a client by using curl
http://localhost/get-software.sh | bash and detecting the compromise
using ./fbash.
The fbash program included in this example fixes https://github.com/draios/falco/issues/46.
Modify falco_test.py to look for a boolean multiplex attribute
'json_output'. If true, examine the lines of the output and for any line
that begins with '{', parse it as json and ensure it has the 4
attributes we expect.
Modify run_regression_tests to have a utility function
prepare_multiplex_fileset that does the work of looping over files in a
directory, along with detect, level, and json output arguments. The
appropriate multiplex attributes are added for each file.
Use that utility function to test json output for the positive and
informational directories along with non-json output. The negative
directory is only tested once.
Instead of using sysdig's json output, which only contains the fields
from the format string without any formatting text, use the string
output to build a json object containing the format string, rule name,
severity, and the event time (converted to a json-friendly ISO8601).
This fixes https://github.com/draios/falco/issues/82.
Add additional rules related to using pipe installers within a fbash
session:
- Modify write_etc to only trigger if *not* in a fbash session. There's
a new rule write_etc_installer which has the same conditions when in
a fbash session, logging at INFO severity.
- A new rule write_rpm_database warns if any non package management
program tries to write below /var/lib/rpm.
- Add a new warning if any program below a fbash session tries to open
an outbound network connection on ports other than http(s) and dns.
- Add INFO level messages when programs in a fbash session try to run
package management binaries (rpm,yum,etc) or service
management (systemctl,chkconfig,etc) binaries.
In order to test these new INFO level rules, make up a third class of
trace files traces-info.zip containing trace files that should result in
info-level messages.
To differentiate warning and info level detection, add an attribute to
the multiplex file "detect_level", which is "Warning" for the files in
traces-positive and "Info" for the files in traces-info. Modify
falco_test.py to look specifically for a non-zero count for the given
detect_level.
Doing this exposed a bug in the way the level-specific counts were being
recorded--they were keeping counts by level name, not number. Fix that.
Update fbash rules to use proc.sname instead of proc.aname and to rely
on sessions instead of process ancestors.
I also wanted to add details on the address/port being listened to but
that's blocked on https://github.com/draios/falco/issues/86.
Along with this change, there are new positive trace files
installer-bash-starts-network-server.scap and
installer-bash-starts-session.scap that test these updated rules.
Do another round of rule cleanups now that we have a larger set of
positive and negative trace files to work with. Outside of this commit,
there are now trace files for all the positive rules, a docker-compose
startup and teardown, and some trace files from the sysdig cloud staging
environment.
Also add a script that runs sysdig with a filter that removes all the
syscalls not handled by falco as well as a few other high-volume,
low-information syscalls. This script was used to create the staging
environment trace files.
Notable rule changes:
- The direction for write_binary_dir/write_etc needs to be exit instead
of enter, as the bin_dir clause works on the file descriptor returned
by the open/openat call.
- Add login as a trusted binary that can read sensitive files (occurs
for direct console logins).
- sshd can read sensitive files well after startup, so exclude it from
the set of binaries that can trigger
read_sensitive_file_trusted_after_startup.
- limit run_shell_untrusted to non-containers.
- Disable the ssh_error_syslog rule for now. With the current
restriction on system calls (no read/write/sendto/recvfrom/etc), you
won't see the ssh error messages. Nevertheless, add a string to look
for to indicate ssh errors and add systemd's true location for the
syslog device.
- Sshd attemps to setuid even when it's not running as root, so exclude
it from the set of binaries to monitor for now.
- Let programs that are direct decendants of systemd spawn user
management tasks for now.
- Temporarily disable the EACCESS rule. This rule is exposing a bug in
sysdig in debug mode, https://github.com/draios/sysdig/issues/598. The
rule is also pretty noisy so I'll keep it disabled until the sysdig bug
is fixed.
- The etc_dir and bin_dir macros both have the problem that they match
pathnames with /etc/, /bin/, etc in the middle of the path, as sysdig
doesn't have a "begins with" comparison. Add notes for that.
- Change spawn_process to spawned_process to indicate that it's for the
exit side of the execve. Also use it in a few places that were
looking for the same conditions without any macro.
- Get rid of adduser_binaries and fold any programs not already present
into shadowutils_binaries.
- Add new groups sysdigcloud_binaries and sysdigcloud_binaries_parent
and add them as exceptions for write_etc/write_binary_dir.
- Add yum as a package management binary and add it as an exception to
write_etc/write_binary_dir.
- Change how db_program_spawned_process works. Since all of the useful
information is on the exit side of the event, you can't really add a
condition based on the process being new. Isntead, have the rule
check for a non-database-related program being spawned by a
database-related program.
- Allow dragent to run shells.
- Add sendmail, sendmail-msp as a program that attempts to setuid.
- Some of the *_binaries macros that were based on dpkg -L accidentally
contained directories in addition to end files. Trim those.
- Add systemd-logind as a login_binary.
- Add unix_chkpwd as a shadowutils_binary.
- Add parentheses around any macros that group items using or. I found
this necessary when the macro is used in the middle of a list of and
conditions.
- Break out system_binaries into a new subset user_mgmt_binaries
containing login_, passwd_, and shadowutils_ binaries. That way you
don't have to pull in all of system_binaries when looking for
sensisitive files or user management activity.
- Rename fs-bash to fbash, thinking ahead to its more likely name.
Start using the Avocado framework for automated regression
testing. Create a test FalcoTest in falco_test.py which can run on a
collection of trace files. The script test/run_regression_tests.sh is
responsible for pulling zip files containing the positive (falco should
detect) and negative (falco should not detect) trace files, creating a
Avocado multiplex file that defines all the tests (one for each trace
file), running avocado on all the trace files, and showing full logs for
any test that didn't pass.
The old regression script, which simply ran falco, has been removed.
Modify falco's stats output to show the total number of events detected
for use in the tests.
In travis.yml, pull a known stable version of avocado and build it,
including installing any dependencies, as a part of the build process.
At shutdown, print stats on the number of rules triggered by severity
and rule name. This is done by a lua function print_stats and the
associated table rule_output_counts.
When passing rules to outputs, update the counts in rule_output_counts.
Add signal handlers for SIGINT/SIGTERM that set a shutdown
flag. Initialize the live inspector with a timeout so the main loop can
watch the flag set by the signal handlers.
Make sure that references to variables that may be paths (which in turn
may contain spaces) are quoted, so cmake won't break on the spaces.
This fixes https://github.com/draios/falco/issues/79.
