Add a new falco rule "Terminal shell in container" that looks for shells
spawned in a container with an attached terminal. This is similar to the
existing "Run shell in container" rule, but doesn't have as many
exceptions as we expect this to be even less rare.
- Sometimes systemd changes its process name to '(systemd)', probably
for a forked daemon process. Add that version to login_binaries.
- Add sv (part of runit) as a program that can write below /etc.
- Allow all /dev/tty* files by moving /dev/tty from the list to a
"startswith /dev/tty" condition.
Tag the existing ruleset to group tags in a meaningful way. The added
tags are:
- filesystem: the rule relates to reading/writing files
- sofware_mgmt: the rule relates to any software/package management
tool like rpm, dpkg, etc.
- process: the rule relates to starting a new process or changing the
state of a current process.
- database: the rule relates to databases
- host: the rule *only* works outside of containers
- shell: the rule specifically relates to starting shells
- container: the rule *only* works inside containers
- cis: the rule is related to the CIS Docker benchmark.
- users: the rule relates to management of users or changing the
identity of a running process.
- network: the rule relates to network activity
Rules can have multiple tags if they relate to multiple of the
above. Rules do not have to have tags, although all the current rules do.
- Add flanneld as a privileged container.
- Add parentheses grouping around many of the "x running y"
containers. I haven't found this strictly necessary with their
current use in rules, but this ensures they will be isolated when
used.
- Allow denyhosts to spawn shells--it runs iptables to add/remove hosts
from its deny list.
This is a rework of a PR made by @juju4 that had a bunch of additions
related to running other security/monitoring products, including aide,
bro, icinga2, nagios, ansible, etc.
This overlapped a lot with changes I had been making to reduce
noisiness, so rather than have @juju4 deal with the conflicts I took the
changes and made a separate commit with the non-conflicting additions.
A summary of the changes:
- Add docker-compose as a docker binary.
- Add showq/critical-stack as setuid binaries.
- Add lxd binaries
- Add some additional package management binaries.
- Add support for host intrustion detection systems like aide.
- Add support for network intrustion detections systems like bro.
- Add support for monitoring systems like nagios, icinga2, npcd.
- Other one-off additions to other lists of mail/etc programs.
A new trace file falco-event-generator.scap contains the result of
running the falco event generator in docker, via:
docker run --security-opt seccomp=unconfined sysdig/falco-event-generator:latest /usr/local/bin/event_generator --once
Make sure this trace file detects the exact set of events we expect for
each rule. This required adding a new verification method
check_detections_by_rule that finds the per-rule counts and compares
them to the expected counts, which are included in the test description
under the key "detect_counts".
This is the first time a trace file for a test is actually in one of the
downloaded zip files. This means it will be tested twice (one for simple
detect-or-not, once for actual counts).
Adding this test showed a problem with Run shell in container
rule--since sysdig/falco-event-generator startswith sysdig/falco, it was
being treated as a trusted container. Modify the macro
trusted_containers to not allow falco-event-generator to be trusted.
- Add a second possible location for denyhosts
- Add PM2 (http://pm2.keymetrics.io/) as a shell spawner.
- There was a bug in use of ansible_running_python. We actually need
two variants depending on whether ansible is the parent or current
process. parent_ansble_running_python is used for Run shell
untrusted, ansible_running_python is used for other rules.
We had added this image while the changes in
https://github.com/draios/falco/pull/177 made it to everyone. This is in
a release now, so we'll remove it from the rule set.
Several changes to reduce spurious alerts when managing machines via
ansible:
- Add ansible_running_python (that is, ansible-spawned python scripts)
as scripts that can read sensitive files and write below
/etc. Notably this is the user ansible module.
- Also add comments to ansible_running_python suggesting users make it
more strict by specifically naming the root directory for ansible
scripts.
- Add pypy as a python variant that can run ansible-related scripts.
Also other changes to reduce FPs:
- add apt-add-reposit, apt-auto-remova (truncation intentional),
apt-get, apt, apt-key as package management programs, and add package
management binaries to the set of shell spawners. The overlapping
binaries that were in known_shell_spawn_binaries were removed.
- add passwd_binaries, gpg, insserv, apparmor_parser, update-mime,
tzdata.{config,postinst}, systemd-machine, and debconf-show to
the set of binaries that can write below /etc.
- Add vsftpd as a program that can read sensitive files.
- Add additional programs (incl. python support programs like pip,
pycompile) as ones that can spawn shells.
- Allow privileged containers to spawn shells.
- Break out the set of files below /dev that are written to with O_CREAT
into a separate list, and add /dev/random,urandom,console to the list.
- Add python running denyhosts as a program that can write below /etc.
- Also add binaries starting with linux-image- as ones that can spawn
shells. These are perl scripts run as a part of installing
linux-image-N.N packages.
Changes to allow shells spawned by ansible. In general this is actually
pretty difficult--on the remote managed machine, ansible performs
actions simply by running python over ssh without any explicit ansible
helper or command line.
One (weak) hint is that the python scripts being run are usually under a
directory with ansible in the name. So use that as the basis for a macro
ansible_running_python. In turn, that macro is used as a negative
condition for the run shell untrusted rule.
This is a pretty fragile and easily exploited condition, so add a note
to the macro saying so.
Feedback from a falco user:
--
to more findings from last night:
logrotate cronjob (Debian default):
Shell spawned by untrusted binary (user=root shell=sh parent=logrotate cmdline=sh -c invoke-rc.d rsyslog rotate > /dev/null logrotate_script /var/log/syslog)
passwd cronjob (Debian default):
Sensitive file opened for reading by non-trusted program (user=root name=cmp command=cmp -s shadow.bak /etc/shadow file=/etc/shadow)
--
New macro cmp_cp_by_passwd allows cmp/cp to be run by passwd to examine
sensitive files. Add logrotate as a program that can spawn a shell.
Also do some cleanups, moving items to lists and splitting long
single-line conditions into multiple lines.
Add cchh/sysdig as a trusted container. We'll probably remove this once
the next agent release occurs that has the fix
https://github.com/draios/falco/pull/177.
Also reformat to avoid long lines.
Periodically both apt and apt-get will spawn shells to update success timestamps and motd.
falco-CLA-1.0-signed-off-by: Jonathan Coetzee <jon@thancoetzee.com>
SSH'ing into an Ubuntu 16.04 box triggers a bunch of "Sensitive file opened for reading by non-trusted program" errors caused by systemd
falco-CLA-1.0-signed-off-by: Jonathan Coetzee jon@thancoetzee.com
Add google_containers/kube-proxy as a trusted image (can be run
privileged, can mount sensitive filesystems). While our k8s deployments
run kube-proxy via the hyperkube image, evidently it's sometimes run via
its own image.
This is one of the fixes for #156.
Also update the output message for this rule.
Make sure falco doesn't detect the things draios-agent does as
suspicious. It's possible that you might run open source falco alongside
sysdig cloud.
App checks spawned by sysdig cloud binaries might also change namespace,
so also allow children of sysdigcloud binaries to call setns.
Add a new list k8s_binaries and allow those binaries to do things like
setns/spawn shells. It's not the case that all of these binaries
actually do these things, but keeping it as a single list makes
management easier.
Copy handling of -pk/-pm/-pc/-k/-m arguments from sysdig. All of the
relevant code was already in the inspector so that was easy.
The information from k8s/mesos/containers is used in two ways:
- In rule outputs, if the format string contains %container.info, that
is replaced with the value from -pk/-pm/-pc, if one of those options
was provided. If no option was provided, %container.info is replaced
with a generic %container.name (id=%container.id) instead.
- If the format string does not contain %container.info, and one of
-pk/-pm/-pc was provided, that is added to the end of the formatting
string.
- If -p was specified with a general value (i.e. not
kubernetes/mesos/container), the value is simply added to the end and
any %container.info is replaced with the generic value.
The new privileged falco rule was noisy when running kubernetes, which
can run privileged. Add it to the trusted_containers list.
Also eliminate a couple spurious warnings related to spawning shells in
containers.
New rule 'File Open by Privileged Container' triggers when a container
that is running privileged opens a file.
New rule 'Sensitive Mount by Container' triggers when a container that
has a sensitive mount opens a file. Currently, a sensitive mount is a
mount of /proc.
This depends on https://github.com/draios/sysdig/pull/655.
Falco itself spawns a shell when using program notifications, so add
falco to the set of trusted programs. (Also add some other programs like
make, awk, configure, that are run while building).
New variable FALCO_RULES_DEST_FILENAME allows the rules file to be
installed with a different filename. Not set in the falco repo, but in
the agent repo it's installed as falco_rules.default.yaml.
Improve ruleset after using with falco event_generator:
- Instead of assuming all shells are bash, add a list shell_binaries
and macro shell_procs, and replace references to bash with
shell_procs. This revealed some other programs that can spawn shells.
- Add "login" as an interactive command. systemd-login isn't in alpine
linux, which is the linux distro used for the container.
- Move read_sensitive_file_untrusted before
read_sensitive_file_trusted_after_startup, so it can hit first.
Docker 1.12 split docker into docker and dockerd, so add dockerd as a
docker binary. Also be consistent about using docker_binares instead of
just references to docker.
Also add ldconfig as a program that can write to files below /etc.
Move the c++ and lua code implementing falco engine/falco common to its
own directory userspace/engine. It's compiled as a static library
libfalco_engine.a, and has its own CMakeLists.txt so it can be included
by other projects.
The engine's CMakeLists.txt has a add_subdirectory for the falco rules
directory, so including the engine also builds the rules.
The variables you need to set to use the engine's CMakeLists.txt are:
- CMAKE_INSTALL_PREFIX: the root directory below which everything is
installed.
- FALCO_ETC_DIR: where to install the rules file.
- FALCO_SHARE_DIR: where to install lua code, relative to the
- install/package root.
- LUAJIT_INCLUDE: where to find header files for lua.
- FALCO_SINSP_LIBRARY: the library containing sinsp code. It will be
- considered a dependency of the engine.
- LPEG_LIB/LYAML_LIB/LIBYAML_LIB: locations for third-party libraries.
- FALCO_COMPONENT: if set, will be included as a part of any install()
commands.
Instead of specifying /usr/share/falco in config_falco_*.h.in, use
CMAKE_INSTALL_PREFIX and FALCO_SHARE_DIR.
The lua code for the engine has also moved, so the two lua source
directories (userspace/engine/lua and userspace/falco/lua) need to be
available separately via falco_common, so make it an argument to
falco_common::init.
As a part of making it easy to include in another project, also clean up
LPEG build/defs. Modify build-lpeg to add a PREFIX argument to allow for
object files/libraries being in an alternate location, and when building
lpeg, put object files in a build/ subdirectory.
In modify_binary_dirs, move the bin_dir_rename check before modify,
which is just a bunch of evt.type checks and is handled by evttype
filters.
Change create_files_below_dev to put the directory check first.
- Move evt.type checks to the front of rules. This is necessary to avoid
warnings now that event types are automatically extracted during rule
parsing and used to bind each rule with a specific set of events.
- Explicitly specify open for O_CREAT. With the change to event-specific
filters, it's necessary to associate a search for O_CREAT with
evt.type=open.
Make changes to rules to improve performance and reduce FPs:
- Rely on https://github.com/draios/sysdig/pull/610 that allows
specifying an open/openat for reading/writing without having to search
through all the flags individually.
- For a two-item list (open, openat), and thinking ahead to
https://github.com/draios/sysdig/pull/624, check the event type
individually instead of as a set membership test, which is a bit
faster.
- Switch to consistently using evt.type instead of syscall.type.
- Move positive tests like etc_dir, bin_dir, sensitive_files,
proc.sname, etc., which are most likely to not succeed, to the
beginning of rules, so they have a greater chance to cause the rest of
the rule to be skipped, which saves time.
- Using exim as a mail program--exim also can suid to root.
- add a new macro for ssl management binaries and allow them to write
below /etc and read sensitive files.
- add a new macro for dhcp client binaries and allow them to write below
/etc.
- Add exe (docker-related program) as a program that can set a namespace
using setns.
- Don't count /dev/tty as an important file under /dev.
https://github.com/draios/sysdig/pull/623 adds support for a startswith
operator to allow for string prefix matching. Modify the parser to
recognize that operator, and use that operator for rules that really
want to check the beginning of a pathname, directory, etc. to make them
faster and avoid FPs.
Once sysdig adds support for handling "in (...)" filter expressions as
set membership tests, it will be advantageous to combine lists of items
together into a single list so they can all be checked in a single set
membership test.
This commit adds support for a new yaml item type "list" containing a
field "name" and field "items" containing a list of items. These are
represented as a yaml list, which allows yaml to handle some of the
initial parsing with the list items maintained natively in lua.
Allow lists to contain list references by expanding any references to
the items in the list, before storing the list items in
state.lists.
When parsing macro or rule conditions, replace all references to a list
name with the list items as a comma separated string.
Modify the falco rules to switch to lists whenever possible. The
new convention is to use the suffix _binaries for lists of program names
and _procs for macros that define a filter expression using the list.
Adding docker-compose based example of man-in-the-middle attack against
installation scripts and how it can be detected using sysdig falco.
The docker-compose environment starts a good web server, compromised
nginx installation, evil web server, and a copy of sysdig falco. The
README walks through the process of compromising a client by using curl
http://localhost/get-software.sh | bash and detecting the compromise
using ./fbash.
The fbash program included in this example fixes https://github.com/draios/falco/issues/46.
Add additional rules related to using pipe installers within a fbash
session:
- Modify write_etc to only trigger if *not* in a fbash session. There's
a new rule write_etc_installer which has the same conditions when in
a fbash session, logging at INFO severity.
- A new rule write_rpm_database warns if any non package management
program tries to write below /var/lib/rpm.
- Add a new warning if any program below a fbash session tries to open
an outbound network connection on ports other than http(s) and dns.
- Add INFO level messages when programs in a fbash session try to run
package management binaries (rpm,yum,etc) or service
management (systemctl,chkconfig,etc) binaries.
In order to test these new INFO level rules, make up a third class of
trace files traces-info.zip containing trace files that should result in
info-level messages.
To differentiate warning and info level detection, add an attribute to
the multiplex file "detect_level", which is "Warning" for the files in
traces-positive and "Info" for the files in traces-info. Modify
falco_test.py to look specifically for a non-zero count for the given
detect_level.
Doing this exposed a bug in the way the level-specific counts were being
recorded--they were keeping counts by level name, not number. Fix that.