Add new unit tests to check that list substitution is working as
expected, with test cases for the list substitution occurring at the
beginning, middle, and end of a condition.
Also add tests that verify that overrides on list/macro/rule names
always occur in order.
When performing list substitution, only replace a list name when it is
surrounded by whitespace or expected punctuation characters. Lua
patterns don't have a notion of this-or-that patterns e.g. (^|abc), so
we have 3 versions of the substitution depending on whether he list name
occurs in the beginning, middle, or end of a string.
This fixes#197.
Also validate macros when they are parsed. Macros are also validated as
a part of rules being parsed, but it's possible to have an individual
rules file containing only macros, or a macro not explicitly tied to any
rule. In this case, it's useful to be able to check the macro to see if
it contains dangling macro references.
When parsing condition expressions, if the type of an ast node is
String (aka quoted string), don't trim whitespace from the value. This
ensures that conditions that want to match exact strings e.g. command
lines with leading/trailing spaces will work properly.
This fixes#253.
* Updates from beta customers.
- add anacron as a cron program
* Reorganize package management binaries
Split package_management_binaries into two separate lists rpm_binaries
and deb_binaries. unattended-upgr is common to both worlds so it's still
in package_management_binaries.
Also change Write below rpm database to use rpm_binaries instead of its
own list.
Also add 75-system-updat (truncated) as a shell spawner.
* Add rules for jenkins
Add rules that allow jenkins to spawn shells, both in containers and
directly on the host.
Also handle jenkins slaves that run /tmp/slave.jar.
* Allow npm to run shells.
Not yet allowing node to run shells itself, although we want to add
something to reduce node-related FPs.
* Allow urlgrabber/git-remote to access /etc
urlgrabber and git-remote both try to access the RHEL nss database,
containing shared certificates. I may change this in a more general way
by changing open_read/open_write to only look for successful opens.
* Only look for successful open_read/open_writes
Change the macros open_read/open_write to only trigger on successful
opens (when fd.num > 0). This is a pretty big change to behavior, but
is more intuitive.
This required a small update to the open counts for a couple of unit
tests, but otherwise they still all passed with this change.
* Allow rename_device to write below /dev
Part of udev.
* Allow cloud-init to spawn shells.
Part of https://cloud-init.io/
* Allow python to run a shell that runs sdchecks
sdchecks is a part of the sysdig monitor agent.
* Allow dev creation binaries to write below etc.
Specifically this includes blkid and /etc/blkid/blkid.tab.
* Allow git binaries to spawn shells.
They were already allowed to run shells in a container.
* Add /dev/kmsg as an allowed /dev file
Allows userspace programs to write to kernel log.
* Allow other make programs to spawn shells.
Also allow gmake/cmake to spawn shells and put them in their own list
make_binaries.
* Add better mesos support.
Mesos slaves appear to be in a container due to their cgroup and can run
programs mesos-health-check/mesos-docker-exec to monitor the containers
on the slave, so allow them to run shells.
Add mesos-agent, mesos-logrotate, mesos-fetch as shell spawners both in
and out of containers.
Add gen_resolvconf. (short for gen_resolvconf.py) as a program that can
write to /etc.
Add toybox (used by mesos, part of http://landley.net/toybox/about.html)
as a shell spawner.
* systemd can listen on network ports.
Systemd can listen on network ports to launch daemons on demand, so
allow it to perform network activity.
* Let docker binaries setuid.
Let docker binaries setuid and add docker-entrypoi (truncation
intentional) to the set of docker binaries.
* Change cis-related rules to be less noisy
Change the two cis-related falco rules "File Open by Privileged
Container" and "Sensitive Mount by Container" to be less noisy. We found
in practice that tracking every open still results in too many falco
notifications.
For now, change the rules to only track the initial process start in the
container by looking for vpid=1. This should result in only triggering
when a privileged/sensitive mount container is started. This is slightly
less coverage but is far less noisy.
* Add quay.io/sysdig as trusted containers
These are used for sysdig cloud onpremise deployments.
* Add gitlab-runner-b(uild) as a gitlab binary.
Add gitlab-runner-b (truncated gitlab-runner-build) as a gitlab binary.
* Add ceph as a shell spawner.
Also allow ceph to spawn shells in a container.
* Allow some shells by command line.
For some mesos containers, where the container doesn't have an image and
is just a tarball in a cgroup/namespace, we don't have any image to work
with. In those cases, allow specific command lines.
* Allow user 'nobody' to setuid.
Allow the user nobody to setuid. This depends on the user nobody being
set up in the first place to have no access, but that should be an ok
assumption.
* Additional allowed shell commandlines
* Add additional shells.
* Allow multiple users to become themself.
Add rule somebody_becoming_themself that handles cases of nobody and
www-data trying to setuid to themself. The sysdig filter language
doesn't support template/variable values to allow "user.name=X and
evt.arg.uid=X for a given X", so we have to enumerate the users.
* More known spawn command lines
* Let make binaries be run in containers.
Some CI/CD pipelines build in containers.
* Add additional shell spawning command lines
* Add additional apt program apt-listchanges.
* Add gitlab-ce as shell spawning container.
* Allow PM2 to spawn shells in containers.
Was already in the general list, seen in some customers, so adding to
the in containers list.
* Clean up pass to fix long lines.
Take a pass through the rules making sure each line is < 120 characters.
* Change tests for privileged container rules.
Change unit tests to reflect the new privileged/sensitive mount
container rules that only detect container launch.
The default falco ruleset now has a wider variety of priorities, so
adjust the automated tests to match:
- Instead of creating a generic test yaml entry for every trace file in
traces-{positive,negative,info} with assumptions about detect levels,
add a new falco_traces.yaml.in multiplex file that has specific
information about the detect priorities and rule detect counts for each
trace file.
- If a given trace file doesn't have a corresponding entry in
falco_traces.yaml.in, a generic entry is added with a simple
detect: (True|False) value and level. That way you can get specific
detect levels/counts for existing trace files, but if you forget to
add a trace to falco_traces.yaml.in, you'll still get some coverage.
- falco_tests.yaml.in isn't added to any longer, so rename it to
falco_tests.yaml.
- Avocado is now run twice--once on each yaml file. The final test
passes if both avocado runs pass.
Review the priorities used by each rule and try to use a consistent set
that uses more of the possible priorities. The general guidelines I used
were:
- If a rule is related to a write of state (i.e. filesystem, etc.),
its priority is ERROR.
- If a rule is related to an unauthorized read of state (i.e. reading
sensitive filees, etc.), its priority is WARNING.
- If a rule is related to unexpected behavior (spawning an unexpected
shell in a container, opening an unexpected network connection, etc.), its priority
is NOTICE.
- If a rule is related to behaving against good practices (unexpected
privileged containers, containers with sensitive mounts, running
interactive commands as root), its priority is INFO.
One exception is that the most FP-prone rule (Run shell untrusted) has a
priority of DEBUG.
Allow the sysdig cloud agent to call setns to collect java process
metrics.
We've also seen cases where some of the intermediate processes created
below runc appear to call setns. It appears that this only should happen
if some events (like the execve that spawns the intermediate processes)
are lost, but just to be safe allow processes starting with "runc:" to
call setns.
Add a new falco rule "Terminal shell in container" that looks for shells
spawned in a container with an attached terminal. This is similar to the
existing "Run shell in container" rule, but doesn't have as many
exceptions as we expect this to be even less rare.
Add automated tests for running falco from a package and container. As a
result, this will also test building the kernel module as well as
runnning falco-probe-loader as a backup.
In travis.yml, switch to the docker-enabled vm and install dkms. This
changed the environment slightly, so change how avocado's python
dependencies are installed. After building falco, copy the .deb package
to docker/local and build a local docker image based on that package.
Add the following new tests:
- docker_package: this uses "docker run" to run the image created in
travis.yml. This includes using dkms to build the kernel module and
load it. In addition, the conf directory is mounted to /host/conf, the
rules directory is mounted to /host/rules, and the traces directory is
mounted to /host/traces.
- docker_package_local_driver: this disables dkms via a volume mount
that maps /dev/null to /usr/sbin/dkms and copies the kernel module by
hand into the container to /root/.sysdig/falco-probe-....ko. As a
result, falco-probe-loader will use the local kernel module instead
of building one itself.
- debian_package: this installs the .deb package and runs the installed
version of falco.
Ideally, there'd also be a test for downloading the driver, but since
the driver depends on the kernel as well as the falco version string,
you can't put a single driver on download.draios.com that will work
long-term.
These tests depend on the following new test attributes:
- package: if present, this points to the docker image/debian package
to install.
- addl_docker_run_args: if present, will be added to the docker run
command.
- copy_local_driver: if present, will copy the built kernel module to
~/.sysdig. ~/.sysdig/* is always cleared out before each test.
- run_duration: maps to falco's -M <secs> flag
- trace_file is now optional.
Also add some misc general test changes:
- Clean up our use of process.run. By default it will fail a test if the
run program returns non-zero, so we don't have to grab the exit
status. In addition, get rid of sudo in the command lines and use the
sudo attribute instead.
- Fix some tests that were writing to files below /tmp/falco_outputs
by creating the directory first. Useful when running avocado directly.
