falco/CHANGELOG.md

Change Log

This file documents all notable changes to Falco. The release numbering uses semantic versioning.

v0.12.0

Released 2018-09-11

Major Changes

• Improved IPv6 Support to fully support use of IPv6 addresses in events, connections and filters [#sysdig/1204]

• Ability to associate connections with dns names: new filterchecks fd.*ip.name allow looking up the DNS name for a connection's IP address. This can be used to identify or restrict connections by dns names e.g. evt.type=connect and fd.sip.name=github.com. [#412] [#sysdig/1213]

• New filterchecks user.loginuid and user.loginname can be used to match the login uid, which stays consistent across sudo/su. This can be used to find the actual user running a given process [#sysdig/1189]

Minor Changes

• Upgrade zlib to 1.2.11, openssl to 1.0.2n, and libcurl to 7.60.0 to address software vulnerabilities [#402]
• New endswith operator can be used for suffix matching on strings [#sysdig/1209]

Bug Fixes

• Better control of specifying location of lua source code [#406]

Rule Changes

• None for this release.

v0.11.1

Released 2018-07-31

Bug Fixes

• Fix a problem that caused the kernel module to not load on certain kernel versions [#397] [#394]

v0.11.0

Released 2018-07-24

Major Changes

• EBPF Support (Beta): Falco can now read events via an ebpf program loaded into the kernel instead of the falco-probe kernel module. Full docs here. [#365]

Minor Changes

• Rules may now have an skip-if-unknown-filter property. If set to true, a rule will be skipped if its condition/output property refers to a filtercheck (e.g. fd.some-new-attibute) that is not present in the current falco version. [#364] [#345]
• Small changes to Falco COPYING file so github automatically recognizes license [#380]
• New example integration showing how to connect Falco with Anchore to dynamically create falco rules based on negative scan results [#390]
• New example integration showing how to connect Falco, nats, and K8s to run flexible "playbooks" based on Falco events [#389]

Bug Fixes

• Ensure all rules are enabled by default [#379]
• Fix libcurl compilation problems [#374]
• Add gcc-6 to docker container, which improves compatibility when building kernel module [#382] [#371]
• Ensure the /lib/modules symlink to /host/lib/modules is set correctly [#392]

Rule Changes

• Add additional binary writing programs [#366]
• Add additional package management programs [#388] [#366]
• Expand write_below_etc handling for additional programs [#388] [#366]
• Expand set of programs allowed to write to /etc/pki [#388]
• Expand set of root written directories/files [#388] [#366]
• Let pam-config read sensitive files [#388]
• Add additional trusted containers: openshift, datadog, docker ucp agent, gliderlabs logspout [#388]
• Let coreos update-ssh-keys write to /home/core/.ssh [#388]
• Expand coverage for MS OMS [#388] [#387]
• Expand the set of shell spawning programs [#366]
• Add additional mysql programs/directories [#366]
• Let program id open network connections [#366]
• Opt-in rule for protecting tomcat shell spawns [#366]
• New rule Write below monitored directory [#366]

v0.10.0

Released 2018-04-24

Major Changes

• Rules Directory Support: Falco will read rules files from /etc/falco/rules.d in addition to /etc/falco/falco_rules.yaml and /etc/falco/falco_rules.local.yaml. Also, when the argument to -r/falco.yaml rules_file is a directory, falco will read rules files from that directory. [#348] [#187]
• Properly support all syscalls (e.g. those without parameter extraction by the kernel module) in falco conditions, so they can be included in evt.type=<name> conditions. [#352]
• When packaged as a container, start building kernel module with gcc 5.0 instead of gcc 4.9. [#331]
• New example puppet module for falco. [#341] [#115]
• When signaled with USR1, falco will close/reopen log files. Include a logrotate example that shows how to use this feature for log rotation. [#347] [#266]
• To improve resource usage, further restrict the set of system calls available to falco [#351] [draios/sysdig#1105]

Minor Changes

• Add gdb to the development Docker image (sysdig/falco:dev) to aid in debugging. [#323]
• You can now specify -V multiple times on the command line to validate multiple rules files at once. [#329]
• When run with -v, falco will print dangling macros/lists that are not used by any rules. [#329]
• Add an example demonstrating cryptomining attack that exploits an open docker daemon using host mounts. [#336]
• New falco.yaml option json_include_output_property controls whether the formatted string "output" is included in the json object when json output is enabled. [#342]
• Centralize testing event types for consideration by falco into a single function [draios/sysdig#1105) [#356]
• If a rule has an attribute warn_evttypes, falco will not complain about evt.type restrictions on that rule [#355]
• When run with -i, print all ignored events/syscalls and exit. [#359]

Bug Fixes

• Minor bug fixes to k8s daemonset configuration. [#325] [#296] [#295]
• Ensure --validate can be used interchangeably with -V. [#334] [#322]
• Rule conditions like fd.net can now be used with the in operator e.g. evt.type=connect and fd.net in ("127.0.0.1/24"). [draios/sysdig#1091] [#343]
• Ensure that keep_alive can be used both with file and program output at the same time. [#335]
• Make it possible to append to a skipped macro/rule without falco complaining [#346] [#305]
• Ensure rule order is preserved even when rules do not contain any evt.type restriction. [#354] [#355]

Rule Changes

• Make it easier to extend the Change thread namespace rule via a user_known_change_thread_namespace_binaries list. [#324]
• Various FP fixes from users. [#321] [#326] [#344] [#350]
• New rule Disallowed SSH Connection detects attempts ssh connection attempts to hosts outside of an expected set. In order to be effective, you need to override the macro allowed_ssh_hosts in a user rules file. [#321]
• New rule Unexpected K8s NodePort Connection detects attempts to contact the K8s NodePort range from a program running inside a container. In order to be effective, you need to override the macro nodeport_containers in a user rules file. [#321]
• Improve Modify binary dirs rule to work with new syscalls [#353]
• New rule Unexpected UDP Traffic checks for udp traffic not on a list of expected ports. Somewhat FP-prone, so it must be explicitly enabled by overriding the macro do_unexpected_udp_check in a user rules file. [#320] [#357]

v0.9.0

Released 2018-01-18

Bug Fixes

• Fix driver incompatibility problems with some linux kernel versions that can disable pagefault tracepoints [#sysdig/1034]
• Fix OSX Build incompatibility with latest version of libcurl [#291]

Minor Changes

• Updated the Kubernetes example to provide an additional example: Daemon Set using RBAC and a ConfigMap for configuration. Also expanded the documentation for both the RBAC and non-RBAC examples. [#309]

Rule Changes

• Refactor the shell-related rules to reduce false positives. These changes significantly decrease the scope of the rules so they trigger only for shells spawned below specific processes instead of anywhere. [#301] [#304]
• Lots of rule changes based on feedback from Sysdig Secure community [#293] [#298] [#300] [#307] [#315]

v0.8.1

Released 2017-10-10

Bug Fixes

• Fix packaging to specify correct built-in config file [#288]

v0.8.0

Released 2017-10-10

Important: the location for falco's configuration file has moved from /etc/falco.yaml to /etc/falco/falco.yaml. The default rules file has moved from /etc/falco_rules.yaml to /etc/falco/falco_rules.yaml. In addition, 0.8.0 has added a local ruls file to /etc/falco/falco_rules.local.yaml. See the documentation for more details.

Major Changes

• Add the ability to append one list to another list by setting an append: true attribute. [#264]
• Add the ability to append one macro/rule to another list by setting an append: true attribute. [#277]
• Ensure that falco rules/config files are preserved across package upgrades/removes if modified. [#278]
• Add the notion of a "local" rules file that should contain modifications to the default falco rules file. [#278]
• When using json output, separately include the individual templated fields in the json object. [#282]
• Add the ability to keep a file/program pipe handle open across rule notifications. [#283]
• New argument -V validates rules file and immediately exits. [#286]

Minor Changes

• Minor updates to falco example programs [#248] [#275]
• Also validate macros at rule parse time. [#257]
• Minor README typo fixes [#276]
• Add a government CLA (contributor license agreement). [#263]
• Add ability to only run rules with a priority >= some threshold [#281]
• Add ability to make output channels unbuffered [#285]

Bug Fixes

• Fix installation of falco on OSX [#252]
• Fix a bug that caused the trailing whitespace of a quoted string to be accidentally removed [#254]
• When multiple sets of kernel headers are installed, find the one for the running kernel [#260]
• Allow pathnames in rule/macro conditions to contain '.' characters [#262]
• Fix a bug where a list named "foo" would be substituted even if it were a substring of a longer word like "my_foo" [#258]
• Remove extra trailing newlines from rule output strings [#265]
• Improve build pathnames to avoid relative paths when possible [#284]

Rule Changes

• Significant changes to default ruleset to address FPs. These changes resulted from hundreds of hours of use in actual customer environments. [#247] [#259]
• Add official gitlab EE docker image to list of known shell spawning images. Thanks @dkerwin! [#270]
• Add keepalived to list of shell spawning binaries. Thanks @dkerwin! [#269]

v0.7.0

Released 2017-05-30

Major Changes

• Update the priorities of falco rules to use a wider range of priorities rather than just ERROR/WARNING. More info on the use of priorities in the ruleset can be found here. [#244]

Minor Changes

None.

Bug Fixes

• Fix typos in various markdown files. Thanks @sublimino! [#241]

Rule Changes

• Add gitlab-mon as a gitlab binary, which allows it to run shells, etc. Thanks @dkerwin! [#237]
• A new rule Terminal shell in container" that looks for shells spawned in a container with an attached terminal. [#242]
• Fix some FPs related to the sysdig monitor agent. [#243]
• Fix some FPs related to stating containers combined with missed events [#243]

v0.6.1

Released 2017-05-15

Major Changes

None

Minor Changes

• Small changes to token bucket used to throttle falco events [#234] [#235] [#236] [#238]

Bug Fixes

• Update the falco driver to work with kernel 4.11 [#829]

Rule Changes

• Don't allow apache2 to spawn shells in containers [#231] [#232]

v0.6.0

Released 2017-03-29

Major Changes

• Add the notion of tagged falco rules. Full documentation for this feature is available on the wiki. [#58] [#59] [#60] [#206]
• Falco now has its own dedicated kernel module. Previously, it would depend on sysdig being installed and would use sysdig's sysdig-probe kernel module. This ensures you can upgrade sysdig and falco without kernel driver compatibility problems. More details on the kernel module and its installation are on the wiki. [#215] [#223] [#224]
• When providing multiple rules files by specifying `-r' multiple times, make sure that you can override rules/lists/macros. Previously, a list/macro/rule specified in an earlier file could not be overridden in a later file. [#176] [#177]
• Add example k8s yaml files that show how to run falco as a k8s DaemonSet, and how to run falco-event-generator as a deployment running on one node. [#222] [#225] [#226]
• Update third party libraries to address security vulnerabilities. [#182]
• Falco can now be built on OSX. Like sysdig, on OSX it is limited to reading existing trace files. [#210]

Minor Changes

• Several changes to falco-event-generator to improve usability. [#205]
• Switch to a formatter cache provided by sysdig code instead of using our own. [#212]
• Add automated tests that use locally-built docker images. [#188]

Bug Fixes

• Make sure output strings are not truncated when a given %field expression has a NULL value. [#180] [#181]
• Allow ASSERTs when running travisci tests. [#199]
• Fix make dependencies for lyaml. [#204] [#130]
• (This was a change in sysdig, but affected falco). Prevent hangs when traversing malformed parent thread state. [#208]

Rule Changes

• Add confd as a program that can write files below /etc and fleetctl as a program that can spawn shells. [#175]
• Add exechealthz, a k8s liveness checking utility, to the list of shell spawners. [#190]
• Eliminate FPs related to weekly ubuntu cron jobs. [#192]
• Allow shells spawned by ansible, and eliminate FPs when managing machines via ansible. [#193] [#196] [#202]
• Eliminate FPs related to use of other security products. Thanks to @juju4 for the useful rule updates. [#200]
• Add additional possible locations for denyhosts, add PM2 as a shell spawner. [#202]
• Add flanneld as a privileged container, improve grouping for the "x running y" macros, allow denyhosts to spawn shells. [#207]
• Handle systemd changing its name to "(systemd)", add sv (part of runit) as a program that can write below /etc, allow writing to all /dev/tty* files. [#209]
• Add erl_child_setup as a shell spawner. Thanks to @dkerwin for the useful rule updates. [#218] [#221]
• Add support for gitlab omnibus containers/pods. Thanks to @dkerwin for the useful rule updates. [#220]

v0.5.0

Released 2016-12-22

Starting with this release, we're adding a new section "Rule Changes" devoted to changes to the default ruleset falco_rules.yaml.

Major Changes

• Cache event formatting objects so they are not re-created for every falco notification. This can result in significant speedups when the ruleset results in lots of notifications. [#158]
• Falco notifications are now throttled by a token bucket, preventing a flood of notifications when many events match a rule. Controlled by the outputs, rate and outputs, max_burst options. [#161]

Minor Changes

• When run from a container, you can provide the environment variable SYSDIG_SKIP_LOAD to skip the process of building/loading the kernel module. Thanks @carlsverre for the fix. [#145]
• Fully implement USE_BUNDLED_DEPS within CMakeFiles so you can build with external third-party libraries. [#147]
• Improve error messages that result when trying to load a rule with a malformed output: attribute [#150] [#151]
• Add the ability to write event capture statistics to a file via the -s <statsfile> option. [#155]
• New configuration option log_level controls the verbosity of falco's logging. [#160]

Bug Fixes

• Improve compatibility with Sysdig Cloud Agent build [#148]

Rule Changes

• Add DNF as non-alerting for RPM and package management. Thanks @djcross for the fix. [#153]
• Make google_containers/kube-proxy a trusted image, affecting the File Open by Privileged Container/Sensitive Mount by Container rules. [#159]
• Add fail2ban-server as a program that can spawn shells. Thanks @jcoetzee for the fix. [#168]
• Add systemd as a program that can access sensitive files. Thanks @jcoetzee for the fix. [#169]
• Add apt/apt-get as programs that can spawn shells. Thanks @jcoetzee for the fix. [#170]

v0.4.0

Released 2016-10-25

As falco depends heavily on sysdig, many changes here were actually made to sysdig and pulled in as a part of the build process. Issues/PRs starting with sysdig/#XXX are sysdig changes.

Major Changes

• Improved visibility into containers: ** New filter container.privileged to match containers running in privileged mode [sysdig/#655] [sysdig/#658] ** New rules utilizing privileged state [#121] ** New filters container.mount* to match container mount points [sysdig/#655] ** New rules utilizing container mount points [#120] ** New filter container.image.id to match container image id [sysdig/#661]

• Improved visibility into orchestration environments: ** New k8s.deployment.* and k8s.rs.* filters to support latest kubernetes features [sysdg/#dbf9b5c] ** Rule changes to avoid FPs when monitoring k8s environments [#138] ** Add new options -pc/-pk/-pm/-k/-m analogous to sysdig command line options. These options pull metadata information from k8s/mesos servers and adjust default falco notification outputs to contain container/orchestration information when applicable. [#131] [#134]

• Improved ability to work with file pathnames: ** Added glob operator for strings, works as classic shell glob path matcher [sysdig/#653] ** Added pmatch operator to efficiently test a subject pathname against a set of target pathnames, to see if the subject is a prefix of any target [sysdig/#660] [#125]

Minor Changes

• Add an event generator program that simulates suspicious activity that can be detected by falco. This is also available as a docker image [sysdig/falco-event-generator]. [#113] [#132]
• Changed rule names to be human readable [#116]
• Add Copyright notice to all source files [#126]
• Changes to docker images to make it easier to massage JSON output for webhooks [#133]
• When run with -v, print statistics on the number of events processed and dropped [#139]
• Add ability to write trace files with -w. This can be useful to write a trace file in parallel with live event monitoring so you can reproduce it later. [#140]
• All rules can now take an optional enabled flag. With enabled: false, a rule will not be loaded or run against events. By default all rules are enabled [#119]

Bug Fixes

• Fixed rule FPs related to docker's docker/dockerd split in 1.12 [#112]
• Fixed rule FPs related to sysdigcloud agent software [#141]
• Minor changes to node.js example to avoid falco false positives [#111]
• Fixed regression that broke configurable outputs [#117]. This was not broken in 0.3.0, just between 0.3.0 and 0.4.0.
• Fixed a lua stack leak that could cause problems when matching millions of events against a large set of rules [#123]
• Update docker files to reflect changes to debian:unstable docker image [#124]
• Fixed logic for detecting config files to ensure config files in /etc/falco.yaml are properly detected [#135] [#136]
• Don't alert on falco spawning a shell for program output notifications [#137]

v0.3.0

Released 2016-08-05

Major Changes

Significantly improved performance, involving changes in the falco and sysdig repositories:

• Reordering a rule condition's operators to put likely-to-fail operators at the beginning and expensive operators at the end. [#95] [#104]
• Adding the ability to perform x in (a, b, c, ...) as a single set membership test instead of individual comparisons between x=a, x=b, etc. [#624] [#98]
• Avoid unnecessary string manipulations. [#625]
• Using startswith as a string comparison operator when possible. [#623]
• Use is_open_read/is_open_write when possible instead of searching through open flags. [#610]
• Group rules by event type, which allows for an initial filter using event type before going through each rule's condition. [#627] [#101]

All of these changes result in dramatically reduced CPU usage. Here are some comparisons between 0.2.0 and 0.3.0 for the following workloads:

• Phoronix's pts/apache and pts/dbench tests.
• Sysdig Cloud Kubernetes Demo: Starts a kubernetes environment using docker with apache and wordpress instances + synthetic workloads.
• Juttle-engine examples : Several elasticsearch, node.js, logstash, mysql, postgres, influxdb instances run under docker-compose.

Workload	0.2.0 CPU Usage	0.3.0 CPU Usage
pts/apache	24%	7%
pts/dbench	70%	5%
Kubernetes-Demo (Running)	6%	2%
Kubernetes-Demo (During Teardown)	15%	3%
Juttle-examples	3%	1%

As a part of these changes, falco now prefers rule conditions that have at least one evt.type= operator, at the beginning of the condition, before any negative operators (i.e. not or !=). If a condition does not have any evt.type= operator, falco will log a warning like:

Rule no_evttype: warning (no-evttype):
proc.name=foo
     did not contain any evt.type restriction, meaning it will run for all event types.
     This has a significant performance penalty. Consider adding an evt.type restriction if possible.

If a rule has a evt.type operator in the later portion of the condition, falco will log a warning like:

Rule evttype_not_equals: warning (trailing-evttype):
evt.type!=execve
     does not have all evt.type restrictions at the beginning of the condition,
     or uses a negative match (i.e. "not"/"!=") for some evt.type restriction.
     This has a performance penalty, as the rule can not be limited to specific event types.
     Consider moving all evt.type restrictions to the beginning of the rule and/or
     replacing negative matches with positive matches if possible.

Minor Changes

• Several sets of rule cleanups to reduce false positives. [#95]
• Add example of how falco can detect abuse of a badly designed REST API. [#97]
• Add a new output type "program" that writes a formatted event to a configurable program. Each notification results in one invocation of the program. A common use of this output type would be to send an email for every falco notification. [#105] [#99]
• Add the ability to run falco on all events, including events that are flagged with EF_DROP_FALCO. (These events are high-volume, low-value events that are ignored by default to improve performance). [#107] [#102]

Bug Fixes

• Add third-party jq library now that sysdig requires it. [#96]

v0.2.0

Released 2016-06-09

For full handling of setsid system calls and session id tracking using proc.sname, falco requires a sysdig version >= 0.10.0.

Major Changes

• Add TravisCI regression tests. Testing involves a variety of positive, negative, and informational trace files with both plain and json output. [#76] [#83]
• Fairly big rework of ruleset to improve coverage, reduce false positives, and handle installation environments effectively [#83] [#87]
• Not directly a code change, but mentioning it here--the Wiki has now been populated with an initial set of articles, migrating content from the README and adding detail when necessary. [#90]

Minor Changes

• Improve JSON output to include the rule name, full output string, time, and severity [#89]

Bug Fixes

• Improve CMake quote handling [#84]
• Remove unnecessary NULL check of a delete [#85]

v0.1.0

Released 2016-05-17

Major Changes

• Initial release. Subsequent releases will have "Major Changes", "Minor Changes", and "Bug Fixes" sections, with links to github issues/pull requests as appropriate.