Merge branch 'dev' into agent-master

When run with -i, print out all ignored syscalls/event names and exit.

CHANGELOG.md

@@ -2,6 +2,48 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
+## v0.10.0
+
+Released 2018-04-24
+
+## Major Changes
+
+* **Rules Directory Support**: Falco will read rules files from `/etc/falco/rules.d` in addition to `/etc/falco/falco_rules.yaml` and `/etc/falco/falco_rules.local.yaml`. Also, when the argument to `-r`/falco.yaml `rules_file` is a directory, falco will read rules files from that directory. [[#348](https://github.com/draios/falco/pull/348)] [[#187](https://github.com/draios/falco/issues/187)]
+* Properly support all syscalls (e.g. those without parameter extraction by the kernel module) in falco conditions, so they can be included in `evt.type=<name>` conditions. [[#352](https://github.com/draios/falco/pull/352)]
+* When packaged as a container, start building kernel module with gcc 5.0 instead of gcc 4.9. [[#331](https://github.com/draios/falco/pull/331)]
+* New example puppet module for falco. [[#341](https://github.com/draios/falco/pull/341)] [[#115](https://github.com/draios/falco/issues/115)]
+* When signaled with `USR1`, falco will close/reopen log files. Include a [logrotate](https://github.com/logrotate/logrotate) example that shows how to use this feature for log rotation. [[#347](https://github.com/draios/falco/pull/347)] [[#266](https://github.com/draios/falco/issues/266)]
+* To improve resource usage, further restrict the set of system calls available to falco [[#351](https://github.com/draios/falco/pull/351)] [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)]
+
+## Minor Changes
+
+* Add gdb to the development Docker image (sysdig/falco:dev) to aid in debugging. [[#323](https://github.com/draios/falco/pull/323)]
+* You can now specify -V multiple times on the command line to validate multiple rules files at once. [[#329](https://github.com/draios/falco/pull/329)]
+* When run with `-v`, falco will print *dangling* macros/lists that are not used by any rules. [[#329](https://github.com/draios/falco/pull/329)]
+* Add an example demonstrating cryptomining attack that exploits an open docker daemon using host mounts. [[#336](https://github.com/draios/falco/pull/336)]
+* New falco.yaml option `json_include_output_property` controls whether the formatted string "output" is included in the json object when json output is enabled. [[#342](https://github.com/draios/falco/pull/342)]
+* Centralize testing event types for consideration by falco into a single function [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)) [[#356](https://github.com/draios/falco/pull/356)]
+* If a rule has an attribute `warn_evttypes`, falco will not complain about `evt.type` restrictions on that rule [[#355](https://github.com/draios/falco/pull/355)]
+* When run with `-i`, print all ignored events/syscalls and exit. [[#359](https://github.com/draios/falco/pull/359)]
+
+## Bug Fixes
+
+* Minor bug fixes to k8s daemonset configuration. [[#325](https://github.com/draios/falco/pull/325)] [[#296](https://github.com/draios/falco/pull/296)] [[#295](https://github.com/draios/falco/pull/295)]
+* Ensure `--validate` can be used interchangeably with `-V`. [[#334](https://github.com/draios/falco/pull/334)] [[#322](https://github.com/draios/falco/issues/322)]
+* Rule conditions like `fd.net` can now be used with the `in` operator e.g. `evt.type=connect and fd.net in ("127.0.0.1/24")`. [[draios/sysdig#1091](https://github.com/draios/sysdig/pull/1091)] [[#343](https://github.com/draios/falco/pull/343)]
+* Ensure that `keep_alive` can be used both with file and program output at the same time. [[#335](https://github.com/draios/falco/pull/335)]
+* Make it possible to append to a skipped macro/rule without falco complaining [[#346](https://github.com/draios/falco/pull/346)] [[#305](https://github.com/draios/falco/issues/305)]
+* Ensure rule order is preserved even when rules do not contain any `evt.type` restriction. [[#354](https://github.com/draios/falco/issues/354)] [[#355](https://github.com/draios/falco/pull/355)]
+
+## Rule Changes
+
+* Make it easier to extend the `Change thread namespace` rule via a `user_known_change_thread_namespace_binaries` list. [[#324](https://github.com/draios/falco/pull/324)]
+* Various FP fixes from users. [[#321](https://github.com/draios/falco/pull/321)] [[#326](https://github.com/draios/falco/pull/326)] [[#344](https://github.com/draios/falco/pull/344)] [[#350](https://github.com/draios/falco/pull/350)]
+* New rule `Disallowed SSH Connection` detects attempts ssh connection attempts to hosts outside of an expected set. In order to be effective, you need to override the macro `allowed_ssh_hosts` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
+* New rule `Unexpected K8s NodePort Connection` detects attempts to contact the K8s NodePort range from a program running inside a container. In order to be effective, you need to override the macro `nodeport_containers` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
+* Improve `Modify binary dirs` rule to work with new syscalls [[#353](https://github.com/draios/falco/pull/353)]
+* New rule `Unexpected UDP Traffic` checks for udp traffic not on a list of expected ports. Somewhat FP-prone, so it must be explicitly enabled by overriding the macro `do_unexpected_udp_check` in a user rules file. [[#320](https://github.com/draios/falco/pull/320)] [[#357](https://github.com/draios/falco/pull/357)]
+
 ## v0.9.0
 
 Released 2018-01-18

README.md

@@ -2,7 +2,7 @@
 
 #### Latest release
 
-**v0.9.0**
+**v0.10.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />

cpack/debian/conffiles

@@ -1,4 +1,4 @@
 /etc/falco/falco.yaml
 /etc/falco/falco_rules.yaml
-/etc/falco/application_rules.yaml
+/etc/falco/rules.available/application_rules.yaml
 /etc/falco/falco_rules.local.yaml

docker/event-generator/event_generator.cpp

@@ -296,23 +296,21 @@ void system_user_interactive() {
 }
 
 void network_activity() {
-	printf("Opening a listening socket on port 8192...\n");
+	printf("Connecting a udp socket to 10.2.3.4:8192...\n");
 	int rc;
 	int sock = socket(PF_INET, SOCK_DGRAM, 0);
 	struct sockaddr_in localhost;
 
 	localhost.sin_family = AF_INET;
 	localhost.sin_port = htons(8192);
-	inet_aton("127.0.0.1", &(localhost.sin_addr));
+	inet_aton("10.2.3.4", &(localhost.sin_addr));
 
-	if((rc = bind(sock, (struct sockaddr *) &localhost, sizeof(localhost))) != 0)
+	if((rc = connect(sock, (struct sockaddr *) &localhost, sizeof(localhost))) != 0)
 	{
 		fprintf(stderr, "Could not bind listening socket to localhost: %s\n", strerror(errno));
 		return;
 	}
 
-	listen(sock, 1);
-
 	close(sock);
 }
 

examples/logrotate/falco

@@ -0,0 +1,7 @@
+/var/log/falco-events.log {
+    rotate 5
+    size 1M
+    postrotate
+        /usr/bin/killall -USR1 falco
+    endscript
+}

falco.yaml

@@ -1,4 +1,7 @@
-# File(s) containing Falco rules, loaded at startup.
+# File(s) or Directories containing Falco rules, loaded at startup.
+# The name "rules_file" is only for backwards compatibility.
+# If the entry is a file, it will be read directly. If the entry is a directory,
+# every file in that directory will be read, in alphabetical order.
 #
 # falco_rules.yaml ships with the falco package and is overridden with
 # every new software version. falco_rules.local.yaml is only created
@@ -10,6 +13,7 @@
 rules_file:
  - /etc/falco/falco_rules.yaml
  - /etc/falco/falco_rules.local.yaml
+ - /etc/falco/rules.d
 
 # Whether to output events in json or text
 json_output: false
@@ -66,6 +70,10 @@ syslog_output:
 # continuously written to, with each output message on its own
 # line. If keep_alive is set to false, the file will be re-opened
 # for each output message.
+#
+# Also, the file will be closed and reopened if falco is signaled with
+# SIGUSR1.
+
 file_output:
   enabled: false
   keep_alive: false
@@ -86,7 +94,9 @@ stdout_output:
 # continuously written to, with each output message on its own
 # line. If keep_alive is set to false, the program will be re-spawned
 # for each output message.
-
+#
+# Also, the program will be closed and reopened if falco is signaled with
+# SIGUSR1.
 program_output:
   enabled: false
   keep_alive: false

rules/CMakeLists.txt

@@ -31,7 +31,9 @@ install(FILES falco_rules.local.yaml
 	RENAME "${FALCO_LOCAL_RULES_DEST_FILENAME}")
 
 install(FILES application_rules.yaml
-	DESTINATION "${FALCO_ETC_DIR}"
+	DESTINATION "/etc/falco/rules.available"
 	RENAME "${FALCO_APP_RULES_DEST_FILENAME}")
+
+install(DIRECTORY DESTINATION "/etc/falco/rules.d")
 endif()
 

rules/falco_rules.yaml

@@ -15,6 +15,9 @@
 - macro: never_true
   condition: (evt.num=0)
 
+- macro: always_true
+  condition: (evt.num=>0)
+
 # In some cases, such as dropped system call events, information about
 # the process name may be missing. For some rules that really depend
 # on the identity of the process performing an action such as opening
@@ -23,7 +26,7 @@
   condition: (proc.name!="<NA>")
 
 - macro: rename
-  condition: evt.type = rename
+  condition: evt.type in (rename, renameat)
 - macro: mkdir
   condition: evt.type = mkdir
 - macro: remove
@@ -237,12 +240,27 @@
 
 # Network
 - macro: inbound
-  condition: ((evt.type=listen and evt.dir=>) or (evt.type=accept and evt.dir=<))
+  condition: >
+    (((evt.type in (accept,listen) and evt.dir=<)) or
+     (fd.typechar = 4 or fd.typechar = 6) and
+     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
+     (evt.rawres >= 0 or evt.res = EINPROGRESS))
 
-# Currently sendto is an ignored syscall, otherwise this could also
-# check for (evt.type=sendto and evt.dir=>)
 - macro: outbound
-  condition: evt.type=connect and evt.dir=< and (fd.typechar=4 or fd.typechar=6)
+  condition: >
+    (((evt.type = connect and evt.dir=<)) or
+     (fd.typechar = 4 or fd.typechar = 6) and
+     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
+     (evt.rawres >= 0 or evt.res = EINPROGRESS))
+
+# Very similar to inbound/outbound, but combines the tests together
+# for efficiency.
+- macro: inbound_outbound
+  condition: >
+    (((evt.type in (accept,listen,connect) and evt.dir=<)) or
+     (fd.typechar = 4 or fd.typechar = 6) and
+     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
+     (evt.rawres >= 0 or evt.res = EINPROGRESS))
 
 - macro: ssh_port
   condition: fd.sport=22
@@ -262,7 +280,7 @@
 
 - rule: Disallowed SSH Connection
   desc: Detect any new ssh connection to a host other than those in an allowed group of hosts
-  condition: (outbound or inbound) and ssh_port and not allowed_ssh_hosts
+  condition: (inbound_outbound) and ssh_port and not allowed_ssh_hosts
   output: Disallowed SSH Connection (command=%proc.cmdline connection=%fd.name user=%user.name)
   priority: NOTICE
   tags: [network]
@@ -350,6 +368,13 @@
        proc.pcmdline startswith "node /root/.config/yarn" or
        proc.pcmdline startswith "node /opt/yarn/bin/yarn.js"))
 
+
+- macro: httpd_writing_ssl_conf
+  condition: >
+    (proc.pname=run-httpd and
+     (proc.cmdline startswith "sed -ri" or proc.cmdline startswith "sed -i") and
+     (fd.name startswith /etc/httpd/conf.d/ or fd.name startswith /etc/httpd/conf))
+
 - macro: parent_Xvfb_running_xkbcomp
   condition: (proc.pname=Xvfb and proc.cmdline startswith 'sh -c "/usr/bin/xkbcomp"')
 
@@ -708,7 +733,7 @@
     and not proc.pname in (sysdigcloud_binaries, mail_config_binaries, hddtemp.postins, sshkit_script_binaries, locales.postins, deb_binaries, dhcp_binaries)
     and not fd.name pmatch (safe_etc_dirs)
     and not fd.name in (/etc/container_environment.sh, /etc/container_environment.json, /etc/motd, /etc/motd.svc)
-    and not exe_running_docker_save    
+    and not exe_running_docker_save
     and not ansible_running_python
     and not python_running_denyhosts
     and not fluentd_writing_conf_files
@@ -765,6 +790,7 @@
     and not centrify_writing_krb
     and not cockpit_writing_conf
     and not ipsec_writing_conf
+    and not httpd_writing_ssl_conf
 
 - rule: Write below etc
   desc: an attempt to write to any file below /etc
@@ -904,7 +930,12 @@
   condition: (proc.aname[2]=redis-server and (proc.cmdline contains "redis-server.post-up.d" or proc.cmdline contains "redis-server.pre-up.d"))
 
 - macro: rabbitmq_running_scripts
-  condition: (proc.pname=beam.smp and (proc.cmdline startswith "sh -c exec ps" or proc.cmdline startswith "sh -c exec inet_gethost"))
+  condition: >
+    (proc.pname=beam.smp and
+    (proc.cmdline startswith "sh -c exec ps" or
+     proc.cmdline startswith "sh -c exec inet_gethost" or
+     proc.cmdline= "sh -s unix:cmd" or
+     proc.cmdline= "sh -c exec /bin/sh -s unix:cmd 2>&1"))
 
 - macro: rabbitmqctl_running_scripts
   condition: (proc.aname[2]=rabbitmqctl and proc.cmdline startswith "sh -c ")
@@ -926,7 +957,7 @@
 
 - rule: Modify binary dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: bin_dir_rename and modify and not package_mgmt_procs and not exe_running_docker_save
+  condition: (bin_dir_rename) and modify and not package_mgmt_procs and not exe_running_docker_save
   output: >
     File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline
     operation=%evt.type file=%fd.name %evt.args)
@@ -1298,7 +1329,7 @@
   desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
   condition: >
     (fd.sockfamily = ip and system_procs)
-    and (inbound or outbound)
+    and (inbound_outbound)
     and not proc.name in (systemd, hostid)
     and not login_doing_dns_lookup
   output: >
@@ -1307,6 +1338,47 @@
   priority: NOTICE
   tags: [network]
 
+- list: openvpn_udp_ports
+  items: [1194, 1197, 1198, 8080, 9201]
+
+- list: l2tp_udp_ports
+  items: [500, 1701, 4500, 10000]
+
+- list: statsd_ports
+  items: [8125]
+
+- list: ntp_ports
+  items: [123]
+
+# Some applications will connect a udp socket to an address only to
+# test connectivity. Assuming the udp connect works, they will follow
+# up with a tcp connect that actually sends/receives data.
+#
+# With that in mind, we listed a few commonly seen ports here to avoid
+# some false positives. In addition, we make the main rule opt-in, so
+# it's disabled by default.
+
+- list: test_connect_ports
+  items: [0, 9, 80, 3306]
+
+- macro: do_unexpected_udp_check
+  condition: (never_true)
+
+- list: expected_udp_ports
+  items: [53, openvpn_udp_ports, l2tp_udp_ports, statsd_ports, ntp_ports, test_connect_ports]
+
+- macro: expected_udp_traffic
+  condition: fd.port in (expected_udp_ports)
+
+- rule: Unexpected UDP Traffic
+  desc: UDP traffic not on port 53 (DNS) or other commonly used ports
+  condition: (inbound_outbound) and do_unexpected_udp_check and fd.l4proto=udp and not expected_udp_traffic
+  output: >
+    Unexpected UDP Traffic Seen
+    (user=%user.name command=%proc.cmdline connection=%fd.name proto=%fd.l4proto evt=%evt.type %evt.args)
+  priority: NOTICE
+  tags: [network]
+
 # With the current restriction on system calls handled by falco
 # (e.g. excluding read/write/sendto/recvfrom/etc, this rule won't
 # trigger).
@@ -1457,7 +1529,7 @@
 
 - rule: Unexpected K8s NodePort Connection
   desc: Detect attempts to use K8s NodePorts from a container
-  condition: (outbound or inbound) and fd.sport >= 30000 and fd.sport <= 32767 and container and not nodeport_containers
+  condition: (inbound_outbound) and fd.sport >= 30000 and fd.sport <= 32767 and container and not nodeport_containers
   output: Unexpected K8s NodePort Connection (command=%proc.cmdline connection=%fd.name)
   priority: NOTICE
   tags: [network, k8s, container]

test/falco_test.py

@@ -31,6 +31,7 @@ class FalcoTest(Test):
 
         self.json_output = self.params.get('json_output', '*', default=False)
         self.json_include_output_property = self.params.get('json_include_output_property', '*', default=True)
+        self.all_events = self.params.get('all_events', '*', default=False)
         self.priority = self.params.get('priority', '*', default='debug')
         self.rules_file = self.params.get('rules_file', '*', default=os.path.join(self.basedir, '../rules/falco_rules.yaml'))
 
@@ -213,7 +214,7 @@ class FalcoTest(Test):
         triggered_rules = match.group(1)
 
         for rule, count in self.detect_counts.iteritems():
-            expected = '{}: (\d+)'.format(rule)
+            expected = '\s{}: (\d+)'.format(rule)
             match = re.search(expected, triggered_rules)
 
             if match is None:
@@ -365,6 +366,9 @@ class FalcoTest(Test):
         if self.run_duration:
             cmd += ' -M {}'.format(self.run_duration)
 
+        if self.all_events:
+            cmd += ' -A'
+
         self.falco_proc = process.SubProcess(cmd)
 
         res = self.falco_proc.run(timeout=180, sig=9)

test/falco_tests.yaml

@@ -128,6 +128,18 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
+    all_events: True
+
+  rules_directory:
+    detect: True
+    detect_level:
+      - WARNING
+      - INFO
+      - ERROR
+    rules_file:
+      - rules/rules_dir
+    trace_file: trace_files/cat_write.scap
+    all_events: True
 
   multiple_rules_suppress_info:
     detect: True
@@ -143,6 +155,7 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
+    all_events: True
 
   multiple_rules_overriding:
     detect: False
@@ -642,6 +655,14 @@ trace_files: !mux
       - rules/rule_append_failure.yaml
     trace_file: trace_files/cat_write.scap
 
+  rule_append_skipped:
+    detect: False
+    priority: ERROR
+    rules_file:
+      - rules/single_rule.yaml
+      - rules/append_single_rule.yaml
+    trace_file: trace_files/cat_write.scap
+
   rule_append:
     detect: True
     detect_level: WARNING
@@ -670,4 +691,25 @@ trace_files: !mux
     detect_level: INFO
     rules_file:
       - rules/detect_connect_using_in.yaml
-    trace_file: trace_files/connect_localhost.scap
+    trace_file: trace_files/connect_localhost.scap
+
+  syscalls:
+    detect: True
+    detect_level: INFO
+    rules_file:
+      - rules/syscalls.yaml
+    detect_counts:
+      - detect_madvise: 2
+      - detect_open: 2
+    trace_file: trace_files/syscall.scap
+    all_events: True
+
+  catchall_order:
+    detect: True
+    detect_level: INFO
+    rules_file:
+      - rules/catchall_order.yaml
+    detect_counts:
+      - open_dev_null: 1
+        dev_null: 0
+    trace_file: trace_files/cat_write.scap

test/rules/append_single_rule.yaml

@@ -0,0 +1,3 @@
+- rule: open_from_cat
+  append: true
+  condition: and fd.name=/tmp

test/rules/catchall_order.yaml

@@ -0,0 +1,12 @@
+- rule: open_dev_null
+  desc: Any open of the file /dev/null
+  condition: evt.type=open and fd.name=/dev/null
+  output: An open of /dev/null was seen (command=%proc.cmdline evt=%evt.type %evt.args)
+  priority: INFO
+
+- rule: dev_null
+  desc: Anything related to /dev/null
+  condition: fd.name=/dev/null
+  output: Something related to /dev/null was seen (command=%proc.cmdline evt=%evt.type %evt.args)
+  priority: INFO
+  warn_evttypes: false

test/rules/rules_dir/000-single_rule.yaml

@@ -0,0 +1,14 @@
+- list: cat_binaries
+  items: [cat]
+
+- list: cat_capable_binaries
+  items: [cat_binaries]
+
+- macro: is_cat
+  condition: proc.name in (cat_capable_binaries)
+
+- rule: open_from_cat
+  desc: A process named cat does an open
+  condition: evt.type=open and is_cat
+  output: "An open was seen (command=%proc.cmdline)"
+  priority: WARNING

test/rules/rules_dir/001-double_rule.yaml

@@ -0,0 +1,13 @@
+# This ruleset depends on the is_cat macro defined in single_rule.yaml
+
+- rule: exec_from_cat
+  desc: A process named cat does execve
+  condition: evt.type=execve and is_cat
+  output: "An exec was seen (command=%proc.cmdline)"
+  priority: ERROR
+
+- rule: access_from_cat
+  desc: A process named cat does an access
+  condition: evt.type=access and is_cat
+  output: "An access was seen (command=%proc.cmdline)"
+  priority: INFO

test/rules/syscalls.yaml

@@ -0,0 +1,11 @@
+- rule: detect_madvise
+  desc: Detect any call to madvise
+  condition: evt.type=madvise and evt.dir=<
+  output: A madvise syscall was seen (command=%proc.cmdline evt=%evt.type)
+  priority: INFO
+
+- rule: detect_open
+  desc: Detect any call to open
+  condition: evt.type=open and evt.dir=< and fd.name=/dev/null
+  output: An open syscall was seen (command=%proc.cmdline evt=%evt.type file=%fd.name)
+  priority: INFO

userspace/engine/falco_engine.cpp

@@ -162,6 +162,13 @@ void falco_engine::evttypes_for_ruleset(std::vector<bool> &evttypes, const std::
 	return m_evttype_filter->evttypes_for_ruleset(evttypes, ruleset_id);
 }
 
+void falco_engine::syscalls_for_ruleset(std::vector<bool> &syscalls, const std::string &ruleset)
+{
+	uint16_t ruleset_id = find_ruleset_id(ruleset);
+
+	return m_evttype_filter->syscalls_for_ruleset(syscalls, ruleset_id);
+}
+
 unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev, uint16_t ruleset_id)
 {
 	if(should_drop_evt())
@@ -237,10 +244,11 @@ void falco_engine::print_stats()
 
 void falco_engine::add_evttype_filter(string &rule,
 				      set<uint32_t> &evttypes,
+				      set<uint32_t> &syscalls,
 				      set<string> &tags,
 				      sinsp_filter* filter)
 {
-	m_evttype_filter->add(rule, evttypes, tags, filter);
+	m_evttype_filter->add(rule, evttypes, syscalls, tags, filter);
 }
 
 void falco_engine::clear_filters()

userspace/engine/falco_engine.h

@@ -91,6 +91,12 @@ public:
 	//
 	void evttypes_for_ruleset(std::vector<bool> &evttypes, const std::string &ruleset);
 
+	//
+	// Given a ruleset, fill in a bitset containing the syscalls
+	// for which this ruleset can run.
+	//
+	void syscalls_for_ruleset(std::vector<bool> &syscalls, const std::string &ruleset);
+
 	//
 	// Given an event, check it against the set of rules in the
 	// engine and if a matching rule is found, return details on
@@ -122,10 +128,11 @@ public:
 
 	//
 	// Add a filter, which is related to the specified set of
-	// event types, to the engine.
+	// event types/syscalls, to the engine.
 	//
 	void add_evttype_filter(std::string &rule,
 				std::set<uint32_t> &evttypes,
+				std::set<uint32_t> &syscalls,
 				std::set<std::string> &tags,
 				sinsp_filter* filter);
 

userspace/engine/lua/compiler.lua

@@ -191,19 +191,20 @@ function check_for_ignored_syscalls_events(ast, filter_type, source)
    parser.traverse_ast(ast, {BinaryRelOp=1}, cb)
 end
 
--- Examine the ast and find the event types for which the rule should
--- run. All evt.type references are added as event types up until the
--- first "!=" binary operator or unary not operator. If no event type
--- checks are found afterward in the rule, the rule is considered
--- optimized and is associated with the event type(s).
+-- Examine the ast and find the event types/syscalls for which the
+-- rule should run. All evt.type references are added as event types
+-- up until the first "!=" binary operator or unary not operator. If
+-- no event type checks are found afterward in the rule, the rule is
+-- considered optimized and is associated with the event type(s).
 --
 -- Otherwise, the rule is associated with a 'catchall' category and is
--- run for all event types. (Also, a warning is printed).
+-- run for all event types/syscalls. (Also, a warning is printed).
 --
 
-function get_evttypes(name, ast, source)
+function get_evttypes_syscalls(name, ast, source, warn_evttypes)
 
    local evttypes = {}
+   local syscallnums = {}
    local evtnames = {}
    local found_event = false
    local found_not = false
@@ -226,17 +227,45 @@ function get_evttypes(name, ast, source)
 	    if node.operator == "in" or node.operator == "pmatch" then
 	       for i, v in ipairs(node.right.elements) do
 		  if v.type == "BareString" then
+
+                     -- The event must be a known event
+                     if events[v.value] == nil and syscalls[v.value] == nil then
+                        error("Unknown event/syscall \""..v.value.."\" in filter: "..source)
+                     end
+
 		     evtnames[v.value] = 1
-		     for id in string.gmatch(events[v.value], "%S+") do
-			evttypes[id] = 1
+		     if events[v.value] ~= nil then
+			for id in string.gmatch(events[v.value], "%S+") do
+			   evttypes[id] = 1
+			end
+		     end
+
+		     if syscalls[v.value] ~= nil then
+			for id in string.gmatch(syscalls[v.value], "%S+") do
+			   syscallnums[id] = 1
+			end
 		     end
 		  end
 	       end
 	    else
 	       if node.right.type == "BareString" then
+
+		  -- The event must be a known event
+		  if events[node.right.value] == nil and syscalls[node.right.value] == nil then
+		     error("Unknown event/syscall \""..node.right.value.."\" in filter: "..source)
+		  end
+
 		  evtnames[node.right.value] = 1
-		  for id in string.gmatch(events[node.right.value], "%S+") do
-		     evttypes[id] = 1
+		  if events[node.right.value] ~= nil then
+		     for id in string.gmatch(events[node.right.value], "%S+") do
+			evttypes[id] = 1
+		     end
+		  end
+
+		  if syscalls[node.right.value] ~= nil then
+		     for id in string.gmatch(syscalls[node.right.value], "%S+") do
+			syscallnums[id] = 1
+		     end
 		  end
 	       end
 	    end
@@ -247,23 +276,29 @@ function get_evttypes(name, ast, source)
    parser.traverse_ast(ast.filter.value, {BinaryRelOp=1, UnaryBoolOp=1} , cb)
 
    if not found_event then
-      io.stderr:write("Rule "..name..": warning (no-evttype):\n")
-      io.stderr:write(source.."\n")
-      io.stderr:write("         did not contain any evt.type restriction, meaning it will run for all event types.\n")
-      io.stderr:write("         This has a significant performance penalty. Consider adding an evt.type restriction if possible.\n")
+      if warn_evttypes == true then
+	 io.stderr:write("Rule "..name..": warning (no-evttype):\n")
+	 io.stderr:write(source.."\n")
+	 io.stderr:write("         did not contain any evt.type restriction, meaning it will run for all event types.\n")
+	 io.stderr:write("         This has a significant performance penalty. Consider adding an evt.type restriction if possible.\n")
+      end
       evttypes = {}
+      syscallnums = {}
       evtnames = {}
    end
 
    if found_event_after_not then
-      io.stderr:write("Rule "..name..": warning (trailing-evttype):\n")
-      io.stderr:write(source.."\n")
-      io.stderr:write("         does not have all evt.type restrictions at the beginning of the condition,\n")
-      io.stderr:write("         or uses a negative match (i.e. \"not\"/\"!=\") for some evt.type restriction.\n")
-      io.stderr:write("         This has a performance penalty, as the rule can not be limited to specific event types.\n")
-      io.stderr:write("         Consider moving all evt.type restrictions to the beginning of the rule and/or\n")
-      io.stderr:write("         replacing negative matches with positive matches if possible.\n")
+      if warn_evttypes == true then
+	 io.stderr:write("Rule "..name..": warning (trailing-evttype):\n")
+	 io.stderr:write(source.."\n")
+	 io.stderr:write("         does not have all evt.type restrictions at the beginning of the condition,\n")
+	 io.stderr:write("         or uses a negative match (i.e. \"not\"/\"!=\") for some evt.type restriction.\n")
+	 io.stderr:write("         This has a performance penalty, as the rule can not be limited to specific event types.\n")
+	 io.stderr:write("         Consider moving all evt.type restrictions to the beginning of the rule and/or\n")
+	 io.stderr:write("         replacing negative matches with positive matches if possible.\n")
+      end
       evttypes = {}
+      syscallnums = {}
       evtnames = {}
    end
 
@@ -281,10 +316,10 @@ function get_evttypes(name, ast, source)
    table.sort(evtnames_only)
 
    if compiler.verbose then
-      io.stderr:write("Event types for rule "..name..": "..table.concat(evtnames_only, ",").."\n")
+      io.stderr:write("Event types/Syscalls for rule "..name..": "..table.concat(evtnames_only, ",").."\n")
    end
 
-   return evttypes
+   return evttypes, syscallnums
 end
 
 function compiler.expand_lists_in(source, list_defs)
@@ -344,7 +379,7 @@ end
 --[[
    Parses a single filter, then expands macros using passed-in table of definitions. Returns resulting AST.
 --]]
-function compiler.compile_filter(name, source, macro_defs, list_defs)
+function compiler.compile_filter(name, source, macro_defs, list_defs, warn_evttypes)
 
    source = compiler.expand_lists_in(source, list_defs)
 
@@ -371,9 +406,9 @@ function compiler.compile_filter(name, source, macro_defs, list_defs)
       error("Unexpected top-level AST type: "..ast.type)
    end
 
-   evttypes = get_evttypes(name, ast, source)
+   evttypes, syscallnums = get_evttypes_syscalls(name, ast, source, warn_evttypes)
 
-   return ast, evttypes
+   return ast, evttypes, syscallnums
 end
 
 

userspace/engine/lua/rule_loader.lua

@@ -132,7 +132,8 @@ end
 -- object. The by_name index is used for things like describing rules,
 -- and the by_idx index is used to map the relational node index back
 -- to a rule.
-local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, macros_by_name={}, lists_by_name={},
+local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={},
+	       skipped_rules_by_name={}, macros_by_name={}, lists_by_name={},
 	       n_rules=0, rules_by_idx={}, ordered_rule_names={}, ordered_macro_names={}, ordered_list_names={}}
 
 local function reset_rules(rules_mgr)
@@ -291,11 +292,13 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	    end
 
 	    if state.rules_by_name[v['rule']] == nil then
-	       error ("Rule " ..v['rule'].. " has 'append' key but no rule by that name already exists")
+	       if state.skipped_rules_by_name[v['rule']] == nil then
+		  error ("Rule " ..v['rule'].. " has 'append' key but no rule by that name already exists")
+	       end
+	    else
+	       state.rules_by_name[v['rule']]['condition'] = state.rules_by_name[v['rule']]['condition'] .. " " .. v['condition']
 	    end
 
-	    state.rules_by_name[v['rule']]['condition'] = state.rules_by_name[v['rule']]['condition'] .. " " .. v['condition']
-
 	 else
 
 	    for i, field in ipairs({'condition', 'output', 'desc', 'priority'}) do
@@ -320,6 +323,8 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	       v['output'] = compiler.trim(v['output'])
 
 	       state.rules_by_name[v['rule']] = v
+	    else
+	       state.skipped_rules_by_name[v['rule']] = v
 	    end
 	 end
       else
@@ -368,8 +373,14 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 
       local v = state.rules_by_name[name]
 
-      local filter_ast, evttypes = compiler.compile_filter(v['rule'], v['condition'],
-							   state.macros, state.lists)
+      warn_evttypes = true
+      if v['warn_evttypes'] ~= nil then
+	 warn_evttypes = v['warn_evttypes']
+      end
+
+      local filter_ast, evttypes, syscallnums = compiler.compile_filter(v['rule'], v['condition'],
+									state.macros, state.lists,
+									warn_evttypes)
 
       if (filter_ast.type == "Rule") then
 	 state.n_rules = state.n_rules + 1
@@ -390,7 +401,7 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	 end
 
 	 -- Pass the filter and event types back up
-	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes, v['tags'])
+	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes, syscallnums, v['tags'])
 
 	 -- Rule ASTs are merged together into one big AST, with "OR" between each
 	 -- rule.

userspace/engine/rules.cpp

@@ -66,8 +66,9 @@ void falco_rules::clear_filters()
 
 int falco_rules::add_filter(lua_State *ls)
 {
-	if (! lua_islightuserdata(ls, -4) ||
-	    ! lua_isstring(ls, -3) ||
+	if (! lua_islightuserdata(ls, -5) ||
+	    ! lua_isstring(ls, -4) ||
+	    ! lua_istable(ls, -3) ||
 	    ! lua_istable(ls, -2) ||
 	    ! lua_istable(ls, -1))
 	{
@@ -75,16 +76,28 @@ int falco_rules::add_filter(lua_State *ls)
 		lua_error(ls);
 	}
 
-	falco_rules *rules = (falco_rules *) lua_topointer(ls, -4);
-	const char *rulec = lua_tostring(ls, -3);
+	falco_rules *rules = (falco_rules *) lua_topointer(ls, -5);
+	const char *rulec = lua_tostring(ls, -4);
 
 	set<uint32_t> evttypes;
 
+	lua_pushnil(ls);  /* first key */
+	while (lua_next(ls, -4) != 0) {
+                // key is at index -2, value is at index
+                // -1. We want the keys.
+		evttypes.insert(luaL_checknumber(ls, -2));
+
+		// Remove value, keep key for next iteration
+		lua_pop(ls, 1);
+	}
+
+	set<uint32_t> syscalls;
+
 	lua_pushnil(ls);  /* first key */
 	while (lua_next(ls, -3) != 0) {
                 // key is at index -2, value is at index
                 // -1. We want the keys.
-		evttypes.insert(luaL_checknumber(ls, -2));
+		syscalls.insert(luaL_checknumber(ls, -2));
 
 		// Remove value, keep key for next iteration
 		lua_pop(ls, 1);
@@ -95,7 +108,7 @@ int falco_rules::add_filter(lua_State *ls)
 	lua_pushnil(ls);  /* first key */
 	while (lua_next(ls, -2) != 0) {
                 // key is at index -2, value is at index
-                // -1. We want the keys.
+                // -1. We want the values.
 		tags.insert(lua_tostring(ls, -1));
 
 		// Remove value, keep key for next iteration
@@ -103,19 +116,19 @@ int falco_rules::add_filter(lua_State *ls)
 	}
 
 	std::string rule = rulec;
-	rules->add_filter(rule, evttypes, tags);
+	rules->add_filter(rule, evttypes, syscalls, tags);
 
 	return 0;
 }
 
-void falco_rules::add_filter(string &rule, set<uint32_t> &evttypes, set<string> &tags)
+void falco_rules::add_filter(string &rule, set<uint32_t> &evttypes, set<uint32_t> &syscalls, set<string> &tags)
 {
 	// While the current rule was being parsed, a sinsp_filter
 	// object was being populated by lua_parser. Grab that filter
 	// and pass it to the engine.
 	sinsp_filter *filter = m_lua_parser->get_filter(true);
 
-	m_engine->add_evttype_filter(rule, evttypes, tags, filter);
+	m_engine->add_evttype_filter(rule, evttypes, syscalls, tags, filter);
 }
 
 int falco_rules::enable_rule(lua_State *ls)
@@ -183,6 +196,35 @@ void falco_rules::load_rules(const string &rules_content,
 
 		lua_setglobal(m_ls, m_lua_events.c_str());
 
+		map<string,string> syscalls_by_name;
+		for(uint32_t j = 0; j < PPM_SC_MAX; j++)
+		{
+			auto it = syscalls_by_name.find(stable[j].name);
+
+			if (it == syscalls_by_name.end())
+			{
+				syscalls_by_name[stable[j].name] = to_string(j);
+			}
+			else
+			{
+				string cur = it->second;
+				cur += " ";
+				cur += to_string(j);
+				syscalls_by_name[stable[j].name] = cur;
+			}
+		}
+
+		lua_newtable(m_ls);
+
+		for( auto kv : syscalls_by_name)
+		{
+			lua_pushstring(m_ls, kv.first.c_str());
+			lua_pushstring(m_ls, kv.second.c_str());
+			lua_settable(m_ls, -3);
+		}
+
+		lua_setglobal(m_ls, m_lua_syscalls.c_str());
+
 		// Create a table containing the syscalls/events that
 		// are ignored by the kernel module. load_rules will
 		// return an error if any rule references one of these

userspace/engine/rules.h

@@ -45,7 +45,7 @@ class falco_rules
 
  private:
 	void clear_filters();
-	void add_filter(string &rule, std::set<uint32_t> &evttypes, std::set<string> &tags);
+	void add_filter(string &rule, std::set<uint32_t> &evttypes, std::set<uint32_t> &syscalls, std::set<string> &tags);
 	void enable_rule(string &rule, bool enabled);
 
 	lua_parser* m_lua_parser;
@@ -57,5 +57,6 @@ class falco_rules
 	string m_lua_ignored_syscalls = "ignored_syscalls";
 	string m_lua_ignored_events = "ignored_events";
 	string m_lua_events = "events";
+	string m_lua_syscalls = "syscalls";
 	string m_lua_describe_rule = "describe_rule";
 };

userspace/falco/configuration.cpp

@@ -16,6 +16,13 @@ You should have received a copy of the GNU General Public License
 along with falco.  If not, see <http://www.gnu.org/licenses/>.
 */
 
+#include <algorithm>
+
+#include <dirent.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
 #include "configuration.h"
 #include "logger.h"
 
@@ -62,7 +69,7 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 		struct stat buffer;
 		if(stat(file.c_str(), &buffer) == 0)
 		{
-			m_rules_filenames.push_back(file);
+			read_rules_file_directory(file, m_rules_filenames);
 		}
 	}
 
@@ -150,6 +157,70 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 	falco_logger::log_syslog = m_config->get_scalar<bool>("log_syslog", true);
 }
 
+void falco_configuration::read_rules_file_directory(const string &path, list<string> &rules_filenames)
+{
+	struct stat st;
+
+	int rc = stat(path.c_str(), &st);
+
+	if(rc != 0)
+	{
+		std::cerr << "Could not get info on rules file " << path << ": " << strerror(errno) << std::endl;
+		exit(-1);
+	}
+
+	if(st.st_mode & S_IFDIR)
+	{
+		// It's a directory. Read the contents, sort
+		// alphabetically, and add every path to
+		// rules_filenames
+		vector<string> dir_filenames;
+
+		DIR *dir = opendir(path.c_str());
+
+		if(!dir)
+		{
+			std::cerr << "Could not get read contents of directory " << path << ": " << strerror(errno) << std::endl;
+			exit(-1);
+		}
+
+		for (struct dirent *ent = readdir(dir); ent; ent = readdir(dir))
+		{
+			string efile = path + "/" + ent->d_name;
+
+			rc = stat(efile.c_str(), &st);
+
+			if(rc != 0)
+			{
+				std::cerr << "Could not get info on rules file " << efile << ": " << strerror(errno) << std::endl;
+				exit(-1);
+			}
+
+			if(st.st_mode & S_IFREG)
+			{
+				dir_filenames.push_back(efile);
+			}
+		}
+
+		closedir(dir);
+
+		std::sort(dir_filenames.begin(),
+			  dir_filenames.end());
+
+		for (string &ent : dir_filenames)
+		{
+			rules_filenames.push_back(ent);
+		}
+	}
+	else
+	{
+		// Assume it's a file and just add to
+		// rules_filenames. If it can't be opened/etc that
+		// will be reported later..
+		rules_filenames.push_back(path);
+	}
+}
+
 static bool split(const string &str, char delim, pair<string,string> &parts)
 {
 	size_t pos;

userspace/falco/configuration.h

@@ -165,6 +165,8 @@ class falco_configuration
 	void init(std::string conf_filename, std::list<std::string> &cmdline_options);
 	void init(std::list<std::string> &cmdline_options);
 
+	static void read_rules_file_directory(const string &path, list<string> &rules_filenames);
+
 	std::list<std::string> m_rules_filenames;
 	bool m_json_output;
 	bool m_json_include_output_property;

userspace/falco/falco.cpp

@@ -22,6 +22,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <fstream>
 #include <set>
 #include <list>
+#include <vector>
+#include <algorithm>
 #include <string>
 #include <signal.h>
 #include <fcntl.h>
@@ -32,6 +34,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <sinsp.h>
 
 #include "logger.h"
+#include "utils.h"
 
 #include "configuration.h"
 #include "falco_engine.h"
@@ -39,6 +42,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "statsfilewriter.h"
 
 bool g_terminate = false;
+bool g_reopen_outputs = false;
+
 //
 // Helper functions
 //
@@ -47,6 +52,11 @@ static void signal_callback(int signal)
 	g_terminate = true;
 }
 
+static void reopen_outputs(int signal)
+{
+	g_reopen_outputs = true;
+}
+
 //
 // Program help
 //
@@ -99,8 +109,9 @@ static void usage()
 	   "                               of %%container.info in rule output fields\n"
 	   "                               See the examples section below for more info.\n"
 	   " -P, --pidfile <pid_file>      When run as a daemon, write pid to specified file\n"
-           " -r <rules_file>               Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml).\n"
-	   "                               Can be specified multiple times to read from multiple files.\n"
+           " -r <rules_file>               Rules file/directory (defaults to value set in configuration file,\n"
+           "                               or /etc/falco_rules.yaml). Can be specified multiple times to read\n"
+           "                               from multiple files/directories.\n"
 	   " -s <stats_file>               If specified, write statistics related to falco's reading/processing of events\n"
 	   "                               to this file. (Only useful in live mode).\n"
 	   " -T <tag>                      Disable any rules with a tag=<tag>. Can be specified multiple times.\n"
@@ -143,7 +154,8 @@ uint64_t do_inspect(falco_engine *engine,
 		    falco_outputs *outputs,
 		    sinsp* inspector,
 		    uint64_t duration_to_tot_ns,
-		    string &stats_filename)
+		    string &stats_filename,
+		    bool all_events)
 {
 	uint64_t num_evts = 0;
 	int32_t res;
@@ -171,6 +183,12 @@ uint64_t do_inspect(falco_engine *engine,
 
 		writer.handle();
 
+		if(g_reopen_outputs)
+		{
+			outputs->reopen_outputs();
+			g_reopen_outputs = false;
+		}
+
 		if (g_terminate)
 		{
 			break;
@@ -204,8 +222,7 @@ uint64_t do_inspect(falco_engine *engine,
 			}
 		}
 
-		if(!inspector->is_debug_enabled() &&
-			ev->get_category() & EC_INTERNAL)
+		if(!ev->falco_consider() && !all_events)
 		{
 			continue;
 		}
@@ -227,6 +244,47 @@ uint64_t do_inspect(falco_engine *engine,
 	return num_evts;
 }
 
+static void print_all_ignored_events(sinsp *inspector)
+{
+	sinsp_evttables* einfo = inspector->get_event_info_tables();
+	const struct ppm_event_info* etable = einfo->m_event_info;
+	const struct ppm_syscall_desc* stable = einfo->m_syscall_info_table;
+
+	std::set<string> ignored_event_names;
+	for(uint32_t j = 0; j < PPM_EVENT_MAX; j++)
+	{
+		if(!sinsp::falco_consider_evtnum(j))
+		{
+			std::string name = etable[j].name;
+			// Ignore event names NA*
+			if(name.find("NA") != 0)
+			{
+				ignored_event_names.insert(name);
+			}
+		}
+	}
+
+	for(uint32_t j = 0; j < PPM_SC_MAX; j++)
+	{
+		if(!sinsp::falco_consider_syscallid(j))
+		{
+			std::string name = stable[j].name;
+			// Ignore event names NA*
+			if(name.find("NA") != 0)
+			{
+				ignored_event_names.insert(name);
+			}
+		}
+	}
+
+	printf("Ignored Event(s):");
+	for(auto it : ignored_event_names)
+	{
+		printf(" %s", it.c_str());
+	}
+	printf("\n");
+}
+
 //
 // ARGUMENT PARSING AND PROGRAM SETUP
 //
@@ -256,6 +314,7 @@ int falco_init(int argc, char **argv)
 	string output_format = "";
 	bool replace_container_info = false;
 	int duration_to_tot = 0;
+	bool print_ignored_events = false;
 
 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -285,6 +344,7 @@ int falco_init(int argc, char **argv)
 		{"version", no_argument, 0, 0 },
 		{"validate", required_argument, 0, 'V' },
 		{"writefile", required_argument, 0, 'w' },
+		{"ignored-events", no_argument, 0, 'i'},
 
 		{0, 0, 0, 0}
 	};
@@ -301,7 +361,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
+                                        "hc:AdD:e:ik:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -327,6 +387,9 @@ int falco_init(int argc, char **argv)
 				k8s_api = new string();
 				mesos_api = new string();
 				break;
+			case 'i':
+				print_ignored_events = true;
+				break;
 			case 'k':
 				k8s_api = new string(optarg);
 				break;
@@ -378,7 +441,7 @@ int falco_init(int argc, char **argv)
 				}
 				break;
 			case 'r':
-				rules_filenames.push_back(optarg);
+				falco_configuration::read_rules_file_directory(string(optarg), rules_filenames);
 				break;
 			case 's':
 				stats_filename = optarg;
@@ -417,12 +480,20 @@ int falco_init(int argc, char **argv)
 			return EXIT_SUCCESS;
 		}
 
-
 		inspector = new sinsp();
+
+		if(print_ignored_events)
+		{
+			print_all_ignored_events(inspector);
+			delete(inspector);
+			return EXIT_SUCCESS;
+		}
+
 		engine = new falco_engine();
 		engine->set_inspector(inspector);
 		engine->set_extra(output_format, replace_container_info);
 
+
 		outputs = new falco_outputs();
 		outputs->set_inspector(inspector);
 
@@ -503,13 +574,19 @@ int falco_init(int argc, char **argv)
 
 		if(config.m_rules_filenames.size() == 0)
 		{
-			throw std::invalid_argument("You must specify at least one rules file via -r or a rules_file entry in falco.yaml");
+			throw std::invalid_argument("You must specify at least one rules file/directory via -r or a rules_file entry in falco.yaml");
+		}
+
+		falco_logger::log(LOG_DEBUG, "Configured rules filenames:\n");
+		for (auto filename : config.m_rules_filenames)
+		{
+			falco_logger::log(LOG_DEBUG, string("   ") + filename + "\n");
 		}
 
 		for (auto filename : config.m_rules_filenames)
 		{
+			falco_logger::log(LOG_INFO, "Loading rules from file " + filename + ":\n");
 			engine->load_rules_file(filename, verbose, all_events);
-			falco_logger::log(LOG_INFO, "Parsed rules from file " + filename + "\n");
 		}
 
 		// You can't both disable and enable rules
@@ -554,6 +631,7 @@ int falco_init(int argc, char **argv)
 		if(!all_events)
 		{
 			inspector->set_drop_event_flags(EF_DROP_FALCO);
+			inspector->start_dropping_mode(1);
 		}
 
 		if (describe_all_rules)
@@ -589,6 +667,13 @@ int falco_init(int argc, char **argv)
 			goto exit;
 		}
 
+		if(signal(SIGUSR1, reopen_outputs) == SIG_ERR)
+		{
+			fprintf(stderr, "An error occurred while setting SIGUSR1 signal handler.\n");
+			result = EXIT_FAILURE;
+			goto exit;
+		}
+
 		if (scap_filename.size())
 		{
 			inspector->open(scap_filename);
@@ -733,7 +818,8 @@ int falco_init(int argc, char **argv)
 				      outputs,
 				      inspector,
 				      uint64_t(duration_to_tot*ONE_SECOND_IN_NS),
-				      stats_filename);
+				      stats_filename,
+				      all_events);
 
 		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;
 

userspace/falco/falco_outputs.cpp

@@ -142,3 +142,19 @@ void falco_outputs::handle_event(sinsp_evt *ev, string &rule, falco_common::prio
 	}
 
 }
+
+void falco_outputs::reopen_outputs()
+{
+	lua_getglobal(m_ls, m_lua_output_reopen.c_str());
+
+	if(!lua_isfunction(m_ls, -1))
+	{
+		throw falco_exception("No function " + m_lua_output_reopen + " found. ");
+	}
+
+	if(lua_pcall(m_ls, 0, 0, 0) != 0)
+	{
+		const char* lerr = lua_tostring(m_ls, -1);
+		throw falco_exception(string(lerr));
+	}
+}

userspace/falco/falco_outputs.h

@@ -53,6 +53,8 @@ public:
 	//
 	void handle_event(sinsp_evt *ev, std::string &rule, falco_common::priority_type priority, std::string &format);
 
+	void reopen_outputs();
+
 private:
 	bool m_initialized;
 
@@ -64,5 +66,6 @@ private:
 	std::string m_lua_add_output = "add_output";
 	std::string m_lua_output_event = "output_event";
 	std::string m_lua_output_cleanup = "output_cleanup";
+	std::string m_lua_output_reopen = "output_reopen";
 	std::string m_lua_main_filename = "output.lua";
 };

userspace/falco/lua/output.lua

@@ -20,8 +20,8 @@ local mod = {}
 
 local outputs = {}
 
-function mod.stdout(priority, priority_num, buffered, msg)
-   if buffered == 0 then
+function mod.stdout(priority, priority_num, msg, options)
+   if options.buffered == 0 then
       io.stdout:setvbuf 'no'
    end
    print (msg)
@@ -31,6 +31,10 @@ function mod.stdout_cleanup()
    io.stdout:flush()
 end
 
+-- Note: not actually closing/reopening stdout
+function mod.stdout_reopen(options)
+end
+
 function mod.file_validate(options)
    if (not type(options.filename) == 'string') then
       error("File output needs to be configured with a valid filename")
@@ -44,73 +48,98 @@ function mod.file_validate(options)
 
 end
 
-function mod.file(priority, priority_num, buffered, msg, options)
-   if options.keep_alive == "true" then
-      if file == nil then
-	 file = io.open(options.filename, "a+")
-	 if buffered == 0 then
-	    file:setvbuf 'no'
-	 end
+function mod.file_open(options)
+   if ffile == nil then
+      ffile = io.open(options.filename, "a+")
+      if options.buffered == 0 then
+	 ffile:setvbuf 'no'
       end
+   end
+end
+
+function mod.file(priority, priority_num, msg, options)
+   if options.keep_alive == "true" then
+      mod.file_open(options)
    else
-      file = io.open(options.filename, "a+")
+      ffile = io.open(options.filename, "a+")
    end
 
-   file:write(msg, "\n")
+   ffile:write(msg, "\n")
 
    if options.keep_alive == nil or
-      options.keep_alive ~= "true" then
-	 file:close()
-	 file = nil
+          options.keep_alive ~= "true" then
+      ffile:close()
+      ffile = nil
    end
 end
 
 function mod.file_cleanup()
-   if file ~= nil then
-      file:flush()
-      file:close()
-      file = nil
+   if ffile ~= nil then
+      ffile:flush()
+      ffile:close()
+      ffile = nil
    end
 end
 
-function mod.syslog(priority, priority_num, buffered, msg, options)
+function mod.file_reopen(options)
+   if options.keep_alive == "true" then
+      mod.file_cleanup()
+      mod.file_open(options)
+   end
+end
+
+function mod.syslog(priority, priority_num, msg, options)
    falco.syslog(priority_num, msg)
 end
 
 function mod.syslog_cleanup()
 end
 
-function mod.program(priority, priority_num, buffered, msg, options)
+function mod.syslog_reopen()
+end
+
+function mod.program_open(options)
+   if pfile == nil then
+      pfile = io.popen(options.program, "w")
+      if options.buffered == 0 then
+	 pfile:setvbuf 'no'
+      end
+   end
+end
+
+function mod.program(priority, priority_num, msg, options)
    -- XXX Ideally we'd check that the program ran
    -- successfully. However, the luajit we're using returns true even
    -- when the shell can't run the program.
 
    -- Note: options are all strings
    if options.keep_alive == "true" then
-      if file == nil then
-	 file = io.popen(options.program, "w")
-	 if buffered == 0 then
-	    file:setvbuf 'no'
-	 end
-      end
+      mod.program_open(options)
    else
-      file = io.popen(options.program, "w")
+      pfile = io.popen(options.program, "w")
    end
 
-   file:write(msg, "\n")
+   pfile:write(msg, "\n")
 
    if options.keep_alive == nil or
-      options.keep_alive ~= "true" then
-	 file:close()
-	 file = nil
+          options.keep_alive ~= "true" then
+      pfile:close()
+      pfile = nil
    end
 end
 
 function mod.program_cleanup()
-   if file ~= nil then
-      file:flush()
-      file:close()
-      file = nil
+   if pfile ~= nil then
+      pfile:flush()
+      pfile:close()
+      pfile = nil
+   end
+end
+
+function mod.program_reopen(options)
+   if options.keep_alive == "true" then
+      mod.program_cleanup()
+      mod.program_open(options)
    end
 end
 
@@ -126,7 +155,7 @@ function output_event(event, rule, priority, priority_num, format)
    msg = formats.format_event(event, rule, priority, format)
 
    for index,o in ipairs(outputs) do
-      o.output(priority, priority_num, o.buffered, msg, o.config)
+      o.output(priority, priority_num, msg, o.options)
    end
 end
 
@@ -137,20 +166,33 @@ function output_cleanup()
    end
 end
 
-function add_output(output_name, buffered, config)
+function output_reopen()
+   for index,o in ipairs(outputs) do
+      o.reopen(o.options)
+   end
+end
+
+function add_output(output_name, buffered, options)
    if not (type(mod[output_name]) == 'function') then
       error("rule_loader.add_output(): invalid output_name: "..output_name)
    end
 
    -- outputs can optionally define a validation function so that we don't
-   -- find out at runtime (when an event finally matches a rule!) that the config is invalid
+   -- find out at runtime (when an event finally matches a rule!) that the options are invalid
    if (type(mod[output_name.."_validate"]) == 'function') then
-     mod[output_name.."_validate"](config)
+     mod[output_name.."_validate"](options)
    end
 
+   if options == nil then
+      options = {}
+   end
+
+   options.buffered = buffered
+
    table.insert(outputs, {output = mod[output_name],
 			  cleanup = mod[output_name.."_cleanup"],
-			  buffered=buffered, config=config})
+			  reopen = mod[output_name.."_reopen"],
+			  options=options})
 end
 
 return mod