Merge remote-tracking branch 'origin/dev' into agent-master

the commit.

CHANGELOG.md

@@ -2,48 +2,6 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
-## v0.10.0
-
-Released 2018-04-24
-
-## Major Changes
-
-* **Rules Directory Support**: Falco will read rules files from `/etc/falco/rules.d` in addition to `/etc/falco/falco_rules.yaml` and `/etc/falco/falco_rules.local.yaml`. Also, when the argument to `-r`/falco.yaml `rules_file` is a directory, falco will read rules files from that directory. [[#348](https://github.com/draios/falco/pull/348)] [[#187](https://github.com/draios/falco/issues/187)]
-* Properly support all syscalls (e.g. those without parameter extraction by the kernel module) in falco conditions, so they can be included in `evt.type=<name>` conditions. [[#352](https://github.com/draios/falco/pull/352)]
-* When packaged as a container, start building kernel module with gcc 5.0 instead of gcc 4.9. [[#331](https://github.com/draios/falco/pull/331)]
-* New example puppet module for falco. [[#341](https://github.com/draios/falco/pull/341)] [[#115](https://github.com/draios/falco/issues/115)]
-* When signaled with `USR1`, falco will close/reopen log files. Include a [logrotate](https://github.com/logrotate/logrotate) example that shows how to use this feature for log rotation. [[#347](https://github.com/draios/falco/pull/347)] [[#266](https://github.com/draios/falco/issues/266)]
-* To improve resource usage, further restrict the set of system calls available to falco [[#351](https://github.com/draios/falco/pull/351)] [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)]
-
-## Minor Changes
-
-* Add gdb to the development Docker image (sysdig/falco:dev) to aid in debugging. [[#323](https://github.com/draios/falco/pull/323)]
-* You can now specify -V multiple times on the command line to validate multiple rules files at once. [[#329](https://github.com/draios/falco/pull/329)]
-* When run with `-v`, falco will print *dangling* macros/lists that are not used by any rules. [[#329](https://github.com/draios/falco/pull/329)]
-* Add an example demonstrating cryptomining attack that exploits an open docker daemon using host mounts. [[#336](https://github.com/draios/falco/pull/336)]
-* New falco.yaml option `json_include_output_property` controls whether the formatted string "output" is included in the json object when json output is enabled. [[#342](https://github.com/draios/falco/pull/342)]
-* Centralize testing event types for consideration by falco into a single function [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)) [[#356](https://github.com/draios/falco/pull/356)]
-* If a rule has an attribute `warn_evttypes`, falco will not complain about `evt.type` restrictions on that rule [[#355](https://github.com/draios/falco/pull/355)]
-* When run with `-i`, print all ignored events/syscalls and exit. [[#359](https://github.com/draios/falco/pull/359)]
-
-## Bug Fixes
-
-* Minor bug fixes to k8s daemonset configuration. [[#325](https://github.com/draios/falco/pull/325)] [[#296](https://github.com/draios/falco/pull/296)] [[#295](https://github.com/draios/falco/pull/295)]
-* Ensure `--validate` can be used interchangeably with `-V`. [[#334](https://github.com/draios/falco/pull/334)] [[#322](https://github.com/draios/falco/issues/322)]
-* Rule conditions like `fd.net` can now be used with the `in` operator e.g. `evt.type=connect and fd.net in ("127.0.0.1/24")`. [[draios/sysdig#1091](https://github.com/draios/sysdig/pull/1091)] [[#343](https://github.com/draios/falco/pull/343)]
-* Ensure that `keep_alive` can be used both with file and program output at the same time. [[#335](https://github.com/draios/falco/pull/335)]
-* Make it possible to append to a skipped macro/rule without falco complaining [[#346](https://github.com/draios/falco/pull/346)] [[#305](https://github.com/draios/falco/issues/305)]
-* Ensure rule order is preserved even when rules do not contain any `evt.type` restriction. [[#354](https://github.com/draios/falco/issues/354)] [[#355](https://github.com/draios/falco/pull/355)]
-
-## Rule Changes
-
-* Make it easier to extend the `Change thread namespace` rule via a `user_known_change_thread_namespace_binaries` list. [[#324](https://github.com/draios/falco/pull/324)]
-* Various FP fixes from users. [[#321](https://github.com/draios/falco/pull/321)] [[#326](https://github.com/draios/falco/pull/326)] [[#344](https://github.com/draios/falco/pull/344)] [[#350](https://github.com/draios/falco/pull/350)]
-* New rule `Disallowed SSH Connection` detects attempts ssh connection attempts to hosts outside of an expected set. In order to be effective, you need to override the macro `allowed_ssh_hosts` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
-* New rule `Unexpected K8s NodePort Connection` detects attempts to contact the K8s NodePort range from a program running inside a container. In order to be effective, you need to override the macro `nodeport_containers` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
-* Improve `Modify binary dirs` rule to work with new syscalls [[#353](https://github.com/draios/falco/pull/353)]
-* New rule `Unexpected UDP Traffic` checks for udp traffic not on a list of expected ports. Somewhat FP-prone, so it must be explicitly enabled by overriding the macro `do_unexpected_udp_check` in a user rules file. [[#320](https://github.com/draios/falco/pull/320)] [[#357](https://github.com/draios/falco/pull/357)]
-
 ## v0.9.0
 
 Released 2018-01-18

README.md

@@ -2,7 +2,7 @@
 
 #### Latest release
 
-**v0.10.0**
+**v0.9.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />

cpack/debian/conffiles

@@ -1,4 +1,4 @@
 /etc/falco/falco.yaml
 /etc/falco/falco_rules.yaml
-/etc/falco/rules.available/application_rules.yaml
+/etc/falco/application_rules.yaml
 /etc/falco/falco_rules.local.yaml

docker/event-generator/event_generator.cpp

@@ -296,21 +296,23 @@ void system_user_interactive() {
 }
 
 void network_activity() {
-	printf("Connecting a udp socket to 10.2.3.4:8192...\n");
+	printf("Opening a listening socket on port 8192...\n");
 	int rc;
 	int sock = socket(PF_INET, SOCK_DGRAM, 0);
 	struct sockaddr_in localhost;
 
 	localhost.sin_family = AF_INET;
 	localhost.sin_port = htons(8192);
-	inet_aton("10.2.3.4", &(localhost.sin_addr));
+	inet_aton("127.0.0.1", &(localhost.sin_addr));
 
-	if((rc = connect(sock, (struct sockaddr *) &localhost, sizeof(localhost))) != 0)
+	if((rc = bind(sock, (struct sockaddr *) &localhost, sizeof(localhost))) != 0)
 	{
 		fprintf(stderr, "Could not bind listening socket to localhost: %s\n", strerror(errno));
 		return;
 	}
 
+	listen(sock, 1);
+
 	close(sock);
 }
 

examples/logrotate/falco

@@ -1,7 +0,0 @@
-/var/log/falco-events.log {
-    rotate 5
-    size 1M
-    postrotate
-        /usr/bin/killall -USR1 falco
-    endscript
-}

falco.yaml

@@ -1,7 +1,4 @@
-# File(s) or Directories containing Falco rules, loaded at startup.
-# The name "rules_file" is only for backwards compatibility.
-# If the entry is a file, it will be read directly. If the entry is a directory,
-# every file in that directory will be read, in alphabetical order.
+# File(s) containing Falco rules, loaded at startup.
 #
 # falco_rules.yaml ships with the falco package and is overridden with
 # every new software version. falco_rules.local.yaml is only created
@@ -13,7 +10,6 @@
 rules_file:
  - /etc/falco/falco_rules.yaml
  - /etc/falco/falco_rules.local.yaml
- - /etc/falco/rules.d
 
 # Whether to output events in json or text
 json_output: false
@@ -70,10 +66,6 @@ syslog_output:
 # continuously written to, with each output message on its own
 # line. If keep_alive is set to false, the file will be re-opened
 # for each output message.
-#
-# Also, the file will be closed and reopened if falco is signaled with
-# SIGUSR1.
-
 file_output:
   enabled: false
   keep_alive: false
@@ -94,9 +86,7 @@ stdout_output:
 # continuously written to, with each output message on its own
 # line. If keep_alive is set to false, the program will be re-spawned
 # for each output message.
-#
-# Also, the program will be closed and reopened if falco is signaled with
-# SIGUSR1.
+
 program_output:
   enabled: false
   keep_alive: false

rules/CMakeLists.txt

@@ -31,9 +31,7 @@ install(FILES falco_rules.local.yaml
 	RENAME "${FALCO_LOCAL_RULES_DEST_FILENAME}")
 
 install(FILES application_rules.yaml
-	DESTINATION "/etc/falco/rules.available"
+	DESTINATION "${FALCO_ETC_DIR}"
 	RENAME "${FALCO_APP_RULES_DEST_FILENAME}")
-
-install(DIRECTORY DESTINATION "/etc/falco/rules.d")
 endif()
 

rules/falco_rules.yaml

@@ -15,9 +15,6 @@
 - macro: never_true
   condition: (evt.num=0)
 
-- macro: always_true
-  condition: (evt.num=>0)
-
 # In some cases, such as dropped system call events, information about
 # the process name may be missing. For some rules that really depend
 # on the identity of the process performing an action such as opening
@@ -26,7 +23,7 @@
   condition: (proc.name!="<NA>")
 
 - macro: rename
-  condition: evt.type in (rename, renameat)
+  condition: evt.type = rename
 - macro: mkdir
   condition: evt.type = mkdir
 - macro: remove
@@ -240,27 +237,12 @@
 
 # Network
 - macro: inbound
-  condition: >
-    (((evt.type in (accept,listen) and evt.dir=<)) or
-     (fd.typechar = 4 or fd.typechar = 6) and
-     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
-     (evt.rawres >= 0 or evt.res = EINPROGRESS))
+  condition: ((evt.type=listen and evt.dir=>) or (evt.type=accept and evt.dir=<))
 
+# Currently sendto is an ignored syscall, otherwise this could also
+# check for (evt.type=sendto and evt.dir=>)
 - macro: outbound
-  condition: >
-    (((evt.type = connect and evt.dir=<)) or
-     (fd.typechar = 4 or fd.typechar = 6) and
-     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
-     (evt.rawres >= 0 or evt.res = EINPROGRESS))
-
-# Very similar to inbound/outbound, but combines the tests together
-# for efficiency.
-- macro: inbound_outbound
-  condition: >
-    (((evt.type in (accept,listen,connect) and evt.dir=<)) or
-     (fd.typechar = 4 or fd.typechar = 6) and
-     (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
-     (evt.rawres >= 0 or evt.res = EINPROGRESS))
+  condition: evt.type=connect and evt.dir=< and (fd.typechar=4 or fd.typechar=6)
 
 - macro: ssh_port
   condition: fd.sport=22
@@ -280,7 +262,7 @@
 
 - rule: Disallowed SSH Connection
   desc: Detect any new ssh connection to a host other than those in an allowed group of hosts
-  condition: (inbound_outbound) and ssh_port and not allowed_ssh_hosts
+  condition: (outbound or inbound) and ssh_port and not allowed_ssh_hosts
   output: Disallowed SSH Connection (command=%proc.cmdline connection=%fd.name user=%user.name)
   priority: NOTICE
   tags: [network]
@@ -368,13 +350,6 @@
        proc.pcmdline startswith "node /root/.config/yarn" or
        proc.pcmdline startswith "node /opt/yarn/bin/yarn.js"))
 
-
-- macro: httpd_writing_ssl_conf
-  condition: >
-    (proc.pname=run-httpd and
-     (proc.cmdline startswith "sed -ri" or proc.cmdline startswith "sed -i") and
-     (fd.name startswith /etc/httpd/conf.d/ or fd.name startswith /etc/httpd/conf))
-
 - macro: parent_Xvfb_running_xkbcomp
   condition: (proc.pname=Xvfb and proc.cmdline startswith 'sh -c "/usr/bin/xkbcomp"')
 
@@ -733,7 +708,7 @@
     and not proc.pname in (sysdigcloud_binaries, mail_config_binaries, hddtemp.postins, sshkit_script_binaries, locales.postins, deb_binaries, dhcp_binaries)
     and not fd.name pmatch (safe_etc_dirs)
     and not fd.name in (/etc/container_environment.sh, /etc/container_environment.json, /etc/motd, /etc/motd.svc)
-    and not exe_running_docker_save
+    and not exe_running_docker_save    
     and not ansible_running_python
     and not python_running_denyhosts
     and not fluentd_writing_conf_files
@@ -790,7 +765,6 @@
     and not centrify_writing_krb
     and not cockpit_writing_conf
     and not ipsec_writing_conf
-    and not httpd_writing_ssl_conf
 
 - rule: Write below etc
   desc: an attempt to write to any file below /etc
@@ -930,12 +904,7 @@
   condition: (proc.aname[2]=redis-server and (proc.cmdline contains "redis-server.post-up.d" or proc.cmdline contains "redis-server.pre-up.d"))
 
 - macro: rabbitmq_running_scripts
-  condition: >
-    (proc.pname=beam.smp and
-    (proc.cmdline startswith "sh -c exec ps" or
-     proc.cmdline startswith "sh -c exec inet_gethost" or
-     proc.cmdline= "sh -s unix:cmd" or
-     proc.cmdline= "sh -c exec /bin/sh -s unix:cmd 2>&1"))
+  condition: (proc.pname=beam.smp and (proc.cmdline startswith "sh -c exec ps" or proc.cmdline startswith "sh -c exec inet_gethost"))
 
 - macro: rabbitmqctl_running_scripts
   condition: (proc.aname[2]=rabbitmqctl and proc.cmdline startswith "sh -c ")
@@ -957,7 +926,7 @@
 
 - rule: Modify binary dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: (bin_dir_rename) and modify and not package_mgmt_procs and not exe_running_docker_save
+  condition: bin_dir_rename and modify and not package_mgmt_procs and not exe_running_docker_save
   output: >
     File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline
     operation=%evt.type file=%fd.name %evt.args)
@@ -1329,7 +1298,7 @@
   desc: any network activity performed by system binaries that are not expected to send or receive any network traffic
   condition: >
     (fd.sockfamily = ip and system_procs)
-    and (inbound_outbound)
+    and (inbound or outbound)
     and not proc.name in (systemd, hostid)
     and not login_doing_dns_lookup
   output: >
@@ -1338,47 +1307,6 @@
   priority: NOTICE
   tags: [network]
 
-- list: openvpn_udp_ports
-  items: [1194, 1197, 1198, 8080, 9201]
-
-- list: l2tp_udp_ports
-  items: [500, 1701, 4500, 10000]
-
-- list: statsd_ports
-  items: [8125]
-
-- list: ntp_ports
-  items: [123]
-
-# Some applications will connect a udp socket to an address only to
-# test connectivity. Assuming the udp connect works, they will follow
-# up with a tcp connect that actually sends/receives data.
-#
-# With that in mind, we listed a few commonly seen ports here to avoid
-# some false positives. In addition, we make the main rule opt-in, so
-# it's disabled by default.
-
-- list: test_connect_ports
-  items: [0, 9, 80, 3306]
-
-- macro: do_unexpected_udp_check
-  condition: (never_true)
-
-- list: expected_udp_ports
-  items: [53, openvpn_udp_ports, l2tp_udp_ports, statsd_ports, ntp_ports, test_connect_ports]
-
-- macro: expected_udp_traffic
-  condition: fd.port in (expected_udp_ports)
-
-- rule: Unexpected UDP Traffic
-  desc: UDP traffic not on port 53 (DNS) or other commonly used ports
-  condition: (inbound_outbound) and do_unexpected_udp_check and fd.l4proto=udp and not expected_udp_traffic
-  output: >
-    Unexpected UDP Traffic Seen
-    (user=%user.name command=%proc.cmdline connection=%fd.name proto=%fd.l4proto evt=%evt.type %evt.args)
-  priority: NOTICE
-  tags: [network]
-
 # With the current restriction on system calls handled by falco
 # (e.g. excluding read/write/sendto/recvfrom/etc, this rule won't
 # trigger).
@@ -1529,7 +1457,7 @@
 
 - rule: Unexpected K8s NodePort Connection
   desc: Detect attempts to use K8s NodePorts from a container
-  condition: (inbound_outbound) and fd.sport >= 30000 and fd.sport <= 32767 and container and not nodeport_containers
+  condition: (outbound or inbound) and fd.sport >= 30000 and fd.sport <= 32767 and container and not nodeport_containers
   output: Unexpected K8s NodePort Connection (command=%proc.cmdline connection=%fd.name)
   priority: NOTICE
   tags: [network, k8s, container]

test/falco_test.py

@@ -31,7 +31,6 @@ class FalcoTest(Test):
 
         self.json_output = self.params.get('json_output', '*', default=False)
         self.json_include_output_property = self.params.get('json_include_output_property', '*', default=True)
-        self.all_events = self.params.get('all_events', '*', default=False)
         self.priority = self.params.get('priority', '*', default='debug')
         self.rules_file = self.params.get('rules_file', '*', default=os.path.join(self.basedir, '../rules/falco_rules.yaml'))
 
@@ -214,7 +213,7 @@ class FalcoTest(Test):
         triggered_rules = match.group(1)
 
         for rule, count in self.detect_counts.iteritems():
-            expected = '\s{}: (\d+)'.format(rule)
+            expected = '{}: (\d+)'.format(rule)
             match = re.search(expected, triggered_rules)
 
             if match is None:
@@ -366,9 +365,6 @@ class FalcoTest(Test):
         if self.run_duration:
             cmd += ' -M {}'.format(self.run_duration)
 
-        if self.all_events:
-            cmd += ' -A'
-
         self.falco_proc = process.SubProcess(cmd)
 
         res = self.falco_proc.run(timeout=180, sig=9)

test/falco_tests.yaml

@@ -128,18 +128,6 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
-    all_events: True
-
-  rules_directory:
-    detect: True
-    detect_level:
-      - WARNING
-      - INFO
-      - ERROR
-    rules_file:
-      - rules/rules_dir
-    trace_file: trace_files/cat_write.scap
-    all_events: True
 
   multiple_rules_suppress_info:
     detect: True
@@ -155,7 +143,6 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
-    all_events: True
 
   multiple_rules_overriding:
     detect: False
@@ -655,14 +642,6 @@ trace_files: !mux
       - rules/rule_append_failure.yaml
     trace_file: trace_files/cat_write.scap
 
-  rule_append_skipped:
-    detect: False
-    priority: ERROR
-    rules_file:
-      - rules/single_rule.yaml
-      - rules/append_single_rule.yaml
-    trace_file: trace_files/cat_write.scap
-
   rule_append:
     detect: True
     detect_level: WARNING
@@ -691,25 +670,4 @@ trace_files: !mux
     detect_level: INFO
     rules_file:
       - rules/detect_connect_using_in.yaml
-    trace_file: trace_files/connect_localhost.scap
-
-  syscalls:
-    detect: True
-    detect_level: INFO
-    rules_file:
-      - rules/syscalls.yaml
-    detect_counts:
-      - detect_madvise: 2
-      - detect_open: 2
-    trace_file: trace_files/syscall.scap
-    all_events: True
-
-  catchall_order:
-    detect: True
-    detect_level: INFO
-    rules_file:
-      - rules/catchall_order.yaml
-    detect_counts:
-      - open_dev_null: 1
-        dev_null: 0
-    trace_file: trace_files/cat_write.scap
+    trace_file: trace_files/connect_localhost.scap

test/rules/append_single_rule.yaml

@@ -1,3 +0,0 @@
-- rule: open_from_cat
-  append: true
-  condition: and fd.name=/tmp

test/rules/catchall_order.yaml

@@ -1,12 +0,0 @@
-- rule: open_dev_null
-  desc: Any open of the file /dev/null
-  condition: evt.type=open and fd.name=/dev/null
-  output: An open of /dev/null was seen (command=%proc.cmdline evt=%evt.type %evt.args)
-  priority: INFO
-
-- rule: dev_null
-  desc: Anything related to /dev/null
-  condition: fd.name=/dev/null
-  output: Something related to /dev/null was seen (command=%proc.cmdline evt=%evt.type %evt.args)
-  priority: INFO
-  warn_evttypes: false

test/rules/rules_dir/000-single_rule.yaml

@@ -1,14 +0,0 @@
-- list: cat_binaries
-  items: [cat]
-
-- list: cat_capable_binaries
-  items: [cat_binaries]
-
-- macro: is_cat
-  condition: proc.name in (cat_capable_binaries)
-
-- rule: open_from_cat
-  desc: A process named cat does an open
-  condition: evt.type=open and is_cat
-  output: "An open was seen (command=%proc.cmdline)"
-  priority: WARNING

test/rules/rules_dir/001-double_rule.yaml

@@ -1,13 +0,0 @@
-# This ruleset depends on the is_cat macro defined in single_rule.yaml
-
-- rule: exec_from_cat
-  desc: A process named cat does execve
-  condition: evt.type=execve and is_cat
-  output: "An exec was seen (command=%proc.cmdline)"
-  priority: ERROR
-
-- rule: access_from_cat
-  desc: A process named cat does an access
-  condition: evt.type=access and is_cat
-  output: "An access was seen (command=%proc.cmdline)"
-  priority: INFO

test/rules/syscalls.yaml

@@ -1,11 +0,0 @@
-- rule: detect_madvise
-  desc: Detect any call to madvise
-  condition: evt.type=madvise and evt.dir=<
-  output: A madvise syscall was seen (command=%proc.cmdline evt=%evt.type)
-  priority: INFO
-
-- rule: detect_open
-  desc: Detect any call to open
-  condition: evt.type=open and evt.dir=< and fd.name=/dev/null
-  output: An open syscall was seen (command=%proc.cmdline evt=%evt.type file=%fd.name)
-  priority: INFO

userspace/engine/falco_engine.cpp

@@ -162,13 +162,6 @@ void falco_engine::evttypes_for_ruleset(std::vector<bool> &evttypes, const std::
 	return m_evttype_filter->evttypes_for_ruleset(evttypes, ruleset_id);
 }
 
-void falco_engine::syscalls_for_ruleset(std::vector<bool> &syscalls, const std::string &ruleset)
-{
-	uint16_t ruleset_id = find_ruleset_id(ruleset);
-
-	return m_evttype_filter->syscalls_for_ruleset(syscalls, ruleset_id);
-}
-
 unique_ptr<falco_engine::rule_result> falco_engine::process_event(sinsp_evt *ev, uint16_t ruleset_id)
 {
 	if(should_drop_evt())
@@ -244,11 +237,10 @@ void falco_engine::print_stats()
 
 void falco_engine::add_evttype_filter(string &rule,
 				      set<uint32_t> &evttypes,
-				      set<uint32_t> &syscalls,
 				      set<string> &tags,
 				      sinsp_filter* filter)
 {
-	m_evttype_filter->add(rule, evttypes, syscalls, tags, filter);
+	m_evttype_filter->add(rule, evttypes, tags, filter);
 }
 
 void falco_engine::clear_filters()

userspace/engine/falco_engine.h

@@ -91,12 +91,6 @@ public:
 	//
 	void evttypes_for_ruleset(std::vector<bool> &evttypes, const std::string &ruleset);
 
-	//
-	// Given a ruleset, fill in a bitset containing the syscalls
-	// for which this ruleset can run.
-	//
-	void syscalls_for_ruleset(std::vector<bool> &syscalls, const std::string &ruleset);
-
 	//
 	// Given an event, check it against the set of rules in the
 	// engine and if a matching rule is found, return details on
@@ -128,11 +122,10 @@ public:
 
 	//
 	// Add a filter, which is related to the specified set of
-	// event types/syscalls, to the engine.
+	// event types, to the engine.
 	//
 	void add_evttype_filter(std::string &rule,
 				std::set<uint32_t> &evttypes,
-				std::set<uint32_t> &syscalls,
 				std::set<std::string> &tags,
 				sinsp_filter* filter);
 

userspace/engine/lua/compiler.lua

@@ -191,20 +191,19 @@ function check_for_ignored_syscalls_events(ast, filter_type, source)
    parser.traverse_ast(ast, {BinaryRelOp=1}, cb)
 end
 
--- Examine the ast and find the event types/syscalls for which the
--- rule should run. All evt.type references are added as event types
--- up until the first "!=" binary operator or unary not operator. If
--- no event type checks are found afterward in the rule, the rule is
--- considered optimized and is associated with the event type(s).
+-- Examine the ast and find the event types for which the rule should
+-- run. All evt.type references are added as event types up until the
+-- first "!=" binary operator or unary not operator. If no event type
+-- checks are found afterward in the rule, the rule is considered
+-- optimized and is associated with the event type(s).
 --
 -- Otherwise, the rule is associated with a 'catchall' category and is
--- run for all event types/syscalls. (Also, a warning is printed).
+-- run for all event types. (Also, a warning is printed).
 --
 
-function get_evttypes_syscalls(name, ast, source, warn_evttypes)
+function get_evttypes(name, ast, source)
 
    local evttypes = {}
-   local syscallnums = {}
    local evtnames = {}
    local found_event = false
    local found_not = false
@@ -227,45 +226,17 @@ function get_evttypes_syscalls(name, ast, source, warn_evttypes)
 	    if node.operator == "in" or node.operator == "pmatch" then
 	       for i, v in ipairs(node.right.elements) do
 		  if v.type == "BareString" then
-
-                     -- The event must be a known event
-                     if events[v.value] == nil and syscalls[v.value] == nil then
-                        error("Unknown event/syscall \""..v.value.."\" in filter: "..source)
-                     end
-
 		     evtnames[v.value] = 1
-		     if events[v.value] ~= nil then
-			for id in string.gmatch(events[v.value], "%S+") do
-			   evttypes[id] = 1
-			end
-		     end
-
-		     if syscalls[v.value] ~= nil then
-			for id in string.gmatch(syscalls[v.value], "%S+") do
-			   syscallnums[id] = 1
-			end
+		     for id in string.gmatch(events[v.value], "%S+") do
+			evttypes[id] = 1
 		     end
 		  end
 	       end
 	    else
 	       if node.right.type == "BareString" then
-
-		  -- The event must be a known event
-		  if events[node.right.value] == nil and syscalls[node.right.value] == nil then
-		     error("Unknown event/syscall \""..node.right.value.."\" in filter: "..source)
-		  end
-
 		  evtnames[node.right.value] = 1
-		  if events[node.right.value] ~= nil then
-		     for id in string.gmatch(events[node.right.value], "%S+") do
-			evttypes[id] = 1
-		     end
-		  end
-
-		  if syscalls[node.right.value] ~= nil then
-		     for id in string.gmatch(syscalls[node.right.value], "%S+") do
-			syscallnums[id] = 1
-		     end
+		  for id in string.gmatch(events[node.right.value], "%S+") do
+		     evttypes[id] = 1
 		  end
 	       end
 	    end
@@ -276,29 +247,23 @@ function get_evttypes_syscalls(name, ast, source, warn_evttypes)
    parser.traverse_ast(ast.filter.value, {BinaryRelOp=1, UnaryBoolOp=1} , cb)
 
    if not found_event then
-      if warn_evttypes == true then
-	 io.stderr:write("Rule "..name..": warning (no-evttype):\n")
-	 io.stderr:write(source.."\n")
-	 io.stderr:write("         did not contain any evt.type restriction, meaning it will run for all event types.\n")
-	 io.stderr:write("         This has a significant performance penalty. Consider adding an evt.type restriction if possible.\n")
-      end
+      io.stderr:write("Rule "..name..": warning (no-evttype):\n")
+      io.stderr:write(source.."\n")
+      io.stderr:write("         did not contain any evt.type restriction, meaning it will run for all event types.\n")
+      io.stderr:write("         This has a significant performance penalty. Consider adding an evt.type restriction if possible.\n")
       evttypes = {}
-      syscallnums = {}
       evtnames = {}
    end
 
    if found_event_after_not then
-      if warn_evttypes == true then
-	 io.stderr:write("Rule "..name..": warning (trailing-evttype):\n")
-	 io.stderr:write(source.."\n")
-	 io.stderr:write("         does not have all evt.type restrictions at the beginning of the condition,\n")
-	 io.stderr:write("         or uses a negative match (i.e. \"not\"/\"!=\") for some evt.type restriction.\n")
-	 io.stderr:write("         This has a performance penalty, as the rule can not be limited to specific event types.\n")
-	 io.stderr:write("         Consider moving all evt.type restrictions to the beginning of the rule and/or\n")
-	 io.stderr:write("         replacing negative matches with positive matches if possible.\n")
-      end
+      io.stderr:write("Rule "..name..": warning (trailing-evttype):\n")
+      io.stderr:write(source.."\n")
+      io.stderr:write("         does not have all evt.type restrictions at the beginning of the condition,\n")
+      io.stderr:write("         or uses a negative match (i.e. \"not\"/\"!=\") for some evt.type restriction.\n")
+      io.stderr:write("         This has a performance penalty, as the rule can not be limited to specific event types.\n")
+      io.stderr:write("         Consider moving all evt.type restrictions to the beginning of the rule and/or\n")
+      io.stderr:write("         replacing negative matches with positive matches if possible.\n")
       evttypes = {}
-      syscallnums = {}
       evtnames = {}
    end
 
@@ -316,10 +281,10 @@ function get_evttypes_syscalls(name, ast, source, warn_evttypes)
    table.sort(evtnames_only)
 
    if compiler.verbose then
-      io.stderr:write("Event types/Syscalls for rule "..name..": "..table.concat(evtnames_only, ",").."\n")
+      io.stderr:write("Event types for rule "..name..": "..table.concat(evtnames_only, ",").."\n")
    end
 
-   return evttypes, syscallnums
+   return evttypes
 end
 
 function compiler.expand_lists_in(source, list_defs)
@@ -379,7 +344,7 @@ end
 --[[
    Parses a single filter, then expands macros using passed-in table of definitions. Returns resulting AST.
 --]]
-function compiler.compile_filter(name, source, macro_defs, list_defs, warn_evttypes)
+function compiler.compile_filter(name, source, macro_defs, list_defs)
 
    source = compiler.expand_lists_in(source, list_defs)
 
@@ -406,9 +371,9 @@ function compiler.compile_filter(name, source, macro_defs, list_defs, warn_evtty
       error("Unexpected top-level AST type: "..ast.type)
    end
 
-   evttypes, syscallnums = get_evttypes_syscalls(name, ast, source, warn_evttypes)
+   evttypes = get_evttypes(name, ast, source)
 
-   return ast, evttypes, syscallnums
+   return ast, evttypes
 end
 
 

userspace/engine/lua/rule_loader.lua

@@ -132,8 +132,7 @@ end
 -- object. The by_name index is used for things like describing rules,
 -- and the by_idx index is used to map the relational node index back
 -- to a rule.
-local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={},
-	       skipped_rules_by_name={}, macros_by_name={}, lists_by_name={},
+local state = {macros={}, lists={}, filter_ast=nil, rules_by_name={}, macros_by_name={}, lists_by_name={},
 	       n_rules=0, rules_by_idx={}, ordered_rule_names={}, ordered_macro_names={}, ordered_list_names={}}
 
 local function reset_rules(rules_mgr)
@@ -292,13 +291,11 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	    end
 
 	    if state.rules_by_name[v['rule']] == nil then
-	       if state.skipped_rules_by_name[v['rule']] == nil then
-		  error ("Rule " ..v['rule'].. " has 'append' key but no rule by that name already exists")
-	       end
-	    else
-	       state.rules_by_name[v['rule']]['condition'] = state.rules_by_name[v['rule']]['condition'] .. " " .. v['condition']
+	       error ("Rule " ..v['rule'].. " has 'append' key but no rule by that name already exists")
 	    end
 
+	    state.rules_by_name[v['rule']]['condition'] = state.rules_by_name[v['rule']]['condition'] .. " " .. v['condition']
+
 	 else
 
 	    for i, field in ipairs({'condition', 'output', 'desc', 'priority'}) do
@@ -323,8 +320,6 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	       v['output'] = compiler.trim(v['output'])
 
 	       state.rules_by_name[v['rule']] = v
-	    else
-	       state.skipped_rules_by_name[v['rule']] = v
 	    end
 	 end
       else
@@ -373,14 +368,8 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 
       local v = state.rules_by_name[name]
 
-      warn_evttypes = true
-      if v['warn_evttypes'] ~= nil then
-	 warn_evttypes = v['warn_evttypes']
-      end
-
-      local filter_ast, evttypes, syscallnums = compiler.compile_filter(v['rule'], v['condition'],
-									state.macros, state.lists,
-									warn_evttypes)
+      local filter_ast, evttypes = compiler.compile_filter(v['rule'], v['condition'],
+							   state.macros, state.lists)
 
       if (filter_ast.type == "Rule") then
 	 state.n_rules = state.n_rules + 1
@@ -401,7 +390,7 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	 end
 
 	 -- Pass the filter and event types back up
-	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes, syscallnums, v['tags'])
+	 falco_rules.add_filter(rules_mgr, v['rule'], evttypes, v['tags'])
 
 	 -- Rule ASTs are merged together into one big AST, with "OR" between each
 	 -- rule.

userspace/engine/rules.cpp

@@ -66,9 +66,8 @@ void falco_rules::clear_filters()
 
 int falco_rules::add_filter(lua_State *ls)
 {
-	if (! lua_islightuserdata(ls, -5) ||
-	    ! lua_isstring(ls, -4) ||
-	    ! lua_istable(ls, -3) ||
+	if (! lua_islightuserdata(ls, -4) ||
+	    ! lua_isstring(ls, -3) ||
 	    ! lua_istable(ls, -2) ||
 	    ! lua_istable(ls, -1))
 	{
@@ -76,28 +75,16 @@ int falco_rules::add_filter(lua_State *ls)
 		lua_error(ls);
 	}
 
-	falco_rules *rules = (falco_rules *) lua_topointer(ls, -5);
-	const char *rulec = lua_tostring(ls, -4);
+	falco_rules *rules = (falco_rules *) lua_topointer(ls, -4);
+	const char *rulec = lua_tostring(ls, -3);
 
 	set<uint32_t> evttypes;
 
-	lua_pushnil(ls);  /* first key */
-	while (lua_next(ls, -4) != 0) {
-                // key is at index -2, value is at index
-                // -1. We want the keys.
-		evttypes.insert(luaL_checknumber(ls, -2));
-
-		// Remove value, keep key for next iteration
-		lua_pop(ls, 1);
-	}
-
-	set<uint32_t> syscalls;
-
 	lua_pushnil(ls);  /* first key */
 	while (lua_next(ls, -3) != 0) {
                 // key is at index -2, value is at index
                 // -1. We want the keys.
-		syscalls.insert(luaL_checknumber(ls, -2));
+		evttypes.insert(luaL_checknumber(ls, -2));
 
 		// Remove value, keep key for next iteration
 		lua_pop(ls, 1);
@@ -108,7 +95,7 @@ int falco_rules::add_filter(lua_State *ls)
 	lua_pushnil(ls);  /* first key */
 	while (lua_next(ls, -2) != 0) {
                 // key is at index -2, value is at index
-                // -1. We want the values.
+                // -1. We want the keys.
 		tags.insert(lua_tostring(ls, -1));
 
 		// Remove value, keep key for next iteration
@@ -116,19 +103,19 @@ int falco_rules::add_filter(lua_State *ls)
 	}
 
 	std::string rule = rulec;
-	rules->add_filter(rule, evttypes, syscalls, tags);
+	rules->add_filter(rule, evttypes, tags);
 
 	return 0;
 }
 
-void falco_rules::add_filter(string &rule, set<uint32_t> &evttypes, set<uint32_t> &syscalls, set<string> &tags)
+void falco_rules::add_filter(string &rule, set<uint32_t> &evttypes, set<string> &tags)
 {
 	// While the current rule was being parsed, a sinsp_filter
 	// object was being populated by lua_parser. Grab that filter
 	// and pass it to the engine.
 	sinsp_filter *filter = m_lua_parser->get_filter(true);
 
-	m_engine->add_evttype_filter(rule, evttypes, syscalls, tags, filter);
+	m_engine->add_evttype_filter(rule, evttypes, tags, filter);
 }
 
 int falco_rules::enable_rule(lua_State *ls)
@@ -196,35 +183,6 @@ void falco_rules::load_rules(const string &rules_content,
 
 		lua_setglobal(m_ls, m_lua_events.c_str());
 
-		map<string,string> syscalls_by_name;
-		for(uint32_t j = 0; j < PPM_SC_MAX; j++)
-		{
-			auto it = syscalls_by_name.find(stable[j].name);
-
-			if (it == syscalls_by_name.end())
-			{
-				syscalls_by_name[stable[j].name] = to_string(j);
-			}
-			else
-			{
-				string cur = it->second;
-				cur += " ";
-				cur += to_string(j);
-				syscalls_by_name[stable[j].name] = cur;
-			}
-		}
-
-		lua_newtable(m_ls);
-
-		for( auto kv : syscalls_by_name)
-		{
-			lua_pushstring(m_ls, kv.first.c_str());
-			lua_pushstring(m_ls, kv.second.c_str());
-			lua_settable(m_ls, -3);
-		}
-
-		lua_setglobal(m_ls, m_lua_syscalls.c_str());
-
 		// Create a table containing the syscalls/events that
 		// are ignored by the kernel module. load_rules will
 		// return an error if any rule references one of these

userspace/engine/rules.h

@@ -45,7 +45,7 @@ class falco_rules
 
  private:
 	void clear_filters();
-	void add_filter(string &rule, std::set<uint32_t> &evttypes, std::set<uint32_t> &syscalls, std::set<string> &tags);
+	void add_filter(string &rule, std::set<uint32_t> &evttypes, std::set<string> &tags);
 	void enable_rule(string &rule, bool enabled);
 
 	lua_parser* m_lua_parser;
@@ -57,6 +57,5 @@ class falco_rules
 	string m_lua_ignored_syscalls = "ignored_syscalls";
 	string m_lua_ignored_events = "ignored_events";
 	string m_lua_events = "events";
-	string m_lua_syscalls = "syscalls";
 	string m_lua_describe_rule = "describe_rule";
 };

userspace/falco/configuration.cpp

@@ -16,13 +16,6 @@ You should have received a copy of the GNU General Public License
 along with falco.  If not, see <http://www.gnu.org/licenses/>.
 */
 
-#include <algorithm>
-
-#include <dirent.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <unistd.h>
-
 #include "configuration.h"
 #include "logger.h"
 
@@ -69,7 +62,7 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 		struct stat buffer;
 		if(stat(file.c_str(), &buffer) == 0)
 		{
-			read_rules_file_directory(file, m_rules_filenames);
+			m_rules_filenames.push_back(file);
 		}
 	}
 
@@ -157,70 +150,6 @@ void falco_configuration::init(string conf_filename, list<string> &cmdline_optio
 	falco_logger::log_syslog = m_config->get_scalar<bool>("log_syslog", true);
 }
 
-void falco_configuration::read_rules_file_directory(const string &path, list<string> &rules_filenames)
-{
-	struct stat st;
-
-	int rc = stat(path.c_str(), &st);
-
-	if(rc != 0)
-	{
-		std::cerr << "Could not get info on rules file " << path << ": " << strerror(errno) << std::endl;
-		exit(-1);
-	}
-
-	if(st.st_mode & S_IFDIR)
-	{
-		// It's a directory. Read the contents, sort
-		// alphabetically, and add every path to
-		// rules_filenames
-		vector<string> dir_filenames;
-
-		DIR *dir = opendir(path.c_str());
-
-		if(!dir)
-		{
-			std::cerr << "Could not get read contents of directory " << path << ": " << strerror(errno) << std::endl;
-			exit(-1);
-		}
-
-		for (struct dirent *ent = readdir(dir); ent; ent = readdir(dir))
-		{
-			string efile = path + "/" + ent->d_name;
-
-			rc = stat(efile.c_str(), &st);
-
-			if(rc != 0)
-			{
-				std::cerr << "Could not get info on rules file " << efile << ": " << strerror(errno) << std::endl;
-				exit(-1);
-			}
-
-			if(st.st_mode & S_IFREG)
-			{
-				dir_filenames.push_back(efile);
-			}
-		}
-
-		closedir(dir);
-
-		std::sort(dir_filenames.begin(),
-			  dir_filenames.end());
-
-		for (string &ent : dir_filenames)
-		{
-			rules_filenames.push_back(ent);
-		}
-	}
-	else
-	{
-		// Assume it's a file and just add to
-		// rules_filenames. If it can't be opened/etc that
-		// will be reported later..
-		rules_filenames.push_back(path);
-	}
-}
-
 static bool split(const string &str, char delim, pair<string,string> &parts)
 {
 	size_t pos;

userspace/falco/configuration.h

@@ -165,8 +165,6 @@ class falco_configuration
 	void init(std::string conf_filename, std::list<std::string> &cmdline_options);
 	void init(std::list<std::string> &cmdline_options);
 
-	static void read_rules_file_directory(const string &path, list<string> &rules_filenames);
-
 	std::list<std::string> m_rules_filenames;
 	bool m_json_output;
 	bool m_json_include_output_property;

userspace/falco/falco.cpp

@@ -22,8 +22,6 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <fstream>
 #include <set>
 #include <list>
-#include <vector>
-#include <algorithm>
 #include <string>
 #include <signal.h>
 #include <fcntl.h>
@@ -34,7 +32,6 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <sinsp.h>
 
 #include "logger.h"
-#include "utils.h"
 
 #include "configuration.h"
 #include "falco_engine.h"
@@ -42,8 +39,6 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include "statsfilewriter.h"
 
 bool g_terminate = false;
-bool g_reopen_outputs = false;
-
 //
 // Helper functions
 //
@@ -52,11 +47,6 @@ static void signal_callback(int signal)
 	g_terminate = true;
 }
 
-static void reopen_outputs(int signal)
-{
-	g_reopen_outputs = true;
-}
-
 //
 // Program help
 //
@@ -109,9 +99,8 @@ static void usage()
 	   "                               of %%container.info in rule output fields\n"
 	   "                               See the examples section below for more info.\n"
 	   " -P, --pidfile <pid_file>      When run as a daemon, write pid to specified file\n"
-           " -r <rules_file>               Rules file/directory (defaults to value set in configuration file,\n"
-           "                               or /etc/falco_rules.yaml). Can be specified multiple times to read\n"
-           "                               from multiple files/directories.\n"
+           " -r <rules_file>               Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml).\n"
+	   "                               Can be specified multiple times to read from multiple files.\n"
 	   " -s <stats_file>               If specified, write statistics related to falco's reading/processing of events\n"
 	   "                               to this file. (Only useful in live mode).\n"
 	   " -T <tag>                      Disable any rules with a tag=<tag>. Can be specified multiple times.\n"
@@ -154,8 +143,7 @@ uint64_t do_inspect(falco_engine *engine,
 		    falco_outputs *outputs,
 		    sinsp* inspector,
 		    uint64_t duration_to_tot_ns,
-		    string &stats_filename,
-		    bool all_events)
+		    string &stats_filename)
 {
 	uint64_t num_evts = 0;
 	int32_t res;
@@ -183,12 +171,6 @@ uint64_t do_inspect(falco_engine *engine,
 
 		writer.handle();
 
-		if(g_reopen_outputs)
-		{
-			outputs->reopen_outputs();
-			g_reopen_outputs = false;
-		}
-
 		if (g_terminate)
 		{
 			break;
@@ -222,7 +204,8 @@ uint64_t do_inspect(falco_engine *engine,
 			}
 		}
 
-		if(!ev->falco_consider() && !all_events)
+		if(!inspector->is_debug_enabled() &&
+			ev->get_category() & EC_INTERNAL)
 		{
 			continue;
 		}
@@ -244,47 +227,6 @@ uint64_t do_inspect(falco_engine *engine,
 	return num_evts;
 }
 
-static void print_all_ignored_events(sinsp *inspector)
-{
-	sinsp_evttables* einfo = inspector->get_event_info_tables();
-	const struct ppm_event_info* etable = einfo->m_event_info;
-	const struct ppm_syscall_desc* stable = einfo->m_syscall_info_table;
-
-	std::set<string> ignored_event_names;
-	for(uint32_t j = 0; j < PPM_EVENT_MAX; j++)
-	{
-		if(!sinsp::falco_consider_evtnum(j))
-		{
-			std::string name = etable[j].name;
-			// Ignore event names NA*
-			if(name.find("NA") != 0)
-			{
-				ignored_event_names.insert(name);
-			}
-		}
-	}
-
-	for(uint32_t j = 0; j < PPM_SC_MAX; j++)
-	{
-		if(!sinsp::falco_consider_syscallid(j))
-		{
-			std::string name = stable[j].name;
-			// Ignore event names NA*
-			if(name.find("NA") != 0)
-			{
-				ignored_event_names.insert(name);
-			}
-		}
-	}
-
-	printf("Ignored Event(s):");
-	for(auto it : ignored_event_names)
-	{
-		printf(" %s", it.c_str());
-	}
-	printf("\n");
-}
-
 //
 // ARGUMENT PARSING AND PROGRAM SETUP
 //
@@ -314,7 +256,6 @@ int falco_init(int argc, char **argv)
 	string output_format = "";
 	bool replace_container_info = false;
 	int duration_to_tot = 0;
-	bool print_ignored_events = false;
 
 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -344,7 +285,6 @@ int falco_init(int argc, char **argv)
 		{"version", no_argument, 0, 0 },
 		{"validate", required_argument, 0, 'V' },
 		{"writefile", required_argument, 0, 'w' },
-		{"ignored-events", no_argument, 0, 'i'},
 
 		{0, 0, 0, 0}
 	};
@@ -361,7 +301,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:ik:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
+                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -387,9 +327,6 @@ int falco_init(int argc, char **argv)
 				k8s_api = new string();
 				mesos_api = new string();
 				break;
-			case 'i':
-				print_ignored_events = true;
-				break;
 			case 'k':
 				k8s_api = new string(optarg);
 				break;
@@ -441,7 +378,7 @@ int falco_init(int argc, char **argv)
 				}
 				break;
 			case 'r':
-				falco_configuration::read_rules_file_directory(string(optarg), rules_filenames);
+				rules_filenames.push_back(optarg);
 				break;
 			case 's':
 				stats_filename = optarg;
@@ -480,20 +417,12 @@ int falco_init(int argc, char **argv)
 			return EXIT_SUCCESS;
 		}
 
+
 		inspector = new sinsp();
-
-		if(print_ignored_events)
-		{
-			print_all_ignored_events(inspector);
-			delete(inspector);
-			return EXIT_SUCCESS;
-		}
-
 		engine = new falco_engine();
 		engine->set_inspector(inspector);
 		engine->set_extra(output_format, replace_container_info);
 
-
 		outputs = new falco_outputs();
 		outputs->set_inspector(inspector);
 
@@ -574,19 +503,13 @@ int falco_init(int argc, char **argv)
 
 		if(config.m_rules_filenames.size() == 0)
 		{
-			throw std::invalid_argument("You must specify at least one rules file/directory via -r or a rules_file entry in falco.yaml");
-		}
-
-		falco_logger::log(LOG_DEBUG, "Configured rules filenames:\n");
-		for (auto filename : config.m_rules_filenames)
-		{
-			falco_logger::log(LOG_DEBUG, string("   ") + filename + "\n");
+			throw std::invalid_argument("You must specify at least one rules file via -r or a rules_file entry in falco.yaml");
 		}
 
 		for (auto filename : config.m_rules_filenames)
 		{
-			falco_logger::log(LOG_INFO, "Loading rules from file " + filename + ":\n");
 			engine->load_rules_file(filename, verbose, all_events);
+			falco_logger::log(LOG_INFO, "Parsed rules from file " + filename + "\n");
 		}
 
 		// You can't both disable and enable rules
@@ -631,7 +554,6 @@ int falco_init(int argc, char **argv)
 		if(!all_events)
 		{
 			inspector->set_drop_event_flags(EF_DROP_FALCO);
-			inspector->start_dropping_mode(1);
 		}
 
 		if (describe_all_rules)
@@ -667,13 +589,6 @@ int falco_init(int argc, char **argv)
 			goto exit;
 		}
 
-		if(signal(SIGUSR1, reopen_outputs) == SIG_ERR)
-		{
-			fprintf(stderr, "An error occurred while setting SIGUSR1 signal handler.\n");
-			result = EXIT_FAILURE;
-			goto exit;
-		}
-
 		if (scap_filename.size())
 		{
 			inspector->open(scap_filename);
@@ -818,8 +733,7 @@ int falco_init(int argc, char **argv)
 				      outputs,
 				      inspector,
 				      uint64_t(duration_to_tot*ONE_SECOND_IN_NS),
-				      stats_filename,
-				      all_events);
+				      stats_filename);
 
 		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;
 

userspace/falco/falco_outputs.cpp

@@ -142,19 +142,3 @@ void falco_outputs::handle_event(sinsp_evt *ev, string &rule, falco_common::prio
 	}
 
 }
-
-void falco_outputs::reopen_outputs()
-{
-	lua_getglobal(m_ls, m_lua_output_reopen.c_str());
-
-	if(!lua_isfunction(m_ls, -1))
-	{
-		throw falco_exception("No function " + m_lua_output_reopen + " found. ");
-	}
-
-	if(lua_pcall(m_ls, 0, 0, 0) != 0)
-	{
-		const char* lerr = lua_tostring(m_ls, -1);
-		throw falco_exception(string(lerr));
-	}
-}

userspace/falco/falco_outputs.h

@@ -53,8 +53,6 @@ public:
 	//
 	void handle_event(sinsp_evt *ev, std::string &rule, falco_common::priority_type priority, std::string &format);
 
-	void reopen_outputs();
-
 private:
 	bool m_initialized;
 
@@ -66,6 +64,5 @@ private:
 	std::string m_lua_add_output = "add_output";
 	std::string m_lua_output_event = "output_event";
 	std::string m_lua_output_cleanup = "output_cleanup";
-	std::string m_lua_output_reopen = "output_reopen";
 	std::string m_lua_main_filename = "output.lua";
 };

userspace/falco/lua/output.lua

@@ -20,8 +20,8 @@ local mod = {}
 
 local outputs = {}
 
-function mod.stdout(priority, priority_num, msg, options)
-   if options.buffered == 0 then
+function mod.stdout(priority, priority_num, buffered, msg)
+   if buffered == 0 then
       io.stdout:setvbuf 'no'
    end
    print (msg)
@@ -31,10 +31,6 @@ function mod.stdout_cleanup()
    io.stdout:flush()
 end
 
--- Note: not actually closing/reopening stdout
-function mod.stdout_reopen(options)
-end
-
 function mod.file_validate(options)
    if (not type(options.filename) == 'string') then
       error("File output needs to be configured with a valid filename")
@@ -48,98 +44,73 @@ function mod.file_validate(options)
 
 end
 
-function mod.file_open(options)
-   if ffile == nil then
-      ffile = io.open(options.filename, "a+")
-      if options.buffered == 0 then
-	 ffile:setvbuf 'no'
-      end
-   end
-end
-
-function mod.file(priority, priority_num, msg, options)
+function mod.file(priority, priority_num, buffered, msg, options)
    if options.keep_alive == "true" then
-      mod.file_open(options)
+      if file == nil then
+	 file = io.open(options.filename, "a+")
+	 if buffered == 0 then
+	    file:setvbuf 'no'
+	 end
+      end
    else
-      ffile = io.open(options.filename, "a+")
+      file = io.open(options.filename, "a+")
    end
 
-   ffile:write(msg, "\n")
+   file:write(msg, "\n")
 
    if options.keep_alive == nil or
-          options.keep_alive ~= "true" then
-      ffile:close()
-      ffile = nil
+      options.keep_alive ~= "true" then
+	 file:close()
+	 file = nil
    end
 end
 
 function mod.file_cleanup()
-   if ffile ~= nil then
-      ffile:flush()
-      ffile:close()
-      ffile = nil
+   if file ~= nil then
+      file:flush()
+      file:close()
+      file = nil
    end
 end
 
-function mod.file_reopen(options)
-   if options.keep_alive == "true" then
-      mod.file_cleanup()
-      mod.file_open(options)
-   end
-end
-
-function mod.syslog(priority, priority_num, msg, options)
+function mod.syslog(priority, priority_num, buffered, msg, options)
    falco.syslog(priority_num, msg)
 end
 
 function mod.syslog_cleanup()
 end
 
-function mod.syslog_reopen()
-end
-
-function mod.program_open(options)
-   if pfile == nil then
-      pfile = io.popen(options.program, "w")
-      if options.buffered == 0 then
-	 pfile:setvbuf 'no'
-      end
-   end
-end
-
-function mod.program(priority, priority_num, msg, options)
+function mod.program(priority, priority_num, buffered, msg, options)
    -- XXX Ideally we'd check that the program ran
    -- successfully. However, the luajit we're using returns true even
    -- when the shell can't run the program.
 
    -- Note: options are all strings
    if options.keep_alive == "true" then
-      mod.program_open(options)
+      if file == nil then
+	 file = io.popen(options.program, "w")
+	 if buffered == 0 then
+	    file:setvbuf 'no'
+	 end
+      end
    else
-      pfile = io.popen(options.program, "w")
+      file = io.popen(options.program, "w")
    end
 
-   pfile:write(msg, "\n")
+   file:write(msg, "\n")
 
    if options.keep_alive == nil or
-          options.keep_alive ~= "true" then
-      pfile:close()
-      pfile = nil
+      options.keep_alive ~= "true" then
+	 file:close()
+	 file = nil
    end
 end
 
 function mod.program_cleanup()
-   if pfile ~= nil then
-      pfile:flush()
-      pfile:close()
-      pfile = nil
-   end
-end
-
-function mod.program_reopen(options)
-   if options.keep_alive == "true" then
-      mod.program_cleanup()
-      mod.program_open(options)
+   if file ~= nil then
+      file:flush()
+      file:close()
+      file = nil
    end
 end
 
@@ -155,7 +126,7 @@ function output_event(event, rule, priority, priority_num, format)
    msg = formats.format_event(event, rule, priority, format)
 
    for index,o in ipairs(outputs) do
-      o.output(priority, priority_num, msg, o.options)
+      o.output(priority, priority_num, o.buffered, msg, o.config)
    end
 end
 
@@ -166,33 +137,20 @@ function output_cleanup()
    end
 end
 
-function output_reopen()
-   for index,o in ipairs(outputs) do
-      o.reopen(o.options)
-   end
-end
-
-function add_output(output_name, buffered, options)
+function add_output(output_name, buffered, config)
    if not (type(mod[output_name]) == 'function') then
       error("rule_loader.add_output(): invalid output_name: "..output_name)
    end
 
    -- outputs can optionally define a validation function so that we don't
-   -- find out at runtime (when an event finally matches a rule!) that the options are invalid
+   -- find out at runtime (when an event finally matches a rule!) that the config is invalid
    if (type(mod[output_name.."_validate"]) == 'function') then
-     mod[output_name.."_validate"](options)
+     mod[output_name.."_validate"](config)
    end
 
-   if options == nil then
-      options = {}
-   end
-
-   options.buffered = buffered
-
    table.insert(outputs, {output = mod[output_name],
 			  cleanup = mod[output_name.."_cleanup"],
-			  reopen = mod[output_name.."_reopen"],
-			  options=options})
+			  buffered=buffered, config=config})
 end
 
 return mod