Merge branch 'dev' into agent-master

* Add ability to skip rules for unknown filters

Add the ability to skip a rule if its condition refers to a filtercheck
that doesn't exist. This allows defining a rules file that contains new
conditions that can still has limited backward compatibility with older
falco versions.

When compiling a filter, return a list of filtercheck names that are
present in the ast (which also includes filterchecks from any
macros). This set of filtercheck names is matched against the set of
filterchecks known to sinsp, expressed as lua patterns, and in the
global table defined_filters. If no match is found, the rule loader
throws an error.

The pattern changes slightly depending on whether the filter has
arguments or not. Two filters (proc.apid/proc.aname) can work with or
without arguments, so both styles of patterns are used.

If the rule has an attribute "skip-if-unknown-filter", the rule will be
skipped instead.

* Unit tests for skipping unknown filter

New unit test for skipping unknown filter. Test cases:

 - A rule that refers to an unknown filter results in an error.
 - A rule that refers to an unknown filter, but has
   "skip-if-unknown-filter: true", can be read, but doesn't match any events.
 - A rule that refers to an unknown filter, but has
   "skip-if-unknown-filter: false", returns an error.

Also test the case of a filtercheck like evt.arg.xxx working properly
with the embedded patterns as well as proc.aname/apid which work both ways.

CHANGELOG.md

@@ -2,6 +2,48 @@
 
 This file documents all notable changes to Falco. The release numbering uses [semantic versioning](http://semver.org).
 
+## v0.10.0
+
+Released 2018-04-24
+
+## Major Changes
+
+* **Rules Directory Support**: Falco will read rules files from `/etc/falco/rules.d` in addition to `/etc/falco/falco_rules.yaml` and `/etc/falco/falco_rules.local.yaml`. Also, when the argument to `-r`/falco.yaml `rules_file` is a directory, falco will read rules files from that directory. [[#348](https://github.com/draios/falco/pull/348)] [[#187](https://github.com/draios/falco/issues/187)]
+* Properly support all syscalls (e.g. those without parameter extraction by the kernel module) in falco conditions, so they can be included in `evt.type=<name>` conditions. [[#352](https://github.com/draios/falco/pull/352)]
+* When packaged as a container, start building kernel module with gcc 5.0 instead of gcc 4.9. [[#331](https://github.com/draios/falco/pull/331)]
+* New example puppet module for falco. [[#341](https://github.com/draios/falco/pull/341)] [[#115](https://github.com/draios/falco/issues/115)]
+* When signaled with `USR1`, falco will close/reopen log files. Include a [logrotate](https://github.com/logrotate/logrotate) example that shows how to use this feature for log rotation. [[#347](https://github.com/draios/falco/pull/347)] [[#266](https://github.com/draios/falco/issues/266)]
+* To improve resource usage, further restrict the set of system calls available to falco [[#351](https://github.com/draios/falco/pull/351)] [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)]
+
+## Minor Changes
+
+* Add gdb to the development Docker image (sysdig/falco:dev) to aid in debugging. [[#323](https://github.com/draios/falco/pull/323)]
+* You can now specify -V multiple times on the command line to validate multiple rules files at once. [[#329](https://github.com/draios/falco/pull/329)]
+* When run with `-v`, falco will print *dangling* macros/lists that are not used by any rules. [[#329](https://github.com/draios/falco/pull/329)]
+* Add an example demonstrating cryptomining attack that exploits an open docker daemon using host mounts. [[#336](https://github.com/draios/falco/pull/336)]
+* New falco.yaml option `json_include_output_property` controls whether the formatted string "output" is included in the json object when json output is enabled. [[#342](https://github.com/draios/falco/pull/342)]
+* Centralize testing event types for consideration by falco into a single function [[draios/sysdig#1105](https://github.com/draios/sysdig/pull/1105)) [[#356](https://github.com/draios/falco/pull/356)]
+* If a rule has an attribute `warn_evttypes`, falco will not complain about `evt.type` restrictions on that rule [[#355](https://github.com/draios/falco/pull/355)]
+* When run with `-i`, print all ignored events/syscalls and exit. [[#359](https://github.com/draios/falco/pull/359)]
+
+## Bug Fixes
+
+* Minor bug fixes to k8s daemonset configuration. [[#325](https://github.com/draios/falco/pull/325)] [[#296](https://github.com/draios/falco/pull/296)] [[#295](https://github.com/draios/falco/pull/295)]
+* Ensure `--validate` can be used interchangeably with `-V`. [[#334](https://github.com/draios/falco/pull/334)] [[#322](https://github.com/draios/falco/issues/322)]
+* Rule conditions like `fd.net` can now be used with the `in` operator e.g. `evt.type=connect and fd.net in ("127.0.0.1/24")`. [[draios/sysdig#1091](https://github.com/draios/sysdig/pull/1091)] [[#343](https://github.com/draios/falco/pull/343)]
+* Ensure that `keep_alive` can be used both with file and program output at the same time. [[#335](https://github.com/draios/falco/pull/335)]
+* Make it possible to append to a skipped macro/rule without falco complaining [[#346](https://github.com/draios/falco/pull/346)] [[#305](https://github.com/draios/falco/issues/305)]
+* Ensure rule order is preserved even when rules do not contain any `evt.type` restriction. [[#354](https://github.com/draios/falco/issues/354)] [[#355](https://github.com/draios/falco/pull/355)]
+
+## Rule Changes
+
+* Make it easier to extend the `Change thread namespace` rule via a `user_known_change_thread_namespace_binaries` list. [[#324](https://github.com/draios/falco/pull/324)]
+* Various FP fixes from users. [[#321](https://github.com/draios/falco/pull/321)] [[#326](https://github.com/draios/falco/pull/326)] [[#344](https://github.com/draios/falco/pull/344)] [[#350](https://github.com/draios/falco/pull/350)]
+* New rule `Disallowed SSH Connection` detects attempts ssh connection attempts to hosts outside of an expected set. In order to be effective, you need to override the macro `allowed_ssh_hosts` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
+* New rule `Unexpected K8s NodePort Connection` detects attempts to contact the K8s NodePort range from a program running inside a container. In order to be effective, you need to override the macro `nodeport_containers` in a user rules file. [[#321](https://github.com/draios/falco/pull/321)]
+* Improve `Modify binary dirs` rule to work with new syscalls [[#353](https://github.com/draios/falco/pull/353)]
+* New rule `Unexpected UDP Traffic` checks for udp traffic not on a list of expected ports. Somewhat FP-prone, so it must be explicitly enabled by overriding the macro `do_unexpected_udp_check` in a user rules file. [[#320](https://github.com/draios/falco/pull/320)] [[#357](https://github.com/draios/falco/pull/357)]
+
 ## v0.9.0
 
 Released 2018-01-18

README.md

@@ -2,7 +2,7 @@
 
 #### Latest release
 
-**v0.9.0**
+**v0.10.0**
 Read the [change log](https://github.com/draios/falco/blob/dev/CHANGELOG.md)
 
 Dev Branch: [![Build Status](https://travis-ci.org/draios/falco.svg?branch=dev)](https://travis-ci.org/draios/falco)<br />

docker/dev/Dockerfile

@@ -17,13 +17,16 @@ ADD http://download.draios.com/apt-draios-priority /etc/apt/preferences.d/
 RUN apt-get update \
  && apt-get install -y --no-install-recommends \
 	bash-completion \
-	curl \
-	jq \
-	gnupg2 \
 	ca-certificates \
+	curl \
+	gnupg2 \
 	gcc \
 	gcc-5 \
-	gdb && rm -rf /var/lib/apt/lists/*
+	gdb \
+	jq \
+	libc6-dev \
+	libelf-dev \
+ && rm -rf /var/lib/apt/lists/*
 
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.

docker/local/Dockerfile

@@ -17,13 +17,16 @@ ADD http://download.draios.com/apt-draios-priority /etc/apt/preferences.d/
 RUN apt-get update \
  && apt-get install -y --no-install-recommends \
 	bash-completion \
-	curl \
-	jq \
-	gnupg2 \
 	ca-certificates \
+	curl \
+	dkms \
+	gnupg2 \
 	gcc \
 	gcc-5 \
-	dkms && rm -rf /var/lib/apt/lists/*
+	jq \
+	libc6-dev \
+	libelf-dev \
+ && rm -rf /var/lib/apt/lists/*
 
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.

docker/stable/Dockerfile

@@ -17,12 +17,15 @@ ADD http://download.draios.com/apt-draios-priority /etc/apt/preferences.d/
 RUN apt-get update \
  && apt-get install -y --no-install-recommends \
 	bash-completion \
-	curl \
-	jq \
 	ca-certificates \
+	curl \
 	gnupg2 \
 	gcc \
-	gcc-5 && rm -rf /var/lib/apt/lists/*
+	gcc-5 \
+	jq \
+	libc6-dev \
+	libelf-dev \
+ && rm -rf /var/lib/apt/lists/*
 
 # Since our base Debian image ships with GCC 7 which breaks older kernels, revert the
 # default to gcc-5.

rules/falco_rules.yaml

@@ -15,6 +15,9 @@
 - macro: never_true
   condition: (evt.num=0)
 
+- macro: always_true
+  condition: (evt.num=>0)
+
 # In some cases, such as dropped system call events, information about
 # the process name may be missing. For some rules that really depend
 # on the identity of the process performing an action such as opening
@@ -39,13 +42,6 @@
 - macro: bin_dir
   condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
 
-- macro: bin_dir_resolved
-  condition: >
-    (evt.abspath startswith /bin/ or
-     evt.abspath startswith /sbin/ or
-     evt.abspath startswith /usr/bin/ or
-     evt.abspath startswith /usr/sbin/)
-
 - macro: bin_dir_mkdir
   condition: >
     (evt.arg[1] startswith /bin/ or
@@ -245,18 +241,14 @@
 # Network
 - macro: inbound
   condition: >
-    (((evt.type in (accept,listen) and evt.dir=<) or
-      (evt.type in (recvfrom,recvmsg) and evt.dir=< and
-       fd.l4proto != tcp and fd.connected=false and fd.name_changed=true)) and
+    (((evt.type in (accept,listen) and evt.dir=<)) or
      (fd.typechar = 4 or fd.typechar = 6) and
      (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
      (evt.rawres >= 0 or evt.res = EINPROGRESS))
 
 - macro: outbound
   condition: >
-    (((evt.type = connect and evt.dir=<) or
-      (evt.type in (sendto,sendmsg) and evt.dir=< and
-       fd.l4proto != tcp and fd.connected=false and fd.name_changed=true)) and
+    (((evt.type = connect and evt.dir=<)) or
      (fd.typechar = 4 or fd.typechar = 6) and
      (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
      (evt.rawres >= 0 or evt.res = EINPROGRESS))
@@ -265,9 +257,7 @@
 # for efficiency.
 - macro: inbound_outbound
   condition: >
-    (((evt.type in (accept,listen,connect) and evt.dir=<) or
-      (evt.type in (recvfrom,recvmsg,sendto,sendmsg) and evt.dir=< and
-       fd.l4proto != tcp and fd.connected=false and fd.name_changed=true)) and
+    (((evt.type in (accept,listen,connect) and evt.dir=<)) or
      (fd.typechar = 4 or fd.typechar = 6) and
      (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
      (evt.rawres >= 0 or evt.res = EINPROGRESS))
@@ -378,6 +368,13 @@
        proc.pcmdline startswith "node /root/.config/yarn" or
        proc.pcmdline startswith "node /opt/yarn/bin/yarn.js"))
 
+
+- macro: httpd_writing_ssl_conf
+  condition: >
+    (proc.pname=run-httpd and
+     (proc.cmdline startswith "sed -ri" or proc.cmdline startswith "sed -i") and
+     (fd.name startswith /etc/httpd/conf.d/ or fd.name startswith /etc/httpd/conf))
+
 - macro: parent_Xvfb_running_xkbcomp
   condition: (proc.pname=Xvfb and proc.cmdline startswith 'sh -c "/usr/bin/xkbcomp"')
 
@@ -793,6 +790,7 @@
     and not centrify_writing_krb
     and not cockpit_writing_conf
     and not ipsec_writing_conf
+    and not httpd_writing_ssl_conf
 
 - rule: Write below etc
   desc: an attempt to write to any file below /etc
@@ -932,7 +930,12 @@
   condition: (proc.aname[2]=redis-server and (proc.cmdline contains "redis-server.post-up.d" or proc.cmdline contains "redis-server.pre-up.d"))
 
 - macro: rabbitmq_running_scripts
-  condition: (proc.pname=beam.smp and (proc.cmdline startswith "sh -c exec ps" or proc.cmdline startswith "sh -c exec inet_gethost"))
+  condition: >
+    (proc.pname=beam.smp and
+    (proc.cmdline startswith "sh -c exec ps" or
+     proc.cmdline startswith "sh -c exec inet_gethost" or
+     proc.cmdline= "sh -s unix:cmd" or
+     proc.cmdline= "sh -c exec /bin/sh -s unix:cmd 2>&1"))
 
 - macro: rabbitmqctl_running_scripts
   condition: (proc.aname[2]=rabbitmqctl and proc.cmdline startswith "sh -c ")
@@ -954,7 +957,7 @@
 
 - rule: Modify binary dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: (bin_dir_rename or bin_dir_resolved) and modify and not package_mgmt_procs and not exe_running_docker_save
+  condition: (bin_dir_rename) and modify and not package_mgmt_procs and not exe_running_docker_save
   output: >
     File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline
     operation=%evt.type file=%fd.name %evt.args)
@@ -1344,33 +1347,35 @@
 - list: statsd_ports
   items: [8125]
 
-- list: mysql_ports
-  items: [3306]
-
 - list: ntp_ports
   items: [123]
 
-# 0 is included in the list because some apps connect to an address
-# only to test connectivity.
+# Some applications will connect a udp socket to an address only to
+# test connectivity. Assuming the udp connect works, they will follow
+# up with a tcp connect that actually sends/receives data.
 #
-# mysql_ports is included becuase some versions of the mysql client
-# will attempt a connect using udp + port 3306 before connecting via
-# tcp + port 3306.
-#
-# 80 is included for the same reason as mysql_ports--some apps do a
-# connect using udp before doing a real connect using tcp.
+# With that in mind, we listed a few commonly seen ports here to avoid
+# some false positives. In addition, we make the main rule opt-in, so
+# it's disabled by default.
+
+- list: test_connect_ports
+  items: [0, 9, 80, 3306]
+
+- macro: do_unexpected_udp_check
+  condition: (never_true)
+
 - list: expected_udp_ports
-  items: [0, 53, 80, openvpn_udp_ports, l2tp_udp_ports, statsd_ports, mysql_ports, ntp_ports]
+  items: [53, openvpn_udp_ports, l2tp_udp_ports, statsd_ports, ntp_ports, test_connect_ports]
 
 - macro: expected_udp_traffic
   condition: fd.port in (expected_udp_ports)
 
 - rule: Unexpected UDP Traffic
   desc: UDP traffic not on port 53 (DNS) or other commonly used ports
-  condition: (inbound_outbound) and fd.l4proto=udp and not expected_udp_traffic
+  condition: (inbound_outbound) and do_unexpected_udp_check and fd.l4proto=udp and not expected_udp_traffic
   output: >
     Unexpected UDP Traffic Seen
-    (user=%user.name command=%proc.cmdline connection=%fd.name proto=%fd.l4proto)
+    (user=%user.name command=%proc.cmdline connection=%fd.name proto=%fd.l4proto evt=%evt.type %evt.args)
   priority: NOTICE
   tags: [network]
 

test/falco_test.py

@@ -31,6 +31,7 @@ class FalcoTest(Test):
 
         self.json_output = self.params.get('json_output', '*', default=False)
         self.json_include_output_property = self.params.get('json_include_output_property', '*', default=True)
+        self.all_events = self.params.get('all_events', '*', default=False)
         self.priority = self.params.get('priority', '*', default='debug')
         self.rules_file = self.params.get('rules_file', '*', default=os.path.join(self.basedir, '../rules/falco_rules.yaml'))
 
@@ -365,6 +366,9 @@ class FalcoTest(Test):
         if self.run_duration:
             cmd += ' -M {}'.format(self.run_duration)
 
+        if self.all_events:
+            cmd += ' -A'
+
         self.falco_proc = process.SubProcess(cmd)
 
         res = self.falco_proc.run(timeout=180, sig=9)

test/falco_tests.yaml

@@ -128,6 +128,7 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
+    all_events: True
 
   rules_directory:
     detect: True
@@ -138,6 +139,7 @@ trace_files: !mux
     rules_file:
       - rules/rules_dir
     trace_file: trace_files/cat_write.scap
+    all_events: True
 
   multiple_rules_suppress_info:
     detect: True
@@ -153,6 +155,7 @@ trace_files: !mux
       - rules/single_rule.yaml
       - rules/double_rule.yaml
     trace_file: trace_files/cat_write.scap
+    all_events: True
 
   multiple_rules_overriding:
     detect: False
@@ -699,6 +702,7 @@ trace_files: !mux
       - detect_madvise: 2
       - detect_open: 2
     trace_file: trace_files/syscall.scap
+    all_events: True
 
   catchall_order:
     detect: True
@@ -709,3 +713,30 @@ trace_files: !mux
       - open_dev_null: 1
         dev_null: 0
     trace_file: trace_files/cat_write.scap
+
+  skip_unknown_noevt:
+    detect: False
+    stdout_contains: Skipping rule "Contains Unknown Event And Skipping" that contains unknown filter proc.nobody
+    rules_file:
+      - rules/skip_unknown_evt.yaml
+    trace_file: trace_files/cat_write.scap
+
+  skip_unknown_prefix:
+    detect: False
+    rules_file:
+      - rules/skip_unknown_prefix.yaml
+    trace_file: trace_files/cat_write.scap
+
+  skip_unknown_error:
+    exit_status: 1
+    stderr_contains: Rule "Contains Unknown Event And Not Skipping" contains unknown filter proc.nobody. Exiting.
+    rules_file:
+      - rules/skip_unknown_error.yaml
+    trace_file: trace_files/cat_write.scap
+
+  skip_unknown_unspec_error:
+    exit_status: 1
+    stderr_contains: Rule "Contains Unknown Event And Unspecified" contains unknown filter proc.nobody. Exiting.
+    rules_file:
+      - rules/skip_unknown_unspec.yaml
+    trace_file: trace_files/cat_write.scap

test/rules/skip_unknown_error.yaml

@@ -0,0 +1,6 @@
+- rule: Contains Unknown Event And Not Skipping
+  desc: Contains an unknown event
+  condition: proc.nobody=cat
+  output: Never
+  skip-if-unknown-filter: false
+  priority: INFO

test/rules/skip_unknown_evt.yaml

@@ -0,0 +1,6 @@
+- rule: Contains Unknown Event And Skipping
+  desc: Contains an unknown event
+  condition: evt.type=open and proc.nobody=cat
+  output: Never
+  skip-if-unknown-filter: true
+  priority: INFO

test/rules/skip_unknown_prefix.yaml

@@ -0,0 +1,8 @@
+- rule: Contains Prefix of Filter
+  desc: Testing matching filter prefixes
+  condition: >
+    evt.type=open and evt.arg.path="foo" and evt.arg[0]="foo"
+    and proc.aname="ls" and proc.aname[1]="ls"
+    and proc.apid=10 and proc.apid[1]=10
+  output: Never
+  priority: INFO

test/rules/skip_unknown_unspec.yaml

@@ -0,0 +1,5 @@
+- rule: Contains Unknown Event And Unspecified
+  desc: Contains an unknown event
+  condition: proc.nobody=cat
+  output: Never
+  priority: INFO

userspace/engine/lua/compiler.lua

@@ -322,6 +322,21 @@ function get_evttypes_syscalls(name, ast, source, warn_evttypes)
    return evttypes, syscallnums
 end
 
+function get_filters(ast)
+
+   local filters = {}
+
+   function cb(node)
+      if node.type == "FieldName" then
+	 filters[node.value] = 1
+      end
+   end
+
+   parser.traverse_ast(ast.filter.value, {FieldName=1} , cb)
+
+   return filters
+end
+
 function compiler.expand_lists_in(source, list_defs)
 
    for name, def in pairs(list_defs) do
@@ -408,7 +423,9 @@ function compiler.compile_filter(name, source, macro_defs, list_defs, warn_evtty
 
    evttypes, syscallnums = get_evttypes_syscalls(name, ast, source, warn_evttypes)
 
-   return ast, evttypes, syscallnums
+   filters = get_filters(ast)
+
+   return ast, evttypes, syscallnums, filters
 end
 
 

userspace/engine/lua/rule_loader.lua

@@ -275,6 +275,12 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	    error ("Missing name in rule")
 	 end
 
+	 -- By default, if a rule's condition refers to an unknown
+	 -- filter like evt.type, etc the loader throws an error.
+	 if v['skip-if-unknown-filter'] == nil then
+	    v['skip-if-unknown-filter'] = false
+	 end
+
 	 -- Possibly append to the condition field of an existing rule
 	 append = false
 
@@ -378,9 +384,34 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
 	 warn_evttypes = v['warn_evttypes']
       end
 
-      local filter_ast, evttypes, syscallnums = compiler.compile_filter(v['rule'], v['condition'],
-									state.macros, state.lists,
-									warn_evttypes)
+      local filter_ast, evttypes, syscallnums, filters = compiler.compile_filter(v['rule'], v['condition'],
+										 state.macros, state.lists,
+										 warn_evttypes)
+
+      -- If a filter in the rule doesn't exist, either skip the rule
+      -- or raise an error, depending on the value of
+      -- skip-if-unknown-filter.
+      for filter, _ in pairs(filters) do
+	 found = false
+
+	 for pat, _ in pairs(defined_filters) do
+	    if string.match(filter, pat) ~= nil then
+	       found = true
+	       break
+	    end
+	 end
+
+	 if not found then
+	    if v['skip-if-unknown-filter'] then
+	       if verbose then
+		  print("Skipping rule \""..v['rule'].."\" that contains unknown filter "..filter)
+	       end
+	       goto next_rule
+	    else
+	       error("Rule \""..v['rule'].."\" contains unknown filter "..filter)
+	    end
+	 end
+      end
 
       if (filter_ast.type == "Rule") then
 	 state.n_rules = state.n_rules + 1
@@ -452,6 +483,8 @@ function load_rules(rules_content, rules_mgr, verbose, all_events, extra, replac
       else
 	 error ("Unexpected type in load_rule: "..filter_ast.type)
       end
+
+      ::next_rule::
    end
 
    if verbose then

userspace/engine/rules.cpp

@@ -258,6 +258,63 @@ void falco_rules::load_rules(const string &rules_content,
 
 		lua_setglobal(m_ls, m_lua_ignored_syscalls.c_str());
 
+		// Create a table containing all filtercheck names.
+		lua_newtable(m_ls);
+
+		vector<const filter_check_info*> fc_plugins;
+		sinsp::get_filtercheck_fields_info(&fc_plugins);
+
+		for(uint32_t j = 0; j < fc_plugins.size(); j++)
+		{
+			const filter_check_info* fci = fc_plugins[j];
+
+			if(fci->m_flags & filter_check_info::FL_HIDDEN)
+			{
+				continue;
+			}
+
+			for(int32_t k = 0; k < fci->m_nfields; k++)
+			{
+				const filtercheck_field_info* fld = &fci->m_fields[k];
+
+				if(fld->m_flags & EPF_TABLE_ONLY ||
+				   fld->m_flags & EPF_PRINT_ONLY)
+				{
+					continue;
+				}
+
+				// Some filters can work with or without an argument
+				std::set<string> flexible_filters = {
+					"^proc.aname",
+					"^proc.apid"
+				};
+
+				std::list<string> fields;
+				std::string field_base = string("^") + fld->m_name;
+
+				if(fld->m_flags & EPF_REQUIRES_ARGUMENT ||
+				   flexible_filters.find(field_base) != flexible_filters.end())
+				{
+					fields.push_back(field_base + "[%[%.]");
+				}
+
+				if(!(fld->m_flags & EPF_REQUIRES_ARGUMENT) ||
+				   flexible_filters.find(field_base) != flexible_filters.end())
+				{
+					fields.push_back(field_base + "$");
+				}
+
+				for(auto &field : fields)
+				{
+					lua_pushstring(m_ls, field.c_str());
+					lua_pushnumber(m_ls, 1);
+					lua_settable(m_ls, -3);
+				}
+			}
+		}
+
+		lua_setglobal(m_ls, m_lua_defined_filters.c_str());
+
 		lua_pushstring(m_ls, rules_content.c_str());
 		lua_pushlightuserdata(m_ls, this);
 		lua_pushboolean(m_ls, (verbose ? 1 : 0));

userspace/engine/rules.h

@@ -56,6 +56,7 @@ class falco_rules
 	string m_lua_load_rules = "load_rules";
 	string m_lua_ignored_syscalls = "ignored_syscalls";
 	string m_lua_ignored_events = "ignored_events";
+	string m_lua_defined_filters = "defined_filters";
 	string m_lua_events = "events";
 	string m_lua_syscalls = "syscalls";
 	string m_lua_describe_rule = "describe_rule";

userspace/falco/falco.cpp

@@ -22,6 +22,8 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <fstream>
 #include <set>
 #include <list>
+#include <vector>
+#include <algorithm>
 #include <string>
 #include <signal.h>
 #include <fcntl.h>
@@ -32,6 +34,7 @@ along with falco.  If not, see <http://www.gnu.org/licenses/>.
 #include <sinsp.h>
 
 #include "logger.h"
+#include "utils.h"
 
 #include "configuration.h"
 #include "falco_engine.h"
@@ -151,7 +154,8 @@ uint64_t do_inspect(falco_engine *engine,
 		    falco_outputs *outputs,
 		    sinsp* inspector,
 		    uint64_t duration_to_tot_ns,
-		    string &stats_filename)
+		    string &stats_filename,
+		    bool all_events)
 {
 	uint64_t num_evts = 0;
 	int32_t res;
@@ -218,8 +222,7 @@ uint64_t do_inspect(falco_engine *engine,
 			}
 		}
 
-		if(!inspector->is_debug_enabled() &&
-			ev->get_category() & EC_INTERNAL)
+		if(!ev->falco_consider() && !all_events)
 		{
 			continue;
 		}
@@ -241,6 +244,47 @@ uint64_t do_inspect(falco_engine *engine,
 	return num_evts;
 }
 
+static void print_all_ignored_events(sinsp *inspector)
+{
+	sinsp_evttables* einfo = inspector->get_event_info_tables();
+	const struct ppm_event_info* etable = einfo->m_event_info;
+	const struct ppm_syscall_desc* stable = einfo->m_syscall_info_table;
+
+	std::set<string> ignored_event_names;
+	for(uint32_t j = 0; j < PPM_EVENT_MAX; j++)
+	{
+		if(!sinsp::falco_consider_evtnum(j))
+		{
+			std::string name = etable[j].name;
+			// Ignore event names NA*
+			if(name.find("NA") != 0)
+			{
+				ignored_event_names.insert(name);
+			}
+		}
+	}
+
+	for(uint32_t j = 0; j < PPM_SC_MAX; j++)
+	{
+		if(!sinsp::falco_consider_syscallid(j))
+		{
+			std::string name = stable[j].name;
+			// Ignore event names NA*
+			if(name.find("NA") != 0)
+			{
+				ignored_event_names.insert(name);
+			}
+		}
+	}
+
+	printf("Ignored Event(s):");
+	for(auto it : ignored_event_names)
+	{
+		printf(" %s", it.c_str());
+	}
+	printf("\n");
+}
+
 //
 // ARGUMENT PARSING AND PROGRAM SETUP
 //
@@ -270,6 +314,7 @@ int falco_init(int argc, char **argv)
 	string output_format = "";
 	bool replace_container_info = false;
 	int duration_to_tot = 0;
+	bool print_ignored_events = false;
 
 	// Used for writing trace files
 	int duration_seconds = 0;
@@ -299,6 +344,7 @@ int falco_init(int argc, char **argv)
 		{"version", no_argument, 0, 0 },
 		{"validate", required_argument, 0, 'V' },
 		{"writefile", required_argument, 0, 'w' },
+		{"ignored-events", no_argument, 0, 'i'},
 
 		{0, 0, 0, 0}
 	};
@@ -315,7 +361,7 @@ int falco_init(int argc, char **argv)
 		// Parse the args
 		//
 		while((op = getopt_long(argc, argv,
-                                        "hc:AdD:e:k:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
+                                        "hc:AdD:e:ik:K:Ll:m:M:o:P:p:r:s:T:t:UvV:w:",
                                         long_options, &long_index)) != -1)
 		{
 			switch(op)
@@ -341,6 +387,9 @@ int falco_init(int argc, char **argv)
 				k8s_api = new string();
 				mesos_api = new string();
 				break;
+			case 'i':
+				print_ignored_events = true;
+				break;
 			case 'k':
 				k8s_api = new string(optarg);
 				break;
@@ -431,12 +480,20 @@ int falco_init(int argc, char **argv)
 			return EXIT_SUCCESS;
 		}
 
-
 		inspector = new sinsp();
+
+		if(print_ignored_events)
+		{
+			print_all_ignored_events(inspector);
+			delete(inspector);
+			return EXIT_SUCCESS;
+		}
+
 		engine = new falco_engine();
 		engine->set_inspector(inspector);
 		engine->set_extra(output_format, replace_container_info);
 
+
 		outputs = new falco_outputs();
 		outputs->set_inspector(inspector);
 
@@ -761,7 +818,8 @@ int falco_init(int argc, char **argv)
 				      outputs,
 				      inspector,
 				      uint64_t(duration_to_tot*ONE_SECOND_IN_NS),
-				      stats_filename);
+				      stats_filename,
+				      all_events);
 
 		duration = ((double)clock()) / CLOCKS_PER_SEC - duration;