new(test): read grpc config fields

Co-authored-by: Lorenzo Fontana <lo@linux.com>
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>

ADOPTERS.md

@@ -10,9 +10,6 @@ This is a list of production adopters of Falco (in alphabetical order):
 
 * [League](https://league.com/ca/) - League provides health benefits management services to help employees understand and get the most from their benefits, and employers to provide effective, efficient plans. Falco is used to monitor our deployed services on Kubernetes, protecting against malicious access to containerswhich could lead to leaks of PHI or other sensitive data. The Falco alerts are logged in Stackdriver for grouping and further analysis. In the future, we're hoping for integrations with Prometheus and AlertManager as well.
 
-* [Logz.io](https://logz.io/) - Logz.io is a cloud observability platform for modern engineering teams. The Logz.io platform consists of three products — Log Management, Infrastructure Monitoring, and Cloud SIEM — that work together to unify the jobs of monitoring, troubleshooting, and security. We empower engineers to deliver better software by offering the world's most popular open source observability tools — the ELK Stack, Grafana, and Jaeger — in a single, easy to use, and powerful platform purpose-built for monitoring distributed cloud environments. Cloud SIEM supports data from multiple sources, including Falco's alerts, and offers useful rules and dashboards content to visualize and manage incidents across your systems in a unified UI.
-  * https://logz.io/blog/k8s-security-with-falco-and-cloud-siem/
-
 * [Preferral](https://www.preferral.com) - Preferral is a HIPAA-compliant platform for Referral Management and Online Referral Forms. Preferral streamlines the referral process for patients, specialists and their referral partners. By automating the referral process, referring practices spend less time on the phone, manual efforts are eliminated, and patients get the right care from the right specialist. Preferral leverages Falco to provide a Host Intrusion Detection System to meet their HIPPA compliance requirements.
   * https://hipaa.preferral.com/01-preferral_hipaa_compliance/
 

cmake/modules/sysdig.cmake

@@ -26,8 +26,8 @@ file(MAKE_DIRECTORY ${SYSDIG_CMAKE_WORKING_DIR})
 # To update sysdig version for the next release, change the default below
 # In case you want to test against another sysdig version just pass the variable - ie., `cmake -DSYSDIG_VERSION=dev ..`
 if(NOT SYSDIG_VERSION)
-  set(SYSDIG_VERSION "422ab408c5706fbdd45432646cc197eb79459169")
-  set(SYSDIG_CHECKSUM "SHA256=367db2a480bca327a46f901bcc8384f151231bcddba88c719a06cf13971f4ab5")
+  set(SYSDIG_VERSION "96bd9bc560f67742738eb7255aeb4d03046b8045")
+  set(SYSDIG_CHECKSUM "SHA256=766e8952a36a4198fd976b9d848523e6abe4336612188e4fc911e217d8e8a00d")
 endif()
 set(PROBE_VERSION "${SYSDIG_VERSION}")
 

docker/tester/Dockerfile

@@ -1,16 +1,18 @@
 FROM fedora:31
 
 LABEL name="falcosecurity/falco-tester"
-LABEL usage="docker run -v /boot:/boot:ro -v /var/run/docker.sock:/var/run/docker.sock -v $PWD/..:/source -v $PWD/build:/build -e FALCO_VERSION=<current_falco_version> --name <name> falcosecurity/falco-tester test"
+LABEL usage="docker run -v /boot:/boot:ro -v /var/run/docker.sock:/var/run/docker.sock -v $PWD/..:/source -v $PWD/build:/build --name <name> falcosecurity/falco-tester test"
 LABEL maintainer="cncf-falco-dev@lists.cncf.io"
 
 ENV FALCO_VERSION=
 ENV BUILD_TYPE=release
 
+ADD https://github.com/fullstorydev/grpcurl/releases/download/v1.6.0/grpcurl_1.6.0_linux_x86_64.tar.gz /
 RUN dnf install -y python-pip python docker findutils jq unzip && dnf clean all
 ENV PATH="/root/.local/bin/:${PATH}"
 RUN pip install --user avocado-framework==69.0
 RUN pip install --user avocado-framework-plugin-varianter-yaml-to-mux==69.0
+RUN tar -C /usr/bin -xvf grpcurl_1.6.0_linux_x86_64.tar.gz
 
 COPY ./root /
 

falco.yaml

@@ -139,7 +139,7 @@ stdout_output:
 webserver:
   enabled: true
   listen_port: 8765
-  k8s_audit_endpoint: /k8s-audit
+  k8s_audit_endpoint: /k8s_audit
   ssl_enabled: false
   ssl_certificate: /etc/falco/falco.pem
 

proposals/20200623-rules-required-engine-version-scope.md

@@ -1,72 +0,0 @@
-# Required engine version scoping for rules
-
-<!-- toc -->
-
-- [Summary](#summary)
-- [Motivation](#motivation)
-  * [Goals](#goals)
-  * [Non-Goals](#non-goals)
-  * [Use cases](#use-cases)
-  * [Example of rules file](#example-of-rules-file)
-
-<!-- tocstop -->
-
-## Summary
-
-We want to be able to scope the `required_engine_version` field also for the specific rule/macro
-other than just for the currently rules file.
-
-## Motivation
-
-While the Falco engine and drivers evolve, new fields are added. While new fields are added,
-the upstream rules evolve too. This menas that we need a mechanism to be able to tell the users
-and the engine at load time "Hey, this rule is compatible". We currently do that at file level
-using the `required_engine_version` field.
-
-While this is very handy, this also does not help users to understand what are the rules that require,
-let's say engine verison `6` instead of `5`. It's very likely that 99% of a file is compatible with engine `2` while
-in reality only one rule is not.
-
-This is particularly useful for rules sharing. Users with different Falco versions can share rules containing this field
-and instead of getting `<NA>` they can be informed immediatelly about the incompatibility.
-
-### Goals
-
-- To add a new field `required_engine_version` scoped to the `rule` and `macro` sections.
-- The new fields take priority over the file global `required_engine_version` field.
-
-### Non-Goals
-
-- NONE
-
-### Use cases
-
-- Better understanding of what are the specific rules that need a specific engine version
-- Helps for when we want to make an API to create/delete/modify rules at runtime. In such a dynamic scenarios it's very useful for users to just know in advance if that rule is compatible
-- Makes easier to spot `<NA>` fields happening for `required_engine_version` mismatches  since the incompatibility is immediately reported by the engine.
-
-
-### Example of rules file
-
-```yaml
-- required_engine_version: 2
-
-- list: cat_binaries
-  items: [cat]
-
-- list: cat_capable_binaries
-  items: [cat_binaries]
-
-- macro: is_cat
-  condition: proc.name in (cat_capable_binaries)
-
-- rule: open_from_cat
-  required_engine_version: 4
-  desc: A process named cat does an open
-  condition: evt.type=open and is_cat
-  output: "An open was seen (command=%proc.cmdline)"
-  priority: WARNING
-
-```
-
----

rules/falco_rules.yaml

@@ -55,7 +55,6 @@
 - macro: proc_name_exists
   condition: (proc.name!="<NA>")
 
-# todo(leogr): we miss "renameat2", but it's not yet supported by sinsp
 - macro: rename
   condition: evt.type in (rename, renameat)
 - macro: mkdir
@@ -81,29 +80,17 @@
 
 - macro: bin_dir_mkdir
   condition: >
-     (evt.arg.path startswith /bin/ or
-     evt.arg.path startswith /sbin/ or
-     evt.arg.path startswith /usr/bin/ or
-     evt.arg.path startswith /usr/sbin/)
+    (evt.arg[1] startswith /bin/ or
+     evt.arg[1] startswith /sbin/ or
+     evt.arg[1] startswith /usr/bin/ or
+     evt.arg[1] startswith /usr/sbin/)
 
 - macro: bin_dir_rename
   condition: >
-     (evt.arg.path startswith /bin/ or
-     evt.arg.path startswith /sbin/ or
-     evt.arg.path startswith /usr/bin/ or
-     evt.arg.path startswith /usr/sbin/ or
-     evt.arg.name startswith /bin/ or
-     evt.arg.name startswith /sbin/ or
-     evt.arg.name startswith /usr/bin/ or
-     evt.arg.name startswith /usr/sbin/ or
-     evt.arg.oldpath startswith /bin/ or
-     evt.arg.oldpath startswith /sbin/ or
-     evt.arg.oldpath startswith /usr/bin/ or
-     evt.arg.oldpath startswith /usr/sbin/ or
-     evt.arg.newpath startswith /bin/ or
-     evt.arg.newpath startswith /sbin/ or
-     evt.arg.newpath startswith /usr/bin/ or
-     evt.arg.newpath startswith /usr/sbin/)
+    evt.arg[1] startswith /bin/ or
+    evt.arg[1] startswith /sbin/ or
+    evt.arg[1] startswith /usr/bin/ or
+    evt.arg[1] startswith /usr/sbin/
 
 - macro: etc_dir
   condition: fd.name startswith /etc/
@@ -942,12 +929,6 @@
     NOTICE
   tags: [filesystem, mitre_persistence]
 
-# Users should overwrite this macro to specify conditions under which a
-# write under the binary dir is ignored. For example, it may be okay to
-# install a binary in the context of a ci/cd build.
-- macro: user_known_write_below_binary_dir_activities
-  condition: (never_true)
-
 - rule: Write below binary dir
   desc: an attempt to write to any file below a set of binary directories
   condition: >
@@ -956,7 +937,6 @@
     and not exe_running_docker_save
     and not python_running_get_pip
     and not python_running_ms_oms
-    and not user_known_write_below_binary_dir_activities
   output: >
     File below a known binary directory opened for writing (user=%user.name
     command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2] container_id=%container.id image=%container.image.repository)
@@ -1376,9 +1356,6 @@
 - macro: runc_writing_exec_fifo
   condition: (proc.cmdline="runc:[1:CHILD] init" and fd.name=/exec.fifo)
 
-- macro: runc_writing_var_lib_docker
-  condition: (proc.cmdline="runc:[1:CHILD] init" and evt.arg.filename startswith /var/lib/docker)
-
 - rule: Write below root
   desc: an attempt to write to any file directly below / or /root
   condition: >
@@ -1528,7 +1505,7 @@
 
 - rule: Modify binary dirs
   desc: an attempt to modify any file below a set of binary directories.
-  condition: bin_dir_rename and modify and not package_mgmt_procs and not exe_running_docker_save
+  condition: (bin_dir_rename) and modify and not package_mgmt_procs and not exe_running_docker_save
   output: >
     File below known binary directory renamed/removed (user=%user.name command=%proc.cmdline
     pcmdline=%proc.pcmdline operation=%evt.type file=%fd.name %evt.args container_id=%container.id image=%container.image.repository)
@@ -2248,7 +2225,7 @@
   desc: creating any files below /dev other than known programs that manage devices. Some rootkits hide files in /dev.
   condition: >
     fd.directory = /dev and
-    (evt.type = creat or ((evt.type = open or evt.type = openat) and evt.arg.flags contains O_CREAT))
+    (evt.type = creat or (evt.type = open and evt.arg.flags contains O_CREAT))
     and not proc.name in (dev_creation_binaries)
     and not fd.name in (allowed_dev_files)
     and not fd.name startswith /dev/tty
@@ -2525,7 +2502,7 @@
 - rule: Delete Bash History
   desc: Detect bash history deletion
   condition: >
-    ((spawned_process and proc.name in (shred, rm, mv) and proc.args contains "bash_history") or
+    ((spawned_process and proc.name in (shred, rm, mv) and proc.args contains "bash_history") or 
      (open_write and fd.name contains "bash_history" and evt.arg.flags contains "O_TRUNC"))
   output: >
     Shell history had been deleted or renamed (user=%user.name type=%evt.type command=%proc.cmdline fd.name=%fd.name name=%evt.arg.name path=%evt.arg.path oldpath=%evt.arg.oldpath %container.info)
@@ -2749,7 +2726,7 @@
   output: Packet socket was created in a container (user=%user.name command=%proc.cmdline socket_info=%evt.args container_id=%container.id container_name=%container.name image=%container.image.repository:%container.image.tag)
   priority: NOTICE
   tags: [network, mitre_discovery]
-
+ 
 # Change to (always_true) to enable rule 'Network connection outside local subnet'
 - macro: enabled_rule_network_only_subnet
   condition: (never_true)
@@ -2765,7 +2742,7 @@
 - macro: network_local_subnet
   condition: >
     fd.rnet in (rfc_1918_addresses) or
-    fd.ip = "0.0.0.0" or
+    fd.ip = "0.0.0.0" or 
     fd.net = "127.0.0.0/8"
 
 # # How to test:
@@ -2825,7 +2802,7 @@
     not fd.sport in (authorized_server_port)
   output: >
     Network connection outside authorized port and binary
-    (command=%proc.cmdline connection=%fd.name user=%user.name container_id=%container.id
+    (command=%proc.cmdline connection=%fd.name user=%user.name container_id=%container.id 
     image=%container.image.repository)
   priority: WARNING
   tags: [network]
@@ -2837,46 +2814,6 @@
     Redirect stdout/stdin to network connection (user=%user.name %container.info process=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository fd.name=%fd.name fd.num=%fd.num fd.type=%fd.type fd.sip=%fd.sip)
   priority: WARNING
 
-# The two Container Drift rules below will fire when a new executable is created in a container.
-# There are two ways to create executables - file is created with execution permissions or permissions change of existing file.
-# We will use a new sysdig filter, is_open_exec, to find all files creations with execution permission, and will trace all chmods in a container.
-# The use case we are targeting here is an attempt to execute code that was not shipped as part of a container (drift) -
-# an activity that might be malicious or non-compliant.
-# Two things to pay attention to:
-#   1) In most cases, 'docker cp' will not be identified, but the assumption is that if an attacker gained access to the container runtime daemon, they are already privileged
-#   2) Drift rules will be noisy in environments in which containers are built (e.g. docker build)
-
-- rule: Container Drift Detected (chmod)
-  desc: New executable created in a container due to chmod
-  condition: >
-    chmod and
-    consider_all_chmods and
-    container and
-    not runc_writing_exec_fifo and
-    not runc_writing_var_lib_docker and
-    evt.rawres>=0 and
-    ((evt.arg.mode contains "S_IXUSR") or
-    (evt.arg.mode contains "S_IXGRP") or
-    (evt.arg.mode contains "S_IXOTH"))
-  output: Drift detected (chmod), new executable created in a container (user=%user.name command=%proc.cmdline filename=%evt.arg.filename name=%evt.arg.name mode=%evt.arg.mode event=%evt.type)
-  priority: ERROR
-
-# ****************************************************************************
-# * "Container Drift Detected (open+create)" requires FALCO_ENGINE_VERSION 6 *
-# ****************************************************************************
-- rule: Container Drift Detected (open+create)
-  desc: New executable created in a container due to open+create
-  condition: >
-    evt.type in (open,openat,creat) and
-    evt.is_open_exec=true and
-    container and
-    not runc_writing_exec_fifo and
-    not runc_writing_var_lib_docker and
-    evt.rawres>=0
-  output: Drift detected (open+create), new executable created in a container (user=%user.name command=%proc.cmdline filename=%evt.arg.filename name=%evt.arg.name mode=%evt.arg.mode event=%evt.type)
-  priority: ERROR
-
-
 # Application rules have moved to application_rules.yaml. Please look
 # there if you want to enable them by adding to
 # falco_rules.local.yaml.

rules/k8s_audit_rules.yaml

@@ -186,7 +186,7 @@
 - rule: Anonymous Request Allowed
   desc: >
     Detect any request made by the anonymous user that was allowed
-  condition: kevt and ka.user.name=system:anonymous and ka.auth.decision="allow" and not health_endpoint
+  condition: kevt and ka.user.name=system:anonymous and ka.auth.decision!=reject and not health_endpoint
   output: Request by anonymous user allowed (user=%ka.user.name verb=%ka.verb uri=%ka.uri reason=%ka.auth.reason))
   priority: WARNING
   source: k8s_audit

test/confs/grpc_unix_socket.yaml

@@ -0,0 +1,38 @@
+#
+# Copyright (C) 2020 The Falco Authors.
+#
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Whether to output events in json or text.
+json_output: false
+
+# Send information logs to stderr and/or syslog
+# Note these are *not* security notification logs!
+# These are just Falco lifecycle (and possibly error) logs.
+log_stderr: false
+log_syslog: false
+
+# Where security notifications should go.
+stdout_output:
+  enabled: false
+
+# gRPC server using an unix socket.
+grpc:
+    enabled: true
+    bind_address: "unix:////tmp/falco.sock"
+    threadiness: 8
+
+grpc_output:
+  enabled: true

test/falco_test.py

@@ -195,6 +195,19 @@ class FalcoTest(Test):
                         os.makedirs(filedir)
             self.outputs = outputs
 
+        self.grpc_unix_socket_path = self.params.get('grpc_unix_socket_path', '*', default='/var/run/falco.sock')
+        self.grpc_address = self.params.get('address', 'grpc/*', default='/var/run/falco.sock')
+        if self.grpc_address.startswith("unix://"):
+            self.is_grpc_using_unix_socket = True
+            self.grpc_address = self.grpc_address[len("unix://"):]
+        self.grpc_proto = self.params.get('proto', 'grpc/*', default='')
+        self.grpc_service = self.params.get('service', 'grpc/*', default='')
+        self.grpc_method = self.params.get('method', 'grpc/*', default='')
+        self.grpc_results = self.params.get('results', 'grpc/*', default='')
+        # todo: if string wrap in an array
+        if self.grpc_results == '':
+            self.grpc_results = []
+
         self.disable_tags = self.params.get('disable_tags', '*', default='')
 
         if self.disable_tags == '':

test/falco_tests.yaml

@@ -672,6 +672,20 @@ trace_files: !mux
     outputs:
       - /tmp/falco_outputs/program_output.txt: Warning An open was seen
 
+  grpc_unix_socket_outputs:
+    detect: True
+    detect_level: WARNING
+    rules_file:
+      - rules/single_rule.yaml
+    conf_file: confs/grpc_unix_socket.yaml
+    grpc:
+      address: unix:///tmp/falco.sock
+      proto: output.proto
+      service: falco.output.service
+      method: subscribe
+      results:
+        - "Warning An open was seen"
+
   detect_counts:
     detect: True
     detect_level: WARNING

userspace/engine/falco_engine_version.h

@@ -16,9 +16,9 @@ limitations under the License.
 
 // The version of rules/filter fields/etc supported by this falco
 // engine.
-#define FALCO_ENGINE_VERSION (6)
+#define FALCO_ENGINE_VERSION (5)
 
 // This is the result of running "falco --list -N | sha256sum" and
 // represents the fields supported by this version of falco. It's used
 // at build time to detect a changed set of fields.
-#define FALCO_FIELDS_CHECKSUM "2f324e2e66d4b423f53600e7e0fcf2f0ff72e4a87755c490f2ae8f310aba9386"
+#define FALCO_FIELDS_CHECKSUM "ca9e75fa41fe4480cdfad8cf275cdbbc334e656569f070c066d87cbd2955c1ae"

userspace/falco/logger.cpp

@@ -134,7 +134,7 @@ void falco_logger::log(int priority, const string msg)
 			if(gtm != NULL &&
 			   (strftime(buf, sizeof(buf), "%FT%T%z", gtm) != 0))
 			{
-				fprintf(stderr, "%s: %s", buf, copy.c_str());
+				fprintf(stderr, "%s: %s", buf, msg.c_str());
 			}
 		}
 		else
@@ -151,7 +151,7 @@ void falco_logger::log(int priority, const string msg)
 			{
 				tstr = "N/A";
 			}
-			fprintf(stderr, "%s: %s", tstr.c_str(), copy.c_str());
+			fprintf(stderr, "%s: %s", tstr.c_str(), msg.c_str());
 		}
 	}
 }