bump Ubuntu version on tests-driver-loader-integration job to fix a
verifier issue as a temp workaround
Signed-off-by: Andrea Terzolo <andreaterzolo3@gmail.com>
This commit will unquote URL's allowing them to be supported by
libcurl and eliminate any errors when a valid (quoted) URL is supplied
by a user.
Closes#2579
Signed-off-by: Daniel Wright danielwright@bitgo.com
add send_numeric_zero_values config to allow users to save space
when using metrics option, while still also allowing
to send all keys (especially because we don't document the schema)
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
To supoprt the use of outputs that are documented in the falco
examples (e.g. jq piped to curl) I would like to propose including
these tools in the falco-no-driver image. They add a very minimal
size and dependency to the image but would make things a lot easier
for users getting started.
Closes#2580
Signed-off-by: Daniel Wright <danielwright@bitgo.com>
* rephrase descriptions for numerous config options
without changing the original content, meaning changes
reflect language improvements and minor extensions
(such as adding justifications or what it is) only
* add Falco environment variables section
* add Guidance for Kubernetes container engine command-line args settings
* general rewrap formatting w/ IDE
* minor additional re-ordering of configs
* minor general language adjustments
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
* add an index for logical categories
* move configs around without changing description content,
solely add a uniform header to each config
* indicate "Stable" or "Experimental" for most configs
to indicate current stability or maturity
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
* Ensure each metric field name more consistently adheres to the grammar used in Falco rules:
* `falco.`: new field class representing userspace counters, statistics, resource utilization, or necessary information fields
* `scap.`: new field class represents counters and statistics mostly obtained from Falco's kernel instrumentation before events are sent to userspace, but can include scap userspace stats as well
* minor cleanup
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
* prefix counters and stats belonging to kernel space w/ `k.` else `u.` for userspace
* add n_drops_perc from old stats writer schema
* revert one change: file output shall reflect exact same "output_fields" key as rule output, note that src is already part of the "output_fields" schema.
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
These changes break the old stats file output schema and consolidates
them with the new schema.
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
- avoiding inspector to be allocated for each rule
- use two boolean values for expecting macros and lists
- move items of lists alongside name, under info
- use snake case for json output, like we do for e.g alerts
- correctly retrieve evt names
- consider two levels of lists for exception operators
Signed-off-by: Lorenzo Susini <susinilorenzo1@gmail.com>
Instead, store image tarballs as artifact and download them in `reusable_publish_docker.yaml` workflow,
to be finally merged in multi-arch images and pushed.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Support for user provided CA certificate that can verify the remote server. Users
can provide path to the CA certiface store by providing a path to the dir or to the
CA store file. If needed users can decide to tell Falco to not verify the server.
Signed-off-by: Aldo Lacuku <aldo@lacuku.eu>
The test now take in accoint pre/post-conditions of the actions,
usage of the -A option, and the newly-introduced base_syscall
user configuration. This also makes sure that the event selection
properly handles generic events and options/configs precedence.
Co-authored-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
* Plus minor adjustments to ensure correct state_event_set for all configurations
* Ensure valid check_for_rules_unsupported_events for all configurations
* Remove user input validation warning -> re-introduce in follow up PR
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
* address reviewers feedback
* improve clarity around new -A and -i behavior
* additional cleanup (e.g. use generic set operations only)
* extend unit tests
Note: sinsp ppm sc API is undergoing a refactor, therefore current lookups are interim
and will subsequently be refactored as well.
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Define new -A behavior in configure_interesting_sets
* default: all syscalls in rules included, sinsp state enforcement without high volume I/O syscalls
* -A flag set: all syscalls in rules included, sinsp state enforcement and allowing high volume I/O syscalls
Co-authored-by: Federico Di Pierro <nierro92@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Moreover, now falcoctl service is enabled by default. It being bound to falco.target
allows us to avoid it running when falco.target is not running.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Eg: when building Falco docker image, and installing Falco package, we don't want it to build any driver.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Cleaned up unused vars in postinst scripts.
Finally, only show dialog window in interactive shells.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
AmazonLinux uses `gcc-$Vers`, like gcc-10, but our regex prevented that to work.
Instead, rely on the fact that **real** gcc has some `--version` fixed output.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Manage it via a bash dialog interface.
Moreover, use falco-driver-loader instead of dkms to build bpf/kmod after package install.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
In the copy constructor and assignment operator for falco_source, also
copy the ruleset along with factories/name.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that ASTs contain parse positions, use them when reporting errors
about unknown macros.
When doing the first pass to find all macro references, save macros as
a map<macro name,parse position> instead of a set<macro name>. While
making that change, change the visitor struct to use references
instead of pointers.
In the second pass, when reporting any unresolved macro references,
also report the parse position.
The unit tests also check that the positions of macros are properly
returned in the resolved/unresolved maps.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that ASTs have parse positions and the compiler will return the
position of the last error, use that in falco rules to return errors
within condition strings instead of reporting the position as the
beginning of the condition.
This led to a change in the filter_ruleset interface--now, an ast is
compiled to a filter before being passed to the filter_ruleset
object. That avoids polluting the interface with a lot of details
about rule_loader contexts, errors, etc. The ast is still provided in
case the filter_ruleset wants to do indexing/analysis of the filter.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Docker builder image was updated to remove the libelf and libz deps as they are now properly bundled, in BUNDLED_DEPS mode.
Finally, circleci musl job was updated to enforce the use of alpine-provided libelf package, since it is already static,
and building libelf on musl is pretty cumbersome.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
The optimization in https://github.com/falcosecurity/falco/pull/2210
had a bug when the engine uses multiple sources at the same
time--m_syscall_source is a pointer to an entry in the indexed vector
m_sources, but if add_source is called multiple times, the vector is
resized, which copies the structs but invalidates any pointer to the
vector entries.
So instead of caching m_syscall_source in add_source(), cache it in
process_events(). m_sources won't change once processing events starts.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
`open_params` is read from the falco YAML configuration file and parsed using Go's URL.
For example:
c349be6e84/plugins/k8saudit/pkg/k8saudit/source.go (L41-L42)
Go's URL parser does not handle whitespace, so if a user defines the `open_params` in the falco configuration file as follows
```yaml
open_params: >
/file/path
```
the parser returns an error. To avoid this, we now trim this parameter so no whitespace will be left for Go's URL parser to error out on.
For reference see #2262.
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Adapted API to sinsp::open API break, and simple consumer API break.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Co-authored-by: Andrea Terzolo <andrea.terzolo@polito.it>
Instead of using a falco_rule struct on the stack, use a single value
inside the falco_source struct. It's mutable as find_source returns a
const struct.
At very high event volumes (> 1M syscalls/second), even the tiny time
it takes to create/destroy the struct starts to add up, and this
switch has some small cpu savings.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
When doing some testing of falco on very high event volumes (> 1.5M
events/second), I found that the time taken to look up a falco_source
struct had a non-negligible contribution to cpu usage.
So instead of looking up the source from the source_idx every time,
separately save the source for syscalls in the falco_engine object
directly. The separately saved copy is only used once someone calls
add_source with source="syscall".
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now, the action takes care of inizializing all app inspectors
(just one in capture mode, one for each evt source in live mode), and of
registering and initializing all loaded plugins in the right inspector as needed.
The plugin initialization logic, which also involves the filtercheck list
population and checks, was moved and refactored from the previous
implementation of the load_plugins action.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Now, the action is in charge of loading all plugins and initializing:
- the offline inspector
- the list of loaded event sources
- the list of loaded plugins and their config
After this action runs, plugins are loaded but not yet initialized.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
If there is a file in `/etc/VERSION` the script assumes that it is running in a minikube
enviroment. Furthermore the logic in building the `KERNEL_VERSION` has been updated
according to how the kernel and probe modules are saved in the S3 bucket. The kernel version
for the minikube kernels includes also the minikube version.
Signed-off-by: Aldo Lacuku <aldo@lacuku.eu>
When running falco-driver-loader in local we need to set some config variables to meaningful values.
Those variables usually are set by the CI during the image/packages builds. The changes introduced
by this commit allow to set them at start-up time using env variables
Signed-off-by: Aldo Lacuku <aldo@lacuku.eu>
Update a few tests related to rules loading to use new names for
items (e.g. "rules content" for top level errors instead of "file")
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Use an enum instead of a string for the item_type aka "parts of a
rules file" field of contexts.
The set of values is mostly defined by the contexts that were already
created. There are a couple of forward-looking values for rule
outputs/macro conditions/etc. that may be useful for later.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
In #2098 and #2158, we reworked how rules loading errors/warnings were
returned to provide a richer set of information, including
locations/context for the errors/warnings.
That did *not* include locations within condition expressions,
though. When parsing a condition expression resulted in a
warning/error, the location simply pointed to the condition property
of the rule.
This commit improves this to handle parse errors:
- When libsinsp::filter::parser::parse() throws an exception, use
get_pos() to get the position within the condition string.
- Add a new context() constructor that takes a filter pos_info instead
of a YAML::Mark.
Now that positions aren't always related to the location of yaml
nodes, Make up a generic "position" struct for locations and convert
YAML::Mark and parser positions to a position struct.
Also allow a context to contain an alternate content string which is
used to build the snippet. For contexts related to condition strings,
the content is the condition.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This new model uses an async worker and a concurrent queue to handle
stats writing. This ensures better performance, because the live event
processing loop will just need to do a push on the queue instead of writing
to a file (only when the timer triggers), and should be thread-safe by design.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
In outputs it could be confusing to see a line:
<filename>: Ok
followed by a set of warnings.
To differentiate this, add a top level status "Ok, with warnings" when
rule loading was successful but had warnings.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The methods that throw exceptions from stringified results need to
additionally pass a rules_contents_t struct. This also meant that they
need to call the filename + content version of load_rules.
To avoid some duplicate code between the two load_rules_file methods,
move the work of opening the file into a private method
read_file(). It can throw an exception, which is passed through for
the void return method and caught + converted into a load_result error
for the method that returns a load_result.
Also, to avoid duplicate code between the void load_rules and
load_rules_file methods, add a private method interpret_load_result()
which throws an exception if the result has an error and prints
warnings otherwise if verbose is true.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Application changes to support multiple files when stringifying rules
results:
- In both validate_rules_files and load_rules_files, instead of
loading each file individually and then calling load_rules(), add a
separate step that loads all the files at once. The actual rules
content strings are held in a vector. The map from filename to
content (reference) points to entries in that vector.
- Both actions do the same work for this step, so put the
implementation in a shared application template method read_files
that works on iterators. It uses itertors because the load filenames
are a list and the validate filenames are a vector.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The old version of rules_result assumed that all errors/warnings were
related to a single file. That was generally correct for errors, as
rules parsing always stopped at the first error, so there is only one
relevant file.
However, for warnings that was not the case. When reading multiple
files A and B, you might get a warning from file A *only* after
reading file B. For example, B might redefine a rule in such a way
that you could get unused list/macro warnings from file A.
To properly address this, make some changes to how contexts are
managed:
- Instead of creating snippets at the time the error/warning was
generated, create snippets at the time the error/warning is
converted into a string. This requires passing all rules contents to
as_string()/as_json(), so define a
falco::load_result::rules_contents_t map from filename to rules
content (reference) and pass it in as_string/as_json(). Snippets are
now generated from the rules content matching the filename in the
context.
- When creating warnings/errors, there's no need to pass along the
rules content. This is only used when converting an error into a
string/json.
Also change snippet() to handle potentially very long lines. Instead
of always printing the entire line matching a location, print up to
snippet_width(param, with default 160 chars)/2 characters surrounding
the column from the location.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The latest released falco always prints full details on errors when
used with -r (read rules)/-V (validate rules). However #2098 changed
this to only print full details when verbose is true.
Fix the regression by always printing errors when loading
rules. Warnings will be printed only with -v.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update tests that validated rules files (typically looking for
errors/warnings) to use the new result struct + json based validation:
- When validating rules files, always use json output.
- In test cases, instead of parsing stderr/stdout, use new test
properties "validate_ok", "validate_errors",
"validate_warnings". These parse the json output and look for
specific tuples of (error code, error message, item type, item name)
in the output.
- There were a few tests that were actually validation tests but using
the -r argument to load rules. Convert them to validation tests. In
one case, split the test into two separate tests--one for
validation, one ensuring that the rule doesn't match anything.
- There were a couple of tests that were duplicates of existing
validation tests, just checking for the error in a different
way. Remove them.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the load_rules_files and validate_rules_files actions to use
the new falco_engine methods that return a rules result struct. The
app action interface is the same, returning ::fatal on error,
ok()/exit() otherwise. The difference is how any warnings/errors are
obtained--from the struct instead of an exception.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Changes to the rule loader to support result objects:
- Instead of throwing falco_exception on internal error, throw a
rule_load_exception instead, which contains distinct
error/message/context information.
- A context object contains a chain of location structs chaining from
the document root to the object where the error occurred. Each
location has a file position (as a YAML::Mark), an item
type (e.g. "rule", "list", "exception"), and an item name (e.g. "Write
Below Etc"). This will allow showing the exact location of an
error (e.g. list item/exception field) while also remembering the item
that contained it.
- All the _info structs now contain a context so errors that occur
after yaml parsing can still point to the original location in the
yaml file.
- rule_loader::result is an implementation of the abstract class
defined in falco_load_result. The implementation keeps track of a
list of errors/warnigns that used to be in the configuration object,
- Clean up compile_ methods to just throw rule_load_exceptions or
return nothing, and ensure that all rule_load_exceptions are caught in
compile(). When caught, errors are added to the result object.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Modify rule reader to use a result struct for errors and warnings:
- Instead of throwing a falco_exception to pass back errors, use a
rule_load_exception, which contains distinct error codes, messages,
and a context that points to the location of the error.
- The static method context_yaml_get_context() has moved to a method
of the rule_loader context object + the result as_string() method.
- As objects are parsed, create relevant context objects as reading
drills down into the contents of a rule/list/exception. This will
enable for specific errors in, say, the middle of an exception/list
while remembering the object that contains it.
- Modify decode_val()/decode_seq() to always return errors as
exceptions. Previously, it was a mix of a bool return + some
exceptions.
- decode_val()/decode_seq() are now more consistent about returning
distinct errors for the property not existing, the property existing
but not being a scalar/sequence value, and not being convertable to
the templated value.
- Combine the two nearly identical decode_seq() functions into a
single one that uses a lambda to perform the final step of adding to
the vector/set.
- There are some item-specific decode_xxx functions for specific
item properties like tags, exceptions fields/comps/values, etc.
that call the above functions.
These changes simplify the calls to decode_seq()/decode_val() as they
don't need to add their own errors when returning false. Also some
calls weren't checking the return value.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that we have a result struct and set of warning codes, change the
filter_warning_resolver to use them. This involves populating a set of
warning codes instead of strings.
Also, the methods to format warnings into human-readable strings is
now in the falco_load_result static methods, so move the text there
and remove the methods here.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add new load_rules methods that return a result object instead of
throwing exceptions on error. The existing load_rules methods call the
new methods internally and continue to throw exceptions on
error/return individual values on success.
The result is returned as a unique_ptr so it can be populated while
loading rules (as a part of the configuration object) and then move()d
to the return value.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Define a falco_load_result abstract class for use in new load_rules
methods. It's abstract so the implementation details in
rule_loader/rule_reader can be hidden from someone who wants to use
the API to load rules and work with a result.
The class defines a set of error codes/warning codes and has static
methods to get a short and long description of each error/warning.
There are virtual methods to access the important parts of a result:
- successful or not
- a string representation of the result, suitable for display to
users. Takes a verbose argument. When verbose is true, the string is
multi-line and has full details, including locations, item names,
etc. When verbose is false, the string is single-line and just
returns error codes.
- a json representation of the result, suitable for automated
parsing/interpretation later.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This allows for native-speed build of docker images, reducing CI time spent in
docker buildx qemu cross build for arm64 up to 10x.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
The only use of it was to include in --support output, which is
redundant as the support output already includes the full contents of
each rules file.
Additionally, it wasn't even being updated after the switch from lua
rules loading to c++ rules
loading (https://github.com/falcosecurity/falco/pull/1966/ or
surrounding PRs).
This will simplify follow-on changes to add a real "result" to rules
loading methods, as there will be fewer API variants to support.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Moreover, updated readme using aarch64 instead of arm64 (same that is used in the badge), and adding "-x86_64" suffix to x86 packages.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Note that another bump is required before releasing Falco, since this commit uses alpha versions.
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
In some cases the rule is not triggered when a reverse shell is spawned.
That's because in the rule we are checking that the file descriptor passed
as argument to the dup functions is of type socket and its fd number is "0, 1, or 2"
and the event direction is "enter".
The following event does not trigger the rule: dup2(socket_fd, STDIN_FILENO);
But using the exit event the rule is triggered.
Signed-off-by: Aldo Lacuku <aldo@lacuku.eu>
The Flatcar kernel module build needs to relocate binaries in /host/lib/modules
to run with the host's ld and glibc. While testing with glibc 2.34 that we plan
to introduce soon, we found that the previous dynlinker symlink (ld-xxx.so)
doesn't exist any longer. Update the hostld variable to use a path that is
present in all versions of glibc.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Pre-built kernel modules/eBPF probes for Flatcar use the value of the OS
VERSION_ID field as KERNEL_RELEASE in the filename. A specific kernel release
version does not uniquely identify a Flatcar configuration, because Flatcar is
image-based instead of package-based. Here's a more specific example: the same
kernel version can be part of various Flatcar releases (across channels
alpha/beta/stable) with differences in configuration. This is why we use the
VERSION_ID value during offline builds with driverkit. Flatcar version numbers
are all higher than 1500.0.0, so there is no risk of collision with kernel
version numbers.
When locally building the kernel module on the system, we have access to the
correct kernel build directory at /lib/modules/$(uname -r)/build with the right
configuration and so for that branch, we need to reset KERNEL_RELEASE=$(uname -r).
See also the driverkit PR that introduces a builder for Flatcar:
https://github.com/falcosecurity/driverkit/pull/131
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Relocate necessary tools from the kernel module build system to run using host
dynlinker and libraries, so that compiling falco module on Flatcar works.
Since Flatcar v2983.0.0, Flatcar ships with glibc-2.33, but the
falco-driver-loader container is based on debian:buster and so has a much older
glibc. This prevents some necessary tools within /lib/modules/*/build from
working which causes the falco module to fail to compile using dkms.
To make the tools work, we need to relocate them so we add patchelf to the
falco and local dockerfiles. The relocation is based on the approach done by
the sysdig agent-kmodule build system, but I'm unable to find the source code
for it. The host linker and libs will be found at /host/usr/lib64, so we change
the interpreter and rpath on the tools. The relocation happens on a copy of the
tools which are then bind mounted at the right location. The result allows the
module build to work.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Newest Debian/Ubuntu versions look for this file instead of Release + Release.asc, hitting lots of 404 (and a retry on the old format).
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
They can be pushed with `docker buildx` for various architectures.
Moreover, updated falco-driver-loader to support multiple architectures.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Moreover, add support for aarch64 plugins, even if Falco 0.32 won't be distributed with official arm64 support.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
This updates the engine to comply and work properly with the newly-introduced
interface design.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The filter_ruleset interface its implementation evt_type_index_ruleset
have been modified as follows:
- Only keep track of ruleset ids and not names. The falco engine will take
care of mapping easy-to-remember ruleset names to ruleset ids.
To emphasize this, use ruleset_id everywhere and not ruleset.
Also, make it non-optional.
- Have explicit separate functions to enable/disable rules, instead of a single enable() method combined with a boolean flag.
This does *not* change the falco_engine interface, which has
similar methods, to avoid breaking API changes.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Co-authored-by: Mark Stemm <mark.stemm@gmail.com>
This also fixes a couple of bugs. With the current implementation, the multi-ruleset feature is broken with multiple sources.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This interface will allow us to use different ruleset implementations inside the same engine.
The goal is to define API boundaries that will allow swapping the current evttype-index
ruleset implementation more easily. Key benefits include: smaller component with less responsibilities,
easier substituibility, more testable design, opportunity to adopt different index strategies
depending on the ruleset implementation.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This means that when starting Falco passing to it a folder for its rules, it will properly manage
changes to any file inside the folders, plus any created/deleted file inside it.
Unified list of rules parsing, instead of having it done twice inside cmdline_options and configuration.
Instead, it is done only once, inside load_rules_files.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
When adding an implied "in" comparison to an exception using the
single value form, add it to item, not items.
This fixes#1984.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This include making a coherent use of const, remove private inheritance, and adding virtual destructors.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This involves moving the code in falco_init() into individual files
below app_actions/. falco_init() simply calls app.run() now. When
app.run() returns false, print any erorr. When app.run() sets restart
to true, falco_init() is called again.
app.run() is still inside a catch block to catch any uncaught
exception.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Each file below app_actions/ defines some of the methods declared in
falco::app::application.
Any state that needs to be shared betweeen methods, or between the run
and teardown methods, resides in falco::app::application::state(), so
the moved code stays pretty much as-is, other than replacing stack
variables with member variables in app_state.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Changes to the falco::app::application object to support actions:
- All of the code that was in falco_init is now in methods of
application. (A later commit actually moves the code from falco_init
and into the split-up methods, this commit just declares them).
- Methods return an application::run_result object, which is a tuple
of success/errstr/proceed. proceed=false is used to short circuit
calling methods (think --help, --list, --support, etc.)
- application now has a run() method which runs the methods in an order
that honors any implicit dependencies (e.g. you have to init an
inspector before you open it, you have to do almost everything else
before processing events, etc.)
- There are a few methods devoted to teardown, they are always called
after the group of run methods are called.
- State that needs to be saved between methods, or saved between the
run and teardown functions, is in a
falco::app::application::state object.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Some objects used by falco (falco outputs, falco_formats, etc) were
using raw pointer references, which isn't great.
So convert use of raw pointers (originally passed from falco_init or
functions it called) with shared_ptr, as they are now held in
application state.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
With the split of falco_init into application:: methods, including a
notion of a success/error/proceed result, there is a
more concrete line between runtime errors (e.g. things that throw
exceptions) and errors returned by methods that run the application.
Some of the plugins tests were expecting errors to be returned by
exceptions (e.g. with a leading "Runtime error: and a trailing "
Exiting.").
Now, the errors are just returned directly in application::run(), so
drop the leading and trailing bits in expected test outputs.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The first warnings we support involve the unsafe comparisons with <NA>, which were present
in the legacy regression tests for PSPs.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
At the same time, this also simplifies the unit test cases by using the SCENARIO construct of catch2,
which allows sharing a setup phases between different unit tests, and removes a bunch of repeated LOC in our case.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Once all rule files have been loaded, and all the rules have been compiled into filters and inserted in the engine rulesets, the loader definitions are maintained in memory without really being used. This commit adds a convenience method to clear the loader state and free-up some memory when engine consumers do not require such information in memory anymore.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The rule_reader class is responsible of parsing the YAML ruleset text and of using the rule_loader
to store the new definition in the internal state. This is a first step towards separating the YAML
reading logic from the rule parsing one. Potentially, this will allow us to read rulesets from another
YAML library or from something different than YAML files too.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The rule_loader is now simply responsible of collecting list/macro/rule definitions and then compiling them as falco_rules. The ruleset file reading code will be moved to another class
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a porting of what we had inside the Lua codebase. This now handles the single responsibility
of gathering stats about rule-event matching, and of formatting them to print them to the user.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The function implementation was removed, however it was still defined in the .h header. Moreover,
this will now be required in order to replace its lua equivalent.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
falco_engine::process_event gets called for every inspector event.
Profiling showed that std::map::find takes about 10% of
falco_engine::process_event, and that can easily improved by accessing
the source by index.
Signed-off-by: Angelo Puglisi <angelopuglisi86@gmail.com>
Both the parser.lua and compiler.lua modules are not necessary anymore, because all the logic related
to filter parsing and compilation is handled inside libsinsp now. Accordingly, they have been removed from
the lua-to-cpp.sh scripts. README.md and parse-smoke.sh have been removed since they are not needed anymore:
lpeg is not used by the project, and the smoke tests are implemented in libsisnsp unit test suite.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The lua_filter_helper class is a simple Lua wrapper that can be used in the Lua rule loader to
parse/compile rule filters, and manipulate them to resolve/replace list and macro references.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The Lua parser grabbed from libs chisels is not used anymore, as the compilation logic happen inside the new
filter parser of libsinsp.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a first step towards porting the rule filter building logic that is currently implemented in Lua.
filter_macro_resolver uses the newly introduced AST constructs from libsinsp, and
allow manipulating filter ASTs to resolve/replace macro references. This is meant to be used
at boot time by the rule loader (which we still want to maintain implemented in Lua for now).
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The Lua PEG parser is not longer needed, since we now use the new filter parser implemented
in libsinsp.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is required due to the plugin API version being bumped to 1.0.0 in the framework after
the recent breaking changes. cloudtrail and json will be switched back to a stable build
once they get released to require the plugin API version with the newest major.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Add new test cases for a rule with an unknown source *and* an
exception, and a macro with an unknown source.
The first results in a rule warning (and no error), and the second
prints an error and skips.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Also skip macros with unknown sources. This matters primarily for
macros related to plugins that have a distinct event source.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
If a rule has an unknown source, *and* has exceptions, loading the
rule will result in an error and not skipping the rule. This is
because exceptions are also validated for unknown fields, and that
occurs before the current check for unknown sources.
The fix is to move the check for unknown sources as soon as the rules
object is read.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This adds a new macro `user_known_mount_in_privileged_containers` which
allows the easier user-defined exclusions for the "Mount Launched in
Privileged Container" rule.
This would be cleaner with the exclusions feature, but this feature
is not used in the default ruleset yet, if I understand correctly.
Signed-off-by: Matt Moyer <mmoyer@figma.com>
Update falco's main falco_init() to use a falco::app::application and
falco::app::cmdline_opts object instead of storing all its command
line state in stack variables.
The bulk of the removed code is in usage() (not needed as cxxopt's
help() is self-documenting.) and getopt_long() which is replaced by
app.init(argc, argv).
For the most part, this is simply replacing references to local
variables (e.g. "all_events") to the bound variable inside the
cmdline_opts object (e.g. app.copts().all_events).
There are a few cases where more complex logic was used (output
formats, initializing k8s/mesos with string pointers), and those
changes are still in falco_init().
For the most part, the monolithic parts of falco_init that involve
reading config files, creating the inspector, loading rules, etc are
still present. Those will be addressed in later changes.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
For the most part, replacing getopt() with cxxopts + falco application
had no effect on falco engine/config interfaces. However, there were a
few places where it was wasier to change the interface than add
middleware code that transformed from, for example, vectors to lists.
This commit has those changes.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Fill in an initial falco::app::cmdline_options class using cxxopts
library to hold options:
- falco::app::cmdline_options contains a cxxopts::Options object to
parse options and a cxxopts::ParseResult to hold the result.
- The only meaningful public method is parse() which parses argc/argv
and returns true/false + error.
- The parsed options are all public instance variables of the object
and generally use the same names of the corresponding variables in
the old falco_init(). These variables are all bound to the
corresponding command line option and are updated in parse().
- In a few cases, the command line option does not directly map to a
bound variable (e.g. -b to set buffer format, -p/-pk/-pc to set
extra formatting options, etc.) In these cases the option values are
read after parsing and update the public instance variable.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a notion of a falco application object. Eventually this will
replace the bulk of falco_init and contain methods to:
- Parse/validate command line options
- Parse/validate falco config
- Initialize prerequsites (inspector, falco engine, webserver, etc)
- Load plugins
- Load/validate rules
- Command/subcommand execution (e.g. --list/--list-fields, or
nothing specified to run "main" loop)
For now, it is only responsible for command line options handling,
which is stubbed out.
Currently, the only public methods are init() to initialize everything
and copts() to access command line options.
Command line options are held in a different class
falco::app::cmdline_opts. application::copts() returns a reference to
that object, which allows access to parsed command line options bound
to various public instance variables.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
We'll use this to better manage the fairly large set of command line
options in self-contained objects instead of a scattering of
individual stack variables.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Replace nearly-identical blocks of code that defined individual custom
targets/custom commands to copy files from source to build dir with
the copy_files_to_build_dir function.
This reduces the number of build targets and speeds up/cleans up the
make output.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Define a cmake function copy_files_to_build_dir(source_files,
targetsuffix) that defines a single custom target and single custom
command to copy the set of source files to
CMAKE_CURRENT_BINARY_DIR. All of the real work is done via cmake -E
copy_if_different.
This function will replace the nearly identical cmake code in several
other directories. This function has the advantage of being a single
target for the set of source files instead of a target per-file.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
I noticed that some external projects were being reconfigured/built
with every make, even though no files in the external project had been
updated.
With some debugging I noticed that git based external projects were
re-running their "update" step every time, and that in turn caused the
configure/build/install steps to re-run as well. (Generally the build
step is a no-op as the Makefile/etc. in the external project is well
formed and doesn't do anything, but the configure/install steps still
run).
It seems related to this cmake bug:
https://gitlab.kitware.com/cmake/cmake/-/issues/19703. In short, the
git update step for an external project does not create any "done"
file that denotes that the files are still up-to-date. Without that
"done" file, the update step is always run, and that in turn causes
the other steps for the external project to re-run as well.
The best way to fix this seems to be to skip the update step by
defining an empty UPDATE_COMMAND. As long as the downloaded code for a
given hash/tag/etc does not change, the update step is unnecessary.
And if we *really* wanted to ensure unchanged dependencies, we would
download our own copies anyway.
Making this change significantly cleans up the falco build to avoid
rebuilding git based external dependencies.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Adds `npm` to `package_mgmt_binaries` for detection of "living off the land" style attacks that utilize NPM pull down additional tooling
Signed-off-by: rileydakota <dakotariley2@gmail.com>
New rule to prevent hardlink bypass and symlink rule set to WARNING for consistency
Signed-off-by: Shay Berkovich <sberkovich@blackberry.com>
Co-authored-by: Meera Balsara <mbalsara@blackberry.com>
The `falco_hostnetwork_images` list is unused.
This PR removes it to avoid the warning.
```console
When reading rules content: 1 warnings:
list falco_hostnetwork_images not refered to by any rule/macro/list
```
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
When listing fields with -N (names only), also skip fields with the
EPF_TABLE_ONLY flag. (Skipping fields without -N is handled in libs,
in the as_string() method).
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Instead of having .lua files external to the program responsible for
loading rules, embed the contents of those files into the executable
and load them as strings instead of as files:
Add a cmake custom command below userspace/engine/lua that calls a
bash script lua-to-cpp.sh to generate falco_engine_lua_files.{cpp,hh}
that are compiled into the falco engine library.
The script creates a .cpp file that has const char * symbols for each
file, as well as lists of files that should be loaded when the falco
engine is loaded. There are actually two lists:
- lua_module_strings: these are loaded and also added to the lua
runtime package.preload table, so they are available when lua code
require()s them.
- lua_code_strings: these are loaded *and* evaluated, so the functions
in them are availble to be called from C++.
This simplifies some of the falco_common methods, as there's no need
to keep track of a "main" lua file to load or paths from which the lua
loader should find files for modules, and there's no need to keep
track of an "alternate" lua directory that occurs for debug builds.
Also, there's no need to include any .lua files in the installed
packages, as they're built into the falco binary.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
change LYAML_SRC to LYAML_ROOT, which points to the top source
directory now.
LYAML_LIB and (new) LYAML_LUA_DIR are based relative to that
directory.
There's no install step at all now--the static library and the .lua
files are now used directly from the source tree.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This will distinguish it from rule_loader.lua, which is *not* a module
but lua code with functions that can be called directly.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update json_event_filter_factory::get_fields() to add the new
info (shortdesc, data_type, tags) to field descriptions.
This allows for richer outputs when printing info on the fields.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
With the new implementation of list_fields(), the order of fields
changed slightly. So update the checksum.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Instead of having a falco-specific function to print field info, use
the built-in filter_fieldclass_info::as_string() instead. This is a
better implementation (displays addl info, has better wrapping, wider
output) and having a single implementation allows for consistent
outputs between falco and other potential programs that could use the libs.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Since now, the maximum depth supported to access config fields is two.
This adds support for accessing fields of arbitrary nesting depth.
A formal grammar has been explicited for the regular language representing
the field keys. The accessor methods have been updated accordingly.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a change of direction from the current design, that imposes loading
the configuration from file only, and in the object constructor. Instead,
yaml_configuration objects can now be reused ad can load the YAML config
from either file or string. This also makes it easier to unit test this class.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This allows defining rules that simply enable/disable already defined rules, like the following:
- rule: A rule enabled by default
enabled: false
- rule: A rule disabled by default
enabled: true
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Test infrastructure and sample confs/rules/traces for plugins
automated tests:
New test cases are in falco_tests_plugins.yaml and cover:
- Listing plugins and fields when plugins are loaded.
- Basic cloudtrail + json plugin on a fake cloudtrail json file and a
sample rule that uses both plugins.
- Conflicts between source/extractor plugins
- Incompatible plugin api
- Wrong plugin path
- Checking for warnings when reading rules with unnown sources (e.g. when plugins are not loaded)
Some test-only plugins written in C are in test/plugins and built on
the fly. (They aren't included in packages of course).
The test framework needed some small changes to handle these tests:
- Add a mode to not check detection counts at all (for --list/--list-plugins)
- addl_cmdline_opts to allow specifying --list/--list-plugins
- Using DOTALL when matching stderr/stdout (allows multi-line matches more easily)
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the falco binary to add support for plugins.
- Keep track of an "event source", which is initially "syscall" but
changes to the input plugin's source if an source plugin ends up being
loaded.
- New argument --list-plugins will return info on any loaded plugins,
using sinsp_plugin::plugin_infos.
- Create filter/formatter factories for plugins. This ensures that
filterchecks for syscalls are not used for plugins and vice versa.
- Use sinsp_plugin::register_plugin() to load each plugin found in
config. The first source plugin found (if any) calls
engine->add_source withthe source plugin's event source.
- If a second source plugin is found, exit with an error.
- Extractor plugins must be compatible with the event source (usually
the plugin event source, but could be "syscall"). If not, exit with
an error.
- Multiple Extractor plugins are allowed, but they can not have
overlapping compatible event sources. This is mostly to avoid
confusion, but we might change this later.
- After loading plugins, use engine is_plugin_compatible to ensure
that the plugin is compatible with any required_plugin_version blocks
in falco rules.
- Normally falco would log warnings if too many SCAP_TIMEOUT results
were received. These are more expected when using plugins, so only
log these warnings when using syscalls.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The generic events support already handled most of this, with a
dedicated formatter factory for plugin sources. Just one missing
header include and change the logic slightly for json parsing.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Rules loading changes for plugins:
- parse required_engine_versions from yaml and pass up to rules
loader as a lua table as an additional return value from load_rules().
- c++ rules loader converts to map: plugin -> list of required plugin
versions
- support is_source_valid callback from lua, calls engine method. If
a source is not valid, skip any rules for that source and add a warning.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Mostly plugins are just handled as a new filter/formatter factory with
a new source based on the loaded input plugin, but there are a few
changes at the engine level:
- is_source_valid returns whether a filter/formatter factory exists
for a given source. Will be used by rules loaded to skip rules for
an unknown source.
- the falco engine now holds the required_plugin_version predicates
found in rules files and a method is_plugin_compatible returns whether
a plugin semver is compatible with the predicates in the rules
- Update the falco engine version and fields checksum for plugins
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
These rules can be used when combined with the cloudtrail plugin.
They're installed to /etc/falco like the other rules files.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a cmake module "plugins" that does the following:
- Downloads/installs the plugins artifacts from a known tag
- Copies the resulting cloudtrail/json shared libraries to
CMAKE_CURRENT_BINARY_DIR/plugins
- Installs them to FALCO_SHARE_DIR/plugins
The default config will define the plugins but they will be disabled
by default.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
It took a while, but we remembered to finish moving the token_bucket
from falco engine to libs. There were 2 copies for a while.
This brings over one change to libs--to have an optional timer
function.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update config code/default falco.yaml to add support for plugins:
- Update config parsing methods to support reading plugin config
objects in a list from yaml.
- The default config defines the cloudtrail/json plugins but does not
give them any actual config for init config/open
params (cloudtrail), or init config (json).
- load_plugins is empty so neither plugin is actually loaded by default.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>-
When FALCO_VERSION was provided via a CMake variable, the build would
eventually fail because the partial version variables hadn't been
populated. Move the creation of those outside the check of FALCO_VERSION
being set so they also happen when that is provided too.
Contributes-to: #1654
Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
The trace file traces-positive/run-shell-untrusted.scap has an old
execve event number (PPME_SYSCALL_EXECVE_18), which was replaced by
PPME_SYSCALL_EXECVE_19 in 2018.
Given the changes in https://github.com/falcosecurity/libs/pull/94,
these events are now skipped. So change the test to note that *no*
events will be detected.
As a bit of context, event numbers won't be changing any longer--a
change around the same time 298fbde8029020ce3fbddd07e2910b59cc402b8b
allowed for extending existing events to add new parameters instead of
having to define a new event number just to add a new parameter. So
the notion of "old events" should not exist for any event created
after mid-to-late 2018.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
It turns out that the macro inbound_outbound had a logical bug where
joining the beginning and end of the macro with "or" led to the macro
matching all event types by accident.
Most of the time this isn't harmful but it turns out some trace files
will do operations on inet connection fds like "dup", and those get
mistakenly picked up by this macro, as the fd for the event does
happen to be a network connection fd.
This fixes the macro to only match those event types *and* when the fd
is a inet connection fd.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
With the changes in https://github.com/falcosecurity/libs/pull/74,
there isn't any need to warn about the order of operators and the
evt.type field--the set of event types for a filter should be exact
now regardless of the order of operators.
So update tests that were logging those warnings to note that the
warnings won't occur any more.
Also, some tests more accurately *do* note that they have an overly
permissive evttype (e.g. ones related to syscalls, which are uncommon
and are evaluated for all event types) to reflect the new behavior.
Finally, in unit tests create an actual sinsp filter instead of a
gen_event_filter, which is the base class and shouldn't be created
directly.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This makes the output of --list a bit more precise to only include
filter fields and not output fields.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This step used to be done in the lua rule loading code, but now we can
get it directly from the filters, so do it in falco instead.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the lua side of rule loading to reflect other changes:
- install_filter renamed to create_filter_obj, and takes just a
lua_parser object created via falco_rules.create_lua_parser() and
uses a single lua callback "filter" instead of separate ones for
syscall/k8s_audit. It can return an error, including about
undefined fields
- is_defined_filter, which used to be local and based on the result of
sinsp_rule_utils.check_for_ignored_syscalls_events, is now a
lua_callback falco_rules.is_defined_field().
- Don't need to pass down sinsp_lua_parser/json_lua_parser now,
creating filters is handled via lua callbacks.
- Checking for ignored syscalls/events is now done in falco itself,
after loading rules.
- add_xxx_filter replaced by add_filter + source.
- Use is_format_valid instead of formats.formatter/formats.free_formatter.
- We don't need the functions in sinsp_rule_utils any longer, so
remove the file and don't import it.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a function is_defined_field(source, fldname) that returns whether
a field with name fldname exists for the given event source. This uses
the filter factory to create a filtercheck, and returns true if an
object was created.
This prevents having to push down the entire set of defined fields
before calling load_rules().
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Use the new falco engine interface with support for generic events
instead of event-specific process_xxx_event methods.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update rules loader to be more general purpose by using factories and
the general purpose engine:
- A lua callback create_lua_parser creates a lua_parser with a filter
object of the right type. The lua parser can then iterate the AST
and populate the filter object.
- Like the falco engine, the rules loader is configured with a list of
factories, and add_filter is now general purpose, taking a source.
Given the fix in https://github.com/falcosecurity/libs/pull/72, there
isn't any need to pass down the entire set of sinsp event
types/syscalls and validate that all filter event types are
valid. That job is now handled by the sinsp filter parsing
code. add_filter now returns the number of event types used by the new
filter, and if that number is excessive the lua code will return a
warning.
Format handling is mostly not handled by the rules loader any more. As
a convienence, there's a new lua callback is_format_valid which takes
a source and output string and uses the right formatter factory to
create a formatter. As long as that doesn't throw an exception, the
format is valid.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Instead of having hard-coded support for syscall/k8s_audit events, use
the notions of filter factories/formatter factories to provide generic
support for events having a given source:
- Within the engine, maps m_filter_factories / m_rulesets /
m_format_factories map from a given source to something that can
create filters, hold filters, and create formatters for a given
source. The hard-coded sinsp_factory/json_factory objects are removed.
- The specific add_xxx_filter/process_xxx_event are general purpose
and take an event source.
- A new method create_formatter() takes a source/output format and
provides a shared_ptr to a formatter than can resolve format
strings. This is used by the falco outputs code.
- In falco main, create the syscall/k8s_audit filter and formatter
factories and pass them to the engine. Later, we might make this
configurable/selective.
With all of the above changes, the falco engine doesn't need a direct
inspector any longer, so remove it.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Make json_event_formatter a generic event formatter by inheriting from
gen_event_formatter and implementing its methods.
Most of the actual work is still done by resolve_format (previously
resolve_tokens, to avoid confusion with sinsp formatter, as it behaves
slightly differently).
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Take advantage of the changes in
https://github.com/falcosecurity/libs/pull/75 to have a
general-purpose way to list fields for a given event source.
in the engine, list_fields() now takes a source, iterates over filter
factories, and calls get_fields() for each factory, printing the results.
list_source_fields now calls the engine regardless of source.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Modify falco_formats to only be responsible for resolving a rule's
output string or coming up with a map of field name->field values from
a given output string.
It relies on the changes in
https://github.com/falcosecurity/libs/pull/77 to use generic
formatters for a given source.
Remove lua bindings to create a formatter/free a formatter. Those were
unused as of the changes in
https://github.com/falcosecurity/falco/pull/1451, so finally remove
them now.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Move the code that splits a json object into a list of k8s audit/json
events out of falco engine and into json_evt.
This, along with other changes, allows the falco engine to be more
general purpose and not directly tied to the notion of syscall vs k8s
audit events.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Modify rulesets to not keep track of the event types for a given set
filter. Instead, using the changes in
https://github.com/falcosecurity/libs/pull/74 event types are returned
directly by the filter.
Within each ruleset, there's a vector that maps from event number to
set of filters that are related to that event number. There's also a
general set of filters for all event types.
run() both indexes into the per-event vector as well as iterate over
the all event types set.
Also, used shared_ptr instead of direct pointers, which matches the
updated interface used by lua_parser. This simplifies the bookkeeping
a bit (no more delete when removing rulesets).
Given these changes, there's no need for a separate
falco_sinsp_ruleset class any longer, so remove it.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Description of changes to falcosecurity/libs and /falco to support
plugins to provide events and extract fields from events.
Signed-off-by: Loris Degioanni <loris@sysdig.com>
Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com>
Co-authored-by: Mark Stemm <mark.stemm@gmail.com>
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
The option was added but could not work since MINIMAL_BUILD is not declared in this scope (also not currently needed).
Furthermore, it never took effect since the builder image was never built and pushed. For the same reason, we have not noticed it until now.
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
added list from the survey that allowed mentions of their name publically in the adopters file.
@jonahjon -- please approve or change any verbiage to adding AWS as contributors with all you and the teams work on PROW and all the contributions thus far.
Signed-off-by: Dan Papandrea <dan.papandrea@sysdig.com>
Also, print out the time of the last processed event in the output
fields of the notification.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
Falco uses a shared buffer between the kernel and userspace to receive
the events (eg., system call information) in userspace.
Anyways, the underlying libraries can also timeout for various reasons.
For example, there could have been issues while reading an event.
Or the particular event needs to be skipped.
Normally, it's very unlikely that Falco does not receive events consecutively.
Falco is able to detect such uncommon situation.
Here you can configure the maximum number of consecutive timeouts without an event
after which you want Falco to alert.
By default this value is set to 1000 consecutive timeouts without an event at all.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
The rationale is that in case Falco obtains a consistent number of
consecutive timeouts (in a row) without a valid event, something is
going wrong.
This because, normally, the libs send timeouts to Falco (also) to signal events to discard.
In such cases, which are the majority of cases, `ev` exists and is not
`null`.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
This fix has two major points in it:
- when `std::stoll` is used in parse_as_int64 handle all the exceptions it
can throw (https://en.cppreference.com/w/cpp/string/basic_string/stol)
- when `process_k8s_audit_event` an eventual exception in it does not
stop the webserver process. This is done by doing a catch all handle
outside it and by logging an error message to the caller as well as in
stderr
Signed-off-by: Lorenzo Fontana <lo@linux.com>
In some cases, it might be useful to know what falco engine version a
given falco binary supports. We already have a --support option that
prints info about the system, config, rules files, etc.
Add a engine_info object, with an engine_version property containing the
falco engine version in falco_engine_version.h. In the output, it looks
like this:
...
"engine_info": {
"engine_version": 8
},
...
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
While prepping Falco 0.28.0 I noticed the correct date (YYYY-MM-DD) to provide to GitHub filters is not the day before the previous release. Instead, it's the day the latest release happened.
Also added clearer instructions on how to generate the CHANGELOG.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
We want users to continue using rules without having to use exceptions.
Exceptions are an additional feature for more advanced use-cases, having
a warning in there will mean that everyone now adds an empty exception
to avoid the warning.
Co-Authored-By: Leonardo Grasso <me@leonardograsso.com>
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Exceptions have been introduced in commit 64a231b962
The feature itself is very useful for more complex environments where
the simple conditions are difficult to handle.
However, many users reported that they find them difficult to understand so
we are doing a rollback of them in the default ruleset in favor of the
syntax without exceptions.
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Add a test that verifies that a pod where one container has no security
context and the second container has a security context + privileged
properly matches the Create Privileged Pod falco rule.
There's a very similar test case already in
trace_files/k8s_audit/create_nginx_pod_privileged_2nd_container.json,
but in that case both containers have a securityContext property.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
While testing, I found a case when creating a pod where:
1) the first container had no securityContext value
2) the second container had a security context with privileged=true
and this did not match the default rule Create Privileged Pod, when it
should match.
The rule Create Privileged Pod uses the field
ka.req.pod.containers.privileged, which in turn uses
json_event_filter_check::def_extract(). def_extract() iterates
over a set of json_pointers, potentially expanding arrays as they are
returned. Many k8s audit fields use this extract function.
For ka.req.pod.containers.privileged, the first json_pointer is
/requestObject/spec/containers to find the list of containers, and the
second is /securityContext/privileged to extract the privileged property
out of the securityContext object. What's returned is an array of
true/false noting if each container is privileged.
The problem is that def_extract() aborts when iterating over arrays if
extracting a pointer from an array can't be done.
In this case, the first pointer extracts the array of containers, and
then when iterating over the array of containers, the security context
pointer doesn't extract, causing the whole filter field to abort and
return ::no_value.
The fix is to not abort when iterating over arrays, but use ::no_value
for that array item's value instead. This allows def_extract() to
extract the privileged value out of the second container.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This is needed in systems where raw tracepoints are not available.
Anyways, since this is needed when the inspector open (and actually
loads) the eBPF probe, ideally the mount should not be done by this
script but rather from Falco, or from Falco libs.
Otherwise, users building the eBPF probe theirseleves and not using this script (and having a kernel without raw
tracepoints) may need to mount this fs theirselves.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
When we started to implemented 20200506-artifacts-scope-part-2 proposal
, among a million other things, we renamed `SKIP_MODULE_LOAD` to
`SKIP_DRIVER_LOADER`. We reatained compatibility with `SKIP_MODULE_LOAD`
for a bunch of releases.
Now, after 9 months have passed I think it's time to completely
deprecate it.
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
Falco won't properly parse a rule like this:
---
- rule: Some Rule
desc: Some Desc
condition: evt.type=execve and container.image.repository = 271931939120.dkr
output: Some output
priority: INFO
---
This is the error when validating the rules:
Tue Mar 30 12:00:40 2021: Validating rules file(s):
Tue Mar 30 12:00:40 2021: /home/mstemm/test.yaml
1 errors:
Compilation error when compiling "evt.type=execve and container.image.repository = 271931939120.dkr": 63: syntax error, unexpected 'dkr', expecting 'or', 'and'
The parsing of the string on the right hand side stops at the period
before the dkr. The dkr then doesn't match the grammar, resulting in the
error.
Looking at the parser implementation more closely, the problem is in the
definition of "Number":
---
- Number = C(V "Hex" + V "Float" + V "Int") / function(n)
return tonumber(n)
end,
---
Note that it stops after the number, but does not have any requirement
about what follows.
This changes the definition of number to require that what follows the
number is not an identifier character. With this change, values that are
only numbers are parsed as numbers, and values that start with numbers
don't match the Number definition and are parsed as BareStrings instead.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The `publish/packages-deb-dev` failed because it wrongly required `tests/integration-static` (that has `/build-static` and `/source-static`) instead of `tests/integration` which comes with both `/build` and `/source`.
Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com>
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
Whitelist ibm images for connecting to k8s api server
IBM Observability by Sysdig has a vendored sysdig/agent image.
IBM's Kubernetes Service ships with an operator manager. Example:
19:12:45.090908160: Notice Unexpected connection to K8s API Server from
container (command=catalog -namespace ibm-system
-configmapServerImage=registry.ng.bluemix.net/armada-master/configmap-operator-registry:v1.6.1
k8s.ns=ibm-system k8s.pod=catalog-operator-6495d76869-ncl2z
container=4ad7a04fa1e0
image=registry.ng.bluemix.net/armada-master/olm:0.14.1-IKS-1
connection=172.30.108.219:48200->172.21.0.1:443) k8s.ns=ibm-system
k8s.pod=catalog-operator-6495d76869-ncl2z container=4ad7a04fa1e0
IBM's Kubernetes service also ships with a metrics collecting agent
Signed-off-by: Spencer Krum <nibz@spencerkrum.com>
eks:node-manager is an Amazon EKS internal service role that performs specific operations for managed node groups and Fargate.
Reference: https://github.com/awsdocs/amazon-eks-user-guide/blob/master/doc_source/logging-monitoring.md
Related falco log
```
{"output":"10:56:31.181308928: Warning K8s Operation performed by user not in allowed list of users
(user=eks:node-manager target=aws-auth/configmaps verb=get uri=/api/v1/namespaces/kube-system/configmaps/aws-auth?timeout=19s resp=200)","priority":"Warning","rule":"Disallowed K8s User","time":"2021-01-26T10:56:31.181308928Z", "output_fields":
{"jevt.time":"10:56:31.181308928","ka.response.code":"200","ka.target.name":"aws-auth","ka.target.resource":"configmaps","ka.uri":"/api/v1/namespaces/kube-system/configmaps/aws-auth?timeout=19s","ka.user.name":"eks:node-manager","ka.verb":"get"}}
```
Signed-off-by: ismailyenigul <ismailyenigul@gmail.com>
Donate:
- libsinsp
- libscap
- the kernel module driver
- the eBPF driver sources
by moving them to the Falco project.
Co-authored-by: Lorenzo Fontana <lo@linux.com>
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
When returning a rule_result struct, also include a set of field names
used by all exceptions for this rule. This may make building exception
values a bit easier.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
If a list:
- list: foo
items: [a, b, c]
Was referenced in another list:
- list: bar
items: [foo, d, e, f]
The first list would not be marked as used, when it should.
This avoids mistaken messages like "list xxx not refered to by any rule/macro/list"
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Take advantage of the changes to support exceptions and refactor rules
to use them whenever feasible:
- Define exceptions for every rule. In cases where no practical
exception exists e.g. "K8s <obj> Created/Deleted", define an empty
exception property just to avoid warnings when loading rules.
- Go through all rules and convert macros-used-as-exceptions that
matched against 2-3 filter fields into exceptions. In most cases,
switching from equality (e.g proc.name=nginx) to in (e.g. proc.name
in (nginx)) allowed for better groupings into a smaller set of
exception items.
- In cases where the exception had complex combinations of fields, keep
the macro as is.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
When running falco with -V/valdiate <rules file>, you won't get any
event counts. All prior tests didn't get this far as they also resulted
in rules parsing errors.
However, validating can now result in warnings only. This won't exit but
won't print event counts either.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Handle various positive and negative cases. Should handle every error
and warning path when reading exceptions objects or rule exception
fields, and various positive cases of using exceptions to prevent
alerts.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Support exceptions properties on rules as described in
https://github.com/falcosecurity/falco/pull/1376.
- When parsing rules, add an empty exceptions table if not specified.
- If exceptions are specified, they must contain names and lists of
fields, and optionally can contain lists of comps and lists of lists of
values.
- If comps are not specified, = is used.
- If a rule has exceptions and append:true, add values to the original rule's
exception values with the matching name.
- It's a warning but not an error to have exception values with a name
not matching any fields.
After loading all rules, build the exception condition string based on
any exceptions:
- If an exception has a single value for the "fields" property, values are
combined into a single set to build a condition string like "field
cmp (val1, val2, ...)".
- Otherwise, iterate through each rule's exception
values, finding the matching field names (field1, field2, ...) and
comp operators (cmp1, cmp2, ...), then
iterating over the list of field values (val1a, val1b, ...), (val2a,
val2b, ...), building up a string of the form:
and not ((field1 cmp1 val1a and field2 cmp2 val1b and ...) or
(field1 cmp1 val2a and field2 cmp2 val2b and ...)...
)"
- If a value is not already quoted and contains a space, quote it in the
string.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The format of error responses has changed to include a summary of errors
and/or warnings. This changed many test cases that were looking for
specific outputs.
Update to add counts and other minor formatting changes.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
When parsing a rules file, if a top level object is not one of the known
types rule, macro, list, required_engine_version, instead of failing
parsing, add a warning instead.
This adds some forwards-compatibility to rules files.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add the notion of warnings when loading rules, which are printed if
verbose is true:
- load_rules now returns a tuple (success, required engine version,
error array, warnings array) instead of (true, required engine
version) or (false, error string)
- build_error/build_error_with_context now returns an array instead of
string value.
- warnings are combined across calls to load_rules_doc
- Current warnings include:
- a rule that contains an unknown filter
- a macro not referred to by any rule
- a list not referred to by any rule/macro/list
Any errors/warnings are concatenated into the exception if success was
false. Any errors/warnings will be printed if verbose is true.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
If an exception item has a single value for fields, all the values are
combined together into a single set to build an expression field
cmp (val1, val2, ...)
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Instead of oveloading the exception item name as the key of the object,
just have a flat array of object with a name property.
A bit more verbose, but makes it easier to understand what the schema is.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
A rule exception can now have a comps property that allows fields to be
matched against items using an operator of =. If not defined, equality
is implied.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
- Clean up npm examples so they are valid.
- Small punctuation changes.
- Emphasize that the strings related to field values are arbitrary.
- Emphasize that exceptions only use equality matching.
- Emphasize that you'll need to upgrade falco to use these new features.
- Capitalize Falco everywhere.
- Change language related to backwards compatibility.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This proposes adding exceptions as a first class object to falco rules
files.
It adds a new key "exceptions" to rule objects that allows a rule
writer to define tuples of field names that comprise an exception, and a
new top level object "exception" that contains lists of tuples of field
values that define exceptions to rules.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Fixes#1269
Add two new fields in the version service for falco's engine version and
the checksum of all of the fields it understands.
This will require rebuilding/re-releasing all the clients.
Signed-off-by: Spencer Krum <nibz@spencerkrum.com>
Have some cmake variables (e.g. *_INCLUDE and *_LIB) marked as advanced,
in order to have a cleaner ccmake menu.
Signed-off-by: Angelo Puglisi <angelopuglisi86@gmail.com>
Fix broken outputs.proto link, previously pointing to nonexistent
branch, making it point to master branch.
Signed-off-by: deepskyblue86 <angelopuglisi86@gmail.com>
Please note that the `HOME` env has been added for consistency purposes with the main docker image.
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
The BUILD_BYPRODUCTS for the civetweb target
is needed so that when Falco is built using Ninja
the falco target can have a reference to
understand what target is building the civetweb lib
and do the build automatically without having to do
`ninja civetweb` first.
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Attempting to start falco on a host that had a similarly named module
(e.g., "falcon") would cause the falco-driver-loader to loop attempting
to rmmod falco when falco was not loaded.
falco-driver-loader will now inspect only the first column of lsmod
output and require the whole search string to match
Fixes#1468
Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
Besides all the other improvements, we are really interested
in getting the Make options for other ISAs than x86_64 when it
comes to compiling abseil [0].
This is what happens on aarch64
```
make[4]: *** [Makefile:2968: /root/falco/build-musl/grpc-prefix/src/grpc/objs/opt/third_party/abseil-cpp/absl/base/internal/thread_identity.o] Error 1
c++: error: unrecognized command line option '-maes'
c++: error: unrecognized command line option '-msse4'
c++: error: unrecognized command line option '-msse4'
c++: error: unrecognized command line option '-maes'
```
[0] bf87ec9e44
Signed-off-by: Lorenzo Fontana <fontanalorenz@gmail.com>
This is needed because Luajit does not support many architectures
such as aarch64 and ppcle64.
Note: some operating systems, such as Alpine, already use moonjit as a dropin
replacement for luajit.
Signed-off-by: Lorenzo Fontana <fontanalorenz@gmail.com>
`multipath`, which is run by `systemd-udevd`, writes to
`/etc/multipath/wwids`, `/etc/multipath/bindings` and a few other paths
under `/etc/multipath` as part of its normal operation.
Signed-off-by: Nicolas Marier <nmarier@coveo.com>
Use the right list name in the rule Full K8s Administrative Access--it
was using the nonexistent list admin_k8s_users, so it was just using the
string "admin_k8s_users".
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Previously, formatters were freed by LUA code when re-opening outputs.
Since now, outputs are not controlling anymore the falco_formats class (see #1412), we just free formatters only if were already initialized.
That is needed when the engine restarts (see #1446).
By doing so, we also ensure that correct inspector instance is set to the formatter cache.
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
Like other rules that rely on a process name for exceptions, don't
trigger an event if the process name is missing e.g. "<NA>".
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Also ignore docker programs which would prevent cases where the path is
expressed within the container filesystem (/.bash_history) vs host
filesystem (/var/lib/docker/overlay/.../.bash_history).
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This will be used by the static build to load lua files from
alternate directories that are not tied to the compile flags
Signed-off-by: Radu Andries <radu.andries@sysdig.com>
It turns out if you read this rules file with falco versions 0.24.0 and
earlier, it can't parse the bare string containing colons:
(Ignore the misleading error context, that's a different problem):
```
Thu Sep 10 10:31:23 2020: Falco initialized with configuration file
/etc/falco/falco.yaml
Thu Sep 10 10:31:23 2020: Loading rules from file
/tmp/k8s_audit_rules.yaml:
Thu Sep 10 10:31:23 2020: Runtime error: found unexpected ':'
---
source: k8s_audit
tags: [k8s]
# In a local/user rules file, you could override this macro to
```
I think the change in 0.25.0 to use a bundled libyaml fixed the problem,
as it also upgraded libyaml to a version that fixed
https://github.com/yaml/libyaml/pull/104.
Work around the problem with earlier falco releases by quoting the colon.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
In some cases, when removing a container, dockerd will itself remove the
entire overlay filesystem, including a shell history file:
---
Shell history had been deleted or renamed (user=root type=unlinkat
command=dockerd -H fd://
... name=/var/lib/docker/overlay2/.../root/.bash_history ..
---
To avoid these FPs, skip paths starting with /var/lib/docker.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The falco-driver-loader script calls dkms to compile the kernel
module using the default gcc.
In some systems, and in the falcosecurity/falco container image,
the defult gcc is not the right one to compile it.
The script will try to compile the module by cycling trough all the available GCCs
starting from the default one until the module is compiled the first
time.
The default gcc is the highest priority while trying.
Newer GCCs have the priority over older GCCs.
Co-Authored-By: Leonardo Di Donato <leodidonato@gmail.com>
Signed-off-by: Lorenzo Fontana <fontanalorenz@gmail.com>
Start versioning trace files with a unique date. Any time we need to
create new trace files, change TRACE_FILES_VERSION in this script and
copy to traces-{positive,negative,info}-<VERSION>.zip.
The zip file should unzip to traces-{positive,negative,info}, without
any version.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add system:managed-certificate-controller as a system role that can be
modified. Can be changed as a part of upgrades.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add several images seen in GKE environments that can run in the
kube-system namespace.
Also change the names of the lists to be more specific. The old names
are retained but are kept around for backwards compatibility.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a set of images known to run in the host network. Mostly related to
GKE, sometimes plus metrics collection.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Sort the items in the list falco_privileged_images alphabetically
and also separate them into individual lines. Make it easier to note
changes to the entries in the list using git blame.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Previously any write to a file called sources.list would match the
access_repositories condition, even a file /usr/tmp/..../sources.list.
Change the macro so the files in repository_files must be somewhere
below any of repository_directories.
Also allow programs spawned by package management programs to change
these files, using package_mgmt_ancestor_procs.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Let programs spawned by linux-bench (CIS Linux Benchmark program) read
/etc/shadow. Tests in the benchmark check for permissions of the file
and accounts in the contents of the file.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
In some cases, dropped events around the time a new container is started
can result in missing the exec/clone for a process that does a setns to
enter the namespace of a container. Here's an example from an oss
capture:
```
282273 09:01:22.098095673 30 runc:[0:PARENT] (168555) < setns res=0
282283 09:01:22.098138869 30 runc:[0:PARENT] (168555) < setns res=0
282295 09:01:22.098179685 30 runc:[0:PARENT] (168555) < setns res=0
517284 09:01:30.128723777 13 <NA> (168909) < setns res=0
517337 09:01:30.129054963 13 <NA> (168909) < setns res=0
517451 09:01:30.129560037 2 <NA> (168890) < setns res=0
524597 09:01:30.162741004 19 <NA> (168890) < setns res=0
527433 09:01:30.179786170 18 runc:[0:PARENT] (168927) < setns res=0
527448 09:01:30.179852428 18 runc:[0:PARENT] (168927) < setns res=0
535566 09:01:30.232420372 25 nsenter (168938) < setns res=0
537412 09:01:30.246200357 0 nsenter (168941) < setns res=0
554163 09:01:30.347158783 17 nsenter (168950) < setns res=0
659908 09:01:31.064622960 12 runc:[0:PARENT] (169023) < setns res=0
659919 09:01:31.064665759 12 runc:[0:PARENT] (169023) < setns res=0
732062 09:01:31.608297074 4 nsenter (169055) < setns res=0
812985 09:01:32.217527319 6 runc:[0:PARENT] (169077) < setns res=0
812991 09:01:32.217579396 6 runc:[0:PARENT] (169077) < setns res=0
813000 09:01:32.217632211 6 runc:[0:PARENT] (169077) < setns res=0
```
When this happens, it can cause false positives for the "Change thread
namespace" rule as it allows certain process names like "runc",
"containerd", etc to perform setns calls.
Other rules already use the proc_name_exists macro to require that the
process name exists. This change adds proc_name_exists to the Change
Thread Namespace rule as well.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This update will provide information as to which process uid intitiated the event. This is really important for processes that are started
by a different user name.
Signed-off-by: Chuck Schweizer <chuck.schweizer.lvk2@statefarm.com>
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines in the [CONTRIBUTING.md](CONTRIBUTING.md) file and learn how to compile Falco from source [here](https://falco.org/docs/source).
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines in the https://github.com/falcosecurity/.github/blob/main/CONTRIBUTING.md file.
2. Please label this pull request according to what type of issue you are addressing.
3. . Please add a release note!
3. Please add a release note!
4. If the PR is unfinished while opening it specify a wip in the title before the actual title, for example, "wip: my awesome feature"
-->
@@ -22,11 +21,7 @@
> /kind feature
> If contributing rules or changes to rules, please make sure to also uncomment one of the following line:
> /kind rule-update
> /kind rule-create
> /kind release
<!--
Please remove the leading whitespace before the `/kind <>` you uncommented.
@@ -40,12 +35,12 @@ Please remove the leading whitespace before the `/kind <>` you uncommented.
> /area engine
> /area rules
> /area tests
> /area proposals
> /area CI
<!--
Please remove the leading whitespace before the `/area <>` you uncommented.
-->
@@ -67,11 +62,13 @@ Fixes #
**Does this PR introduce a user-facing change?**:
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below.
If the PR requires additional action from users switching to the new release, prepend the string "action required:".
For example, `action required: change the API interface of the rule engine`.
If NO, just write "NONE" in the release-note block below.
If YES, a release note is required, enter your release note in the block below.
The convention is the same as for commit messages: https://github.com/falcosecurity/.github/blob/main/CONTRIBUTING.md#commit-convention
If the PR introduces non-backward compatible changes, please add a line starting with "BREAKING CHANGE:" and describe what changed.
For example, `BREAKING CHANGE: the API interface of the rule engine has changed`.
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name:Checkout repository
uses:actions/checkout@v3
with:
fetch-depth:0
# Initializes the CodeQL tools for scanning.
- name:Initialize CodeQL
uses:github/codeql-action/init@v2
with:
languages:${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
Known end users with notable contributions to the project include:
* AWS
* IBM
* Red Hat
Falco is being used by numerous other companies, both large and small, to build higher layer products and services. The list includes but is not limited to:
* Equinix Metal
* IEEE
* Lowes
* Reckrut
* Yellow Pepper
* CTx
* Utikal
* Discrete Events
* Agritech Infra
This is a list of production adopters of Falco (in alphabetical order):
* [ASAPP](https://www.asapp.com/) - ASAPP is a pushing the boundaries of fundamental artificial intelligence research. We apply our research into AI-Native® products that make organizations, in the customer experience industry, highly productive, efficient, and effective—by augmenting human activity and automating workflows. We constantly monitor our workloads against different hazards and FALCO helps us extend our threat monitoring boundaries.
* [Booz Allen Hamilton](https://www.boozallen.com/) - BAH leverages Falco as part of their Kubernetes environment to verify that work loads behave as they did in their CD DevSecOps pipelines. BAH offers a solution to internal developers to easily build DevSecOps pipelines for projects. This makes it easy for developers to incorporate Security principles early on in the development cycle. In production, Falco is used to verify that the code the developer ships does not violate any of the production security requirements. BAH [are speaking at Kubecon NA 2019](https://kccncna19.sched.com/event/UaWr/building-reusable-devsecops-pipelines-on-a-secure-kubernetes-platform-steven-terrana-booz-allen-hamilton-michael-ducy-sysdig) on their use of Falco.
* [Coveo](https://www.coveo.com/) - Coveo stitches together content and data, learning from every interaction, to tailor every experience using AI to drive growth, satisfy customers and develop employee proficiency. All Falco events are centralized in our SIEM for analysis. Understanding what is running on production servers, and the context around why things are running is even more tricky now that we have further abstractions with containers and orchestration systems. Falco is giving us a good visibility inside containers and complement other Host and Network Intrusion Detection Systems. In a near future, we expect to deploy serverless functions to take action when Falco identifies patterns worth taking action for.
* [Deckhouse](https://deckhouse.io/) - Deckhouse Platform presents to you the opportunity to create homogeneous Kubernetes clusters anywhere and handles comprehensive, automagical management for them. It supplies all the add-ons you need for auto-scaling, observability, security, and service mesh. Falco is used as a part of the [runtime-audit-engine](https://deckhouse.io/documentation/latest/modules/650-runtime-audit-engine/) module to provide threats detection and enforce security compliance out of the box. By pairing with [shell-operator](https://github.com/flant/shell-operator) Falco can be configured by Kubernetes Custom Resources.
* [Fairwinds](https://fairwinds.com/) - [Fairwinds Insights](https://fairwinds.com/insights), Kubernetes governance software, integrates Falco to offer a single pane of glass view into potential security incidents. Insights adds out-of-the-box integrations and rules filter to reduce alert fatigue and improve security response. The platform adds security prevention, detection, and response capabilities to your existing Kubernetes infrastructure. Security and DevOps teams benefit from a centralized view of container security vulnerability scanning and runtime container security.
* [Frame.io](https://frame.io/) - Frame.io is a cloud-based (SaaS) video review and collaboration platform that enables users to securely upload source media, work-in-progress edits, dailies, and more into private workspaces where they can invite their team and clients to collaborate on projects. Understanding what is running on production servers, and the context around why things are running is even more tricky now that we have further abstractions like Docker and Kubernetes. To get this needed visibility into our system, we rely on Falco. Falco's ability to collect raw system calls such as open, connect, exec, along with their arguments offer key insights on what is happening on the production system and became the foundation of our intrusion detection and alerting system.
* [Giant Swarm](https://www.giantswarm.io/) - Giant Swarm manages Kubernetes clusters and infrastructure for enterprises across multiple cloud providers as well as several flavors of on-premises data centers. Our platform provisions and monitors pure "vanilla" Kubernetes clusters which can be augmented with managed solutions to many common Kubernetes challenges, including security. We use Falco for anomaly detection as part of our collection of entirely open-source tools for securing our own clusters, and offer the same capabilities to our customers as part of our [managed security offering](https://docs.giantswarm.io/app-platform/apps/security/).
* [GitLab](https://about.gitlab.com/direction/defend/container_host_security/) - GitLab is a complete DevOps platform, delivered as a single application, fundamentally changing the way Development, Security, and Ops teams collaborate. GitLab Ultimate provides the single tool teams need to find, triage, and fix vulnerabilities in applications, services, and cloud-native environments enabling them to manage their risk. This provides them with repeatable, defensible processes that automate security and compliance policies. GitLab includes a tight integration with Falco, allowing users to defend their containerized applications from attacks while running in production.
* [League](https://league.com/ca/) - League provides health benefits management services to help employees understand and get the most from their benefits, and employers to provide effective, efficient plans. Falco is used to monitor our deployed services on Kubernetes, protecting against malicious access to containerswhich could lead to leaks of PHI or other sensitive data. The Falco alerts are logged in Stackdriver for grouping and further analysis. In the future, we're hoping for integrations with Prometheus and AlertManager as well.
* [gVisor](https://gvisor.dev/) - gVisor secures Kubernetes, containers, and workloads via an alternate execution environment that handles system calls in user space, blocking security issues before they reach the underlying host. gVisor provides defense-in-depth, protection against untrusted code execution, and a secure-by-default Kubernetes experience where containers are a security boundary. Falco can be used with gVisor to detect unusual or suspicious activity using its threat detection engine on top of gVisor runtime execution information.
* [League](https://league.com/ca/) - League provides health benefits management services to help employees understand and get the most from their benefits, and employers to provide effective, efficient plans. Falco is used to monitor our deployed services on Kubernetes, protecting against malicious access to containers which could lead to leaks of PHI or other sensitive data. The Falco alerts are logged in Stackdriver for grouping and further analysis. In the future, we're hoping for integrations with Prometheus and AlertManager as well.
* [Logz.io](https://logz.io/) - Logz.io is a cloud observability platform for modern engineering teams. The Logz.io platform consists of three products — Log Management, Infrastructure Monitoring, and Cloud SIEM — that work together to unify the jobs of monitoring, troubleshooting, and security. We empower engineers to deliver better software by offering the world's most popular open source observability tools — the ELK Stack, Grafana, and Jaeger — in a single, easy to use, and powerful platform purpose-built for monitoring distributed cloud environments. Cloud SIEM supports data from multiple sources, including Falco's alerts, and offers useful rules and dashboards content to visualize and manage incidents across your systems in a unified UI.
* [Preferral](https://www.preferral.com) - Preferral is a HIPAA-compliant platform for Referral Management and Online Referral Forms. Preferral streamlines the referral process for patients, specialists and their referral partners. By automating the referral process, referring practices spend less time on the phone, manual efforts are eliminated, and patients get the right care from the right specialist. Preferral leverages Falco to provide a Host Intrusion Detection System to meet their HIPPA compliance requirements.
* [MathWorks](https://mathworks.com) - MathWorks develops mathematical computing software for engineers and scientists. MathWorks uses Falco for Kubernetes threat detection, unexpected application behavior, and maps Falco rules to their cloud infrastructure's security kill chain model. MathWorks presented their Falco use case at [KubeCon + CloudNativeCon North America 2020](https://www.youtube.com/watch?v=L-5RYBTV010).
* [Pocteo](https://pocteo.co) - Pocteo helps with Kubernetes adoption in enterprises by providing a variety of services such as training, consulting, auditing and mentoring. We build CI/CD pipelines the GitOps way, as well as design and run k8s clusters. Pocteo uses Falco as a runtime monitoring system to secure clients' workloads against suspicious behavior and ensure k8s pods immutability. We also use Falco to collect, process and act on security events through a response engine and serverless functions.
* [Preferral](https://www.preferral.com) - Preferral is a HIPAA-compliant platform for Referral Management and Online Referral Forms. Preferral streamlines the referral process for patients, specialists and their referral partners. By automating the referral process, referring practices spend less time on the phone, manual efforts are eliminated, and patients get the right care from the right specialist. Preferral leverages Falco to provide a Host Intrusion Detection System to meet their HIPAA compliance requirements.
* [Qonto](https://qonto.com) - Qonto is a modern banking for SMEs and freelancers. Qonto provides a fully featured business account with a simplified accounting flow. Falco is used by our SecOps team to detect suspicious behaviors in our clusters.
* [Raft](https://goraft.tech) - Raft is a government contractor that offers cloud-native solutions across many different agencies including DoD (Department of Defense), HHS (Health and Human Services), as well as within CFPB (Consumer Finance Protection Bureau). Raft leverages Falco to detect threats in our client's Kubernetes clusters and as a Host Intrusion Detection System. Raft proudly recommends Falco across all our different projects.
* [Replicated](https://www.replicated.com/) - Replicated is the modern way to ship on-prem software. Replicated gives software vendors a container-based platform for easily deploying cloud native applications inside customers' environments to provide greater security and control. Replicated uses Falco as runtime security to detect threats in the Kubernetes clusters which host our critical SaaS services.
* [Secureworks](https://www.secureworks.com/) - Secureworks is a leading worldwide cybersecurity company with a cloud-native security product that combines the power of human intellect with security analytics to unify detection and response across cloud, network, and endpoint environments for improved security operations and outcomes. Our Taegis XDR platform and detection system processes petabytes of security relevant data to expose active threats amongst the billions of daily events from our customers. We are proud to protect our platform’s Kubernetes deployments, as well as help our customers protect their own Linux and container environments, using Falco.
* [Shopify](https://www.shopify.com) - Shopify is the leading multi-channel commerce platform. Merchants use Shopify to design, set up, and manage their stores across multiple sales channels, including mobile, web, social media, marketplaces, brick-and-mortar locations, and pop-up shops. The platform also provides merchants with a powerful back-office and a single view of their business, from payments to shipping. The Shopify platform was engineered for reliability and scale, making enterprise-level technology available to businesses of all sizes. Shopify uses Falco to complement its Host and Network Intrusion Detection Systems.
* [Sight Machine](https://www.sightmachine.com) - Sight Machine is the category leader for manufacturing analytics and used by Global 500 companies to make better, faster decisions about their operations. Sight Machine uses Falco to help enforce SOC2 compliance as well as a tool for real time security monitoring and alerting in Kubernetes.
@@ -26,5 +64,24 @@ This is a list of production adopters of Falco (in alphabetical order):
* [Sumo Logic](https://www.sumologic.com/) - Sumo Logic provides a SaaS based log aggregation service that provides dashboards and applications to easily identify and analyze problems in your application and infrastructure. Sumo Logic provides native integrations for many CNCF projects, such as Falco, that allows end users to easily collect Falco events and analyze Falco events on DecSecOps focused dashboards.
* [Sysdig](https://www.sysdig.com/) Sysdig originally created Falco in 2016 to detect unexpected or suspicious activity using a rules engine on top of the data that comes from the sysdig kernel system call probe. Sysdig provides tooling to help with vulnerability management, compliance, detection, incident response and forensics in Cloud-native environments. Sysdig Secure has extended falco to include: a rule library, the ability to update macros, lists & rules via the user interface and API, automated tuning of rules, and rule creation based on profiling known system behavior. On top of the basic Falco rules, Sysdig Secure implements the concept of a "Security policy" that can comprise several rules which are evaluated for a user-define infrastructure scope like Kubernetes namespaces, OpenShift clusters, deployment workload, cloud regions etc.
* [Swissblock Technologies](https://swissblock.net/) At Swissblock we connect the dots by combining cutting-edge algorithmic trading strategies with in-depth market analysis. We route all Falco events to our control systems, both monitoring and logging. Being able to deeply analyse alerts, we can understand what is running on our Kubernetes clusters and check against security policies, specifically defined for each workload. A set of alarms notifies us in case of critical events, letting us react fast. In the near future we plan to build a little application to route Kubernetes internal events directly to Falco, fully leveraging Falco PodSecurityPolicies analyses.
* [Shapesecurity/F5](https://www.shapesecurity.com/) Shapesecurity defends against application fraud attacks like Account Take Over, Credential Stuffing, Fake Accounts, etc. Required by FedRamp certification, we needed to find a FIM solution to help monitor and protect our Kubernetes clusters. Traditional FIM solutions were not scalable and not working for our environment, but with Falco we found the solution we needed. Falco's detection capabilities have helped us identify anomalous behaviour within our clusters. We leverage Sidekick (https://github.com/falcosecurity/charts/tree/master/falcosidekick) to send Falco alerts to a PubSub which in turn publishes those alerts to our SIEM (SumoLogic)
* [Yahoo! JAPAN](https://www.yahoo.co.jp/) Yahoo! JAPAN is a leading company of internet in Japan. We build an AI Platform in our private cloud and provide it to scientists in our company. AI Platform is a multi-tenant Kubernetes environment and more flexible, faster, more efficient Machine Learning environment. Falco is used to detect unauthorized commands and malicious access and our AI Platform is monitored and alerted by Falco.
* [Sysdig](https://www.sysdig.com/) Sysdig originally created Falco in 2016 to detect unexpected or suspicious activity using a rules engine on top of the data that comes from the sysdig kernel system call driver. Sysdig provides tooling to help with vulnerability management, compliance, detection, incident response and forensics in Cloud-native environments. Sysdig Secure has extended Falco to include: a rule library, the ability to update macros, lists & rules via the user interface and API, automated tuning of rules, and rule creation based on profiling known system behavior. On top of the basic Falco rules, Sysdig Secure implements the concept of a "Security policy" that can comprise several rules which are evaluated for a user-defined infrastructure scope like Kubernetes namespaces, OpenShift clusters, deployment workload, cloud regions etc.
* [Xenit AB](https://xenit.se/contact/) Xenit is a growth company with services within cloud and digital transformation. We provide an open-source Kubernetes framework that we leverage to help our customers get their applications to production as quickly and as securely as possible. We use Falco's detection capabilities to identify anomalous behaviour within our clusters in both Azure and AWS.
## Projects that use Falco libs
* [R6/Phoenix](https://r6security.com/) is an attack surface protection company that uses moving target defense to provide fully automated, proactive and devops friendly security to its customers. There are a set of policies you can add to enable the moving target defense capabilities. Some of them are triggered by a combination of Falco's findings. You can kill, restart and rename pods according to the ever changing policies.
* [SysFlow](https://sysflow.io) SysFlow is a cloud-native system telemetry framework that focuses on data abstraction, behavioral analytics, and noise reduction. At its core, SysFlow exposes a compact open telemetry format that records workload behaviors by connecting event and flow representations of process control flows, file interactions, and network communications. The resulting abstraction encodes a graph structure that enables provenance reasoning on host and container environments, and fast retrieval of security-relevant information.
* [StackRox](https://stackrox.io) is the industry’s first Kubernetes-native security platform enabling organizations to build, deploy, and run cloud-native applications securely. The platform works with Kubernetes environments and integrates with DevOps and security tools, enabling teams to operationalize and secure their supply chain, infrastructure, and workloads. StackRox aims to harness containerized applications’ development speed while giving operations and security teams greater context and risk profiling. StackRox leverages cloud-native principles and declarative artifacts to automate DevSecOps best practices.
## Adding a name
If you would like to add your name to this file, submit a pull request with your change.
[Falco](https://falco.org/) is a cloud native runtime security tool for Linux operating systems. It is designed to detect and alert on abnormal behavior and potential security threats in real-time.
At its core, Falco is a kernel event monitoring and detection agent that captures events, such as syscalls, based on custom rules. Falco can enhance these events by integrating metadata from the container runtime and Kubernetes. The collected events can be analyzed off-host in SIEM or data lake systems.
---
Falco, originally created by [Sysdig](https://sysdig.com), is an incubating project under the [Cloud Native Computing Foundation](https://cncf.io) (CNCF) used in producation by various [organisations](https://github.com/falcosecurity/falco/blob/master/ADOPTERS.md).
The Falco Project, originally created by [Sysdig](https://sysdig.com), is an incubating [CNCF](https://cncf.io) open source cloud native runtime security tool.
Falco makes it easy to consume kernel events, and enrich those events with information from Kubernetes and the rest of the cloud native stack.
Falco has a rich rule set of security rules specifically built for Kubernetes, Linux, and cloud-native.
If a rule is violated in a system, Falco will send an alert notifying the user of the violation and its severity.
For detailed technical information and insights into the cyber threats that Falco can detect, visit the official [Falco](https://falco.org/) website.
### Installing Falco
For comprehensive information on the latest updates and changes to the project, please refer to the [change log](CHANGELOG.md). Additionally, we have documented the [release process](RELEASE.md) for delivering new versions of Falco.
If you would like to run Falco in **production** please adhere to the [official installation guide](https://falco.org/docs/installation/).
## Falco Repo: Powering the Core of The Falco Project
##### Kubernetes
This is the main Falco repository which contains the source code for building the Falco binary. By utilizing its [libraries](https://github.com/falcosecurity/libs) and the [falco.yaml](falco.yaml) configuration file, this repository forms the foundation of Falco's functionality. The Falco repository is closely interconnected with the following *core* repositories:
| Helm | [Chart Repository](https://github.com/falcosecurity/charts/tree/master/falco#introduction) | The Falco community offers regular helm chart releases. |
| Minikube | [Tutorial](https://falco.org/docs/third-party/#minikube) | The Falco driver has been baked into minikube for easy deployment. |
| Kind | [Tutorial](https://falco.org/docs/third-party/#kind) | Running Falco with kind requires a driver on the host system. |
| GKE | [Tutorial](https://falco.org/docs/third-party/#gke) | We suggest using the eBPF driver for running Falco on GKE. |
- [falcosecurity/libs](https://github.com/falcosecurity/libs): Falco's libraries are key to its fundamental operations, making up the greater portion of the source code of the Falco binary and providing essential features such as kernel drivers.
- [falcosecurity/rules](https://github.com/falcosecurity/rules): Contains the official ruleset for Falco, providing pre-defined detection rules for various security threats and abnormal behaviors.
- [falcosecurity/plugins](https://github.com/falcosecurity/plugins/): Falco plugins facilitate integration with external services, expand Falco's capabilities beyond syscalls and container events, and are designed to evolve with specialized functionality in future releases.
- [falcosecurity/falcoctl](https://github.com/falcosecurity/falcoctl): Command-line utility for managing and interacting with Falco.
### Developing
For more information, visit the official hub of The Falco Project: [falcosecurity/evolution](https://github.com/falcosecurity/evolution). It provides valuable insights and information about the project's repositories.
Falco is designed to be extensible such that it can be built into cloud-native applications and infrastructure.
## Getting Started with Falco
Falco has a [gRPC](https://falco.org/docs/grpc/) endpoint and an API defined in [protobuf](https://github.com/falcosecurity/falco/blob/update-readme/userspace/falco/outputs.proto).
The Falco Project supports various SDKs for this endpoint.
Carefully review and follow the [official guide and documentation](https://falco.org/docs/getting-started/).
1. Understand dependencies: Assess the environment where you'll run Falco and consider kernel versions and architectures.
2. Define threat detection objectives: Clearly identify the threats you want to detect and evaluate Falco's strengths and limitations.
3. Consider performance and cost: Assess compute performance overhead and align with system administrators or SREs. Budget accordingly.
4. Choose build and customization approach: Decide between the open source Falco build or creating a custom build pipeline. Customize the build and deployment process as necessary, including incorporating unique tests or approaches, to ensure a resilient deployment with fast deployment cycles.
5. Integrate with output destinations: Integrate Falco with SIEM, data lake systems, or other preferred output destinations to establish a robust foundation for comprehensive data analysis and enable effective incident response workflows.
### What can Falco detect?
## How to Contribute
Falco can detect and alert on any behavior that involves making Linux system calls.
Falco alerts can be triggered by the use of specific system calls, their arguments, and by properties of the calling process.
For example, Falco can easily detect incidents including but not limited to:
Please refer to the [contributing guide](https://github.com/falcosecurity/.github/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/falcosecurity/evolution/CODE_OF_CONDUCT.md) for more information on how to contribute.
- A shell is running inside a container or pod in Kubernetes.
- A container is running in privileged mode, or is mounting a sensitive path, such as `/proc`, from the host.
- A server process is spawning a child process of an unexpected type.
- Unexpected read of a sensitive file, such as `/etc/shadow`.
- A non-device file is written to `/dev`.
- A standard system binary, such as `ls`, is making an outbound network connection.
### Documentation
## Join the Community
The [Official Documentation](https://falco.org/docs/) is the best resource to learn about Falco.
To get involved with the Falco Project please visit the [community repository](https://github.com/falcosecurity/community) to find more information and ways to get involved.
### Join the Community
If you have any questions about Falco or contributing, do not hesitate to file an issue or contact the Falco maintainers and community members for assistance.
To get involved with The Falco Project please visit [the community repository](https://github.com/falcosecurity/community) to find more.
How to reach out?
### Contributing
- Join the [#falco](https://kubernetes.slack.com/messages/falco) channel on the [Kubernetes Slack](https://slack.k8s.io).
- Join the [Falco mailing list](https://lists.cncf.io/g/cncf-falco-dev).
- File an [issue](https://github.com/falcosecurity/falco/issues) or make feature requests.
See the [CONTRIBUTING.md](./CONTRIBUTING.md).
## Commitment to Falco's Own Security
### Security Audit
Full reports of various security audits can be found [here](./audits/).
A third party security audit was performed by Cure53, you can see the full report [here](./audits/SECURITY_AUDIT_2019_07.pdf).
In addition, you can refer to the [falco security](https://github.com/falcosecurity/falco/security) and [libs security](https://github.com/falcosecurity/libs/security) sections for detailed updates on security advisories and policies.
### Reporting security vulnerabilities
To report security vulnerabilities, please follow the community process outlined in the documentation found [here](https://github.com/falcosecurity/.github/blob/main/SECURITY.md).
Please report security vulnerabilities following the community process documented [here](https://github.com/falcosecurity/.github/blob/master/SECURITY.md).
## What's next for Falco?
### License Terms
Stay updated with Falco's evolving capabilities by exploring the [Falco Roadmap](https://github.com/orgs/falcosecurity/projects/5), which provides insights into the features currently under development and planned for future releases.
## License
Falco is licensed to you under the [Apache 2.0](./COPYING) open source license.
Our release process is mostly automated, but we still need some manual steps to initiate and complete it.
Changes and new features are grouped in [milestones](https://github.com/falcosecurity/falco/milestones), the milestone with the next version represents what is going to be released.
## Overview
Releases happen on a monthly cadence, towards the 16th of the on-going month, and we need to assign owners for each (usually we pair a new person with an experienced one). Assignees and the due date are proposed during the [weekly community call](https://github.com/falcosecurity/community). Note that hotfix releases can happen as soon as it is needed.
This document provides the process to create a new Falco release. In addition, it provides information about the versioning of the Falco components. At a high level each Falco release consists of the following main components:
Finally, on the proposed due date the assignees for the upcoming release proceed with the processes described below.
- Falco binary (userspace), includes `modern_bpf` driver object code (kernel space) starting with Falco 0.34.x releases
- Falco kernel driver object files, separate artifacts for `kmod` and `bpf` drivers, not applicable for `modern_bpf` driver (kernel space)
- Option 1: Kernel module (`.ko` files)
- Option 2: eBPF (`.o` files)
- Falco config and rules `.yaml` files (userspace)
- Falco plugins (userspace - optional)
> Note: Starting with Falco 0.34.x releases, the Falco userspace binary includes the `modern_bpf` driver object code during the linking process. This integration is made possible by the CO-RE (Compile Once - Run Everywhere) feature of the modern BPF driver. CO-RE allows the driver to function on kernels that have backported BTF (BPF Type Format) support or have a kernel version >= 5.8. For the older `kmod` and `bpf` drivers, separate artifacts are released for the kernel space. This is because these drivers need to be explicitly compiled for the specific kernel release, using the exact kernel headers. This approach ensures that Falco can support a wide range of environments, including multiple kernel versions, distributions, and architectures. (see `libs` [driver - kernel version support matrix](https://github.com/falcosecurity/libs#drivers-officially-supported-architectures)).
The Falco Project manages the release of both the Falco userspace binary and pre-compiled Falco kernel drivers for the most popular kernel versions and distros. The build and publish process is managed by the [test-infra](https://github.com/falcosecurity/test-infra) repo.
The Falco userspace executable includes bundled dependencies, so that it can be run from anywhere.
Falco publishes all sources, enabling users to audit the project's integrity and build kernel drivers for custom or unsupported kernels/distributions, specifically for non-modern BPF drivers (see [driverkit](https://github.com/falcosecurity/driverkit) for more information).
Finally, the release process follows a transparent process described in more detail in the following sections and the official [Falco guide and documentation](https://falco.org/) provide rich information around building, installing and using Falco.
### Falco Binaries, Rules and Sources Artifacts - Quick Links
The Falco project publishes all sources and the Falco userspace binaries as GitHub releases.
-`tgz`, `zip` source code, each ruleset is tagged separately in a mono-repo fashion, see the [rules release guidelines](https://github.com/falcosecurity/rules/blob/main/RELEASE.md)
Alternatively Falco binaries or plugins can be downloaded from the Falco Artifacts repo.
> Note: This section specifically applies to non-modern BPF drivers.
The Falco Project publishes all drivers for each release for popular kernel versions / distros and `x86_64` and `aarch64` architectures to the Falco project's managed Artifacts repo. The Artifacts repo follows standard directory level conventions. The respective driver object file is prefixed by distro and named / versioned by kernel release - `$(uname -r)`. Pre-compiled drivers are released with a [best effort](https://github.com/falcosecurity/falco/blob/master/proposals/20200818-artifacts-storage.md#notice) notice. This is because gcc (`kmod`) and clang (`bpf`) compilers sometimes fail to build the artifacts for a specific kernel version. More details around driver versioning and driver compatibility are provided in the [Falco Components Versioning](#falco-components-versioning) section. Short preview: If you use the standard Falco setup leveraging driver-loader, [driver-loader script](https://github.com/falcosecurity/falco/blob/master/scripts/falco-driver-loader) will fetch the kernel space artifact (object file) corresponding to the default `DRIVER_VERSION` Falco was shipped with.
- Option 1: Kernel module (`.ko` files) - all under same driver version directory
- Option 2: eBPF (`.o` files) - all under same driver version directory
### Timeline
Falco follows a release schedule of three times per year, with releases expected at the end of January, May, and September. Hotfix releases are issued as needed.
Changes and new features are organized into [milestones](https://github.com/falcosecurity/falco/milestones). The milestone corresponding to the next version represents the content that will be included in the upcoming release.
### Procedures
The release process is mostly automated, requiring only a few manual steps to initiate and complete.
Moreover, we assign owners for each release (typically pairing a new person with an experienced one). Assignees and due dates for releases are proposed during the [weekly community call](https://github.com/falcosecurity/community).
At a high level each Falco release needs to follow a pre-determined sequencing of releases and build order:
Assignees are responsible for creating a Falco GitHub issue to track the release tasks and monitor the progress of the release. This issue serves as a central point for communication and provides updates on the release dates. You can refer to the [Falco v0.35 release](https://github.com/falcosecurity/falco/issues/2554) or [Libs Release (0.11.0+5.0.1+driver)](https://github.com/falcosecurity/libs/issues/1092) issues as examples/templates for creating the release issue.
Finally, on the proposed due date, the assignees for the upcoming release proceed with the processes described below.
## Pre-Release Checklist
Before proceeding with the release, make sure to complete the following preparatory steps, which can be easily done using the GitHub UI:
### 1. Release notes
-Let `YYYY-MM-DD` the day before of the [latest release](https://github.com/falcosecurity/falco/releases)
-Find the previous release date (`YYYY-MM-DD`) by looking at the [Falco releases](https://github.com/falcosecurity/falco/releases)
- Check the release note block of every PR matching the `is:pr is:merged closed:>YYYY-MM-DD` [filter](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Amerged+closed%3A%3EYYYY-MM-DD)
- Ensure the release note block follows the [commit convention](https://github.com/falcosecurity/falco/blob/master/CONTRIBUTING.md#commit-convention), otherwise fix its content
- Ensure the release note block follows the [commit convention](https://github.com/falcosecurity/.github/blob/master/CONTRIBUTING.md#commit-convention), otherwise fix its content
- If the PR has no milestone, assign it to the milestone currently undergoing release
- Check issues without a milestone (using [is:pr is:merged no:milestone closed:>YYYT-MM-DD](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Amerged+no%3Amilestone+closed%3A%3EYYYT-MM-DD) filter) and add them to the milestone currently undergoing release
- Double-check that there are no more merged PRs without the target milestone assigned with the `is:pr is:merged no:milestone closed:>YYYT-MM-DD` [filters](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Amerged+no%3Amilestone+closed%3A%3EYYYT-MM-DD), if any, fix them
- Check issues without a milestone (using `is:pr is:merged no:milestone closed:>YYYY-MM-DD` [filter](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Amerged+no%3Amilestone+closed%3A%3EYYYY-MM-DD) ) and add them to the milestone currently undergoing release
- Double-check that there are no more merged PRs without the target milestone assigned with the `is:pr is:merged no:milestone closed:>YYYY-MM-DD` [filter](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Amerged+no%3Amilestone+closed%3A%3EYYYY-MM-DD), if any, update those missing
### 2. Milestones
- Move the [tasks not completed](https://github.com/falcosecurity/falco/pulls?q=is%3Apr+is%3Aopen) to a new minor milestone
### 3. Release PR
### 3. Release branch
Assuming we are releasing a non-patch version (like: Falco 0.34.0), a new release branch needs to be created.
Its naming will be `release/M.m.x`; for example: `release/0.34.x`.
The same branch will then be used for any eventual cherry pick for patch releases.
For patch releases, instead, the `release/M.m.x` branch should already be in place; no more steps are needed.
Double check that any PR that should be part of the tag has been cherry-picked from master!
### 4. Release PR
The release PR is meant to be made against the respective `release/M.m.x` branch, **then cherry-picked on master**.
- Double-check if any hard-coded version number is present in the code, it should be not present anywhere:
- If any, manually correct it then open an issue to automate version number bumping later
- Versions table in the `README.md` update itself automatically
- Generate the change log https://github.com/leodido/rn2md, or https://fs.fntlnz.wtf/falco/milestones-changelog.txt for the lazy people (it updates every 5 minutes)
- Add the lastest changes on top the previous `CHANGELOG.md`
- Versions table in the `README.md` updates itself automatically
- Generate the change log using [rn2md](https://github.com/leodido/rn2md):
- In case `rn2md` emits error try to generate an GitHub OAuth access token and provide it with the `-t` flag
- Add the latest changes on top the previous `CHANGELOG.md`
- Submit a PR with the above modifications
- Await PR approval
- Close the completed milestone as soon PR is merged
- Close the completed milestone as soon as the PR is merged into the release branch
- Cherry pick the PR on master too
## Publishing Pre-Releases (RCs and tagged development versions)
Core maintainers and/or the release manager can decide to publish pre-releases at any time before the final release
is live for development and testing purposes.
The prerelease tag must be formatted as `M.m.p-r`where `r` is the prerelease version information (e.g. `0.35.0-rc1`.)
To do so:
- [Draft a new release](https://github.com/falcosecurity/falco/releases/new)
- Use `M.m.p-r` both as tag version and release title.
- Check the "Set as a pre-release" checkbox and make sure "Set as the latest release" is unchecked
- It is recommended to add a brief description so that other contributors will understand the reason why the prerelease is published
- Publish the prerelease!
- The release pipeline will start automatically. Packages will be uploaded to the `-dev` bucket and container images will be tagged with the specified tag.
In order to check the status of the release pipeline click on the [GitHub Actions tab](https://github.com/falcosecurity/falco/actions?query=event%3Arelease) in the Falco repository and filter by release.
## Release
Let `x.y.z` the new version.
Assume `M.m.p` is the new version.
### 1. Create a tag
- Once the release PR has got merged, and the CI has done its job on the master, git tag the new release
```
git pull
git checkout master
git tag x.y.z
git push origin x.y.z
```
> **N.B.**: do NOT use an annotated tag
- Wait for the CI to complete
### 2. Update the GitHub release
### 1. Create the release with GitHub
- [Draft a new release](https://github.com/falcosecurity/falco/releases/new)
- Use `x.y.z` both as tag version and release title
- Use `M.m.p` both as tag version and release title
- Use the following template to fill the release description:
```
<!-- Copy the relevant part of the changelog here -->
<!-- Substitute M.m.p with the current release version -->
<!-- Substitute <changelog> with the one generated by [rn2md](https://github.com/leodido/rn2md) -->
### Statistics
| Merged PRs | Number |
|-------------------|---------|
| Not user-facing | x |
| Release note | x |
| Total | x |
| Merged PRs | Number |
|--------------- | ------|
| Not user-facing | x |
| Release note | x |
| Total | x |
<!-- Calculate stats and fill the above table -->
#### Release Manager <github handle>
<!-- Substitute GitHub handle with the release manager's one -->
```
- Finally, publish the release!
- The release pipeline will start automatically upon publication and all packages and container images will be uploaded to the stable repositories.
In order to check the status of the release pipeline click on the [GitHub Actions tab](https://github.com/falcosecurity/falco/actions?query=event%3Arelease) in the Falco repository and filter by release.
### 2. Update the meeting notes
For each release we archive the meeting notes in git for historical purposes.
- The notes from the Falco meetings can be [found here](https://hackmd.io/3qYPnZPUQLGKCzR14va_qg).
- Note: There may be other notes from working groups that can optionally be added as well as needed.
- Add the entire content of the document to a new file in [github.com/falcosecurity/community/tree/master/meeting-notes](https://github.com/falcosecurity/community/tree/master/meeting-notes) as a new file labeled `release-M.m.p.md`
- Open up a pull request with the new change.
## Post-Release tasks
Announce the new release to the world!
- Publish a blog on [Falco website](https://github.com/falcosecurity/falco-website) ([example](https://github.com/falcosecurity/falco-website/blob/master/content/en/blog/falco-0-28-1.md))
- Send an announcement to cncf-falco-dev@lists.cncf.io (plain text, please)
- Let folks in the slack #falco channel know about a new release came out
- IFF the on going release introduces a **new minor version**, [archive a snapshot of the Falco website](https://github.com/falcosecurity/falco-website/blob/master/release.md#documentation-versioning)
## Falco Components Versioning
This section provides more details around the versioning of the components that make up Falco's core. It can also be a useful guide for the uninitiated to be more informed about Falco's source. Because `libs` makes up the greater portion of the source code of the Falco binary and is the home of each of the kernel drivers and engines, the [libs release doc](https://github.com/falcosecurity/libs/blob/master/release.md) is an excellent additional resource. In addition, the [plugins release doc](https://github.com/falcosecurity/plugins/blob/master/release.md) provides similar details around Falco's plugins. `SHA256` checksums are provided throughout Falco's source code to empower the end user to perform integrity checks. All Falco releases also contain the sources as part of the packages.
### Falco repo (this repo)
- Falco version is a git tag (`x.y.z`), see [Procedures](#procedures) section. Note that the Falco version is a sem-ver-like schema, but not fully compatible with sem-ver.
- [FALCO_ENGINE_VERSION](https://github.com/falcosecurity/falco/blob/master/userspace/engine/falco_engine_version.h) is not sem-ver and must be bumped either when a backward incompatible change has been introduced to the rules files syntax and/or `FALCO_FIELDS_CHECKSUM` computed via `falco --list -N | sha256sum` has changed. The primary idea is that when new filter / display fields (see currently supported [Falco fields](https://falco.org/docs/rules/supported-fields/)) are introduced, a version change indicates that these fields were not available in previous engine versions. See the [rules release guidelines](https://github.com/falcosecurity/rules/blob/main/RELEASE.md#versioning-a-ruleset) to understand how this affects the versioning of Falco rules. Breaking changes introduced in the Falco engine are not necessarily tied to the drivers or libs versions. Lastly, `FALCO_ENGINE_VERSION` is typically incremented once during a Falco release cycle, while `FALCO_FIELDS_CHECKSUM` is bumped whenever necessary during the development and testing phases of the release cycle.
- During development and release preparation, libs and driver reference commits are often bumped in Falco's cmake setup ([falcosecurity-libs cmake](https://github.com/falcosecurity/falco/blob/master/cmake/modules/falcosecurity-libs.cmake#L30) and [driver cmake](https://github.com/falcosecurity/falco/blob/master/cmake/modules/driver.cmake#L29)) in order to merge new Falco features. In practice, they are mostly bumped at the same time referencing the same `libs` commit. However, for the official Falco build `FALCOSECURITY_LIBS_VERSION` flag that references the stable libs version is used (read below).
- Similarly, Falco plugins versions are bumped in Falco's cmake setup ([plugins cmake](https://github.com/falcosecurity/falco/blob/master/cmake/modules/plugins.cmake)) and those versions are the ones used for the Falco release.
- At release time Plugin, Libs and Driver versions are compatible with Falco.
- If you use the standard Falco setup leveraging driver-loader, [driver-loader script](https://github.com/falcosecurity/falco/blob/master/scripts/falco-driver-loader) will fetch the kernel space artifact (object file) corresponding to the default `DRIVER_VERSION` Falco was shipped with (read more below under Libs).
```
Falco version: x.y.z (sem-ver like)
Libs version: x.y.z (sem-ver like)
Plugin API: x.y.z (sem-ver like)
Engine: x
Driver:
API version: x.y.z (sem-ver)
Schema version: x.y.z (sem-ver)
Default driver: x.y.z+driver (sem-ver like, indirectly encodes compatibility range in addition to default version Falco is shipped with)
```
### Libs repo
- Libs version is a git tag (`x.y.z`) and when building Falco the libs version is set via the `FALCOSECURITY_LIBS_VERSION` flag (see above).
- The driver version is not directly linked to the userspace components of the Falco binary. This is because of the clear separation between userspace and kernel space, which adds an additional layer of complexity. To address this, the concept of a `Default driver` has been introduced, allowing for implicit declaration of compatible driver versions. For example, if the default driver version is `5.0.1+driver`, Falco works with all driver versions >= 5.0.1 and < 6.0.0. This is a consequence of how the driver version is constructed starting from the `Driver API version` and `Driver Schema version`. Driver API and Schema versions are explained in the respective [libs driver doc](https://github.com/falcosecurity/libs/blob/master/driver/README.VERSION.md) -> Falco's `driver-loader` will always fetch the default driver, therefore a Falco release is always "shipped" with the driver version corresponding to the default driver.
- See [libs release doc](https://github.com/falcosecurity/libs/blob/master/release.md) for more information.
### Plugins repo
- Plugins version is a git tag (`x.y.z`)
- See [plugins release doc](https://github.com/falcosecurity/plugins/blob/master/release.md) for more information.
### Rules repo
- Rulesets are versioned individually through git tags
- See [rules release doc](https://github.com/falcosecurity/rules/blob/main/RELEASE.md) for more information.
- See [plugins release doc](https://github.com/falcosecurity/plugins/blob/master/release.md) for more information about plugins rulesets.
This document describes The Falco Project's branding guidelines, language, and message.
Falco is an open source security project whose brand and identity are governed by the [Cloud Native Computing Foundation](https://www.linuxfoundation.org/legal/trademark-usage).
Content in this document can be used to publically share about Falco.
This document describes the official branding guidelines of The Falco Project. Please see the [Falco Branding](https://falco.org/community/falco-brand/) page on our website for further details.
Content in this document can be used to publicly share about Falco.
@@ -15,6 +17,21 @@ There are 3 logos available for use in this directory. Use the primary logo unle
The Falco logo is Apache 2 licensed and free to use in media and publication for the CNCF Falco project.
### Colors
| Name | PMS | RGB |
|-----------|------|-------------|
| Teal | 3125 | 0 174 199 |
| Cool Gray | 11 | 83 86 90 |
| Black | | 0 0 0 |
| Blue-Gray | 7700 | 22 92 125 |
| Gold | 1375 | 255 158 27 |
| Orange | 171 | 255 92 57 |
| Emerald | 3278 | 0 155 119 |
| Green | 360 | 108 194 74 |
The primary colors are those in the first two rows.
### Slogan
> Cloud Native Runtime Security
@@ -41,7 +58,7 @@ If a rule has been violated, Falco triggers an alert.
### How does Falco work?
Falco traces kernel events and reports information about the system calls being executed at runtime.
Falco leverages the extended berkley packet filter (eBPF) which is a kernel feature implemented for dynamic crash-resilient and secure code execution in the kernel.
Falco leverages the extended berkeley packet filter (eBPF) which is a kernel feature implemented for dynamic crash-resilient and secure code execution in the kernel.
Falco enriches these kernel events with information about containers running on the system.
Falco also can consume signals from other input streams such as the containerd socket, the Kubernetes API server and the Kubernetes audit log.
At runtime, Falco will reason about these events and assert them against configured security rules.
@@ -67,7 +84,7 @@ Examples of malicious behavior include:
Falco is capable of [consuming the Kubernetes audit logs](https://kubernetes.io/docs/tasks/debug-application-cluster/falco/#use-falco-to-collect-audit-events).
By adding Kubernetes application context, and Kubernetes audit logs teams can understand who did what.
### Writing about Falco
##### Yes
@@ -98,7 +115,7 @@ Falco ultimately is a security engine. It reasons about signals coming from a sy
##### Anomaly detection
This refers to an event that occurs with something unsual, concerning, or odd occurs.
This refers to an event that occurs with something unusual, concerning, or odd occurs.
We can associate anomalies with unwanted behavior, and alert in their presence.
##### Detection tooling
@@ -107,7 +124,6 @@ Falco does not prevent unwanted behavior.
Falco however alerts when unusual behavior occurs.
This is commonly referred to as **detection** or **forensics**.
---
# Glossary
@@ -128,6 +144,10 @@ Sometimes this word is incorrectly used to refer to a `probe`.
The global term for the software that sends events from the kernel. Such as the eBPF `probe` or the `kernel module`.
#### Plugin
Used to describe a dynamic shared library (`.so` files in Unix, `.dll` files in Windows) that conforms to a documented API and allows to extend Falco's capabilities.
#### Falco
The name of the project, and also the name of [the main engine](https://github.com/falcosecurity/falco) that the rest of the project is built on.
message(STATUS"cppcheck command not found, static code analysis using cppcheck will not be available.")
else()
message(STATUS"cppcheck found at: ${CPPCHECK}")
# we are aware that cppcheck can be run
# along with the software compilation in a single step
# using the CMAKE_CXX_CPPCHECK variables.
# However, for practical needs we want to keep the
# two things separated and have a specific target for it.
# Our cppcheck target reads the compilation database produced by CMake
set(CMAKE_EXPORT_COMPILE_COMMANDSOn)
add_custom_target(
cppcheck
COMMAND${CPPCHECK}
"--enable=all"
"--force"
"--inconclusive"
"--inline-suppr"# allows to specify suppressions directly in source code
"--xml"# we want to generate a report
"--output-file=${CMAKE_CURRENT_BINARY_DIR}/static-analysis-reports/cppcheck/cppcheck.xml"# generate the report under the reports folder in the build folder
"-i${CMAKE_CURRENT_BINARY_DIR}"# exclude the build folder
"${CMAKE_SOURCE_DIR}"
)
endif()# CPPCHECK
if(NOTCPPCHECK_HTMLREPORT)
message(STATUS"cppcheck-htmlreport command not found, will not be able to produce html reports for cppcheck results")
else()
message(STATUS"cppcheck-htmlreport found at: ${CPPCHECK_HTMLREPORT}")
This directory contains various ways to package Falco as a container and related tools.
This directory contains various ways to package Falco as a container and related tools.
## Currently Supported Images
| Name | Directory | Description |
|---|---|---|
| [falcosecurity/falco:latest](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:master](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/falco | Falco (DEB built from git tag or from the master) with all the building toolchain. |
| [falcosecurity/falco-driver-loader:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:master](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader) | docker/driver-loader | `falco-driver-loader` as entrypoint with the building toolchain. |
| [falcosecurity/falco-no-driver:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver), [falcosecurity/falco-no-driver:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver),[falcosecurity/falco-no-driver:master](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver) | docker/no-driver | Falco (TGZ built from git tag or from the master) without the building toolchain. |
| [falcosecurity/falco-builder:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-builder) | docker/builder | The complete build tool chain for compiling Falco from source. See [the documentation](https://falco.org/docs/source/) for more details on building from source. Used to build Falco (CI). |
| [falcosecurity/falco-tester:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-tester) | docker/tester | Container image for running the Falco test suite. Used to run Falco integration tests (CI). |
| _to not be published_ | docker/local | Built on-the-fly and used by falco-tester. |
| [falcosecurity/falco:latest](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:master](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/falco | Falco (DEB built from git tag or from the master) with all the building toolchain. |
| _not yet published (experimental)_ | docker/ubi | Falco (built from RedHat's UBI base image) with the building toolchain. |
| [falcosecurity/falco-driver-loader:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader),[falcosecurity/falco-driver-loader:master](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader) | docker/driver-loader | `falco-driver-loader` as entrypoint with the building toolchain. |
| [falcosecurity/falco-no-driver:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver), [falcosecurity/falco-no-driver:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver),[falcosecurity/falco-no-driver:master](https://hub.docker.com/repository/docker/falcosecurity/falco-no-driver) | docker/no-driver | Falco (TGZ built from git tag or from the master) without the building toolchain. |
| [falcosecurity/falco-builder:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-builder) | docker/builder | The complete build tool chain for compiling Falco from source. See [the documentation](https://falco.org/docs/getting-started/source/) for more details on building from source. Used to build Falco (CI). |
| [falcosecurity/falco-tester:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-tester) | docker/tester | Container image for running the Falco test suite. Used to run Falco integration tests (CI). |
| _not to be published_ | docker/local | Built on-the-fly and used by falco-tester. |
> Note: `falco-builder`, `falco-tester` (and the `docker/local` image that it's built on the fly) are not integrated into the release process because they are development and CI tools that need to be manually pushed only when updated.
* We use `Dockerfile` to build the `centos7` Falco builder image.
* We use `modern-falco-builder.Dockerfile` to build Falco with the modern probe and return it as a Dockerfile output. This Dockerfile doesn't generate a Docker image but returns as output (through the `--output` command):
RUNtest -n "$FALCO_VERSION"||(echo"FALCO_VERSION not set"&&false)
ENVFALCO_VERSION=${FALCO_VERSION}
LABEL"name"="Falco Runtime Security"
LABEL"vendor"="Falco"
LABEL"version"="${FALCO_VERSION}"
LABEL"release"="${FALCO_VERSION}"
LABEL"ubi-version"="${UBI_VERSION}"
LABEL"summary"="Falco is a security policy engine that monitors system calls and cloud events, and fires alerts when security policies are violated."
LABEL"description"="Falco is a security policy engine that monitors system calls and cloud events, and fires alerts when security policies are violated."
LABEL"io.k8s.display-name"="Falco"
LABEL"io.k8s.description"="Falco is a security policy engine that monitors system calls and cloud events, and fires alerts when security policies are violated."
@@ -27,7 +27,7 @@ That's where Falco comes in. We want to make it possible for Falco to perform a
Transparently read a candidate PSP into an equivalent set of Falco rules that can look for the conditions in the PSP.
The PSP is converted into a set of Falco rules which can be either saved as a file for later use/inspection, or loaded directly so they they can monitor system calls and k8s audit activity.
The PSP is converted into a set of Falco rules which can be either saved as a file for later use/inspection, or loaded directly so that they can monitor system calls and k8s audit activity.
### Non-Goals
@@ -51,6 +51,6 @@ No diagrams yet.
* We'll use [inja](https://github.com/pantor/inja) as the templating engine.
* For the most part, we can rely on the existing framework of rules, filter expressions, and output expressions that already exist in Falco. One significant change will be that filter fields can extract more than one "value" per event, and we'll need to define new operators to perform set comparisions betweeen values in an event and values in the comparison right-hand-side.
* For the most part, we can rely on the existing framework of rules, filter expressions, and output expressions that already exist in Falco. One significant change will be that filter fields can extract more than one "value" per event, and we'll need to define new operators to perform set comparisons between values in an event and values in the comparison right-hand-side.
* This will rely heavily on existing support for [K8s Audit Events](https://falco.org/docs/event-sources/kubernetes-audit/) in Falco.
@@ -6,13 +6,13 @@ This is a proposal to better structure the Falco API.
The Falco API is a set of contracts describing how users can interacts with Falco.
By definiing a set of interfaces the Falco Authors intend to decouple Falco from other softwares and data (eg., from the input sources) and, at the same time, make it more extensible.
By defining a set of interfaces the Falco Authors intend to decouple Falco from other software and data (eg., from the input sources) and, at the same time, make it more extensible.
Thus, this document intent is to propose a list of services that contistute the Falco API (targeting the first stable version of Falco, v1.0.0).
Thus, this document intent is to propose a list of services that constitute the Falco API (targeting the first stable version of Falco, v1.0.0).
## Motivation
We want to enable users to use thirdy-party clients to interface with Falco outputs, inputs, rules, and configurations.
We want to enable users to use third-party clients to interface with Falco outputs, inputs, rules, and configurations.
Such ability would enable the community to create a whole set of OSS tools, built on top of Falco.
@@ -94,7 +94,7 @@ This translates in having the following set of `proto` files.
}
```
- one or more `.proto` containing the commond models - ie., the already existing `schema.proto` containing source enum, etc.
- one or more `.proto` containing the command models - ie., the already existing `schema.proto` containing source enum, etc.
@@ -36,7 +36,7 @@ There will be no intention to cover Falco rule syntax in this proposal.
### Use cases
When new PRs are created in the area of rules, reviewers need to examine whether there are new rules, macros or lists are introduced. If yes, check wether follow the naming convention.
When new PRs are created in the area of rules, reviewers need to examine whether there are new rules, macros or lists are introduced. If yes, check whether follow the naming convention.
@@ -4,7 +4,7 @@ The **Falco Artifact Scope** proposal is divided in two parts:
1. the Part 1 - *this document*: the State of Art of Falco artifacts
2. the [Part 2](./20200506-artifacts-scope-part-2.md): the intended state moving forward
## Summary
## Summary
As a project we would like to support the following artifacts.
@@ -16,7 +16,7 @@ Inspired by many previous issues and many of the weekly community calls.
## Terms
**falco**
**falco**
*The Falco binary*
@@ -30,12 +30,12 @@ Inspired by many previous issues and many of the weekly community calls.
**package**
*An installable artifact that is operating system specific. All packages MUST be hosted on bintray.*
*An installable artifact that is operating system specific. All packages MUST be hosted on [bintray](https://bintray.com/falcosecurity).*
**image**
*OCI compliant container image hosted on dockerhub with tags for every release and the current master branch.*
# Packages
@@ -52,11 +52,11 @@ List of currently official container images (for X86 64bits only):
| Name | Directory | Description |
|---|---|---|
| [falcosecurity/falco:latest](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:master](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/stable | Falco (DEB built from git tag or from the master) with all the building toolchain. |
| [falcosecurity/falco:latest-slim](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_-slim](https://hub.docker.com/repository/docker/falcosecurity/falco),[falcosecurity/falco:master-slim](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/slim | Falco (DEB build from git tag or from the master) without the building toolchain. |
| [falcosecurity/falco-driver-loader:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:master](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader) | docker/driver-loader | `falco-driver-loader` as entrypoint with the building toolchain. |
| [falcosecurity/falco-builder:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-builder) | docker/builder | The complete build tool chain for compiling Falco from source. See [the documentation](https://falco.org/docs/source/) for more details on building from source. Used to build Falco (CI). |
| [falcosecurity/falco-tester:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-tester) | docker/tester | Container image for running the Falco test suite. Used to run Falco integration tests (CI). |
| [falcosecurity/falco:latest](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:master](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/stable | Falco (DEB built from git tag or from the master) with all the building toolchain. |
| [falcosecurity/falco:latest-slim](https://hub.docker.com/repository/docker/falcosecurity/falco), [falcosecurity/falco:_tag_-slim](https://hub.docker.com/repository/docker/falcosecurity/falco),[falcosecurity/falco:master-slim](https://hub.docker.com/repository/docker/falcosecurity/falco) | docker/slim | Falco (DEB build from git tag or from the master) without the building toolchain. |
| [falcosecurity/falco-driver-loader:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:_tag_](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader), [falcosecurity/falco-driver-loader:master](https://hub.docker.com/repository/docker/falcosecurity/falco-driver-loader) | docker/driver-loader | `falco-driver-loader` as entrypoint with the building toolchain. |
| [falcosecurity/falco-builder:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-builder) | docker/builder | The complete build tool chain for compiling Falco from source. See [the documentation](https://falco.org/docs/source/) for more details on building from source. Used to build Falco (CI). |
| [falcosecurity/falco-tester:latest](https://hub.docker.com/repository/docker/falcosecurity/falco-tester) | docker/tester | Container image for running the Falco test suite. Used to run Falco integration tests (CI). |
| _to not be published_ | docker/local | Built on-the-fly and used by falco-tester. |
**Note**: `falco-builder`, `falco-tester` (and the `docker/local` image which it's built on the fly by the `falco-tester` one) are not integrated into the release process because they are development and CI tools that need to be manually pushed only when updated.
@@ -76,13 +76,13 @@ This new [contrib](https://github.com/falcosecurity/contrib) repository will be
### repository
"_Incubating level_" projects such as [falco-exporter](https://github.com/falco-exporter) can be promoted from `contrib` to their own repository.
"_Incubating level_" projects such as [falco-exporter](https://github.com/falco-exporter) can be promoted from `contrib` to their own repository.
This is done as needed, and can best be measured by the need to cut a release and use the GitHub release features. Again, this is at the discretion of the Falco open source community.
### official support
As the need for a project grows, it can ultimately achieve the highest and most coveted status within The Falco Project. "_Offical support_."
As the need for a project grows, it can ultimately achieve the highest and most coveted status within The Falco Project. "_Official support_."
The artifacts listed above are part of the official Falco release process. These artifact will be refined and amended by the [Part 2](./20200506-artifacts-scope-part-2.md).
@@ -92,7 +92,7 @@ The *Part 1* is mainly intended as a cleanup process.
For each item not listed above, ask if it needs to be moved or deleted.
After the cleanup process, all items will match the *Part 1* of this proposal.
### Action Items
Here are SOME of the items that would need to be done, for example:
@@ -111,4 +111,4 @@ Update documentation in [falco-website#184](https://github.com/falcosecurity/fal
### Adjusting projects
- YAML manifest documentation to be moved to `contrib`
- Minkube, Kind, Puppet, Ansible, etc documentation to be moved to `contrib`
- Minikube, Kind, Puppet, Ansible, etc documentation to be moved to `contrib`
The Falco packages are **automatically** built and sent to [bintray](https://bintray.com/falcosecurity) in the following cases:
- a pull request gets merged into the master branch (**Falco development releases**)
- a new Falco release (git tag) happens on the master branch (**Falco stable releases**)
The only prerequisite is that the specific Falco source code builds successfully and that the tests pass.
As per [Falco Artifacts Scope (#1)](./20200506-artifacts-scope-part-1.md) proposal we provide three kind of Falco packages:
- DEB
- RPM
- Tarball
Thus, we have three repositories for the Falco stable releases:
- https://bintray.com/falcosecurity/deb
- https://bintray.com/falcosecurity/rpm
- https://bintray.com/falcosecurity/bin
And three repositories for the Falco development releases:
- https://bintray.com/falcosecurity/deb-dev
- https://bintray.com/falcosecurity/rpm-dev
- https://bintray.com/falcosecurity/bin-dev
## Drivers
The process of publishing a set of prebuilt Falco drivers is implemented by the **Drivers Build Grid (DBG)** in the [test-infra](https://github.com/falcosecurity/test-infra/tree/master/driverkit) repository (`driverkit` directory).
This process is driven by the configuration files (YAML) present in the `driverkit/config` directory in the [test-infra](https://github.com/falcosecurity/test-infra/tree/master/driverkit) repository.
Each of these files represents a prebuilt driver (eventually two: kernel module and eBPF probe, when possible) that will be published on [bintray](https://bintray.com/falcosecurity) if it builds correctly.
Every time the `driverkit/config` directory on the master branch has some changes from the previous commit the CI system, which you can find defined in the [.circleci/config.yml](https://github.com/falcosecurity/test-infra/blob/master/.circleci/config.yml) file, takes care of building and publishing all the drivers.
The driver versions we ship prebuilt drivers for are:
- the driver version associated with the last Falco stable version ([see here](https://github.com/falcosecurity/falco/blob/c4b7f17271d1a4ca533b2e672ecaaea5289ccdc5/cmake/modules/sysdig.cmake#L29))
- the driver version associated with the penultimate Falco stable version
The prebuilt drivers get published into [this](https://bintray.com/falcosecurity/driver) generic artifacts repository.
You can also visualize the full list of prebuilt drivers by driver version visiting this [URL](https://dl.bintray.com/falcosecurity/driver).
### Notice
The generation of new prebuilt drivers takes usually place with a frequency of 1-2 weeks, on a **best-effort** basis.
Thus, it can happen the list of available prebuilt drivers does not yet contain the driver version currently on Falco master.
Nevertheless, this process is an open, auditable, and transparent one.
So, by sending a pull-request towards [test-infra](https://github.com/falcosecurity/test-infra) repository containing the configuration YAML files you can help the Falco community stay on track.
Some pull-requests you can look at to create your own are:
While, the documentation of the YAML configuration files can be found [here](https://github.com/falcosecurity/driverkit/blob/master/README.md).
## Container images
As per Falco packages, also the Falco official container images are **automatically** published to the [dockerhub](https://hub.docker.com/r/falcosecurity/falco).
These images are built and published in two cases:
- a pull request gets merged into the master branch (**Falco development releases**)
- a new Falco release (git tag) happens (**Falco stable releases**)
For a detailed explanation of the container images we build and ship look at the following [documentation](https://github.com/falcosecurity/falco/blob/master/docker/README.md).
# Proposal for First Class Structured Exceptions in Falco Rules
## Summary
## Motivation
Almost all Falco Rules have cases where the behavior detected by the
rule should be allowed. For example, The rule Write Below Binary Dir
has exceptions for specific programs that are known to write below
these directories as a part of software installation/management:
```yaml
- rule:Write below binary dir
desc:an attempt to write to any file below a set of binary directories
condition:>
bin_dir and evt.dir = < and open_write
and not package_mgmt_procs
and not exe_running_docker_save
and not python_running_get_pip
and not python_running_ms_oms
and not user_known_write_below_binary_dir_activities
...
```
In most cases, these exceptions are expressed as concatenations to the original rule's condition. For example, looking at the macro package_mgmt_procs:
```yaml
- macro:package_mgmt_procs
condition:proc.name in (package_mgmt_binaries)
```
The result is appending `and not proc.name in (package_mgmt_binaries)` to the condition of the rule.
A more extreme case of this is the write_below_etc macro used by Write below etc rule. It has tens of exceptions:
```
...
and not sed_temporary_file
and not exe_running_docker_save
and not ansible_running_python
and not python_running_denyhosts
and not fluentd_writing_conf_files
and not user_known_write_etc_conditions
and not run_by_centrify
and not run_by_adclient
and not qualys_writing_conf_files
and not git_writing_nssdb
...
```
The exceptions all generally follow the same structure--naming a program and a directory prefix below /etc where that program is allowed to write files.
### Using Appends/Overwrites to Customize Rules
An important way to customize rules and macros is to use `append: true` to add to them, or `append: false` to define a new rule/macro, overwriting the original rule/macro. Here's an example from Update Package Repository:
((open_write and access_repositories) or (modify and modify_repositories))
and not package_mgmt_procs
and not exe_running_docker_save
and not user_known_update_package_registry
```
If someone wanted to add additional exceptions to this rule, they could add the following to the user_rules file:
```yaml
- list:package_mgmt_binaries
items:[puppet]
append:true
- macro:package_mgmt_procs
condition:and not proc.pname=chef
append:true
- macro:user_known_update_package_registry
condition:(proc.name in (npm))
append:false
```
This adds an 3 different exceptions:
* an additional binary to package_mgmt_binaries (because append is true),
* adds to package_mgmt_procs, adding an exception for programs spawned by chef (because append is true)
* overrides the macro user_known_update_package_registry to add an exception for npm (because append is false).
### Problems with Appends/Overrides to Define Exceptions
Although the concepts of macros and lists in condition fields, combined with appending to lists/conditions in macros/rules, is very general purpose, it can be unwieldy:
* Appending to conditions can result in incorrect behavior, unless the original condition has its logical operators set up properly with parentheses. For example:
```yaml
rule:my_rule
condition:(evt.type=open and (fd.name=/tmp/foo or fd.name=/tmp/bar))
rule:my_rule
condition:or fd.name=/tmp/baz
append:true
```
Results in unintended behavior. It will match any fd related event where the name is /tmp/baz, when the intent was probably to add /tmp/baz as an additional opened file.
* A good convention many rules use is to have a clause "and not user_known_xxxx" built into the condition field. However, it's not in all rules and its use is a bit haphazard.
* Appends and overrides can get confusing if you try to apply them multiple times. For example:
```yaml
macro:allowed_files
condition:fd.name=/tmp/foo
...
macro:allowed_files
condition:and fd.name=/tmp/bar
append:true
```
If someone wanted to override the original behavior of allowed_files, they would have to use `append: false` in a third definition of allowed_files, but this would result in losing the append: true override.
## Solution: Exceptions as first class objects
To address some of these problems, we will add the notion of Exceptions as top level objects alongside Rules, Macros, and Lists. A rule that supports exceptions must define a new key `exceptions` in the rule. The exceptions key is a list of identifier plus list of tuples of filtercheck fields. Here's an example:
```yaml
- rule:Write below binary dir
desc:an attempt to write to any file below a set of binary directories
condition:>
bin_dir and evt.dir = < and open_write
and not package_mgmt_procs
and not exe_running_docker_save
and not python_running_get_pip
and not python_running_ms_oms
and not user_known_write_below_binary_dir_activities
exceptions:
- name:proc_writer
fields:[proc.name, fd.directory]
- name:container_writer
fields:[container.image.repository, fd.directory]
comps:[=, startswith]
- name:proc_filenames
fields:[proc.name, fd.name]
comps:[=, in]
- name:filenames
fields:fd.filename
comps:in
```
This rule defines four kinds of exceptions:
* proc_writer: uses a combination of proc.name and fd.directory
* container_writer: uses a combination of container.image.repository and fd.directory
* proc_filenames: uses a combination of process and list of filenames.
* filenames: uses a list of filenames
The specific strings "proc_writer"/"container_writer"/"proc_filenames"/"filenames" are arbitrary strings and don't have a special meaning to the rules file parser. They're only used to link together the list of field names with the list of field values that exist in the exception object.
proc_writer does not have any comps property, so the fields are directly compared to values using the = operator. container_writer does have a comps property, so each field will be compared to the corresponding exception items using the corresponding comparison operator.
proc_filenames uses the in comparison operator, so the corresponding values entry should be a list of filenames.
filenames differs from the others in that it names a single field and single comp operator. This changes how the exception condition snippet is constructed (see below).
Notice that exceptions are defined as a part of the rule. This is important because the author of the rule defines what construes a valid exception to the rule. In this case, an exception can consist of a process and file directory (actor and target), but not a process name only (too broad).
Exception values will most commonly be defined in rules with append: true. Here's an example:
```yaml
- list:apt_files
items:[/bin/ls, /bin/rm]
- rule:Write below binary dir
exceptions:
- name:proc_writer
values:
- [apk, /usr/lib/alpine]
- [npm, /usr/node/bin]
- name:container_writer
values:
- [docker.io/alpine, /usr/libexec/alpine]
- name:proc_filenames
values:
- [apt, apt_files]
- [rpm, [/bin/cp, /bin/pwd]]
- name:filenames
values:[python, go]
```
A rule exception applies if for a given event, the fields in a rule.exception match all of the values in some exception.item. For example, if a program `apk` writes to a file below `/usr/lib/alpine`, the rule will not trigger, even if the condition is met.
Notice that an item in a values list can be a list. This allows building exceptions with operators like "in", "pmatch", etc. that work on a list of items. The item can also be a name of an existing list. If not present surrounding parentheses will be added.
Finally, note that the structure of the values property differs between the items where fields is a list of fields (proc_writer/container_writer/proc_filenames) and when it is a single field (procs_only). This changes how the condition snippet is constructed.
### Implementation
For exception items where the fields property is a list of field names, each exception can be thought of as an implicit "and not (field1 cmp1 val1 and field2 cmp2 val2 and...)" appended to the rule's condition. For exception items where the fields property is a single field name, the exception can be thought of as an implicit "and not field cmp (val1, val2, ...)". In practice, that's how exceptions will be implemented.
When a rule is parsed, the original condition will be wrapped in an extra layer of parentheses and all exception values will be appended to the condition. For example, using the example above, the resulting condition will be:
```
(<Write below binary dir condition>) and not (
(proc.name = apk and fd.directory = /usr/lib/alpine) or (proc.name = npm and fd.directory = /usr/node/bin) or
(container.image.repository = docker.io/alpine and fd.directory startswith /usr/libexec/alpine) or
(proc.name=apt and fd.name in (apt_files))) or
(fd.filename in (python, go))))
```
The exceptions are effectively syntactic sugar that allows expressing sets of exceptions in a concise way.
### Advantages
Adding Exception objects as described here has several advantages:
* All rules will implicitly support exceptions. A rule writer doesn't need to define a user_known_xxx macro and add it to the condition.
* The rule writer has some controls on what defines a valid exception. The rule author knows best what is a good exception, and can define the fields that make up the exception.
* With this approach, it's much easier to add and manage multiple sets of exceptions from multiple sources. You're just combining lists of tuples of filtercheck field values.
## Backwards compatibility
To take advantage of these new features, users will need to upgrade Falco to a version that supports exception objects and exception keys in rules. For the most part, however, the rules file structure is unchanged.
This approach does not remove the ability to append to exceptions nor the existing use of user_xxx macros to define exceptions to rules. It only provides an additional way to express exceptions. Hopefully, we can migrate existing exceptions to use this approach, but there isn't any plan to make wholesale rules changes as a part of this.
This approach is for the most part backwards compatible with older Falco releases. To implement exceptions, we'll add a preprocessing element to rule parsing. The main Falco engine is unchanged.
However, there are a few changes we'll have to make to Falco rules file parsing:
* Currently, Falco will reject files containing anything other than rule/macro/list top-level objects. As a result, `exception` objects would be rejected. We'll probably want to make a one-time change to Falco to allow arbitrary top level objects.
* Similarly, Falco will reject rule objects with exception keys. We'll also probably want to change Falco to allow unknown keys inside rule/macro/list/exception objects.
The [bintray](https://bintray.com/falcosecurity) open-source plan offers 10GB free space for storing artifacts.
They also kindly granted us an additional 5GB of free space.
## Goal
Keep the storage space usage under 15GB by cleaning up the [Falco artifacts](./20200506-artifacts-scope-part-1.md) from the [storage](./20200818-artifacts-storage).
## Status
To be implemented.
## Packages
### Tarballs from Falco master
At the moment of writing this document, this kind of Falco package requires approx. 50MB (maximum detected size) of storage space.
Since, historically, the [bin-dev](https://bintray.com/falcosecurity/bin-dev) repository is the less used one, this document proposes to keep only the last 10 **Falco development releases** it contains.
This means that the [bin-dev](https://bintray.com/falcosecurity/bin-dev) repository will take at maximum 500MB of storage space.
### DEB from Falco master
At the moment of writing this document, this kind of Falco package requires approx. 5.1MB (maximum detected size) of storage space.
Historically, every Falco release is composed by less than 50 merges (upper limit).
So, to theoretically retain all the **Falco development releases** that led to a Falco stable release, this document proposes to keep the last 50 Falco DEB packages.
This means that the [deb-dev](https://bintray.com/falcosecurity/deb-dev) repository will take at maximum 255MB of storage space.
### RPM from Falco master
At the moment of writing this document, this kind of Falco package requires approx. 4.3MB (maximum detected size) of storage space.
For the same exact reasons explained above this document proposes to keep the last 50 Falco RPM packages.
This means that the [rpm-dev](https://bintray.com/falcosecurity/rpm-dev) repository will take at maximum 215MB of storage space.
### Stable releases
This document proposes to retain all the stable releases.
This means that all the Falco packages present in the Falco stable release repositories will be kept.
The [bin](https://bintray.com/falcosecurity/bin) repository contains a Falco tarball package for every release.
This means it grows in space of ~50MB each month.
The [deb](https://bintray.com/falcosecurity/deb) repository contains a Falco DEB package for every release.
This means it grows in space of ~5MB each month.
The [rpm](https://bintray.com/falcosecurity/rpm) repository contains a Falco RPM package for every release.
This means it grows in space of ~4.3MB each month.
### Considerations
Assuming the size of the packages does not surpass the numbers listed in the above sections, the **Falco development releases** will always take less that 1GB of artifacts storage space.
Assuming 12 stable releases at year, at the current size of packages, the **Falco stable releases** will take approx. 720MB of storage space every year.
### Implementation
The Falco CI will have a new CI job - called `cleanup/packages-dev` - responsible for removing the **Falco development releases** depending on the above plan.
This job will be triggered after the `publish/packages-dev` completed successfully.
## Drivers
As explained in the [Artifacts Storage](./20200818-artifacts-storage) proposal, we build the drivers for the **last two driver versions** associated with **latest Falco stable releases**.
Then, we store those drivers into a [generic bintray repository](https://bintray.com/falcosecurity/driver) from which the installation process automatically downloads them, if suitable.
This document proposes to implement a cleanup mechanism that deletes all the other driver versions available.
At the moment of writing, considering only the last two driver versions (**ae104eb**, **85c8895**) associated with the latest Falco stable releases, we ship ~340 eBPF drivers, each accounting for ~3.1MB of storage space, and 1512 kernel modules (~3.1MB size each, too).
Thus, we obtain an estimate of approx. 2.875GB for **each** driver version.
This document proposes to only store the last two driver versions associates with the latest Falco stable releases. And deleting the other ones.
This way, assuming the number of prebuilt drivers does not skyrocket, we can reasonably estimate the storage space used by prebuilt drivers to be around 6GB.
Notice that, in case a Falco stable release will not depend on a new driver version, this means the last two driver versions will, in this case, cover more than the two Falco stable releases.
### Archiving
Since the process of building drivers is time and resource consuming, this document also proposes to move the driver versions in other storage facilities.
The candidate is an AWS S3 bucket responsible for holding the deleted driver version files.
#### Notice
The current mechanism the Falco community uses to store the Falco drivers is explained by the [drivers-storage-s3](https://github.com/falcosecurity/falco/blob/master/proposals/20201025-drivers-storage-s3.md) proposal.
### Implementation
The [test-infra](https://github.com/falcosecurity/test-infra) CI, specifically its part dedicated to run the **Drivers Build Grid** that runs every time it detects changes into the `driverkit` directory of the [test-infra](https://github.com/falcosecurity/test-infra) repository,
will have a new job - called `drivers/cleanup` - responsible for removing all the Falco driver versions except the last two.
This job will be triggered after the `drivers/publish` completed successfully on the master branch.
#### Notice
At the moment of writing (2021 09 28) the `drivers/cleanup` job is no more in place.
Pragmatically, this means that the older Falco drivers will remain available in their [S3 bucket](https://download.falco.org/?prefix=driver/).
In the past days, as many people probably noticed, Bintray started rate-limiting our users, effectively preventing them from downloading any kernel module, rpm/deb package or any pre-built dependency we host there.
This does not only interrupt the workflow of our users but also the workflow of the contributors, since without bintray most of our container images and CMake files can’t download the dependencies we mirror.
### What is the cause?
We had a spike in adoption apparently, either a user with many nodes or an increased number of users. We don’t know this detail specifically yet because bintray does not give us very fine-grained statistics on this.
This is the 30-days history:
As you can see, we can only see that they downloaded the latest kernel module driver version, however we can’t see if:
* It’s a single source or many different users
* What is the kernel/OS they are using
### What do we host on Bintray?
* RPM packages: high traffic but very manageable ~90k downloads a month
* Deb packages:low traffic ~5k downloads a month
* Pre-built image Dependencies: low traffic, will eventually disappear in the future
* Kernel modules: very high traffic, 700k downloads in 10 days, this is what is causing the current problems. They are primarily used by users of our container images.
* eBPF probes: low traffic ~5k downloads a month
### Motivations to go to S3 instead of Bintray for the Drivers
Bintray does an excellent service at building the rpm/deb structures for us, however we also use them for S3-like storage for the drivers. We have ten thousand files hosted there and the combinations are infinite.
Before today, we had many issues with storage even without the spike in users we are seeing since the last ten days.
## Context on AWS
Amazon AWS, recently gave credits to the Falco project to operate some parts of the infrastructure on AWS. The CNCF is providing a sub-account we are already using for the migration of the other pieces (like Prow).
## Interactions with other teams and the CNCF
* The setup on the AWS account side already done, this is all technical work.
* We need to open a CNCF service account ticket for the download.falco.org subdomain to point to the S3 bucket we want to use
## The Plan
We want to propose to move the drivers and the container dependencies to S3.
#### Moving means:
* We create a public S3 bucket with [stats enabled](https://docs.aws.amazon.com/AmazonS3/latest/dev/analytics-storage-class.html)
* We attach the bucket to a cloudfront distribution behind the download.falco.org subdomain
* We move the current content keeping the same web server directory structure
* We change the Falco Dockerfiles and driver loader script accordingly
* We update test-infra to push the drivers to S3
* Once we have the drivers in S3, we can ask bintray to relax the limits for this month so that our users are able to download the other packages we keep there. Otherwise they will have to wait until November 1st. We only want to do that after the moving because otherwise we will hit the limits pretty quickly.
#### The repositories we want to move are:
* [https://bintray.com/falcosecurity/driver](https://bintray.com/falcosecurity/driver) will become https://download.falco.org/driver
* [https://bintray.com/falcosecurity/dependencies](https://bintray.com/falcosecurity/dependencies) will become https://download.falco.org/dependencies
#### Changes in Falco
* [Search for bintray ](https://github.com/falcosecurity/falco/search?p=2&q=bintray)on the Falco repo and replace the URL for the CMake and Docker files.
* It’s very important to change the DRIVERS_REPO environment variable [here](https://github.com/falcosecurity/falco/blob/0a33f555eb8e019806b46fea8b80a6302a935421/CMakeLists.txt#L86) - this is what updates the falco-driver-loader scripts that the users and container images use to fetch the module
#### Changes in Test Infra
* We need to use the S3 cli instead of jfrog cli to upload to the s3 bucket after building [here](https://github.com/falcosecurity/test-infra/blob/master/.circleci/config.yml)
* We can probably remove jfrog from that repo since it only deals with drivers and drivers are being put on S3 now
* Instructions on how to setup the S3 directory structure [here](https://falco.org/docs/installation/#install-driver)
* Changes should not be necessary, we are not updating the way people install Falco but only the driver. The driver is managed by a script we can change.
## Mitigation and next steps for the users
* **The average users should be good to go now, Bintray raised our limits and we have some room to do this without requiring manual steps on your end**
* **Users that can’t wait for us to have the S3 setup done: **can setup an S3 as driver repo themselves, push the drivers they need to it after compiling them (they can use [Driverkit](https://github.com/falcosecurity/driverkit) for that) Instructions on how to setup the S3 directory structure [here](https://falco.org/docs/installation/#install-driver).
* **Users that can’t wait but don’t want to setup a webserver themselves**: the falco-driver-loader script can also compile the module for you. Make sure to install the kernel-headers on your nodes.
* **Users that can wait** we will approve this document and act on the plan described here by providing the DRIVERS_REPO at [https://download.falco.org/driver](https://download.falco.org/driver) that then you can use
### How to use an alternative DRIVERS_REPO ?
**On bash:**
export DRIVERS_REPO=https://your-url-here
**Docker**
Pass it as environment variable using the docker run flag -e - for example:
docker run -e DRIVERS_REPO=[https://your-url-here](https://your-url-here)
**Kubernetes**
spec:
containers:
- env:
- name: DRIVERS_REPO
value: https://your-url-here
## Release
Next release is on December 1st, we want to rollout a hotfix 0.26.2 release that only contains the updated script before that date so that users don’t get confused and we can just tell them "update Falco" to get the thing working again.
Sysdig Inc. intends to donate **libsinsp**, **libscap**, the **kernel module driver** and the **eBPF driver sources** by moving them to the Falco project.
This means that some parts of the [draios/sysdig](https://github.com/draios/sysdig) repository will be moved to a new GitHub repository called [falcosecurity/libs](https://github.com/falcosecurity/libs).
This plan aims to describe and clarify the terms and goals to get the contribution done.
## Motivation
There are two main OSS projects using the libraries and drivers that we are aware of:
- [sysdig](https://github.com/draios/sysdig) the command line tool
- [Falco](https:/github.com/falcosecurity/falco), the CNCF project.
Since the Falco project is a heavy user of the libraries, a lot more than the sysdig cli tool, Sysdig (the company) decided to donate the libraries and the driver to the Falco community.
Sysdig (the command line tool) will continue to use the libraries now provided by the Falco community underneath.
This change is win-win for both parties because of the following reasons:
- The Falco community owns the source code of the three most important parts of the software it distributes.
- Right now it is "only" an engine on top of the libraries. This **contribution** helps in making the scope of the Falco project broader. Having the majority of the source code under an **open governance** in the same organization gives the Falco project more contribution opportunities, helps it in **evolving independently** and makes the whole Falco community a strong owner of the processes and decision making regarding those crucial parts.
- Given the previous point, Sysdig (the command line tool) will benefit from the now **extended contributors base**
- Sysdig (the company) can now focus on the user experience and user space features
- **Contributions** to the libraries and drivers will be **easier** to spread across the Falco community
- By being donated, with their own **release process**, **release artifacts**, and **documentation**, the libraries can now live on their own and possibly be used directly in other projects by becoming fundamental pieces for their success.
## Goals
There are many sub-projects and each of them interacts in a different way in this contribution.
Let's see the goals per sub-project.
### libsinsp
1. Extract libsinsp from `draios/sysdig/userspace/libsinsp` (keeping the commit history) into [falcosecurity/libs](https://github.com/falcosecurity/libs)
2. The migration comes first, then we can do additional PRs for the points below so that we do only one thing at a time and keep the history linear
3. Keep the same code, refactorings will need to be done in subsequent PRs and approved separately
4. Adapt the CMake and build files
5. Install [poiana](https://github.com/poiana) and its workflows on it
6. Define the `OWNERS`
- Owners are chosen from the current major contributors (considering the past two years) to this project, given their availability, commitment is key
7. When possible, migrate issues and PRs to the new repository
8. Distribute the `libsinsp.so` library and headers as an artifact (rpm, deb, tar.gz) following the falcosecurity current process
9. Distribute the `libsinsp.a` library and headers as an artifact (rpm, deb, tar.gz) following the falcosecurity current process
10. Creation of the CI scripts using the Falco CI and Falco Infra
11. The CI scripts will need to publish the artifacts in the current falcosecurity artifacts repository
12. Artifacts will be pushed for every tag (release) and for every master merge (development release)
13. Falco follows a [multi-stage model for adopting new projects](https://github.com/falcosecurity/evolution#falco-project-evolution), in this case we will do an exception since the library is foundational for Falco and it has a very good track record already
14. This project will go already "Official support" once the contribution is completed
15. Contributing, Code of Conduct, Governance, Security, and Support will be the same as the rest of the organization, find them [here](https://github.com/falcosecurity/.github)
16. Every other additional change will need to have its own process with a proposal
17. Implement the release process as described above
18. Propose a change to Falco repository to use the artifacts produced by the libsinsp release process for the build
19. Document the API
### libscap
1. Extract libscap from `draios/sysdig/userspace/libscap` (keeping the commit history) into [falcosecurity/libs](https://github.com/falcosecurity/libs)
2. The migration comes first, then we can do additional PRs for the points below so that we do only one thing at a time and keep the history linear
3. Keep the same code, refactorings will need to be done in subsequent PRs and approved separately
4. Adapt the CMake and build files
5. Install [poiana](https://github.com/poiana) and its workflows on it
6. Define the `OWNERS`
- Owners are chosen from the current major contributors (considering the past two years) to this project, given their availability, commitment is key
7. When possible, migrate issues and PRs to the new repository
8. Distribute the `libscap.so` library and headers as an artifact (rpm, deb, tar.gz) following the falcosecurity current process
9. Distribute the `libscap.a` library and headers as an artifact (rpm, deb, tar.gz) following the falcosecurity current process
10. Creation of the CI scripts using the Falco CI and Falco Infra
11. The CI scripts will need to publish the artifacts in the current falcosecurity artifacts repository
12. Artifacts will be pushed for every tag (release) and for every master merge (development release)
13. Falco follows a [multi-stage model for adopting new projects](https://github.com/falcosecurity/evolution#falco-project-evolution), in this case we will do an exception since the library is foundational for Falco and it has a very good track record already
14. This project will go already "Official support" once the contribution is completed
15. Contributing, Code of Conduct, Governance, Security, and Support will be the same as the rest of the organization, find them [here](https://github.com/falcosecurity/.github)
16. Every other additional change will need to have its own process with a proposal
17. Implement the release process as described above
18. Propose a change to Falco repository to use the artifacts produced by the libscap release process for the build
19. Document the API
### Drivers: Kernel module and eBPF probe
1. Extract them from `draios/sysdig/driver` (keeping the commit history) into [falcosecurity/libs](https://github.com/falcosecurity/libs)
2. The migration comes first, then we can do additional PRs for the point below so that we do only one thing at a time and keep the history linear
3. Keep the same code, refactorings will need to be done in subsequent PRs and approved separately
4. Adapt the Makefiles and build files
5. Install [poiana](https://github.com/poiana) and its workflows on it
6. Define the `OWNERS`
- Owners are chosen from the current major contributors (considering the past two years) to this project, given their availability, commitment is key
7. When possible, migrate issues and PRs to the new repository
8. Falco follows a [multi-stage model for adopting new projects](https://github.com/falcosecurity/evolution#falco-project-evolution), in this case we will do an exception since the library is foundational for Falco and it has a very good track record already. We are just changing maintenance ownership
9. Contributing, Code of Conduct, Governance, Security, and Support will be the same as the rest of the organization, find them [here](https://github.com/falcosecurity/.github)
10. Every other additional change will need to have its own process with a proposal
11. The Falco community already ships driver artifacts using [driverkit](https://github.com/falcosecurity/driverkit) and the [test-infra repository](https://github.com/falcosecurity/test-infra)
- Adapt the place from which [driverkit](https://github.com/falcosecurity/driverkit) grabs the drivers source
12. This project will go already "Official support" once the migration is completed.
### Falco
1. Adapt the CMake files to point to the new homes for libscap, libsinsp and the drivers
2. When distributing the deb and rpm, libscap and libsinsp will need to be install dependencies and not anymore compiled into Falco
### Driverkit
1. Change the source location for the drivers to point to the new driver repository
### pdig
1. The project will need to be adapted to use libscap and libsinsp and the fillers from their new location
This is a proposal to create an infrastructure to extend the functionality of the Falco libraries via plugins.
Plugins will allow users to easily extend the functionality of the libraries and, as a consequence, of Falco and any other tool based on the libraries.
This proposal, in particular, focuses on two types of plugins: source plugins and extractor plugins.
## Motivation
[libscap](https://github.com/falcosecurity/libs/tree/master/userspace/libscap) and [libsinsp](https://github.com/falcosecurity/libs/tree/master/userspace/libsinsp) provide a powerful data capture framework, with a rich set of features that includes:
- data capture
- trace files management
- enrichment
- filtering
- formatting and screen rendering
- Lua scripting (chisels)
These features have been designed with one specific input in mind: system calls. However, they are generically adaptable to a broad set of inputs, such as cloud logs.
With this proposal, we want to dramatically extend the scope of what the libraries, Falco and other tools can be applied to. We want to do it in a way that is easy, efficient and empowers anyone in the community to write a plugin.
## Goals
- To design and implement a plugin framework that makes the libraries more modular and extensible
- To have a framework that is easy to use
- To support dynamic loading of plugins, so that the libraries can be extended without having to be recompiled and relinked
- To enable users to write plugins in any language, with a particular focus on Go, C and C++
- To have an efficient plugin framework so that, performance-wise, writing a plugin is as close as possible as extending the libraries internal source code
- To make it possible to write plugins for Linux, MacOS and Windows
## Non-Goals
- To implement plugins other than source and extractor: to be approached as separate task
- To document the plugin framework and interface: to be approached as separate task
## Proposal
### Plugin Common Information
Both source and extractor plugins have the following:
- A required api version, to ensure compatibility with the plugin framework.
- A name
- A description
- A version
- A contact field for the plugin authors (website, github repo, twitter, etc).
- Functions to initialize and destroy the plugin internal state.
### Plugin types
Initially, we will implement support for two types of plugins: source plugins and extractor plugins.
#### Source Plugin
A source plugin implements a new sinsp/scap event source. It has the ability to "open" and "close" a session that provides events. It also has the ability to return an event to the plugin framework via a next() method. Events returned by source plugins have an "event source", which describes the information in the event. This is distinct from the plugin name to allow for multiple kinds of plugins to generate the same kind of events. For example, there might be plugins gke-audit-bridge, eks-audit-bridge, ibmcloud-audit-bridge, etc. that all fetch [K8s Audit](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) information. The plugins would have different names but would have the same event source "k8s_audit".
Source plugins also have the ability to extract information from events based on fields. For example, a field proc.name extracts a process name from a syscall event. The plugin returns a set of supported fields, and there are functions to extract a value given an event and field. The plugin framework can then build filtering expressions/Falco rule conditions based on these fields combined with relational and/or logical operators. For example, given an expression "ct.name=root and ct.region=us-east-1", the plugin framework handles parsing the expression, calling the plugin to extract values for a given event, and determining the result of the expression. In a Falco output string like "An EC2 Node was created (name=%ec2.name region=%ct.region)", the plugin framework handles parsing the output string, calling the plugin to extract values for a given event, and building the resolved string.
Source plugins also provide an "id", which is globally unique and is used in capture files (see below).
#### Extractor Plugin
An extractor plugin focuses only on field extraction from events generated by other plugins, or by the core libraries. It does *not* provide an event source, but can extract fields from other event sources. An example is json field extraction, where a plugin might be able to extract fields from arbitrary json payloads.
An extractor plugin provides an optional set of event sources. When the framework receives an event with an event source in the plugin's set of event sources, fields in expressions/Falco outputs will be extracted from events using the plugin. An extractor plugin can also *not* name a set of event sources. In this case, fields will be extracted from *all* events, regardless of source. In this case, the extractor plugin must detect the format of arbitrary payloads and be able to return NULL/no value when the payload is not supported.
### Support for Plugin Events in Capture Files.
libscap will define a new event type called "pluginevent" that contains two fields:
* "plugin ID": This uniquely identifies the plugin that generated this event.
* "event_data": This is a variable-length data buffer containing the event data, as returned by the plugin.
Defining an event for plugins allows creating capture files from plugins. These capture files can be saved, read, filtered, etc, like any other capture file, allowing for later analysis/display/etc.
### Plugins format
Plugins are dynamic libraries (.so files in Unix, .dll files in windows) that export a minimum set of functions that the libraries will recognize.
Plugins are versioned using semantic versioning to minimize regressions and compatibility issues.
Plugins can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++.
### Protecting from plugin issues
The libraries will do everything possible to validate the data coming from the plugins and protect Falco and the other consumers from corrupted data. However, for performance reasons, plugins will be "trusted": they will run in the same thread and address space as Falco and they could crash the program. We assume that the user will be in control of plugin loading and will make sure only trusted plugins are loaded/packaged with Falco.
### Plugin/Event Source registries
Every source plugin requires its own, unique plugin ID to interoperate with Falco and the other plugins. The plugin ID will be used by the libs to properly process incoming events (for example, when saving events to file and loading them back), and by plugins to unambiguously recognize their dependencies.
To facilitate the allocation and distribution of plugin IDs, we will require that plugin developers request IDs for their plugins to the Falco organization. The mechanism used for plugin allocation is not determined yet and will be discussed in the future.
Similarly, plugin developers must register event sources with the Falco organization. This allows coordination between plugins that wish to provide compatible payloads, and to allow extractor plugins to know what data format is associated with a given event source.
### golang plugin SDK
To facilitate the development of plugins written in go, an SDK has been developed. We intend this SDK (and future SDKs for other languages) to be part of the Falco organization. For this reason, we submitted the following incubation request: https://github.com/falcosecurity/evolution/issues/62
### Proposed API (subject to change)
```c
// This struct represents an event returned by the plugin, and is used
// below in next()/next_batch().
// - data: pointer to a memory buffer pointer. The plugin will set it
// to point to the memory containing the next event. Once returned,
// the memory is owned by the plugin framework and will be freed via
// a call to free().
// - datalen: pointer to a 32bit integer. The plugin will set it the size of the
// buffer pointed by data.
// - ts: the event timestamp. Can be (uint64_t)-1, in which case the engine will
// automatically fill the event time with the current time.
typedefstructss_plugin_event
{
uint8_t*data;
uint32_tdatalen;
uint64_tts;
}ss_plugin_event;
//
// This is the opaque pointer to the state of a plugin.
// It points to any data that might be needed plugin-wise. It is
// allocated by init() and must be destroyed by destroy().
// It is defined as void because the engine doesn't care what it is
// and it treats is as opaque.
//
typedefvoidss_plugin_t;
//
// This is the opaque pointer to the state of an open instance of the source
// plugin.
// It points to any data that is needed while a capture is running. It is
// allocated by open() and must be destroyed by close().
// It is defined as void because the engine doesn't care what it is
// and it treats is as opaque.
//
typedefvoidss_instance_t;
//
// Interface for a sinsp/scap source plugin
//
//
// NOTE: For all functions below that return a char *, the memory
// pointed to by the char * must be allocated by the plugin using
// malloc() and should be freed by the caller using free().
//
// For each function below, the exported symbol from the dynamic
Falco rules already have the notion of a "source", using the source property in rules objects, and there are currently two kinds of event sources: "syscall" and "k8s_audit". We will use the source property in Falco rules to map a given rule to the event source on which the rule runs.
For example, given a plugin with source "aws_cloudtrail", and a Falco rule with source "aws_cloudtrail", the rule will be evaluated for any events generated by the plugin.
Similarly, an extractor plugin that includes "aws_cloudtrail" in its set of event sources will have the opportunity to extract information from aws_cloudtrail events if a matching field is found in the rule's condition, exception, or output properties.
This, combined with the restrictions below, allows a set of loaded rules files to contain a mix of rules for plugins as well as "core" syscall/k8s_audit events.
We will also make a change to compile rules/macros/lists selectively based on the set of loaded plugins (specifically, their event sources), instead of unconditionally as Falco is started. This is especially important for macros, which do not contain a source property, but might contain fields that are only implemented by a given plugin.
### Handling Duplicate/Overlapping Fields in Plugins/Libraries Core
At an initial glance, adding plugins introduces the possibility of tens/hundreds of new filtercheck fields that could potentially overlap/conflict. For example, what happens if a plugin defines a "proc.name" field? However, the notion of "event source" makes these potential conflicts manageable.
Remember that field extraction is always done in the context of an event, and each event can be mapped back to an event source. So we only need to ensure that filtercheck fields are distinct for a given event source. For example, it's perfectly valid for an AWS Cloudtrail plugin to define a proc.name field, as the events generated by that plugin are wholly separate from syscall events. For syscall events, the AWS Cloudtrail plugin is not involved and the core libraries extract the process name for the tid performing a syscall. For AWS Cloudtrail events, the core libraries are not involved in field extraction and is performed by the AWS Cloudtrail plugin instead.
We only need to ensure the following:
* That only one plugin is loaded at a time that exports a given event source. For example, the libraries can load either a gke-audit-bridge plugin with event source k8s_audit, or eks-audit-bridge with event source k8s_audit, but not both.
* That for a mix of source and extractor plugins having the same event source, that the fields are distinct. For example, a source plugin with source k8s_audit can export ka.* fields, and an extractor plugin with event source k8s_audit can export a jevt.value[/...] field, and the appropriate plugin will be used to extract fields from k8s_audit events as fields are parsed from condition expressions/output format strings.
### Plugin Versions and Falco Rules
To allow rules files to document the plugin versions they are compatible with, we will add a new top-level field `required_plugin_versions` to the Falco rules file format. The field is optional, and if not provided no plugin compatibility checks will be performed. The syntax of `required_plugin_versions` will be the following:
```yaml
- required_plugin_versions:
- name:<plugin_name>
version:x.y.z
...
```
Below required_plugin_versions is a list of objects, where each object has `name` and `version` properties. If a plugin is loaded, and if an entry in `required_plugin_versions` has a matching name, then the loaded plugin version must be semver compatible with the version property.
Falco can load multiple rules files, and each file may contain its own `required_plugin_versions` property. In this case, name+version pairs across all files will be merged, and in the case of duplicate names all provided versions must be compatible.
### Loading the plugins
The mechanics of loading a plugin are implemented in the libraries and leverage the dynamic library functionality of the operating system (dlopen/dlsym in unix, LoadLibrary/GetProcAddress in Windows). The plugin loading code also ensures that:
- the plugin is valid, i.e. that it exports the set of expected symbols
- the plugin has an api version number that is compatible with the libraries instance
- that only one source plugin is loaded at a time for a given event source
- if a mix of source and extractor plugins are loaded for a given event source, that the exported fields have unique names that don't overlap across plugins
#### Loading plugins in falcosecurity/libs
At the libraries level, loading plugins is handled via the static method:
filepath points to a dynamic library containing code that exports plugin API functions. config contains arbitrary config content which is passed to init().
Note that the code using the libraries is responsible for determining the location of plugin libraries and their configuration.
#### Loading plugins in falcosecurity/falco
Falco will control/configure loading plugins via the new "plugins" property in falco.yaml. Here's an example:
```yaml
plugins:
- name: aws_cloudtrail
library_path: aws_cloudtrail/plugin.so
init_config: "..."
open_params: "..."
- name: http_json
library_path: http_json/plugin.so
init_config_file: http_json/config.txt
open_params_file: http_json/params.txt
# Optional
load_plugins: [aws_cloudtrail]
```
A new "plugins" property in falco.yaml will define the set of plugins that can be loaded by Falco. The property contains a list of objects, with the following properties:
* name: Only used for load_plugins, but by convention should be the same as the value returned by the name() api function.
* library_path: a path to the shared library. The path can be relative, in which case it is relative to Falco's "share" directory under a "plugins" subdirectory e.g. /usr/share/falco/plugins.
* init_config: If present, the exact configuration text that will be provided as an argument to the init() function.
* init_config_file: If present, the provided file will be read and the contents will be provided as an argument to the init() function.
* open_params: If present, the exact params text that will be provided as an argument to the open() function.
* open_params_file: If present, the provided file will be read and the contents will be provided as an argument to the open() function.
For a given yaml object in the plugins list, only one of init_config/init_config_file and one of open_params/open_params_file can be provided at a time.
A new "load_plugins" property in falco.yaml will allow for loading a subset of the plugins defined in plugins. If present, only the plugins with the provided names will be loaded.
### Examples
We have an initial version working, consisting of:
* A version of falcosecurity/libs that supports the [plugin framework](https://github.com/falcosecurity/libs/tree/new/plugin-system-api-additions)
* Support code and examples for [writing plugins in go](https://github.com/mstemm/libsinsp-plugin-sdk-go/tree/new/plugin-system-api-additions)
* A [cloudtrail](https://github.com/mstemm/plugins/tree/new/plugin-system-api-additions) plugin that can generate events from cloudtrail logs and extract fields from those events.
* A version of Falco that uses all of the above to [load and evaluate rules with plugins](https://github.com/leogr/falco/tree/new/plugin-system-api-additions)
This proposal aims to define guidelines for the official distribution of artifacts published by Falcosecurity.
Therefore, to create a unified management of the distribution of artifacts, this document supersedes (for the parts concerning the distributions of artifacts) proposals [Falco Artifacts Scope - Part 1](https://github.com/falcosecurity/falco/blob/master/proposals/20200506-artifacts-scope-part-1.md), [Falco Artifacts Scope - Part 2](https://github.com/falcosecurity/falco/blob/master/proposals/20200506-artifacts-scope-part-2.md), and [Falco Drivers Storage S3](https://github.com/falcosecurity/falco/blob/master/proposals/20201025-drivers-storage-s3.md) and also extends and generalizes the proposal [Falco Rules and Plugin distribution](https://github.com/falcosecurity/falcoctl/blob/main/proposals/20220916-rules-and-plugin-distribution.md) for [falcoctl](https://github.com/falcosecurity/falcoctl).
## Goals
- Allow users to consume artifacts in a consistent way
- Define official artifacts
- Unify distribution mechanism, infrastructure and tooling
- Provide generic guidelines applicable to any artifact to be distributed
## Non-Goals
- Infra/CI implementation details
- Supply chain security topics
## Proposal
With officially supported artifacts, we mean that set of artifacts published
by Falcosecurity as part of Falco or its ecosystem.
At the time of writing, the Falcosecurity organization distributes several kinds of artifacts in the form of files or container images. They include:
- Installation packages
- Helm charts
- Drivers (eg, kmod, eBPF)
- Rule files
- Plugins
- Other kinds may be added in the future.
Features shipped with **official artifacts are intended for general availability(GA)**, unless otherwise specified (eg. if experimental or non-production ready features are present, they must be indicated in the release notes).
The same artifacts can be distributed via multiple distribution channels, and each channel can be mirrored. **The [falco.org](https://falco.org/) website must list all official distribution channels and mirrors**. Any distribution channel not listed on our official website must not be considered part of the official distribution. However, maintainers can still use other channels for experimentation or incubating projects eventually.
### Distribution channels
#### HTTP Distribution
Distributing artifacts as plain files via HTTP is mostly intended for **humans, simple and legacy clients** (e.g., a shell script that downloads a file).
The allowed publishing channels are:
- **[download.falco.org](https://download.falco.org/)** where most of the file artifacts lives
- **endpoints made available by GitHub** for the Falcosecurity organization (e.g., release download URL, GitHub pages, etc.).
Typically, all official artifacts that can be shipped as plain files should be published at [download.falco.org](https://download.falco.org/) and available for download.
Using the GitHub platform is allowed as an alternative assuming that artifacts are published under the Falcosecurity organization and the GitHub platform usage limitations are being respected (a notable example is publishing a [Helm chart index file using GitHub pages](https://falcosecurity.github.io/charts/)).
It is allowed to publish other non-official artifacts (for example, [development builds](https://download.falco.org/?prefix=packages/bin-dev/)), taking that those are correctly denoted.
Introducing other HTTP channels is discouraged. Providing mirrors is discouraged unless required for technical reasons.
#### OCI Distribution
Some artifacts are in the form of Open Container Initiative (OCI) images and require OCI registries to be distributed. Nevertheless, since the [OCI Distribution Spec](https://specs.opencontainers.org/distribution-spec/?v=v1.0.0) allows any content, even regular files can be stored in OCI registries and distributed likewise. Notably, the [Helm project in early 2022 started storing charts in OCI](https://helm.sh/blog/storing-charts-in-oci/) registries. One our tool [falcoctl did the same](https://github.com/falcosecurity/falcoctl/blob/main/proposals/20220916-rules-and-plugin-distribution.md) later.
Distributing artifacts via OCI registries is intended for all compatible consumers (i.e., [falcoctl](https://github.com/falcosecurity/falcoctl)). It is **allowed and encouraged for any artifacts**. All official artifacts should be published so.
Both channels are equivalent and may publish the same artifacts. However, for historical reasons and to avoid confusion, the **`docker.io` registry should only be used for container images** and not for other kinds of artifacts (e.g., plugins, rules, etc.).
Mirrors are allowed and encouraged if they facilitate artifacts consumption by our users. This proposal reccomends to enable mirrors on the major public OCI registry, such as [Amazon ECR](https://gallery.ecr.aws/) (which is already implentend in our infra at the time of writing).
Official **channels and mirrors must be listed at [falco.org](https://falco.org/)**.
It is allowed to publish other non-official artifacts, even using image tags, taking that those are correctly denoted.
#### Other channels
At the time of writing, no other distribution channels are present or needed. However, in case a new kind of artifact will require a particular distribution mechanism (for example, in case an existing package manager system need to consume the artifact using its protocol), the rule of thumb is first to use the available GitHub features for the Falcosecurity organization, if possible. Users will quickly recognize the association between the artifact and the publisher (i.e., falcosecurity), and for that reason is usually preferable.
In all other cases, introducing a new distribution channel must require extensive discussion among maintainers. Nevertheless, **introducing too many distribution channels is discouraged** because it disperses the effort and can mislead users.
### Publishing
#### Source repository
Artifacts must always be built starting from the originating source code and thru an auditable and reproducible process that runs on our infra. It's recommended that the naming and versioning of the published artifact consistently match the originating repository's naming and versioning. For example, the package `falco-0.33.0-x86_64.tar.gz` must match the source code of the git tag [0.33.0](https://github.com/falcosecurity/falco/tree/0.33.0) of the [falco](https://github.com/falcosecurity/falco) repository.
It's recommended that **each repository publish only one kind of artifact** associated with it.
Exceptions are allowed for:
- mono repos (notably [charts](https://github.com/falcosecurity/charts) and [plugins](https://github.com/falcosecurity/plugins)),
- or whenever technical constraints impose a different approach (notably, our Driver Build Grid lives on [test-infra](https://github.com/falcosecurity/test-infra), but the source code is in [libs](https://github.com/falcosecurity/libs)).
Exceptions should be documented to avoid the users and contributors might be confused.
#### Namespacing
As a general rule, to avoid name clashing among different projects under the Falcosecurity organization, all **published artifacts should reflect the originating repository name** in their publishing URL. For example, all artifacts generated by the [falcosecurity/plugins](https://github.com/falcosecurity/plugins) repository should have `falcosecurity/plugins` as the URL's base path.
Exceptions are allowed for:
- legacy and already published artifacts (to avoid disruption);
- justified technical reasons.
#### Versioning
All published artifacts must be labeled with version numbers following the **[Semantic Versioning 2 specification](https://semver.org/)**.
For the [HTTP Distribution](#http-distribution), the version number must be reflected in the file name (including build metadata like the targeted arch and platform).
For the [OCI Distribution](#oci-distribution), the version number must be reflected in the image tag (build metadata may be avoided if included in the manifest).
### Tooling
Tooling is essential to deliver a consistent and straightforward UX to our users since the limited set of distribution channels is acceptable to provide just one (or a limited set of) tool(s) capable of working with various artifacts published by the Falcosecurity organization.
In this regard, this proposal follows up the [Falco Rules and Plugin distribution](https://github.com/falcosecurity/falcoctl/blob/main/proposals/20220916-rules-and-plugin-distribution.md) proposal and recommends to use of **[falcoctl](https://github.com/falcosecurity/falcoctl) as the tool to managing artifacts specifically intended for Falco**. The tool's design should consider that other kinds of artifacts may be added in the future.
Likewise, relying on existing **third-party tools for generic or well-known kinds of artifacts** (for example, Helm charts) is recommended.
### Ecosystem
Compatibility with other tools on the broader cloud native ecosystem should be considered when dealing with artifacts and their distribution.
It is also recommended to use third-party solutions and projects that facilitate our users' discovery of published artifacts (for example, https://artifacthub.io/).
## Action items
The following subsections indicate major action items to be executed in order to transition from the current to the desiderate state of the art, as noted in this proposal.
### Move [Falco rules](https://github.com/falcosecurity/falco/tree/master/rules) to their own repo
Falco rules files (i.e., the ruleset for the data source syscall) are currently only distributed in bundles with Falco. However, now falcoctl can manage rules artifacts so that we can ship them separately.
The benefits of having rules living in their repository are:
- dedicated versioning
- rules release will not be tied anymore to a Falco release (e.g., no need to wait for the scheduled Falco release to publish a new rule aiming to detect the latest published CVE)
- consistent installation/update mechanism with other rulesets (plugins rules are already published in their repository and can be consumed by falcoctl)
Note that this change will not introduce a breaking change: Falco will continue shipping the default ruleset by including the published ruleset package.
### Make `falcoctl` official
Considering the centrality of falcoctl for managing official artifacts for Falco, the falcoctl project must be promoted to "Official" status, and its repository assumed to be [core](https://github.com/falcosecurity/evolution/blob/main/GOVERNANCE.md#core-repositories).
### Deprecate `falco-driver-loader`
At the time of writing, `falco-driver-loader` is a shell script shipped in a bundle with Falco that has the responsibility of installing a driver by either downloading it from our distribution channels or trying to build it on-the-fly.
Our experience showed all the limitations of this approach, and it's now clear that such as script is hard to maintain. Furthermore, its responsibility overlaps with our aim to use `falcoctl` as the tool for managing artifacts.
Thus, this proposal mandates to deprecate of `falco-driver-loader` in favor of `falcoctl.`
However, to avoid user disruption and breaking legacy use case, it's recommended to provide still a faced script that exposes the same command line usage of `falco-driver-loader` but forward its execution to the new tool `falcoctl`.
This implicitly requires that `falcoctl` be shipped in a bundle with Falco.
### Update the documentation
This proposal mandates making use of official documentation (i.e., falco.org) to state official items, such as artifacts, distribution channels, and mirrors.
For that reason, it becomes imperative to update the documentation periodically concerning the list of officially supported distribution channels and mirrors.
### Usage of GitHub Packages
Since GitHub is the primary platform where the Falcosecurity organization hosts its code and infrastructure, its provided features should be preferred whenever possible.
This proposal recommends using the GitHub Packages feature when the need to distribute a new kind of artifact arises. Such as convention should be adopted among all repositories of the organization.
This document proposes the introduction of a structured process for managing Falco's roadmap and implementing related changes in our development process. The goal is to ensure the efficient execution of our roadmap objectives.
### Goals
The pillars of this proposal are:
- Define processes for release cycles and development iterations
- Provide guidelines for planning and prioritizing efforts
- Introduce regular meetings for core maintainers
- Using *GitHub Project* as the primary tool for managing *The Falco Project* roadmap
### Non-Goals
- Providing an exact set of criteria for task prioritization
- Detailing testing procedures
- Providing detailed instructions for GitHub Project usage
- Addressing hotfix releases
### Scope of this Proposal
Primarily, the roadmap targets the planning of Falco development and releases. However, given Falco's dependence on numerous components, it's inevitable that scheduling and planning activities span across multiple repositories. We anticipate that all [core repositories](https://github.com/falcosecurity/evolution#official) will be interconnected with the roadmap, making it comprehensive enough to incorporate items from all related [Falcosecurity repositories](https://github.com/falcosecurity) as necessary.
This proposal does **not apply to hotfix releases** that may happen whenever needed at the maintainers' discretion.
## Release Cycles and Development Iterations
Falco releases happen 3 times per year. Each release cycle completes, respectively, by the end of January, May, and September.
A **release cycle is a 16-week time frame** between two subsequent releases.
Using this schema, in a 52-week calendar year, we allocate 48 weeks for scheduled activities (16 weeks *x* 3 releases), leaving 4 weeks for breaks.
The 16-week release cycle is further divided into three distinct iterations:
| Release Preparation | 4 weeks | Release preparation, testing, bug fixing, no new feature |
### Targeted Release Date
The final week of the *Release Preparation* should conclude before the *last Monday of the release month* (ie. January/May/September). This *last Monday* is designated as the **targeted release date** (when the release is being published), and the remaining part of the week is considered a break period.
### Milestones
For each release, we create a [GitHub Milestone](https://github.com/falcosecurity/falco/milestones) (whose due date must be equal to the target release date). We use the milestone to collect all items to be tentatively completed within the release.
### Alignment of Falco Components
The release schedule of the [components Falco depends on](https://github.com/falcosecurity/falco/blob/master/RELEASE.md#falco-components-versioning) needs to be synchronized to conform to these stipulations. For instance, a [falcosecurity/libs](https://github.com/falcosecurity/libs) release may be required at least one week prior to the termination of each iteration.
The maintainers are responsible for adapting those components' release schedules and procedures to release cycles and development iterations of Falco. Furthermore, all release processes must be documented and provide clear expectations regarding release dates.
## Project Roadmap
We use the [GitHub Project called *Falco Roadmap*](https://github.com/orgs/falcosecurity/projects/5) to plan and track the progress of each release cycle. The GitHub Project needs to be configured with the above mentioned iterations and break periods, compiled with actual dates. It's recommended to preconfigure the GitHub Project to accommodate the current plus the following three release cycles.
### Roadmap Planning
The roadmap serves as a strategic planning tool that outlines the goals and objectives for Falco. Its purpose is to visually represent the overall direction and timeline, enhance transparency and engage the community.
The onus is on the [Core Maintainers](https://github.com/falcosecurity/evolution/blob/main/GOVERNANCE.md#core-maintainers) to manage the roadmap. In this regard, Core Maintainers meet in **planning sessions on the first week of each calendar month**.
During these planning sessions, tasks are allocated to the current iteration or postponed to one of the following iterations. The assigned iteration indicates the projected completion date for a particular workstream.
When a session matches with the commencement of an iteration, maintainers convene to assess the planning and prioritize tasks for the iteration. The first planning session of a release cycle must define top priorities for the related release.
## Testing and Quality Assurance (QA)
Each iteration's output must include at least one Falco pre-release (or a viable development build) designated for testing and QA activities. While it's acceptable for these builds to contain unfinished features or known bugs, they must enable any community member to contribute to the testing and QA efforts.
The targeted schedule for these Testing/QA activities should be the **last week of each iteration** (or earlier during the *Release Preparation*).
Testing and Quality Assurance criteria and procedures must be defined and documented across relevant repositories.
Furthermore, given the strong reliance of Falco on [falcosecurity/libs](https://github.com/falcosecurity/libs), the above-mentioned pre-release/build for Testing/QA purposes must be based on the most recent *libs* development for the intended iteration. This means that during each interaction, a *libs* release (either pre or stable) must happen early enough to be used for this purpose.
## Next Steps and Conclusions
The Falco 0.36 release cycle, running from June to September 2023, will mark the initiation of the new process. This cycle will also serve as an experimental phase for refining the process.
Furthermore, as soon as possible, we will kick off a Working Group specifically to ensure smooth execution. This group will involve community members in assisting maintainers with roadmap management. It will provide curated feature suggestions for the roadmap, informed by community needs. This approach would facilitate the core maintainers' decisions, as they would mostly need just to review and adopt these pre-vetted recommendations, enhancing efficiency.
The Working Group's responsibilities will include (non-exhaustive list):
- Address input from the [2023-04-27 Core Maintainers meeting](https://github.com/falcosecurity/community/blob/main/meeting-notes/2023-04-27-Falco-Roadmap-Discussion.md)
- Sorting and reviewing pending issues to identify key topics for discussion and potential inclusion in the roadmap
- Establishing protocols not explicitly covered in this document
- Updating the documentation accordingly
- Supporting Core Maintainers in managing the [Falco Roadmap GitHub project](https://github.com/orgs/falcosecurity/projects/5)
- Gathering suggestions from all involved stakeholders to put forward potential enhancements
Finally, we anticipate the need for minor adjustments, which will become apparent only after an initial period of experimentation. Thus we have to intend this process to be flexible enough to adapt to emerging needs and improvements as long as the fundamental spirit of this proposal is upheld.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
