* address reviewers feedback
* improve clarity around new -A and -i behavior
* additional cleanup (e.g. use generic set operations only)
* extend unit tests
Note: sinsp ppm sc API is undergoing a refactor, therefore current lookups are interim
and will subsequently be refactored as well.
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Define new -A behavior in configure_interesting_sets
* default: all syscalls in rules included, sinsp state enforcement without high volume I/O syscalls
* -A flag set: all syscalls in rules included, sinsp state enforcement and allowing high volume I/O syscalls
Co-authored-by: Federico Di Pierro <nierro92@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
In the copy constructor and assignment operator for falco_source, also
copy the ruleset along with factories/name.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that ASTs contain parse positions, use them when reporting errors
about unknown macros.
When doing the first pass to find all macro references, save macros as
a map<macro name,parse position> instead of a set<macro name>. While
making that change, change the visitor struct to use references
instead of pointers.
In the second pass, when reporting any unresolved macro references,
also report the parse position.
The unit tests also check that the positions of macros are properly
returned in the resolved/unresolved maps.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that ASTs have parse positions and the compiler will return the
position of the last error, use that in falco rules to return errors
within condition strings instead of reporting the position as the
beginning of the condition.
This led to a change in the filter_ruleset interface--now, an ast is
compiled to a filter before being passed to the filter_ruleset
object. That avoids polluting the interface with a lot of details
about rule_loader contexts, errors, etc. The ast is still provided in
case the filter_ruleset wants to do indexing/analysis of the filter.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The optimization in https://github.com/falcosecurity/falco/pull/2210
had a bug when the engine uses multiple sources at the same
time--m_syscall_source is a pointer to an entry in the indexed vector
m_sources, but if add_source is called multiple times, the vector is
resized, which copies the structs but invalidates any pointer to the
vector entries.
So instead of caching m_syscall_source in add_source(), cache it in
process_events(). m_sources won't change once processing events starts.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
`open_params` is read from the falco YAML configuration file and parsed using Go's URL.
For example:
c349be6e84/plugins/k8saudit/pkg/k8saudit/source.go (L41-L42)
Go's URL parser does not handle whitespace, so if a user defines the `open_params` in the falco configuration file as follows
```yaml
open_params: >
/file/path
```
the parser returns an error. To avoid this, we now trim this parameter so no whitespace will be left for Go's URL parser to error out on.
For reference see #2262.
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
improve also some logs for `m_syscall_buf_size_preset` configuration errors
Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Adapted API to sinsp::open API break, and simple consumer API break.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Co-authored-by: Andrea Terzolo <andrea.terzolo@polito.it>
Instead of using a falco_rule struct on the stack, use a single value
inside the falco_source struct. It's mutable as find_source returns a
const struct.
At very high event volumes (> 1M syscalls/second), even the tiny time
it takes to create/destroy the struct starts to add up, and this
switch has some small cpu savings.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
When doing some testing of falco on very high event volumes (> 1.5M
events/second), I found that the time taken to look up a falco_source
struct had a non-negligible contribution to cpu usage.
So instead of looking up the source from the source_idx every time,
separately save the source for syscalls in the falco_engine object
directly. The separately saved copy is only used once someone calls
add_source with source="syscall".
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
According to the FHS 3.0 (https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s15.html), transient UNIX-domain sockets should be placed under the directory /run, so this commit updates the implicit value generated by the application.
Signed-off-by: Vicente J. Jiménez Miras <vjjmiras@gmail.com>
Now, the action takes care of inizializing all app inspectors
(just one in capture mode, one for each evt source in live mode), and of
registering and initializing all loaded plugins in the right inspector as needed.
The plugin initialization logic, which also involves the filtercheck list
population and checks, was moved and refactored from the previous
implementation of the load_plugins action.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Now, the action is in charge of loading all plugins and initializing:
- the offline inspector
- the list of loaded event sources
- the list of loaded plugins and their config
After this action runs, plugins are loaded but not yet initialized.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Use an enum instead of a string for the item_type aka "parts of a
rules file" field of contexts.
The set of values is mostly defined by the contexts that were already
created. There are a couple of forward-looking values for rule
outputs/macro conditions/etc. that may be useful for later.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
In #2098 and #2158, we reworked how rules loading errors/warnings were
returned to provide a richer set of information, including
locations/context for the errors/warnings.
That did *not* include locations within condition expressions,
though. When parsing a condition expression resulted in a
warning/error, the location simply pointed to the condition property
of the rule.
This commit improves this to handle parse errors:
- When libsinsp::filter::parser::parse() throws an exception, use
get_pos() to get the position within the condition string.
- Add a new context() constructor that takes a filter pos_info instead
of a YAML::Mark.
Now that positions aren't always related to the location of yaml
nodes, Make up a generic "position" struct for locations and convert
YAML::Mark and parser positions to a position struct.
Also allow a context to contain an alternate content string which is
used to build the snippet. For contexts related to condition strings,
the content is the condition.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This new model uses an async worker and a concurrent queue to handle
stats writing. This ensures better performance, because the live event
processing loop will just need to do a push on the queue instead of writing
to a file (only when the timer triggers), and should be thread-safe by design.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
In outputs it could be confusing to see a line:
<filename>: Ok
followed by a set of warnings.
To differentiate this, add a top level status "Ok, with warnings" when
rule loading was successful but had warnings.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The methods that throw exceptions from stringified results need to
additionally pass a rules_contents_t struct. This also meant that they
need to call the filename + content version of load_rules.
To avoid some duplicate code between the two load_rules_file methods,
move the work of opening the file into a private method
read_file(). It can throw an exception, which is passed through for
the void return method and caught + converted into a load_result error
for the method that returns a load_result.
Also, to avoid duplicate code between the void load_rules and
load_rules_file methods, add a private method interpret_load_result()
which throws an exception if the result has an error and prints
warnings otherwise if verbose is true.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Application changes to support multiple files when stringifying rules
results:
- In both validate_rules_files and load_rules_files, instead of
loading each file individually and then calling load_rules(), add a
separate step that loads all the files at once. The actual rules
content strings are held in a vector. The map from filename to
content (reference) points to entries in that vector.
- Both actions do the same work for this step, so put the
implementation in a shared application template method read_files
that works on iterators. It uses itertors because the load filenames
are a list and the validate filenames are a vector.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The old version of rules_result assumed that all errors/warnings were
related to a single file. That was generally correct for errors, as
rules parsing always stopped at the first error, so there is only one
relevant file.
However, for warnings that was not the case. When reading multiple
files A and B, you might get a warning from file A *only* after
reading file B. For example, B might redefine a rule in such a way
that you could get unused list/macro warnings from file A.
To properly address this, make some changes to how contexts are
managed:
- Instead of creating snippets at the time the error/warning was
generated, create snippets at the time the error/warning is
converted into a string. This requires passing all rules contents to
as_string()/as_json(), so define a
falco::load_result::rules_contents_t map from filename to rules
content (reference) and pass it in as_string/as_json(). Snippets are
now generated from the rules content matching the filename in the
context.
- When creating warnings/errors, there's no need to pass along the
rules content. This is only used when converting an error into a
string/json.
Also change snippet() to handle potentially very long lines. Instead
of always printing the entire line matching a location, print up to
snippet_width(param, with default 160 chars)/2 characters surrounding
the column from the location.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The latest released falco always prints full details on errors when
used with -r (read rules)/-V (validate rules). However #2098 changed
this to only print full details when verbose is true.
Fix the regression by always printing errors when loading
rules. Warnings will be printed only with -v.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the load_rules_files and validate_rules_files actions to use
the new falco_engine methods that return a rules result struct. The
app action interface is the same, returning ::fatal on error,
ok()/exit() otherwise. The difference is how any warnings/errors are
obtained--from the struct instead of an exception.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Changes to the rule loader to support result objects:
- Instead of throwing falco_exception on internal error, throw a
rule_load_exception instead, which contains distinct
error/message/context information.
- A context object contains a chain of location structs chaining from
the document root to the object where the error occurred. Each
location has a file position (as a YAML::Mark), an item
type (e.g. "rule", "list", "exception"), and an item name (e.g. "Write
Below Etc"). This will allow showing the exact location of an
error (e.g. list item/exception field) while also remembering the item
that contained it.
- All the _info structs now contain a context so errors that occur
after yaml parsing can still point to the original location in the
yaml file.
- rule_loader::result is an implementation of the abstract class
defined in falco_load_result. The implementation keeps track of a
list of errors/warnigns that used to be in the configuration object,
- Clean up compile_ methods to just throw rule_load_exceptions or
return nothing, and ensure that all rule_load_exceptions are caught in
compile(). When caught, errors are added to the result object.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Modify rule reader to use a result struct for errors and warnings:
- Instead of throwing a falco_exception to pass back errors, use a
rule_load_exception, which contains distinct error codes, messages,
and a context that points to the location of the error.
- The static method context_yaml_get_context() has moved to a method
of the rule_loader context object + the result as_string() method.
- As objects are parsed, create relevant context objects as reading
drills down into the contents of a rule/list/exception. This will
enable for specific errors in, say, the middle of an exception/list
while remembering the object that contains it.
- Modify decode_val()/decode_seq() to always return errors as
exceptions. Previously, it was a mix of a bool return + some
exceptions.
- decode_val()/decode_seq() are now more consistent about returning
distinct errors for the property not existing, the property existing
but not being a scalar/sequence value, and not being convertable to
the templated value.
- Combine the two nearly identical decode_seq() functions into a
single one that uses a lambda to perform the final step of adding to
the vector/set.
- There are some item-specific decode_xxx functions for specific
item properties like tags, exceptions fields/comps/values, etc.
that call the above functions.
These changes simplify the calls to decode_seq()/decode_val() as they
don't need to add their own errors when returning false. Also some
calls weren't checking the return value.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Now that we have a result struct and set of warning codes, change the
filter_warning_resolver to use them. This involves populating a set of
warning codes instead of strings.
Also, the methods to format warnings into human-readable strings is
now in the falco_load_result static methods, so move the text there
and remove the methods here.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add new load_rules methods that return a result object instead of
throwing exceptions on error. The existing load_rules methods call the
new methods internally and continue to throw exceptions on
error/return individual values on success.
The result is returned as a unique_ptr so it can be populated while
loading rules (as a part of the configuration object) and then move()d
to the return value.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Define a falco_load_result abstract class for use in new load_rules
methods. It's abstract so the implementation details in
rule_loader/rule_reader can be hidden from someone who wants to use
the API to load rules and work with a result.
The class defines a set of error codes/warning codes and has static
methods to get a short and long description of each error/warning.
There are virtual methods to access the important parts of a result:
- successful or not
- a string representation of the result, suitable for display to
users. Takes a verbose argument. When verbose is true, the string is
multi-line and has full details, including locations, item names,
etc. When verbose is false, the string is single-line and just
returns error codes.
- a json representation of the result, suitable for automated
parsing/interpretation later.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The only use of it was to include in --support output, which is
redundant as the support output already includes the full contents of
each rules file.
Additionally, it wasn't even being updated after the switch from lua
rules loading to c++ rules
loading (https://github.com/falcosecurity/falco/pull/1966/ or
surrounding PRs).
This will simplify follow-on changes to add a real "result" to rules
loading methods, as there will be fewer API variants to support.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Signed-off-by: Luca Guerra <luca@guerra.sh>
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
Perform the validate_rules action before the load_rules action. This
ensures that *only* the rules files named with -V arguments are
validated.
This fixes https://github.com/falcosecurity/falco/issues/2087.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This updates the engine to comply and work properly with the newly-introduced
interface design.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The filter_ruleset interface its implementation evt_type_index_ruleset
have been modified as follows:
- Only keep track of ruleset ids and not names. The falco engine will take
care of mapping easy-to-remember ruleset names to ruleset ids.
To emphasize this, use ruleset_id everywhere and not ruleset.
Also, make it non-optional.
- Have explicit separate functions to enable/disable rules, instead of a single enable() method combined with a boolean flag.
This does *not* change the falco_engine interface, which has
similar methods, to avoid breaking API changes.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Co-authored-by: Mark Stemm <mark.stemm@gmail.com>
This also fixes a couple of bugs. With the current implementation, the multi-ruleset feature is broken with multiple sources.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This interface will allow us to use different ruleset implementations inside the same engine.
The goal is to define API boundaries that will allow swapping the current evttype-index
ruleset implementation more easily. Key benefits include: smaller component with less responsibilities,
easier substituibility, more testable design, opportunity to adopt different index strategies
depending on the ruleset implementation.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This means that when starting Falco passing to it a folder for its rules, it will properly manage
changes to any file inside the folders, plus any created/deleted file inside it.
Unified list of rules parsing, instead of having it done twice inside cmdline_options and configuration.
Instead, it is done only once, inside load_rules_files.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
When adding an implied "in" comparison to an exception using the
single value form, add it to item, not items.
This fixes#1984.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This include making a coherent use of const, remove private inheritance, and adding virtual destructors.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This involves moving the code in falco_init() into individual files
below app_actions/. falco_init() simply calls app.run() now. When
app.run() returns false, print any erorr. When app.run() sets restart
to true, falco_init() is called again.
app.run() is still inside a catch block to catch any uncaught
exception.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Each file below app_actions/ defines some of the methods declared in
falco::app::application.
Any state that needs to be shared betweeen methods, or between the run
and teardown methods, resides in falco::app::application::state(), so
the moved code stays pretty much as-is, other than replacing stack
variables with member variables in app_state.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Changes to the falco::app::application object to support actions:
- All of the code that was in falco_init is now in methods of
application. (A later commit actually moves the code from falco_init
and into the split-up methods, this commit just declares them).
- Methods return an application::run_result object, which is a tuple
of success/errstr/proceed. proceed=false is used to short circuit
calling methods (think --help, --list, --support, etc.)
- application now has a run() method which runs the methods in an order
that honors any implicit dependencies (e.g. you have to init an
inspector before you open it, you have to do almost everything else
before processing events, etc.)
- There are a few methods devoted to teardown, they are always called
after the group of run methods are called.
- State that needs to be saved between methods, or saved between the
run and teardown functions, is in a
falco::app::application::state object.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Some objects used by falco (falco outputs, falco_formats, etc) were
using raw pointer references, which isn't great.
So convert use of raw pointers (originally passed from falco_init or
functions it called) with shared_ptr, as they are now held in
application state.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The first warnings we support involve the unsafe comparisons with <NA>, which were present
in the legacy regression tests for PSPs.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
At the same time, this also simplifies the unit test cases by using the SCENARIO construct of catch2,
which allows sharing a setup phases between different unit tests, and removes a bunch of repeated LOC in our case.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Once all rule files have been loaded, and all the rules have been compiled into filters and inserted in the engine rulesets, the loader definitions are maintained in memory without really being used. This commit adds a convenience method to clear the loader state and free-up some memory when engine consumers do not require such information in memory anymore.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The rule_reader class is responsible of parsing the YAML ruleset text and of using the rule_loader
to store the new definition in the internal state. This is a first step towards separating the YAML
reading logic from the rule parsing one. Potentially, this will allow us to read rulesets from another
YAML library or from something different than YAML files too.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The rule_loader is now simply responsible of collecting list/macro/rule definitions and then compiling them as falco_rules. The ruleset file reading code will be moved to another class
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a porting of what we had inside the Lua codebase. This now handles the single responsibility
of gathering stats about rule-event matching, and of formatting them to print them to the user.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The function implementation was removed, however it was still defined in the .h header. Moreover,
this will now be required in order to replace its lua equivalent.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
falco_engine::process_event gets called for every inspector event.
Profiling showed that std::map::find takes about 10% of
falco_engine::process_event, and that can easily improved by accessing
the source by index.
Signed-off-by: Angelo Puglisi <angelopuglisi86@gmail.com>
Both the parser.lua and compiler.lua modules are not necessary anymore, because all the logic related
to filter parsing and compilation is handled inside libsinsp now. Accordingly, they have been removed from
the lua-to-cpp.sh scripts. README.md and parse-smoke.sh have been removed since they are not needed anymore:
lpeg is not used by the project, and the smoke tests are implemented in libsisnsp unit test suite.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The lua_filter_helper class is a simple Lua wrapper that can be used in the Lua rule loader to
parse/compile rule filters, and manipulate them to resolve/replace list and macro references.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The Lua parser grabbed from libs chisels is not used anymore, as the compilation logic happen inside the new
filter parser of libsinsp.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a first step towards porting the rule filter building logic that is currently implemented in Lua.
filter_macro_resolver uses the newly introduced AST constructs from libsinsp, and
allow manipulating filter ASTs to resolve/replace macro references. This is meant to be used
at boot time by the rule loader (which we still want to maintain implemented in Lua for now).
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
The Lua PEG parser is not longer needed, since we now use the new filter parser implemented
in libsinsp.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Also skip macros with unknown sources. This matters primarily for
macros related to plugins that have a distinct event source.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
If a rule has an unknown source, *and* has exceptions, loading the
rule will result in an error and not skipping the rule. This is
because exceptions are also validated for unknown fields, and that
occurs before the current check for unknown sources.
The fix is to move the check for unknown sources as soon as the rules
object is read.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update falco's main falco_init() to use a falco::app::application and
falco::app::cmdline_opts object instead of storing all its command
line state in stack variables.
The bulk of the removed code is in usage() (not needed as cxxopt's
help() is self-documenting.) and getopt_long() which is replaced by
app.init(argc, argv).
For the most part, this is simply replacing references to local
variables (e.g. "all_events") to the bound variable inside the
cmdline_opts object (e.g. app.copts().all_events).
There are a few cases where more complex logic was used (output
formats, initializing k8s/mesos with string pointers), and those
changes are still in falco_init().
For the most part, the monolithic parts of falco_init that involve
reading config files, creating the inspector, loading rules, etc are
still present. Those will be addressed in later changes.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
For the most part, replacing getopt() with cxxopts + falco application
had no effect on falco engine/config interfaces. However, there were a
few places where it was wasier to change the interface than add
middleware code that transformed from, for example, vectors to lists.
This commit has those changes.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Fill in an initial falco::app::cmdline_options class using cxxopts
library to hold options:
- falco::app::cmdline_options contains a cxxopts::Options object to
parse options and a cxxopts::ParseResult to hold the result.
- The only meaningful public method is parse() which parses argc/argv
and returns true/false + error.
- The parsed options are all public instance variables of the object
and generally use the same names of the corresponding variables in
the old falco_init(). These variables are all bound to the
corresponding command line option and are updated in parse().
- In a few cases, the command line option does not directly map to a
bound variable (e.g. -b to set buffer format, -p/-pk/-pc to set
extra formatting options, etc.) In these cases the option values are
read after parsing and update the public instance variable.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a notion of a falco application object. Eventually this will
replace the bulk of falco_init and contain methods to:
- Parse/validate command line options
- Parse/validate falco config
- Initialize prerequsites (inspector, falco engine, webserver, etc)
- Load plugins
- Load/validate rules
- Command/subcommand execution (e.g. --list/--list-fields, or
nothing specified to run "main" loop)
For now, it is only responsible for command line options handling,
which is stubbed out.
Currently, the only public methods are init() to initialize everything
and copts() to access command line options.
Command line options are held in a different class
falco::app::cmdline_opts. application::copts() returns a reference to
that object, which allows access to parsed command line options bound
to various public instance variables.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This workaround an issue in libs, targeting Falco 0.31.0.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Jason Dellaluce <jasondellaluce@gmail.com>
When listing fields with -N (names only), also skip fields with the
EPF_TABLE_ONLY flag. (Skipping fields without -N is handled in libs,
in the as_string() method).
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This fixes the parser introduced in https://github.com/falcosecurity/falco/pull/1792.
Now, nested fields such as `arr[1].subval` are supported, whereas the parser used
to recognize the `.` as an unexpected character.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Instead of having .lua files external to the program responsible for
loading rules, embed the contents of those files into the executable
and load them as strings instead of as files:
Add a cmake custom command below userspace/engine/lua that calls a
bash script lua-to-cpp.sh to generate falco_engine_lua_files.{cpp,hh}
that are compiled into the falco engine library.
The script creates a .cpp file that has const char * symbols for each
file, as well as lists of files that should be loaded when the falco
engine is loaded. There are actually two lists:
- lua_module_strings: these are loaded and also added to the lua
runtime package.preload table, so they are available when lua code
require()s them.
- lua_code_strings: these are loaded *and* evaluated, so the functions
in them are availble to be called from C++.
This simplifies some of the falco_common methods, as there's no need
to keep track of a "main" lua file to load or paths from which the lua
loader should find files for modules, and there's no need to keep
track of an "alternate" lua directory that occurs for debug builds.
Also, there's no need to include any .lua files in the installed
packages, as they're built into the falco binary.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This will distinguish it from rule_loader.lua, which is *not* a module
but lua code with functions that can be called directly.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update json_event_filter_factory::get_fields() to add the new
info (shortdesc, data_type, tags) to field descriptions.
This allows for richer outputs when printing info on the fields.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
With the new implementation of list_fields(), the order of fields
changed slightly. So update the checksum.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Instead of having a falco-specific function to print field info, use
the built-in filter_fieldclass_info::as_string() instead. This is a
better implementation (displays addl info, has better wrapping, wider
output) and having a single implementation allows for consistent
outputs between falco and other potential programs that could use the libs.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Since now, the maximum depth supported to access config fields is two.
This adds support for accessing fields of arbitrary nesting depth.
A formal grammar has been explicited for the regular language representing
the field keys. The accessor methods have been updated accordingly.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
This is a change of direction from the current design, that imposes loading
the configuration from file only, and in the object constructor. Instead,
yaml_configuration objects can now be reused ad can load the YAML config
from either file or string. This also makes it easier to unit test this class.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>
Return warnings after validation of rule exceptions
Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>
Update FALCO_ENGINE_VERSION
Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>
This allows defining rules that simply enable/disable already defined rules, like the following:
- rule: A rule enabled by default
enabled: false
- rule: A rule disabled by default
enabled: true
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
When MUSL_OPTIMIZED_BUILD is specified, falco is statically linked under
musl, and can't dlopen() files: see
https://inbox.vuxu.org/musl/20200423162406.GV11469@brightrain.aerifal.cx/T/
So skip listing/loading/testing plugins when MUSL_OPTIMIZED_BUILD is specified.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the falco binary to add support for plugins.
- Keep track of an "event source", which is initially "syscall" but
changes to the input plugin's source if an source plugin ends up being
loaded.
- New argument --list-plugins will return info on any loaded plugins,
using sinsp_plugin::plugin_infos.
- Create filter/formatter factories for plugins. This ensures that
filterchecks for syscalls are not used for plugins and vice versa.
- Use sinsp_plugin::register_plugin() to load each plugin found in
config. The first source plugin found (if any) calls
engine->add_source withthe source plugin's event source.
- If a second source plugin is found, exit with an error.
- Extractor plugins must be compatible with the event source (usually
the plugin event source, but could be "syscall"). If not, exit with
an error.
- Multiple Extractor plugins are allowed, but they can not have
overlapping compatible event sources. This is mostly to avoid
confusion, but we might change this later.
- After loading plugins, use engine is_plugin_compatible to ensure
that the plugin is compatible with any required_plugin_version blocks
in falco rules.
- Normally falco would log warnings if too many SCAP_TIMEOUT results
were received. These are more expected when using plugins, so only
log these warnings when using syscalls.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The generic events support already handled most of this, with a
dedicated formatter factory for plugin sources. Just one missing
header include and change the logic slightly for json parsing.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Rules loading changes for plugins:
- parse required_engine_versions from yaml and pass up to rules
loader as a lua table as an additional return value from load_rules().
- c++ rules loader converts to map: plugin -> list of required plugin
versions
- support is_source_valid callback from lua, calls engine method. If
a source is not valid, skip any rules for that source and add a warning.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Mostly plugins are just handled as a new filter/formatter factory with
a new source based on the loaded input plugin, but there are a few
changes at the engine level:
- is_source_valid returns whether a filter/formatter factory exists
for a given source. Will be used by rules loaded to skip rules for
an unknown source.
- the falco engine now holds the required_plugin_version predicates
found in rules files and a method is_plugin_compatible returns whether
a plugin semver is compatible with the predicates in the rules
- Update the falco engine version and fields checksum for plugins
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a cmake module "plugins" that does the following:
- Downloads/installs the plugins artifacts from a known tag
- Copies the resulting cloudtrail/json shared libraries to
CMAKE_CURRENT_BINARY_DIR/plugins
- Installs them to FALCO_SHARE_DIR/plugins
The default config will define the plugins but they will be disabled
by default.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
It took a while, but we remembered to finish moving the token_bucket
from falco engine to libs. There were 2 copies for a while.
This brings over one change to libs--to have an optional timer
function.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update config code/default falco.yaml to add support for plugins:
- Update config parsing methods to support reading plugin config
objects in a list from yaml.
- The default config defines the cloudtrail/json plugins but does not
give them any actual config for init config/open
params (cloudtrail), or init config (json).
- load_plugins is empty so neither plugin is actually loaded by default.
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>-
This makes the output of --list a bit more precise to only include
filter fields and not output fields.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
This step used to be done in the lua rule loading code, but now we can
get it directly from the filters, so do it in falco instead.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the lua side of rule loading to reflect other changes:
- install_filter renamed to create_filter_obj, and takes just a
lua_parser object created via falco_rules.create_lua_parser() and
uses a single lua callback "filter" instead of separate ones for
syscall/k8s_audit. It can return an error, including about
undefined fields
- is_defined_filter, which used to be local and based on the result of
sinsp_rule_utils.check_for_ignored_syscalls_events, is now a
lua_callback falco_rules.is_defined_field().
- Don't need to pass down sinsp_lua_parser/json_lua_parser now,
creating filters is handled via lua callbacks.
- Checking for ignored syscalls/events is now done in falco itself,
after loading rules.
- add_xxx_filter replaced by add_filter + source.
- Use is_format_valid instead of formats.formatter/formats.free_formatter.
- We don't need the functions in sinsp_rule_utils any longer, so
remove the file and don't import it.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Add a function is_defined_field(source, fldname) that returns whether
a field with name fldname exists for the given event source. This uses
the filter factory to create a filtercheck, and returns true if an
object was created.
This prevents having to push down the entire set of defined fields
before calling load_rules().
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Use the new falco engine interface with support for generic events
instead of event-specific process_xxx_event methods.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
