Commit Graph

1334 Commits

Author SHA1 Message Date
Federico Di Pierro
1a485c3447 update(userspace/engine,userspace/falco): improved some string warnings.
Always print warnings while loading rules.
Print a single line when warning for ignored events.

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2022-01-24 17:52:31 +01:00
yoshi314
a9e7512936 fix setting the variable of User-Agent, it was missing the prefix. Switched to dedicated curl's method to do this
Signed-off-by: Marcin Kowalski <marcin.kowalski@assecobs.pl>
2022-01-18 09:49:34 +01:00
Marcin Kowalski
f67e8bdad7 fix indentation in outputs_http.cpp
add sample config entry for user-agent variable

Signed-off-by: Marcin Kowalski <marcin.kowalski@assecobs.pl>
2022-01-18 09:49:34 +01:00
Marcin Kowalski
a94e6de458 add useragent string to output
Signed-off-by: Marcin Kowalski <marcin.kowalski@assecobs.pl>
2022-01-18 09:49:34 +01:00
Leonardo Grasso
3e9f8c1ef1 chore(userpsace/engine): update fields checksum
Signed-off-by: Leonardo Grasso <me@leonardograsso.com>
2022-01-17 18:15:43 +01:00
Mark Stemm
d20a326e09 Skip EPF_TABLE_ONLY fields with --list -N
When listing fields with -N (names only), also skip fields with the
EPF_TABLE_ONLY flag. (Skipping fields without -N is handled in libs,
in the as_string() method).

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-17 18:15:43 +01:00
Federico Di Pierro
ae57718bda update(build): updated libs to latest master version. Updated plugins versions. Updated falco engine version.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2022-01-17 17:20:33 +01:00
Jason Dellaluce
4ab8d6db98 refactor(configuration): remove plugin config loading from file feature
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-17 14:55:11 +01:00
Jason Dellaluce
5e354859a9 new(configuration): allow defining plugin config as YAML maps
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-17 14:55:11 +01:00
Jason Dellaluce
f4b79296fc fix: improve nested configuration field support
This fixes the parser introduced in https://github.com/falcosecurity/falco/pull/1792.
Now, nested fields such as `arr[1].subval` are supported, whereas the parser used
to recognize the `.` as an unexpected character.

Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-17 14:55:11 +01:00
Jason Dellaluce
6bf8f34d9f fix(engine): correctly format json output in json_event
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-14 13:29:33 +01:00
vadim.zyarko
f8f053c7fa Add an emty line to sattisfy the rules tests
Signed-off-by: vadim.zyarko <vadim.zyarko@sysdig.com>
2022-01-13 09:44:57 +01:00
VadimZy
b88a1cbb09 replace .. with table concat
Signed-off-by: vadim.zyarko <vadim.zyarko@sysdig.com>
2022-01-13 09:44:57 +01:00
Mark Stemm
c86615f68c Embed .lua files into falco executable
Instead of having .lua files external to the program responsible for
loading rules, embed the contents of those files into the executable
and load them as strings instead of as files:

Add a cmake custom command below userspace/engine/lua that calls a
bash script lua-to-cpp.sh to generate falco_engine_lua_files.{cpp,hh}
that are compiled into the falco engine library.

The script creates a .cpp file that has const char * symbols for each
file, as well as lists of files that should be loaded when the falco
engine is loaded. There are actually two lists:

- lua_module_strings: these are loaded and also added to the lua
  runtime package.preload table, so they are available when lua code
  require()s them.

- lua_code_strings: these are loaded *and* evaluated, so the functions
  in them are availble to be called from C++.

This simplifies some of the falco_common methods, as there's no need
to keep track of a "main" lua file to load or paths from which the lua
loader should find files for modules, and there's no need to keep
track of an "alternate" lua directory that occurs for debug builds.

Also, there's no need to include any .lua files in the installed
packages, as they're built into the falco binary.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-13 09:26:35 +01:00
Mark Stemm
10512b9ef9 Move compiler/parser lua files to a "modules" subdir
This will distinguish it from rule_loader.lua, which is *not* a module
but lua code with functions that can be called directly.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-13 09:26:35 +01:00
Jason Dellaluce
0e52ef9971 fix(grpc): ignore protobuf deprecation warning
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-12 00:16:49 +01:00
Jason Dellaluce
a371a995b4 update(outputs): adapt grpc output to new protobuf definitions
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-12 00:16:49 +01:00
Jason Dellaluce
0f984c4dbe update(grpc): substitute and deprecate enum source field from protobuf
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2022-01-12 00:16:49 +01:00
Federico Di Pierro
48a23121df new(userspace/falco): add support for kernel side simple consumer.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2022-01-10 10:58:44 +01:00
Federico Di Pierro
475ed0dbeb fix(userspace/engine,userspace/falco): set http output contenttype to text/plain when json output is disabled
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
2022-01-10 10:57:44 +01:00
Lorenzo Susini
cef2c2d5c1 chore: improve --list output using is_source_valid
Signed-off-by: Lorenzo Susini <susinilorenzo1@gmail.com>
2022-01-10 10:53:44 +01:00
Mark Stemm
455be15b0b Fill in new shortdesc/data_type/tags for json fields
Update json_event_filter_factory::get_fields() to add the new
info (shortdesc, data_type, tags) to field descriptions.

This allows for richer outputs when printing info on the fields.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-12-23 17:05:39 +01:00
Mark Stemm
64e8feb200 Update fields checksum (no changes, order only)
With the new implementation of list_fields(), the order of fields
changed slightly. So update the checksum.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-12-23 17:05:39 +01:00
Mark Stemm
eded1062cd Use filter_fieldclass_info::as_string to print field info
Instead of having a falco-specific function to print field info, use
the built-in filter_fieldclass_info::as_string() instead. This is a
better implementation (displays addl info, has better wrapping, wider
output) and having a single implementation allows for consistent
outputs between falco and other potential programs that could use the libs.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-12-23 17:05:39 +01:00
Federico Di Pierro
bb8b75a2cd update(userspace/falco): enforce check that content-type actually starts with "application/json" string.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
2021-12-09 21:04:47 +01:00
Federico Di Pierro
b359f71511 fix(userspace/falco): accept 'Content-Type' header that contains "application/json", but it is not strictly equal to it.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-12-09 21:04:47 +01:00
Federico Di Pierro
9dcd8bccac fix(userspace/falco): in case output_file cannot be opened, throw a falco exception.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-12-09 21:02:48 +01:00
Jason Dellaluce
c005af22cc fix: set config value and create node if not existing
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-12-06 19:04:15 +01:00
Jason Dellaluce
1a7611a761 chore(engine): using is_defined config method instead of private get_node
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-12-06 19:04:15 +01:00
Jason Dellaluce
7fb61ba4a3 refactor(engine): access config fields with new key syntax
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-12-06 19:04:15 +01:00
Jason Dellaluce
9ab810f431 update(engine): support accessing nested config fields
Since now, the maximum depth supported to access config fields is two.
This adds support for accessing fields of arbitrary nesting depth.
A formal grammar has been explicited for the regular language representing
the field keys. The accessor methods have been updated accordingly.

Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-12-06 19:04:15 +01:00
Jason Dellaluce
7781385769 refactor(engine): support string config loading and add ad-hoc methods
This is a change of direction from the current design, that imposes loading
the configuration from file only, and in the object constructor. Instead,
yaml_configuration objects can now be reused ad can load the YAML config
from either file or string. This also makes it easier to unit test this class.

Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-12-06 19:04:15 +01:00
Jason Dellaluce
85db078dc4 chore: renaming comment references
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
Co-authored-by: Federico Di Pierro <nierro92@gmail.com>
Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
2021-11-18 16:26:18 +01:00
sai-arigeli
23706da75e Allow append of new exceptions to rules
Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>

Return warnings after validation of rule exceptions

Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>

Update FALCO_ENGINE_VERSION

Signed-off-by: Sai Arigeli <saiharisharigeli@gmail.com>
2021-11-18 09:11:20 +01:00
Federico Di Pierro
8a603c3c5d update(build): latest libs correctly set OPENSSL_LIBRARIES for us.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-11-17 16:18:23 +01:00
Federico Di Pierro
5f1d04ec82 fix(build): build civetweb using cmake and linking to static openssl built by us.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-11-17 16:18:23 +01:00
Luca Guerra
09799e125d chore(build): update libs version to 7906f7ec416a8b67b82d92d37b25f28d545bcb8f
Signed-off-by: Luca Guerra <luca@guerra.sh>
2021-11-16 19:02:21 +01:00
Jason Dellaluce
eec2f5062f update(userspace/engine): allow overwriting rules with enabled flag only
This allows defining rules that simply enable/disable already defined rules, like the following:
- rule: A rule enabled by default
  enabled: false
- rule: A rule disabled by default
  enabled: true

Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-11-16 16:37:45 +01:00
Federico Di Pierro
7dcf8f4bf7 update(userspace/engine): use s_ prefix for static var.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Co-authored-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-16 15:34:12 +01:00
Federico Di Pierro
bea91ca844 fix(userspace/engine): properly constify m_default_ruleset.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-11-16 15:34:12 +01:00
Federico Di Pierro
ea2ca56d5b style(userspace/engine): avoid creating multiple versions of methods only to assume default ruleset. Use a default argument instead.
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
2021-11-16 15:34:12 +01:00
Mark Stemm
cb51522423 Skip plugins list/load/tests for MUSL_OPTIMIZED_BUILD
When MUSL_OPTIMIZED_BUILD is specified, falco is statically linked under
musl, and can't dlopen() files: see
https://inbox.vuxu.org/musl/20200423162406.GV11469@brightrain.aerifal.cx/T/

So skip listing/loading/testing plugins when MUSL_OPTIMIZED_BUILD is specified.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
6a1f4f7374 Plugins support in falco executable
Update the falco binary to add support for plugins.

- Keep track of an "event source", which is initially "syscall" but
  changes to the input plugin's source if an source plugin ends up being
  loaded.

- New argument --list-plugins will return info on any loaded plugins,
  using sinsp_plugin::plugin_infos.

- Create filter/formatter factories for plugins. This ensures that
  filterchecks for syscalls are not used for plugins and vice versa.

- Use sinsp_plugin::register_plugin() to load each plugin found in
  config. The first source plugin found (if any) calls
  engine->add_source withthe source plugin's event source.

- If a second source plugin is found, exit with an error.

- Extractor plugins must be compatible with the event source (usually
  the plugin event source, but could be "syscall"). If not, exit with
  an error.

- Multiple Extractor plugins are allowed, but they can not have
  overlapping compatible event sources. This is mostly to avoid
  confusion, but we might change this later.

- After loading plugins, use engine is_plugin_compatible to ensure
  that the plugin is compatible with any required_plugin_version blocks
  in falco rules.

- Normally falco would log warnings if too many SCAP_TIMEOUT results
  were received. These are more expected when using plugins, so only
  log these warnings when using syscalls.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
98599d5e25 Plugins support (outputs)
The generic events support already handled most of this, with a
dedicated formatter factory for plugin sources. Just one missing
header include and change the logic slightly for json parsing.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
e7d41f8166 Rules loading changes for plugins
Rules loading changes for plugins:

 - parse required_engine_versions from yaml and pass up to rules
   loader as a lua table as an additional return value from load_rules().
 - c++ rules loader converts to map: plugin -> list of required plugin
   versions
 - support is_source_valid callback from lua, calls engine method. If
   a source is not valid, skip any rules for that source and add a warning.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
9075eea62f Falco engine support for plugins
Mostly plugins are just handled as a new filter/formatter factory with
a new source based on the loaded input plugin, but there are a few
changes at the engine level:

- is_source_valid returns whether a filter/formatter factory exists
  for a given source. Will be used by rules loaded to skip rules for
  an unknown source.

- the falco engine now holds the required_plugin_version predicates
  found in rules files and a method is_plugin_compatible returns whether
  a plugin semver is compatible with the predicates in the rules

- Update the falco engine version and fields checksum for plugins

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
38a7f7ada0 cmake/build changes for plugins
Add a cmake module "plugins" that does the following:

 - Downloads/installs the plugins artifacts from a known tag
 - Copies the resulting cloudtrail/json shared libraries to
   CMAKE_CURRENT_BINARY_DIR/plugins
 - Installs them to FALCO_SHARE_DIR/plugins

The default config will define the plugins but they will be disabled
by default.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
6a4e4eaa4f Finish moving token bucket impl from falco to libs
It took a while, but we remembered to finish moving the token_bucket
from falco engine to libs. There were 2 copies for a while.

This brings over one change to libs--to have an optional timer
function.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2021-11-12 18:27:59 +01:00
Mark Stemm
1313e77113 Falco yaml config for plugins
Update config code/default falco.yaml to add support for plugins:

- Update config parsing methods to support reading plugin config
  objects in a list from yaml.

- The default config defines the cloudtrail/json plugins but does not
  give them any actual config for init config/open
  params (cloudtrail), or init config (json).

- load_plugins is empty so neither plugin is actually loaded by default.

Co-authored-by: Leonardo Grasso <me@leonardograsso.com>
Co-authored-by: Loris Degioanni <loris@sysdig.com>
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>-
2021-11-12 18:27:59 +01:00
Jason Dellaluce
28d6a293fc update(userspace/engine): support jsonpointer escaping in rule parser
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
2021-11-11 18:36:21 +01:00