This new model uses an async worker and a concurrent queue to handle
stats writing. This ensures better performance, because the live event
processing loop will just need to do a push on the queue instead of writing
to a file (only when the timer triggers), and should be thread-safe by design.
Signed-off-by: Jason Dellaluce <jasondellaluce@gmail.com>
In outputs it could be confusing to see a line:
<filename>: Ok
followed by a set of warnings.
To differentiate this, add a top level status "Ok, with warnings" when
rule loading was successful but had warnings.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The methods that throw exceptions from stringified results need to
additionally pass a rules_contents_t struct. This also meant that they
need to call the filename + content version of load_rules.
To avoid some duplicate code between the two load_rules_file methods,
move the work of opening the file into a private method
read_file(). It can throw an exception, which is passed through for
the void return method and caught + converted into a load_result error
for the method that returns a load_result.
Also, to avoid duplicate code between the void load_rules and
load_rules_file methods, add a private method interpret_load_result()
which throws an exception if the result has an error and prints
warnings otherwise if verbose is true.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Application changes to support multiple files when stringifying rules
results:
- In both validate_rules_files and load_rules_files, instead of
loading each file individually and then calling load_rules(), add a
separate step that loads all the files at once. The actual rules
content strings are held in a vector. The map from filename to
content (reference) points to entries in that vector.
- Both actions do the same work for this step, so put the
implementation in a shared application template method read_files
that works on iterators. It uses itertors because the load filenames
are a list and the validate filenames are a vector.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The old version of rules_result assumed that all errors/warnings were
related to a single file. That was generally correct for errors, as
rules parsing always stopped at the first error, so there is only one
relevant file.
However, for warnings that was not the case. When reading multiple
files A and B, you might get a warning from file A *only* after
reading file B. For example, B might redefine a rule in such a way
that you could get unused list/macro warnings from file A.
To properly address this, make some changes to how contexts are
managed:
- Instead of creating snippets at the time the error/warning was
generated, create snippets at the time the error/warning is
converted into a string. This requires passing all rules contents to
as_string()/as_json(), so define a
falco::load_result::rules_contents_t map from filename to rules
content (reference) and pass it in as_string/as_json(). Snippets are
now generated from the rules content matching the filename in the
context.
- When creating warnings/errors, there's no need to pass along the
rules content. This is only used when converting an error into a
string/json.
Also change snippet() to handle potentially very long lines. Instead
of always printing the entire line matching a location, print up to
snippet_width(param, with default 160 chars)/2 characters surrounding
the column from the location.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
The latest released falco always prints full details on errors when
used with -r (read rules)/-V (validate rules). However #2098 changed
this to only print full details when verbose is true.
Fix the regression by always printing errors when loading
rules. Warnings will be printed only with -v.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Update the load_rules_files and validate_rules_files actions to use
the new falco_engine methods that return a rules result struct. The
app action interface is the same, returning ::fatal on error,
ok()/exit() otherwise. The difference is how any warnings/errors are
obtained--from the struct instead of an exception.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
Changes to the rule loader to support result objects:
- Instead of throwing falco_exception on internal error, throw a
rule_load_exception instead, which contains distinct
error/message/context information.
- A context object contains a chain of location structs chaining from
the document root to the object where the error occurred. Each
location has a file position (as a YAML::Mark), an item
type (e.g. "rule", "list", "exception"), and an item name (e.g. "Write
Below Etc"). This will allow showing the exact location of an
error (e.g. list item/exception field) while also remembering the item
that contained it.
- All the _info structs now contain a context so errors that occur
after yaml parsing can still point to the original location in the
yaml file.
- rule_loader::result is an implementation of the abstract class
defined in falco_load_result. The implementation keeps track of a
list of errors/warnigns that used to be in the configuration object,
- Clean up compile_ methods to just throw rule_load_exceptions or
return nothing, and ensure that all rule_load_exceptions are caught in
compile(). When caught, errors are added to the result object.
Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
