Compare commits

...

21 Commits

Author SHA1 Message Date
Mark Stemm
cdd4f51db0 Better fix for falco tests after rebase 2022-01-31 11:55:11 -08:00
Mark Stemm
1b112d752a Fixing falco tests after rebase 2022-01-31 11:49:20 -08:00
Mark Stemm
eb86768dfb Fixing falco bugs after rebase 2022-01-31 11:49:04 -08:00
Mark Stemm
b55df884ef falco_engine fix typos 2022-01-31 11:15:46 -08:00
Mark Stemm
debcb1e729 Update test output matches to match new aligned code
The prior falco changes made output printing more consistent by moving
it into a standlone function instead of scattered printf()s.

Some test cases relied on the (inconsistent) print strings, so update
them to the (hopefully more consistent) outputs used now.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-31 11:15:43 -08:00
Mark Stemm
9f88c7cbd0 Falco cleanups. this should pass all tests 2022-01-31 11:15:03 -08:00
Mark Stemm
946a431e55 swappable falco engine cleanups. this should pass all tests 2022-01-31 11:14:07 -08:00
Mark Stemm
7e37fc8210 Rules loading cleanups. This version should pass all tests 2022-01-31 11:14:07 -08:00
Mark Stemm
1ed2bec4d7 Remove temp debug logs (falco_engine) 2022-01-31 11:14:07 -08:00
Mark Stemm
cc4332c8ce More falco_engine cleanups. this should pass all tests 2022-01-31 11:14:04 -08:00
Mark Stemm
c648f2fcfd falco cleanups. this passes most tests 2022-01-31 11:07:33 -08:00
Mark Stemm
03d826d249 swappable falco engine cleanups. this passes mosts tests 2022-01-31 11:07:33 -08:00
Mark Stemm
83fe8d649a Rules loading cleanups. This passes mosts tests 2022-01-31 11:07:33 -08:00
Mark Stemm
4356307412 falco_engine cleanups. this passes most tests 2022-01-31 11:07:33 -08:00
Mark Stemm
d338185524 Move validation to after swengine init
A bigger reorg is probably in order, but this gets validation with -v
to work, at least
2022-01-31 11:07:32 -08:00
Mark Stemm
54dea70482 This version builds 2022-01-31 11:07:30 -08:00
Mark Stemm
08a67b77d6 This version builds 2022-01-31 11:06:29 -08:00
Mark Stemm
22e6205921 Add grpc methods to reload/validate rules files
Add grpc methods to reload/validate a set of rules files. This is only
stubs at the moment, but the implementation will consist of:

- creating a new falco engine
- doing any required initialization
- loading each rules file
- enabling/disabling rules based on command line options
- (for reload) using swappable_falco_engine::replace() to update the
  falco engine.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-31 11:06:29 -08:00
Mark Stemm
91ee079ea6 Use swappable_falco_engine to hold falco engine
Use an instance of swappable_falco_engine obj to hold the falco
engine.

This generally involves:

 - Passing around a reference to a swappable_falco_engine instead of a
   pointer to a falco_engine
 - Using swengine.engine() to access the current falco engine instead
   of the falco_engine pointer.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-31 11:06:25 -08:00
Mark Stemm
4cc05d6f4a Add a notion for a swappable falco engine
New class swappable_falco_engine contains a shared_ptr to a
falco_engine object and has two main methods:

 - engine(): retrieve the current shared pointer
 - replace(): update the engine with a new one

The implementation allows for replace() and engine() to occur on
different threads, using a tbb::concurrent_queue. replace() pushes
onto the queue(), and engine() pops from the queue, replacing the
current engine.

This will be used to replace the falco engine on the fly when
reloading rules on the fly via the grpc interface.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-31 11:03:33 -08:00
Mark Stemm
06b7427ede Move falco formats code out of engine
This isn't used by the engine itself anymore, now that it uses
factories to provide formatters.

This is in preparation for other changes to make the falco engine
hot-swappable while running.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
2022-01-31 11:03:33 -08:00
23 changed files with 897 additions and 314 deletions

View File

@@ -408,7 +408,8 @@ trace_files: !mux
invalid_overwrite_macro:
exit_status: 1
stdout_contains: |+
.*invalid_base_macro.yaml: Ok
.*invalid_base_macro.yaml: 1 warnings:
macro some macro not refered to by any rule/macro
.*invalid_overwrite_macro.yaml: 1 errors:
Compilation error when compiling "foo": Undefined macro 'foo' used in filter.
---
@@ -424,7 +425,8 @@ trace_files: !mux
invalid_append_macro:
exit_status: 1
stdout_contains: |+
.*invalid_base_macro.yaml: Ok
.*invalid_base_macro.yaml: 1 warnings:
macro some macro not refered to by any rule/macro
.*invalid_append_macro.yaml: 1 errors:
Compilation error when compiling "evt.type=execve foo": 17: syntax error, unexpected 'foo', expecting 'or', 'and'
---
@@ -1181,7 +1183,7 @@ trace_files: !mux
skip_unknown_error:
exit_status: 1
stderr_contains: |+
Could not load rules file.*skip_unknown_error.yaml: 1 errors:
When loading rules: 1 errors:
Rule Contains Unknown Event And Not Skipping: error filter_check called with nonexistent field proc.nobody
---
- rule: Contains Unknown Event And Not Skipping
@@ -1198,7 +1200,7 @@ trace_files: !mux
skip_unknown_unspec_error:
exit_status: 1
stderr_contains: |+
Could not load rules file .*skip_unknown_unspec.yaml: 1 errors:
When loading rules: 1 errors:
Rule Contains Unknown Event And Unspecified: error filter_check called with nonexistent field proc.nobody
---
- rule: Contains Unknown Event And Unspecified

View File

@@ -82,7 +82,9 @@ trace_files: !mux
incompat_plugin_rules_version:
exit_status: 1
stderr_contains: "Runtime error: Plugin cloudtrail version .* not compatible with required plugin version 100000.0.0. Exiting."
stderr_contains: |+
Runtime error: When loading rules: 1 errors:
Plugin cloudtrail version .* not compatible with required plugin version 100000.0.0
conf_file: BUILD_DIR/test/confs/plugins/cloudtrail_json_create_instances.yaml
rules_file:
- rules/plugins/cloudtrail_incompat_plugin_version.yaml

View File

@@ -18,8 +18,7 @@ set(FALCO_ENGINE_SOURCE_FILES
falco_engine.cpp
falco_utils.cpp
json_evt.cpp
ruleset.cpp
formats.cpp)
ruleset.cpp)
add_library(falco_engine STATIC ${FALCO_ENGINE_SOURCE_FILES})
add_dependencies(falco_engine njson lyaml lpeg string-view-lite)

View File

@@ -20,14 +20,11 @@ limitations under the License.
#include <fstream>
#include <sinsp.h>
#include <plugin.h>
#include "falco_engine.h"
#include "falco_utils.h"
#include "falco_engine_version.h"
#include "formats.h"
extern "C" {
#include "lpeg.h"
#include "lyaml.h"
@@ -55,8 +52,6 @@ falco_engine::falco_engine(bool seed_rng)
falco_common::init();
falco_rules::init(m_ls);
m_required_plugin_versions.clear();
if(seed_rng)
{
srandom((unsigned) getpid());
@@ -148,6 +143,11 @@ void falco_engine::list_fields(std::string &source, bool verbose, bool names_onl
}
}
void falco_engine::set_plugin_infos(std::list<sinsp_plugin::info> &plugin_infos)
{
m_plugin_infos = plugin_infos;
}
void falco_engine::load_rules(const string &rules_content, bool verbose, bool all_events)
{
uint64_t dummy;
@@ -157,18 +157,20 @@ void falco_engine::load_rules(const string &rules_content, bool verbose, bool al
void falco_engine::load_rules(const string &rules_content, bool verbose, bool all_events, uint64_t &required_engine_version)
{
if(!m_rules)
rulesfile rf;
rf.content = rules_content;
load_result res(rf);
load_rules(rf, verbose, all_events, res);
if(!res.successful)
{
m_rules.reset(new falco_rules(this,
m_ls));
bool include_filenames = false;
for(auto const &it : m_filter_factories)
{
m_rules->add_filter_factory(it.first, it.second);
}
// This version assembles the errors and/or warnings into an exception.
throw falco_exception(res.as_string(include_filenames, verbose));
}
m_rules->load_rules(rules_content, verbose, all_events, m_extra, m_replace_container_info, m_min_priority, required_engine_version, m_required_plugin_versions);
}
void falco_engine::load_rules_file(const string &rules_filename, bool verbose, bool all_events)
@@ -196,6 +198,105 @@ void falco_engine::load_rules_file(const string &rules_filename, bool verbose, b
load_rules(rules_content, verbose, all_events, required_engine_version);
}
falco_engine::rulesfile::rulesfile()
{
}
falco_engine::load_result::load_result(const rulesfile &rulesfile)
: rf(rulesfile)
{
}
falco_engine::load_result::~load_result()
{
}
std::string falco_engine::load_result::as_string(bool include_filenames, bool include_warnings)
{
std::ostringstream os;
if(include_filenames)
{
os << rf.name << ": ";
}
if((errors.size() + warnings.size()) == 0)
{
os << "Ok" << std::endl;
} else {
if (errors.size() > 0)
{
os << errors.size() << " errors:" << std::endl;
for(auto err : errors)
{
os << err << std::endl;
}
}
if (include_warnings && warnings.size() > 0)
{
os << warnings.size() << " warnings:" << std::endl;
for(auto warn : warnings)
{
os << warn << std::endl;
}
}
}
return os.str();
}
bool falco_engine::rulesfile::load(const std::string &filename, std::string &errstr)
{
std::ifstream is;
is.open(filename);
if (!is.is_open())
{
errstr = "Could not open rules filename " +
filename + " " + "for reading";
return false;
}
name = filename;
content.assign((istreambuf_iterator<char>(is)),
istreambuf_iterator<char>());
return true;
}
falco_engine::rulesfile::~rulesfile()
{
}
void falco_engine::load_rules(falco_engine::rulesfile &rf,
bool verbose, bool all_events,
falco_engine::load_result &result)
{
if(!m_rules)
{
m_rules.reset(new falco_rules(this,
m_ls));
for(auto const &it : m_filter_factories)
{
m_rules->add_filter_factory(it.first, it.second);
}
}
uint64_t required_engine_version;
std::map<std::string, std::list<std::string>> required_plugin_versions;
result.successful = m_rules->load_rules(rf.content, verbose, all_events,
m_extra, m_replace_container_info,
m_min_priority,
result.warnings,
result.errors,
required_engine_version,
m_plugin_infos);
}
void falco_engine::enable_rule(const string &substring, bool enabled, const string &ruleset)
{
uint16_t ruleset_id = find_ruleset_id(ruleset);
@@ -428,36 +529,6 @@ bool falco_engine::is_source_valid(const std::string &source)
return (m_rulesets.find(source) != m_rulesets.end());
}
bool falco_engine::is_plugin_compatible(const std::string &name,
const std::string &version,
std::string &required_version)
{
sinsp_plugin::version plugin_version(version);
if(!plugin_version.m_valid)
{
throw falco_exception(string("Plugin version string ") + version + " not valid");
}
if(m_required_plugin_versions.find(name) == m_required_plugin_versions.end())
{
// No required engine versions, so no restrictions. Compatible.
return true;
}
for(auto &rversion : m_required_plugin_versions[name])
{
sinsp_plugin::version req_version(rversion);
if (!plugin_version.check(req_version))
{
required_version = rversion;
return false;
}
}
return true;
}
void falco_engine::clear_filters()
{
m_rulesets.clear();
@@ -467,8 +538,6 @@ void falco_engine::clear_filters()
std::shared_ptr<falco_ruleset> ruleset(new falco_ruleset());
m_rulesets[it.first] = ruleset;
}
m_required_plugin_versions.clear();
}
void falco_engine::set_sampling_ratio(uint32_t sampling_ratio)

View File

@@ -56,6 +56,12 @@ public:
// If source is non-empty, only fields for the provided source are printed.
void list_fields(std::string &source, bool verbose, bool names_only);
// Optional--pass info on the currently loaded set of
// plugins. If provided, when loading rules content plugin
// info will be checked against any required_plugin_version
// blocks in the rules content.
void set_plugin_infos(std::list<sinsp_plugin::info> &plugin_infos);
//
// Load rules either directly or from a filename.
//
@@ -69,6 +75,44 @@ public:
void load_rules_file(const std::string &rules_filename, bool verbose, bool all_events, uint64_t &required_engine_version);
void load_rules(const std::string &rules_content, bool verbose, bool all_events, uint64_t &required_engine_version);
// Represents a block of rules. The name is only for
// logging/identification purposes, but generally is the path
// to the file that contained content.
class rulesfile {
public:
rulesfile();
virtual ~rulesfile();
bool load(const std::string &filename, std::string &errstr);
std::string name;
std::string content;
};
// Represents the result of loading a block of rules.
// - successful: true if the file was loaded, false otherwise
// - warnings: any warnings from loading the rules
// - errors: any errors from loading the rules
class load_result {
public:
load_result(const rulesfile &rulesfile);
~load_result();
std::string as_string(bool include_filenames, bool include_warnings);
const rulesfile &rf;
bool successful;
std::list<std::string> warnings;
std::list<std::string> errors;
};
// Improved variant that explicitly passes back a load_result
// struct containing lists of errors/warnings, etc. instead of
// throwing an exception on error.
void load_rules(rulesfile &rf,
bool verbose, bool all_events,
load_result &result);
//
// Enable/Disable any rules matching the provided substring.
// If the substring is "", all rules are enabled/disabled.
@@ -206,12 +250,6 @@ public:
std::shared_ptr<gen_event_formatter> create_formatter(const std::string &source,
const std::string &output);
// Return whether the provided plugin name + version is
// compatible with the current set of loaded rules files.
// required_version will be filled in with the required
// version when the method returns false.
bool is_plugin_compatible(const std::string &name, const std::string &version, std::string &required_version);
private:
//
@@ -221,6 +259,9 @@ private:
//
inline bool should_drop_evt();
// Optional--info on any loaded plugins.
std::list<sinsp_plugin::info> m_plugin_infos;
// Maps from event source to object that can generate filters from rules
std::map<std::string, std::shared_ptr<gen_event_filter_factory>> m_filter_factories;
@@ -235,10 +276,6 @@ private:
std::map<string, uint16_t> m_known_rulesets;
falco_common::priority_type m_min_priority;
// Maps from plugin to a list of required plugin versions
// found in any loaded rules files.
std::map<std::string, std::list<std::string>> m_required_plugin_versions;
void populate_rule_result(unique_ptr<struct rule_result> &res, gen_event *ev);
//

View File

@@ -397,13 +397,17 @@ static void get_lua_table_list_values(lua_State *ls,
}
void falco_rules::load_rules(const string &rules_content,
bool falco_rules::load_rules(const string &rules_content,
bool verbose, bool all_events,
string &extra, bool replace_container_info,
falco_common::priority_type min_priority,
std::list<std::string> &warnings,
std::list<std::string> &errors,
uint64_t &required_engine_version,
std::map<std::string, std::list<std::string>> &required_plugin_versions)
std::list<sinsp_plugin::info> &plugin_infos)
{
std::map<std::string, std::list<std::string>> required_plugin_versions;
lua_getglobal(m_ls, m_lua_load_rules.c_str());
if(lua_isfunction(m_ls, -1))
{
@@ -432,46 +436,42 @@ void falco_rules::load_rules(const string &rules_content,
bool successful = lua_toboolean(m_ls, -5);
required_engine_version = lua_tonumber(m_ls, -4);
get_lua_table_list_values(m_ls, -3, required_plugin_versions);
std::list<std::string> errors = get_lua_table_values(m_ls, -2);
std::list<std::string> warnings = get_lua_table_values(m_ls, -1);
errors = get_lua_table_values(m_ls, -2);
warnings = get_lua_table_values(m_ls, -1);
// Concatenate errors/warnings
std::ostringstream os;
if (errors.size() > 0)
lua_pop(m_ls, 4);
// Also check plugin compatibility. Only meaningful if
// plugin_infos is non-empty.
for(auto &pinfo : plugin_infos)
{
os << errors.size() << " errors:" << std::endl;
for(auto err : errors)
if(required_plugin_versions.find(pinfo.name) == required_plugin_versions.end())
{
os << err << std::endl;
// No required engine versions, so no restrictions. Compatible.
continue;
}
}
if (warnings.size() > 0)
{
os << warnings.size() << " warnings:" << std::endl;
for(auto warn : warnings)
for(auto &rversion : required_plugin_versions[pinfo.name])
{
os << warn << std::endl;
sinsp_plugin::version req_version(rversion.c_str());
if(!pinfo.plugin_version.check(req_version))
{
errors.push_back(std::string("Plugin ") + pinfo.name + " version " + pinfo.plugin_version.as_string() + " not compatible with required plugin version " + rversion);
successful = false;
}
}
}
if(!successful)
{
throw falco_exception(os.str());
return false;
}
if (verbose && os.str() != "") {
// We don't really have a logging callback
// from the falco engine, but this would be a
// good place to use it.
fprintf(stderr, "When reading rules content: %s", os.str().c_str());
}
lua_pop(m_ls, 4);
} else {
throw falco_exception("No function " + m_lua_load_rules + " found in lua rule module");
}
return true;
}
void falco_rules::describe_rule(std::string *rule)

View File

@@ -39,11 +39,13 @@ class falco_rules
void add_filter_factory(const std::string &source,
std::shared_ptr<gen_event_filter_factory> factory);
void load_rules(const string &rules_content, bool verbose, bool all_events,
bool load_rules(const string &rules_content, bool verbose, bool all_events,
std::string &extra, bool replace_container_info,
falco_common::priority_type min_priority,
std::list<std::string> &warnings,
std::list<std::string> &errors,
uint64_t &required_engine_version,
std::map<std::string, std::list<std::string>> &required_plugin_versions);
std::list<sinsp_plugin::info> &plugin_infos);
void describe_rule(string *rule);
bool is_source_valid(const std::string &source);

View File

@@ -25,6 +25,8 @@ set(
event_drops.cpp
statsfilewriter.cpp
falco.cpp
formats.cpp
swappable_falco_engine.cpp
)
set(
@@ -76,6 +78,8 @@ if(NOT MINIMAL_BUILD)
${CMAKE_CURRENT_BINARY_DIR}/version.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/outputs.grpc.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/outputs.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/rules.grpc.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/rules.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/schema.pb.cc
)
@@ -150,6 +154,10 @@ if(NOT MINIMAL_BUILD)
${CMAKE_CURRENT_BINARY_DIR}/outputs.grpc.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/outputs.grpc.pb.h
${CMAKE_CURRENT_BINARY_DIR}/outputs.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/rules.grpc.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/rules.grpc.pb.h
${CMAKE_CURRENT_BINARY_DIR}/rules.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/rules.pb.h
${CMAKE_CURRENT_BINARY_DIR}/outputs.pb.h
${CMAKE_CURRENT_BINARY_DIR}/schema.pb.cc
${CMAKE_CURRENT_BINARY_DIR}/schema.pb.h
@@ -166,6 +174,12 @@ if(NOT MINIMAL_BUILD)
COMMAND ${PROTOC} -I ${CMAKE_CURRENT_SOURCE_DIR} --grpc_out=. --plugin=protoc-gen-grpc=${GRPC_CPP_PLUGIN}
${CMAKE_CURRENT_SOURCE_DIR}/outputs.proto
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
# Falco gRPC Rules API
DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/rules.proto
COMMAND ${PROTOC} -I ${CMAKE_CURRENT_SOURCE_DIR} --cpp_out=. ${CMAKE_CURRENT_SOURCE_DIR}/rules.proto
COMMAND ${PROTOC} -I ${CMAKE_CURRENT_SOURCE_DIR} --grpc_out=. --plugin=protoc-gen-grpc=${GRPC_CPP_PLUGIN}
${CMAKE_CURRENT_SOURCE_DIR}/rules.proto
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
endif()

View File

@@ -46,6 +46,7 @@ limitations under the License.
#include "configuration.h"
#include "falco_engine.h"
#include "falco_engine_version.h"
#include "swappable_falco_engine.h"
#include "config_falco.h"
#include "statsfilewriter.h"
#ifndef MINIMAL_BUILD
@@ -209,7 +210,7 @@ std::list<string> cmdline_options;
#ifndef MINIMAL_BUILD
// Read a jsonl file containing k8s audit events and pass each to the engine.
void read_k8s_audit_trace_file(falco_engine *engine,
void read_k8s_audit_trace_file(swappable_falco_engine &swengine,
falco_outputs *outputs,
string &trace_filename)
{
@@ -229,7 +230,7 @@ void read_k8s_audit_trace_file(falco_engine *engine,
continue;
}
if(!k8s_audit_handler::accept_data(engine, outputs, line, errstr))
if(!k8s_audit_handler::accept_data(swengine, outputs, line, errstr))
{
falco_logger::log(LOG_ERR, "Could not read k8s audit event line #" + to_string(line_num) + ", \"" + line + "\": " + errstr + ", stopping");
return;
@@ -250,7 +251,7 @@ static std::string read_file(std::string filename)
//
// Event processing loop
//
uint64_t do_inspect(falco_engine *engine,
uint64_t do_inspect(swappable_falco_engine &swengine,
falco_outputs *outputs,
sinsp* inspector,
std::string &event_source,
@@ -383,7 +384,7 @@ uint64_t do_inspect(falco_engine *engine,
// engine, which will match the event against the set
// of rules. If a match is found, pass the event to
// the outputs.
unique_ptr<falco_engine::rule_result> res = engine->process_event(event_source, ev);
unique_ptr<falco_engine::rule_result> res = swengine.engine()->process_event(event_source, ev);
if(res)
{
outputs->handle_event(res->evt, res->rule, res->source, res->priority_num, res->format, res->tags);
@@ -436,13 +437,13 @@ static void print_all_ignored_events(sinsp *inspector)
printf("\n");
}
static void check_for_ignored_events(sinsp &inspector, falco_engine &engine)
static void check_for_ignored_events(sinsp &inspector, swappable_falco_engine &swengine)
{
std::set<uint16_t> evttypes;
sinsp_evttables* einfo = inspector.get_event_info_tables();
const struct ppm_event_info* etable = einfo->m_event_info;
engine.evttypes_for_ruleset(syscall_source, evttypes);
swengine.engine()->evttypes_for_ruleset(syscall_source, evttypes);
// Save event names so we don't warn for both the enter and exit event.
std::set<std::string> warn_event_names;
@@ -484,14 +485,14 @@ static void check_for_ignored_events(sinsp &inspector, falco_engine &engine)
}
}
static void list_source_fields(falco_engine *engine, bool verbose, bool names_only, std::string &source)
static void list_source_fields(swappable_falco_engine &swengine, bool verbose, bool names_only, std::string &source)
{
if(source.size() > 0 &&
!engine->is_source_valid(source))
!swengine.engine()->is_source_valid(source))
{
throw std::invalid_argument("Value for --list must be a valid source type");
}
engine->list_fields(source, verbose, names_only);
swengine.engine()->list_fields(source, verbose, names_only);
}
//
@@ -502,7 +503,9 @@ int falco_init(int argc, char **argv)
int result = EXIT_SUCCESS;
sinsp* inspector = NULL;
sinsp_evt::param_fmt event_buffer_format = sinsp_evt::PF_NORMAL;
falco_engine *engine = NULL;
swappable_falco_engine::config engine_config;
swappable_falco_engine swengine;
std::string errstr;
falco_outputs *outputs = NULL;
syscall_evt_drop_mgr sdropmgr;
int op;
@@ -519,7 +522,6 @@ int falco_init(int argc, char **argv)
list<string> validate_rules_filenames;
string stats_filename = "";
uint64_t stats_interval = 5000;
bool verbose = false;
bool names_only = false;
bool all_events = false;
#ifndef MINIMAL_BUILD
@@ -528,9 +530,7 @@ int falco_init(int argc, char **argv)
string *k8s_node_name = 0;
string* mesos_api = 0;
#endif
string output_format = "";
uint32_t snaplen = 0;
bool replace_container_info = false;
int duration_to_tot = 0;
bool print_ignored_events = false;
bool list_flds = false;
@@ -539,9 +539,6 @@ int falco_init(int argc, char **argv)
bool print_support = false;
string cri_socket_path;
bool cri_async = true;
set<string> disable_sources;
bool disable_syscall = false;
bool disable_k8s_audit = false;
bool userspace = false;
// Used for writing trace files
@@ -552,14 +549,13 @@ int falco_init(int argc, char **argv)
bool compress = false;
bool buffered_outputs = true;
bool buffered_cmdline = false;
std::map<string,uint64_t> required_engine_versions;
// Used for stats
double duration;
scap_stats cstats;
#ifndef MINIMAL_BUILD
falco_webserver webserver;
falco_webserver webserver(swengine);
falco::grpc::server grpc_server;
std::thread grpc_server_thread;
#endif
@@ -594,11 +590,7 @@ int falco_init(int argc, char **argv)
try
{
set<string> disabled_rule_substrings;
string substring;
string all_rules;
set<string> disabled_rule_tags;
set<string> enabled_rule_tags;
//
// Parse the args
@@ -626,7 +618,7 @@ int falco_init(int argc, char **argv)
break;
case 'D':
substring = optarg;
disabled_rule_substrings.insert(substring);
engine_config.disabled_rule_substrings.insert(substring);
break;
case 'e':
trace_filename = optarg;
@@ -679,23 +671,23 @@ int falco_init(int argc, char **argv)
case 'p':
if(string(optarg) == "c" || string(optarg) == "container")
{
output_format = "container=%container.name (id=%container.id)";
replace_container_info = true;
engine_config.output_format = "container=%container.name (id=%container.id)";
engine_config.replace_container_info = true;
}
else if(string(optarg) == "k" || string(optarg) == "kubernetes")
{
output_format = "k8s.ns=%k8s.ns.name k8s.pod=%k8s.pod.name container=%container.id";
replace_container_info = true;
engine_config.output_format = "k8s.ns=%k8s.ns.name k8s.pod=%k8s.pod.name container=%container.id";
engine_config.replace_container_info = true;
}
else if(string(optarg) == "m" || string(optarg) == "mesos")
{
output_format = "task=%mesos.task.name container=%container.id";
replace_container_info = true;
engine_config.output_format = "task=%mesos.task.name container=%container.id";
engine_config.replace_container_info = true;
}
else
{
output_format = optarg;
replace_container_info = false;
engine_config.output_format = optarg;
engine_config.replace_container_info = false;
}
break;
case 'r':
@@ -708,10 +700,10 @@ int falco_init(int argc, char **argv)
stats_filename = optarg;
break;
case 'T':
disabled_rule_tags.insert(optarg);
engine_config.disabled_rule_tags.insert(optarg);
break;
case 't':
enabled_rule_tags.insert(optarg);
engine_config.enabled_rule_tags.insert(optarg);
break;
case 'U':
buffered_outputs = false;
@@ -721,7 +713,7 @@ int falco_init(int argc, char **argv)
userspace = true;
break;
case 'v':
verbose = true;
engine_config.verbose = true;
break;
case 'V':
validate_rules_filenames.push_back(optarg);
@@ -786,7 +778,7 @@ int falco_init(int argc, char **argv)
{
if(optarg != NULL)
{
disable_sources.insert(optarg);
engine_config.event_sources.erase(std::string(optarg));
}
}
break;
@@ -797,6 +789,10 @@ int falco_init(int argc, char **argv)
}
/////////////////////////////////////////////////////////
// Create and configure inspector
/////////////////////////////////////////////////////////
inspector = new sinsp();
inspector->set_buffer_format(event_buffer_format);
@@ -824,39 +820,6 @@ int falco_init(int argc, char **argv)
return EXIT_SUCCESS;
}
engine = new falco_engine(true);
engine->set_extra(output_format, replace_container_info);
// Create "factories" that can create filters/formatters for
// syscalls and k8s audit events.
std::shared_ptr<gen_event_filter_factory> syscall_filter_factory(new sinsp_filter_factory(inspector));
std::shared_ptr<gen_event_filter_factory> k8s_audit_filter_factory(new json_event_filter_factory());
std::shared_ptr<gen_event_formatter_factory> syscall_formatter_factory(new sinsp_evt_formatter_factory(inspector));
std::shared_ptr<gen_event_formatter_factory> k8s_audit_formatter_factory(new json_event_formatter_factory(k8s_audit_filter_factory));
engine->add_source(syscall_source, syscall_filter_factory, syscall_formatter_factory);
engine->add_source(k8s_audit_source, k8s_audit_filter_factory, k8s_audit_formatter_factory);
if(disable_sources.size() > 0)
{
auto it = disable_sources.begin();
while(it != disable_sources.end())
{
if(*it != syscall_source && *it != k8s_audit_source)
{
it = disable_sources.erase(it);
continue;
}
++it;
}
disable_syscall = disable_sources.count(syscall_source) > 0;
disable_k8s_audit = disable_sources.count(k8s_audit_source) > 0;
if (disable_syscall && disable_k8s_audit) {
throw std::invalid_argument("The event source \"syscall\" and \"k8s_audit\" can not be disabled together");
}
}
// Some combinations of arguments are not allowed.
if (daemon && pidfilename == "") {
throw std::invalid_argument("If -d is provided, a pid file must also be provided");
@@ -892,31 +855,6 @@ int falco_init(int argc, char **argv)
}
}
if(validate_rules_filenames.size() > 0)
{
falco_logger::log(LOG_INFO, "Validating rules file(s):\n");
for(auto file : validate_rules_filenames)
{
falco_logger::log(LOG_INFO, " " + file + "\n");
}
for(auto file : validate_rules_filenames)
{
// Only include the prefix if there is more than one file
std::string prefix = (validate_rules_filenames.size() > 1 ? file + ": " : "");
try {
engine->load_rules_file(file, verbose, all_events);
}
catch(falco_exception &e)
{
printf("%s%s", prefix.c_str(), e.what());
throw;
}
printf("%sOk\n", prefix.c_str());
}
falco_logger::log(LOG_INFO, "Ok\n");
goto exit;
}
falco_configuration config;
if (conf_filename.size())
{
@@ -936,19 +874,10 @@ int falco_init(int argc, char **argv)
// plugin was found, the source is the source of that
// plugin.
std::string event_source = syscall_source;
engine_config.json_output = config.m_json_output;
engine_config.min_priority = config.m_min_priority;
// All filterchecks created by plugins go in this
// list. If we ever support multiple event sources at
// the same time, this (and the below factories) will
// have to be a map from event source to filtercheck
// list.
filter_check_list plugin_filter_checks;
// Factories that can create filters/formatters for
// the (single) source supported by the (single) input plugin.
std::shared_ptr<gen_event_filter_factory> plugin_filter_factory(new sinsp_filter_factory(inspector, plugin_filter_checks));
std::shared_ptr<gen_event_formatter_factory> plugin_formatter_factory(new sinsp_evt_formatter_factory(inspector, plugin_filter_checks));
// Load and validate the configured plugins, if any.
std::shared_ptr<sinsp_plugin> input_plugin;
std::list<std::shared_ptr<sinsp_plugin>> extractor_plugins;
for(auto &p : config.m_plugins)
@@ -962,7 +891,7 @@ int falco_init(int argc, char **argv)
plugin = sinsp_plugin::register_plugin(inspector,
p.m_library_path,
(p.m_init_config.empty() ? NULL : (char *)p.m_init_config.c_str()),
plugin_filter_checks);
swengine.plugin_filter_checks());
#endif
if(plugin->type() == TYPE_SOURCE_PLUGIN)
@@ -983,7 +912,9 @@ int falco_init(int argc, char **argv)
inspector->set_input_plugin_open_params(p.m_open_params.c_str());
}
engine->add_source(event_source, plugin_filter_factory, plugin_formatter_factory);
// For now, if an input plugin is configured, the built-in sources are disabled.
engine_config.event_sources.clear();
engine_config.event_sources.insert(splugin->event_source());
} else {
extractor_plugins.push_back(plugin);
@@ -1019,20 +950,25 @@ int falco_init(int argc, char **argv)
}
}
if(config.m_json_output)
// If there are no event sources, due to a combination
// of --disable-source and lack of plugin config,
// raise an error.
if(engine_config.event_sources.empty())
{
syscall_formatter_factory->set_output_format(gen_event_formatter::OF_JSON);
k8s_audit_formatter_factory->set_output_format(gen_event_formatter::OF_JSON);
plugin_formatter_factory->set_output_format(gen_event_formatter::OF_JSON);
throw std::invalid_argument("No event sources configured. Check use of --disable-source and plugin configuration");
}
std::list<sinsp_plugin::info> infos = sinsp_plugin::plugin_infos(inspector);
engine_config.plugin_infos = sinsp_plugin::plugin_infos(inspector);
if (!swengine.init(engine_config, inspector, errstr))
{
throw std::runtime_error("Could not initialize falco engine: " + errstr);
}
if(list_plugins)
{
std::ostringstream os;
for(auto &info : infos)
for(auto &info : engine_config.plugin_infos)
{
os << "Name: " << info.name << std::endl;
os << "Description: " << info.description << std::endl;
@@ -1051,13 +987,13 @@ int falco_init(int argc, char **argv)
os << std::endl;
}
printf("%lu Plugins Loaded:\n\n%s\n", infos.size(), os.str().c_str());
printf("%lu Plugins Loaded:\n\n%s\n", engine_config.plugin_infos.size(), os.str().c_str());
return EXIT_SUCCESS;
}
if(list_flds)
{
list_source_fields(engine, verbose, names_only, list_flds_source);
list_source_fields(swengine, engine_config.verbose, names_only, list_flds_source);
return EXIT_SUCCESS;
}
@@ -1066,8 +1002,6 @@ int falco_init(int argc, char **argv)
config.m_rules_filenames = rules_filenames;
}
engine->set_min_priority(config.m_min_priority);
if(buffered_cmdline)
{
config.m_buffered_outputs = buffered_outputs;
@@ -1078,6 +1012,48 @@ int falco_init(int argc, char **argv)
throw std::invalid_argument("You must specify at least one rules file/directory via -r or a rules_file entry in falco.yaml");
}
if(validate_rules_filenames.size() > 0)
{
std::list<falco_engine::rulesfile> validate_rules;
falco_logger::log(LOG_INFO, "Validating rules file(s):\n");
for(auto file : validate_rules_filenames)
{
falco_logger::log(LOG_INFO, " " + file + "\n");
}
if (!swappable_falco_engine::open_files(validate_rules_filenames, validate_rules, errstr))
{
throw falco_exception(errstr);
}
std::string load_result;
bool ret = swengine.validate(validate_rules, load_result);
if(!ret)
{
// Print to stdout and also throw an error
printf("%s", load_result.c_str());
throw falco_exception(load_result);
}
else
{
// load_result might contain warnings. Print them
// if verbose is true.
if(engine_config.verbose && !load_result.empty())
{
printf("%s", load_result.c_str());
}
else
{
printf("Ok\n");
}
}
falco_logger::log(LOG_INFO, "Ok\n");
goto exit;
}
falco_logger::log(LOG_DEBUG, "Configured rules filenames:\n");
for (auto filename : config.m_rules_filenames)
{
@@ -1087,65 +1063,35 @@ int falco_init(int argc, char **argv)
for (auto filename : config.m_rules_filenames)
{
falco_logger::log(LOG_INFO, "Loading rules from file " + filename + ":\n");
uint64_t required_engine_version;
try {
engine->load_rules_file(filename, verbose, all_events, required_engine_version);
}
catch(falco_exception &e)
{
std::string prefix = "Could not load rules file " + filename + ": ";
throw falco_exception(prefix + e.what());
}
required_engine_versions[filename] = required_engine_version;
}
// Ensure that all plugins are compatible with the loaded set of rules
for(auto &info : infos)
std::list<falco_engine::rulesfile> rulesfiles;
if (!swappable_falco_engine::open_files(config.m_rules_filenames, rulesfiles, errstr))
{
std::string required_version;
throw falco_exception(errstr);
}
if(!engine->is_plugin_compatible(info.name, info.plugin_version.as_string(), required_version))
{
throw std::invalid_argument(std::string("Plugin ") + info.name + " version " + info.plugin_version.as_string() + " not compatible with required plugin version " + required_version);
}
std::string load_result;
bool ret = swengine.replace(rulesfiles, load_result);
if (!ret)
{
throw falco_exception(string("When loading rules: ") + load_result);
}
// load_result might contain warnings. Print them
// if verbose is true.
if(engine_config.verbose && !load_result.empty())
{
fprintf(stderr, "When loading rules: %s", load_result.c_str());
}
// You can't both disable and enable rules
if((disabled_rule_substrings.size() + disabled_rule_tags.size() > 0) &&
enabled_rule_tags.size() > 0) {
if((engine_config.disabled_rule_substrings.size() + engine_config.disabled_rule_tags.size() > 0) &&
engine_config.enabled_rule_tags.size() > 0) {
throw std::invalid_argument("You can not specify both disabled (-D/-T) and enabled (-t) rules");
}
for (auto substring : disabled_rule_substrings)
{
falco_logger::log(LOG_INFO, "Disabling rules matching substring: " + substring + "\n");
engine->enable_rule(substring, false);
}
if(disabled_rule_tags.size() > 0)
{
for(auto tag : disabled_rule_tags)
{
falco_logger::log(LOG_INFO, "Disabling rules with tag: " + tag + "\n");
}
engine->enable_rule_by_tag(disabled_rule_tags, false);
}
if(enabled_rule_tags.size() > 0)
{
// Since we only want to enable specific
// rules, first disable all rules.
engine->enable_rule(all_rules, false);
for(auto tag : enabled_rule_tags)
{
falco_logger::log(LOG_INFO, "Enabling rules with tag: " + tag + "\n");
}
engine->enable_rule_by_tag(enabled_rule_tags, true);
}
if(print_support)
{
nlohmann::json support;
@@ -1176,13 +1122,14 @@ int falco_init(int argc, char **argv)
support["engine_info"]["engine_version"] = FALCO_ENGINE_VERSION;
support["config"] = read_file(conf_filename);
support["rules_files"] = nlohmann::json::array();
for(auto filename : config.m_rules_filenames)
for(auto &rf : rulesfiles)
{
nlohmann::json finfo;
finfo["name"] = filename;
finfo["name"] = rf.name;
nlohmann::json variant;
variant["required_engine_version"] = required_engine_versions[filename];
variant["content"] = read_file(filename);
// XXX/mstemm add this back
//variant["required_engine_version"] = rf.required_engine_version;
variant["content"] = rf.content;
finfo["variants"].push_back(variant);
support["rules_files"].push_back(finfo);
}
@@ -1212,7 +1159,10 @@ int falco_init(int argc, char **argv)
// For syscalls, see if any event types used by the
// loaded rules are ones with the EF_DROP_SIMPLE_CONS
// label.
check_for_ignored_events(*inspector, *engine);
if(engine_config.contains_event_source(syscall_source))
{
check_for_ignored_events(*inspector, swengine);
}
// Drop EF_DROP_SIMPLE_CONS kernel side
inspector->set_simple_consumer();
// Eventually, drop any EF_DROP_SIMPLE_CONS event
@@ -1223,13 +1173,13 @@ int falco_init(int argc, char **argv)
if (describe_all_rules)
{
engine->describe_rule(NULL);
swengine.engine()->describe_rule(NULL);
goto exit;
}
if (describe_rule != "")
{
engine->describe_rule(&describe_rule);
swengine.engine()->describe_rule(&describe_rule);
goto exit;
}
@@ -1322,7 +1272,7 @@ int falco_init(int argc, char **argv)
outputs = new falco_outputs();
outputs->init(engine,
outputs->init(swengine,
config.m_json_output,
config.m_json_include_output_property,
config.m_json_include_tags_property,
@@ -1407,13 +1357,14 @@ int falco_init(int argc, char **argv)
open_t open_f;
// Default mode: both event sources enabled
if (!disable_syscall && !disable_k8s_audit) {
if (engine_config.contains_event_source(syscall_source) &&
engine_config.contains_event_source(k8s_audit_source)) {
open_f = open_cb;
}
if (disable_syscall) {
if (!engine_config.contains_event_source(syscall_source)) {
open_f = open_nodriver_cb;
}
if (disable_k8s_audit) {
if (!engine_config.contains_event_source(k8s_audit_source)) {
open_f = open_cb;
}
@@ -1424,7 +1375,7 @@ int falco_init(int argc, char **argv)
catch(sinsp_exception &e)
{
// If syscall input source is enabled and not through userspace instrumentation
if (!disable_syscall && !userspace)
if (engine_config.contains_event_source(syscall_source) && !userspace)
{
// Try to insert the Falco kernel module
if(system("modprobe " PROBE_NAME " > /dev/null 2> /dev/null"))
@@ -1467,7 +1418,7 @@ int falco_init(int argc, char **argv)
k8s_api_cert = new string(k8s_cert_env);
}
}
inspector->init_k8s_client(k8s_api, k8s_api_cert, k8s_node_name, verbose);
inspector->init_k8s_client(k8s_api, k8s_api_cert, k8s_node_name, engine_config.verbose);
k8s_api = 0;
k8s_api_cert = 0;
}
@@ -1483,7 +1434,7 @@ int falco_init(int argc, char **argv)
}
}
k8s_api = new string(k8s_api_env);
inspector->init_k8s_client(k8s_api, k8s_api_cert, k8s_node_name, verbose);
inspector->init_k8s_client(k8s_api, k8s_api_cert, k8s_node_name, engine_config.verbose);
}
else
{
@@ -1499,14 +1450,14 @@ int falco_init(int argc, char **argv)
//
if(mesos_api)
{
inspector->init_mesos_client(mesos_api, verbose);
inspector->init_mesos_client(mesos_api, engine_config.verbose);
}
else if(char* mesos_api_env = getenv("FALCO_MESOS_API"))
{
if(mesos_api_env != NULL)
{
mesos_api = new string(mesos_api_env);
inspector->init_mesos_client(mesos_api, verbose);
inspector->init_mesos_client(mesos_api, engine_config.verbose);
}
}
delete mesos_api;
@@ -1517,11 +1468,11 @@ int falco_init(int argc, char **argv)
falco_logger::log(LOG_DEBUG, "Setting metadata download watch frequency to " + to_string(config.m_metadata_download_watch_freq_sec) + " seconds\n");
inspector->set_metadata_download_params(config.m_metadata_download_max_mb * 1024 * 1024, config.m_metadata_download_chunk_wait_us, config.m_metadata_download_watch_freq_sec);
if(trace_filename.empty() && config.m_webserver_enabled && !disable_k8s_audit)
if(trace_filename.empty() && config.m_webserver_enabled && engine_config.contains_event_source(k8s_audit_source))
{
std::string ssl_option = (config.m_webserver_ssl_enabled ? " (SSL)" : "");
falco_logger::log(LOG_INFO, "Starting internal webserver, listening on port " + to_string(config.m_webserver_listen_port) + ssl_option + "\n");
webserver.init(&config, engine, outputs);
webserver.init(&config, outputs);
webserver.start();
}
@@ -1548,7 +1499,7 @@ int falco_init(int argc, char **argv)
if(!trace_filename.empty() && !trace_is_scap)
{
#ifndef MINIMAL_BUILD
read_k8s_audit_trace_file(engine,
read_k8s_audit_trace_file(swengine,
outputs,
trace_filename);
#endif
@@ -1557,7 +1508,7 @@ int falco_init(int argc, char **argv)
{
uint64_t num_evts;
num_evts = do_inspect(engine,
num_evts = do_inspect(swengine,
outputs,
inspector,
event_source,
@@ -1573,7 +1524,7 @@ int falco_init(int argc, char **argv)
inspector->get_capture_stats(&cstats);
if(verbose)
if(engine_config.verbose)
{
fprintf(stderr, "Driver Events:%" PRIu64 "\nDriver Drops:%" PRIu64 "\n",
cstats.n_evts,
@@ -1596,7 +1547,7 @@ int falco_init(int argc, char **argv)
}
inspector->close();
engine->print_stats();
swengine.engine()->print_stats();
sdropmgr.print_stats();
#ifndef MINIMAL_BUILD
webserver.stop();
@@ -1626,7 +1577,6 @@ int falco_init(int argc, char **argv)
exit:
delete inspector;
delete engine;
delete outputs;
return result;

View File

@@ -60,7 +60,7 @@ falco_outputs::~falco_outputs()
}
}
void falco_outputs::init(falco_engine *engine,
void falco_outputs::init(swappable_falco_engine &swengine,
bool json_output,
bool json_include_output_property,
bool json_include_tags_property,
@@ -74,7 +74,7 @@ void falco_outputs::init(falco_engine *engine,
throw falco_exception("falco_outputs already initialized");
}
m_formats.reset(new falco_formats(engine, json_include_output_property, json_include_tags_property));
m_formats.reset(new falco_formats(swengine, json_include_output_property, json_include_tags_property));
m_json_output = json_output;

View File

@@ -23,7 +23,7 @@ limitations under the License.
#include "json_evt.h"
#include "falco_common.h"
#include "token_bucket.h"
#include "falco_engine.h"
#include "swappable_falco_engine.h"
#include "outputs.h"
#include "formats.h"
#include "tbb/concurrent_queue.h"
@@ -39,7 +39,7 @@ public:
falco_outputs();
virtual ~falco_outputs();
void init(falco_engine *engine,
void init(swappable_falco_engine &swengine,
bool json_output,
bool json_include_output_property,
bool json_include_tags_property,

View File

@@ -17,13 +17,13 @@ limitations under the License.
#include <json/json.h>
#include "formats.h"
#include "falco_engine.h"
#include "swappable_falco_engine.h"
#include "banned.h" // This raises a compilation error when certain functions are used
falco_formats::falco_formats(falco_engine *engine,
falco_formats::falco_formats(swappable_falco_engine &swengine,
bool json_include_output_property,
bool json_include_tags_property)
: m_falco_engine(engine),
: m_swengine(swengine),
m_json_include_output_property(json_include_output_property),
m_json_include_tags_property(json_include_tags_property)
{
@@ -40,7 +40,7 @@ string falco_formats::format_event(gen_event *evt, const std::string &rule, cons
std::shared_ptr<gen_event_formatter> formatter;
formatter = m_falco_engine->create_formatter(source, format);
formatter = m_swengine.engine()->create_formatter(source, format);
// Format the original output string, regardless of output format
formatter->tostring_withformat(evt, line, gen_event_formatter::OF_NORMAL);
@@ -134,7 +134,7 @@ map<string, string> falco_formats::get_field_values(gen_event *evt, const std::s
{
std::shared_ptr<gen_event_formatter> formatter;
formatter = m_falco_engine->create_formatter(source, format);
formatter = m_swengine.engine()->create_formatter(source, format);
map<string, string> ret;

View File

@@ -28,12 +28,12 @@ extern "C"
#include <gen_filter.h>
#include "falco_engine.h"
#include "swappable_falco_engine.h"
class falco_formats
{
public:
falco_formats(falco_engine *engine,
falco_formats(swappable_falco_engine &swengine,
bool json_include_output_property,
bool json_include_tags_property);
virtual ~falco_formats();
@@ -45,7 +45,7 @@ public:
const std::string &format);
protected:
falco_engine *m_falco_engine;
swappable_falco_engine &m_swengine;
bool m_json_include_output_property;
bool m_json_include_tags_property;
};

View File

@@ -139,6 +139,78 @@ void request_context<version::service, version::request, version::response>::end
start(srv);
}
template<>
void request_context<rules::service, rules::rules_files, rules::reload_response>::start(server* srv)
{
m_state = request_context_base::REQUEST;
m_srv_ctx.reset(new ::grpc::ServerContext);
auto srvctx = m_srv_ctx.get();
m_res_writer.reset(new ::grpc::ServerAsyncResponseWriter<rules::reload_response>(srvctx));
m_req.Clear();
auto cq = srv->m_completion_queue.get();
// Request to start processing given requests.
// Using "this" - ie., the memory address of this context - as the tag that uniquely identifies the request.
// In this way, different contexts can serve different requests concurrently.
(srv->m_rules_svc.*m_request_func)(srvctx, &m_req, m_res_writer.get(), cq, cq, this);
}
template<>
void request_context<rules::service, rules::rules_files, rules::reload_response>::process(server* srv)
{
rules::reload_response res;
(srv->*m_process_func)(m_srv_ctx.get(), m_req, res);
// Notify the gRPC runtime that this processing is done
m_state = request_context_base::FINISH;
// Using "this"- ie., the memory address of this context - to uniquely identify the event.
m_res_writer->Finish(res, ::grpc::Status::OK, this);
}
template<>
void request_context<rules::service, rules::rules_files, rules::reload_response>::end(server* srv, bool error)
{
// todo(leodido) > handle processing errors here
// Ask to start processing requests
start(srv);
}
template<>
void request_context<rules::service, rules::rules_files, rules::validate_response>::start(server* srv)
{
m_state = request_context_base::REQUEST;
m_srv_ctx.reset(new ::grpc::ServerContext);
auto srvctx = m_srv_ctx.get();
m_res_writer.reset(new ::grpc::ServerAsyncResponseWriter<rules::validate_response>(srvctx));
m_req.Clear();
auto cq = srv->m_completion_queue.get();
// Request to start processing given requests.
// Using "this" - ie., the memory address of this context - as the tag that uniquely identifies the request.
// In this way, different contexts can serve different requests concurrently.
(srv->m_rules_svc.*m_request_func)(srvctx, &m_req, m_res_writer.get(), cq, cq, this);
}
template<>
void request_context<rules::service, rules::rules_files, rules::validate_response>::process(server* srv)
{
rules::validate_response res;
(srv->*m_process_func)(m_srv_ctx.get(), m_req, res);
// Notify the gRPC runtime that this processing is done
m_state = request_context_base::FINISH;
// Using "this"- ie., the memory address of this context - to uniquely identify the event.
m_res_writer->Finish(res, ::grpc::Status::OK, this);
}
template<>
void request_context<rules::service, rules::rules_files, rules::validate_response>::end(server* srv, bool error)
{
// todo(leodido) > handle processing errors here
// Ask to start processing requests
start(srv);
}
template<>
void request_bidi_context<outputs::service, outputs::request, outputs::response>::start(server* srv)
{

View File

@@ -194,6 +194,7 @@ void falco::grpc::server::run()
{
m_server_builder.RegisterService(&m_output_svc);
m_server_builder.RegisterService(&m_version_svc);
m_server_builder.RegisterService(&m_rules_svc);
m_completion_queue = m_server_builder.AddCompletionQueue();
m_server = m_server_builder.BuildAndStart();
@@ -213,6 +214,8 @@ void falco::grpc::server::run()
REGISTER_UNARY(version::request, version::response, version::service, version, version, context_num)
REGISTER_STREAM(outputs::request, outputs::response, outputs::service, get, get, context_num)
REGISTER_BIDI(outputs::request, outputs::response, outputs::service, sub, sub, context_num)
REGISTER_UNARY(rules::rules_files, rules::reload_response, rules::service, reload_rules, reload_rules, context_num)
REGISTER_UNARY(rules::rules_files, rules::validate_response, rules::service, validate_rules, validate_rules, context_num)
m_threads.resize(m_threadiness);
int thread_idx = 0;

View File

@@ -46,6 +46,7 @@ public:
outputs::service::AsyncService m_output_svc;
version::service::AsyncService m_version_svc;
rules::service::AsyncService m_rules_svc;
std::unique_ptr<::grpc::ServerCompletionQueue> m_completion_queue;

View File

@@ -77,13 +77,21 @@ void falco::grpc::server_impl::version(const context& ctx, const version::reques
version = FALCO_VERSION;
res.set_engine_version(FALCO_ENGINE_VERSION);
res.set_engine_fields_checksum(FALCO_FIELDS_CHECKSUM);
res.set_engine_fields_checksum(FALCO_FIELDS_CHECKSUM);
res.set_major(FALCO_VERSION_MAJOR);
res.set_minor(FALCO_VERSION_MINOR);
res.set_patch(FALCO_VERSION_PATCH);
}
void falco::grpc::server_impl::reload_rules(const context& ctx, const rules::rules_files& rules_files, rules::reload_response& res)
{
}
void falco::grpc::server_impl::validate_rules(const context& ctx, const rules::rules_files& rules_files, rules::validate_response& res)
{
}
void falco::grpc::server_impl::shutdown()
{
m_stop = true;

View File

@@ -19,6 +19,7 @@ limitations under the License.
#include <atomic>
#include "outputs.grpc.pb.h"
#include "version.grpc.pb.h"
#include "rules.grpc.pb.h"
#include "grpc_context.h"
namespace falco
@@ -43,6 +44,10 @@ protected:
// Version
void version(const context& ctx, const version::request& req, version::response& res);
// Rules
void reload_rules(const context& ctx, const rules::rules_files& rules_files, rules::reload_response& res);
void validate_rules(const context& ctx, const rules::rules_files& rules_files, rules::validate_response& res);
private:
std::atomic<bool> m_stop{false};
};

View File

@@ -0,0 +1,51 @@
/*
Copyright (C) 2020 The Falco Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
syntax = "proto3";
package falco.rules;
option go_package = "github.com/falcosecurity/client-go/pkg/api/rules";
// Functions to reload a set of rules files (replacing any currently
// loaded rules), or validate a set of candidate rules files.
service service {
rpc reload_rules(rules_files) returns (reload_response);
rpc validate_rules(rules_files) returns (validate_response);
}
message rules_file
{
string filename = 1;
string content = 2;
}
message rules_files
{
repeated rules_file files = 1;
}
message reload_response
{
bool successful = 1;
string error = 2;
}
message validate_response
{
bool successful = 1;
string error = 2;
}

View File

@@ -0,0 +1,256 @@
/*
Copyright (C) 2022 The Falco Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
#include <fstream>
#include "logger.h"
#include "swappable_falco_engine.h"
std::string swappable_falco_engine::syscall_source = "syscall";
std::string swappable_falco_engine::k8s_audit_source = "k8s_audit";
swappable_falco_engine::config::config()
: json_output(false), verbose(false), replace_container_info(false),
event_sources{syscall_source, k8s_audit_source}
{
}
swappable_falco_engine::config::~config()
{
}
bool swappable_falco_engine::config::contains_event_source(const std::string &source)
{
return (event_sources.find(source) != event_sources.end());
}
bool swappable_falco_engine::open_files(std::list<std::string> &filenames,
std::list<falco_engine::rulesfile> &rulesfiles,
std::string &errstr)
{
rulesfiles.clear();
for(const auto &filename : filenames)
{
std::string errstr;
rulesfiles.emplace_back();
falco_engine::rulesfile &rf = rulesfiles.back();
if (!rf.load(filename, errstr))
{
return false;
}
}
errstr = "";
return true;
}
swappable_falco_engine::swappable_falco_engine()
{
}
swappable_falco_engine::~swappable_falco_engine()
{
}
bool swappable_falco_engine::init(swappable_falco_engine::config &cfg, sinsp *inspector, std::string &errstr)
{
m_config = cfg;
m_inspector = inspector;
// Initialize some engine with no rules
std::list<falco_engine::rulesfile> empty;
return replace(empty, errstr);
}
std::shared_ptr<falco_engine> swappable_falco_engine::engine()
{
std::shared_ptr<falco_engine> new_engine;
while(m_pending_falco_engine.try_pop(new_engine))
{
m_engine=new_engine;
}
if(m_engine == NULL)
{
throw falco_exception("No engine, must call replace() first");
}
return m_engine;
}
filter_check_list &swappable_falco_engine::plugin_filter_checks()
{
return m_plugin_filter_checks;
}
bool swappable_falco_engine::replace(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result)
{
std::shared_ptr<falco_engine> new_engine;
new_engine = create_new(rulesfiles, load_result);
if (new_engine == NULL)
{
return false;
}
m_pending_falco_engine.push(new_engine);
return true;
}
bool swappable_falco_engine::validate(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result)
{
std::shared_ptr<falco_engine> new_engine;
new_engine = create_new(rulesfiles, load_result);
return (new_engine != NULL);
}
std::shared_ptr<falco_engine> swappable_falco_engine::create_new(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result)
{
std::shared_ptr<falco_engine> ret = make_shared<falco_engine>();
load_result = "";
if(!m_inspector)
{
load_result = "No inspector provided yet";
ret = NULL;
return ret;
}
ret->set_extra(m_config.output_format, m_config.replace_container_info);
ret->set_min_priority(m_config.min_priority);
ret->set_plugin_infos(m_config.plugin_infos);
// Create "factories" that can create filters/formatters for
// each supported source.
for(const auto &source : m_config.event_sources)
{
std::shared_ptr<gen_event_filter_factory> filter_factory;
std::shared_ptr<gen_event_formatter_factory> formatter_factory;
if(source == syscall_source)
{
// This use of m_inspector looks unsafe, as it
// may have been created on a different thread
// than the thread where create_new() was
// called. But the inspector is only *used*
// when evaluating filters, and that is only
// done on the thread processing events and
// calling engine().
filter_factory.reset(new sinsp_filter_factory(m_inspector));
formatter_factory.reset(new sinsp_evt_formatter_factory(m_inspector));
}
else if (source == k8s_audit_source)
{
filter_factory.reset(new json_event_filter_factory());
formatter_factory.reset(new json_event_formatter_factory(filter_factory));
}
else
{
// Assumed to be a source plugin
filter_factory.reset(new sinsp_filter_factory(m_inspector, m_plugin_filter_checks));
formatter_factory.reset(new sinsp_evt_formatter_factory(m_inspector, m_plugin_filter_checks));
}
if(m_config.json_output)
{
formatter_factory->set_output_format(gen_event_formatter::OF_JSON);
}
ret->add_source(source, filter_factory, formatter_factory);
}
// Note that we load all rules files, even if one of them has an error.
bool successful = true;
// We include filenames if there is more than one file
bool include_filenames = (rulesfiles.size() > 1);
// We include warnings if verbose
bool include_warnings = m_config.verbose;
std::ostringstream os;
for(auto &rf : rulesfiles)
{
falco_engine::load_result res(rf);
// XXX/mstemm all_events is actually unused, remove it.
bool all_events = false;
ret->load_rules(rf,
m_config.verbose, all_events,
res);
os << res.as_string(include_filenames, include_warnings);
if(!res.successful)
{
successful = false;
}
}
load_result = os.str();
if(!successful)
{
ret = NULL;
return ret;
}
for (auto substring : m_config.disabled_rule_substrings)
{
falco_logger::log(LOG_INFO, "Disabling rules matching substring: " + substring + "\n");
ret->enable_rule(substring, false);
}
if(m_config.disabled_rule_tags.size() > 0)
{
for(auto tag : m_config.disabled_rule_tags)
{
falco_logger::log(LOG_INFO, "Disabling rules with tag: " + tag + "\n");
}
ret->enable_rule_by_tag(m_config.disabled_rule_tags, false);
}
if(m_config.enabled_rule_tags.size() > 0)
{
string all_rules = "";
// Since we only want to enable specific
// rules, first disable all rules.
ret->enable_rule(all_rules, false);
for(auto tag : m_config.enabled_rule_tags)
{
falco_logger::log(LOG_INFO, "Enabling rules with tag: " + tag + "\n");
}
ret->enable_rule_by_tag(m_config.enabled_rule_tags, true);
}
return ret;
}

View File

@@ -0,0 +1,119 @@
/*
Copyright (C) 2022 The Falco Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
#pragma once
#include <list>
#include <memory>
#include <string>
#include <set>
#include <utility>
#include "tbb/concurrent_queue.h"
#include <sinsp.h>
#include <filter_check_list.h>
#include <falco_engine.h>
class swappable_falco_engine
{
public:
static std::string syscall_source;
static std::string k8s_audit_source;
class config {
public:
config();
virtual ~config();
bool contains_event_source(const std::string &source);
bool json_output;
bool verbose;
std::string output_format;
bool replace_container_info;
std::set<std::string> event_sources;
falco_common::priority_type min_priority;
std::list<sinsp_plugin::info> plugin_infos;
std::set<std::string> disabled_rule_substrings;
std::set<std::string> disabled_rule_tags;
std::set<std::string> enabled_rule_tags;
};
// Helper to load a set of files from filenames
static bool open_files(std::list<std::string> &filenames,
std::list<falco_engine::rulesfile> &rulesfiles,
std::string &errstr);
swappable_falco_engine();
virtual ~swappable_falco_engine();
bool init(config &cfg, sinsp *inspector, std::string &errstr);
std::shared_ptr<falco_engine> engine();
filter_check_list &plugin_filter_checks();
// Create a new engine, configure it using the saved config,
// load the provided set of rules files, and queue it to
// replace the current engine.
//
// This can be called from a different thread than the one
// calling engine().
//
// Returns true on success, false otherwise.
//
// load_result will be filled in with details on any errors (and warnings,
// if config.verbose is true).
bool replace(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result);
// Create a new engine, configure it, load the provided set of
// rules files, but do *not* queue it to replace the current
// engine.
//
// This can be called from a different thread than the one
// calling engine().
//
// Returns true on success, false otherwise.
//
// load_result will be filled in with details on any errors (and warnings,
// if config.verbose is true).
bool validate(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result);
private:
// Does everything but enqueue the new engine. Returns a
// shared_ptr to a new falco_engine on success, an empty
// shared_ptr on failure.
//
// load_result will be filled in with details on any errors (and warnings,
// if config.verbose is true).
std::shared_ptr<falco_engine> create_new(std::list<falco_engine::rulesfile> &rulesfiles,
std::string &load_result);
sinsp *m_inspector;
config m_config;
filter_check_list m_plugin_filter_checks;
std::shared_ptr<falco_engine> m_engine;
// If non-empty the head item will be moved to m_falco_engine
// with the next call to engine()
tbb::concurrent_queue<std::shared_ptr<falco_engine>> m_pending_falco_engine;
};

View File

@@ -27,8 +27,8 @@ using namespace std;
string k8s_audit_handler::m_k8s_audit_event_source = "k8s_audit";
k8s_audit_handler::k8s_audit_handler(falco_engine *engine, falco_outputs *outputs):
m_engine(engine), m_outputs(outputs)
k8s_audit_handler::k8s_audit_handler(swappable_falco_engine &swengine, falco_outputs *outputs):
m_swengine(swengine), m_outputs(outputs)
{
}
@@ -45,7 +45,7 @@ bool k8s_healthz_handler::handleGet(CivetServer *server, struct mg_connection *c
return true;
}
bool k8s_audit_handler::accept_data(falco_engine *engine,
bool k8s_audit_handler::accept_data(swappable_falco_engine &swengine,
falco_outputs *outputs,
std::string &data,
std::string &errstr)
@@ -89,7 +89,7 @@ bool k8s_audit_handler::accept_data(falco_engine *engine,
try
{
res = engine->process_event(m_k8s_audit_event_source, &jev);
res = swengine.engine()->process_event(m_k8s_audit_event_source, &jev);
}
catch(...)
{
@@ -120,7 +120,7 @@ bool k8s_audit_handler::accept_data(falco_engine *engine,
bool k8s_audit_handler::accept_uploaded_data(std::string &post_data, std::string &errstr)
{
return k8s_audit_handler::accept_data(m_engine, m_outputs, post_data, errstr);
return k8s_audit_handler::accept_data(m_swengine, m_outputs, post_data, errstr);
}
bool k8s_audit_handler::handleGet(CivetServer *server, struct mg_connection *conn)
@@ -177,7 +177,8 @@ bool k8s_audit_handler::handlePost(CivetServer *server, struct mg_connection *co
return true;
}
falco_webserver::falco_webserver():
falco_webserver::falco_webserver(swappable_falco_engine &swengine):
m_swengine(swengine),
m_config(NULL)
{
}
@@ -188,11 +189,9 @@ falco_webserver::~falco_webserver()
}
void falco_webserver::init(falco_configuration *config,
falco_engine *engine,
falco_outputs *outputs)
{
m_config = config;
m_engine = engine;
m_outputs = outputs;
}
@@ -214,11 +213,6 @@ void falco_webserver::start()
throw falco_exception("No config provided to webserver");
}
if(!m_engine)
{
throw falco_exception("No engine provided to webserver");
}
if(!m_outputs)
{
throw falco_exception("No outputs provided to webserver");
@@ -253,7 +247,7 @@ void falco_webserver::start()
throw falco_exception("Could not create embedded webserver");
}
m_k8s_audit_handler = make_unique<k8s_audit_handler>(m_engine, m_outputs);
m_k8s_audit_handler = make_unique<k8s_audit_handler>(m_swengine, m_outputs);
m_server->addHandler(m_config->m_webserver_k8s_audit_endpoint, *m_k8s_audit_handler);
m_k8s_healthz_handler = make_unique<k8s_healthz_handler>();
m_server->addHandler(m_config->m_webserver_k8s_healthz_endpoint, *m_k8s_healthz_handler);

View File

@@ -19,26 +19,26 @@ limitations under the License.
#include "CivetServer.h"
#include "configuration.h"
#include "falco_engine.h"
#include "swappable_falco_engine.h"
#include "falco_outputs.h"
class k8s_audit_handler : public CivetHandler
{
public:
k8s_audit_handler(falco_engine *engine, falco_outputs *outputs);
k8s_audit_handler(swappable_falco_engine &swengine, falco_outputs *outputs);
virtual ~k8s_audit_handler();
bool handleGet(CivetServer *server, struct mg_connection *conn);
bool handlePost(CivetServer *server, struct mg_connection *conn);
static bool accept_data(falco_engine *engine,
static bool accept_data(swappable_falco_engine &swengine,
falco_outputs *outputs,
std::string &post_data, std::string &errstr);
static std::string m_k8s_audit_event_source;
private:
falco_engine *m_engine;
swappable_falco_engine &m_swengine;
falco_outputs *m_outputs;
bool accept_uploaded_data(std::string &post_data, std::string &errstr);
};
@@ -60,18 +60,17 @@ public:
class falco_webserver
{
public:
falco_webserver();
falco_webserver(swappable_falco_engine &swengine);
virtual ~falco_webserver();
void init(falco_configuration *config,
falco_engine *engine,
falco_outputs *outputs);
void start();
void stop();
private:
falco_engine *m_engine;
swappable_falco_engine &m_swengine;
falco_configuration *m_config;
falco_outputs *m_outputs;
unique_ptr<CivetServer> m_server;