From d6d975e28c0f03f23504d3fcd20736aec8c7eb2b Mon Sep 17 00:00:00 2001 From: Mark Stemm Date: Tue, 28 Nov 2017 07:04:37 -0800 Subject: [PATCH] Refactor shell rules (#301) * Refactor shell rules to avoid FPs. Refactoring the shell related rules to avoid FPs. Instead of considering all shells suspicious and trying to carve out exceptions for the legitimate uses of shells, only consider shells spawned below certain processes suspicious. The set of processes is a collection of commonly used web servers, databases, nosql document stores, mail programs, message queues, process monitors, application servers, etc. Also, runsv is also considered a top level process that denotes a service. This allows a way for more flexible servers like ad-hoc nodejs express apps, etc to denote themselves as a full server process. * Update event generator to reflect new shell rules spawn_shell is now a silent action. its replacement is spawn_shell_under_httpd, which respawns itself as httpd and then runs a shell. db_program_spawn_binaries now runs ls instead of a shell so it only matches db_program_spawn_process. * Comment out old shell related rules * Modify nodejs example to work w/ new shell rules Start the express server using runit's runsv, which allows falco to consider any shells run by it as suspicious. * Use the updated argument for mkdir In https://github.com/draios/sysdig/pull/757 the path argument for mkdir moved to the second argument. This only became visible in the unit tests once the trace files were updated to reflect the other shell rule changes--the trace files had the old format. * Update unit tests for shell rules changes Shell in container doesn't exist any longer and its functionality has been subsumed by run shell untrusted. * Allow git binaries to run shells In some cases, these are run below a service runsv so we still need exceptions for them. * Let consul agent spawn curl for health checks * Don't protect tomcat There's enough evidence of people spawning general commands that we can't protect it. * Reorder exceptions, add rabbitmq exception Move the nginx exception to the main rule instead of the protected_shell_spawner macro. Also add erl_child_setup (related to rabbitmq) as an allowed shell spawner. * Add additional spawn binaries All off these are either below nginx, httpd, or runsv but should still be allowed to spawn shells. * Exclude shells when ancestor is a pkg mgmt binary Skip shells when any process ancestor (parent, gparent, etc) is a package management binary. This includes the program needrestart. This is a deep search but should prevent a lot of other more detailed exceptions trying to find the specific scripts run as a part of installations. * Skip shells related to serf Serf is a service discovery tool and can in some cases be spawned by apache/nginx. Also allow shells that are just checking the status of pids via kill -0. * Add several exclusions back Add several exclusions back from the shell in container rule. These are all allowed shell spawns that happen to be below nginx/fluentd/apache/etc. * Remove commented-out rules This saves space as well as cleanup. I haven't yet removed the macros/lists used by these rules and not used anywhere else. I'll do that cleanup in a separate step. * Also exclude based on command lines Add back the exclusions based on command lines, using the existing set of command lines. * Add addl exclusions for shells Of note is runsv, which means it can directly run shells (the ./run and ./finish scripts), but the things it runs can not. * Don't trigger on shells spawning shells We'll detect the first shell and not any other shells it spawns. * Allow "runc:" parents to count as a cont entrypnt In some cases, the initial process for a container can have a parent "runc:[0:PARENT]", so also allow those cases to count as a container entrypoint. * Use container_entrypoint macro Use the container_entrypoint macro to denote entering a container and also allow exe to be one of the processes that's the parent of an entrypoint. --- docker/event-generator/event_generator.cpp | 16 +- examples/nodejs-bad-rest-api/demo.yml | 4 +- examples/nodejs-bad-rest-api/run | 2 + rules/falco_rules.yaml | 243 +++++++++------------ test/falco_tests.yaml | 2 +- test/falco_traces.yaml.in | 11 +- 6 files changed, 121 insertions(+), 157 deletions(-) create mode 100755 examples/nodejs-bad-rest-api/run diff --git a/docker/event-generator/event_generator.cpp b/docker/event-generator/event_generator.cpp index 3311b6b0..dd8874c6 100644 --- a/docker/event-generator/event_generator.cpp +++ b/docker/event-generator/event_generator.cpp @@ -50,6 +50,8 @@ void usage(char *program) printf(" then read a sensitive file\n"); printf(" write_rpm_database Write to files below /var/lib/rpm\n"); printf(" spawn_shell Run a shell (bash)\n"); + printf(" Used by spawn_shell_under_httpd below\n"); + printf(" spawn_shell_under_httpd Run a shell (bash) under a httpd process\n"); printf(" db_program_spawn_process As a database program, try to spawn\n"); printf(" another program\n"); printf(" modify_binary_dirs Modify a file below /bin\n"); @@ -64,7 +66,7 @@ void usage(char *program) printf(" non_sudo_setuid Setuid as a non-root user\n"); printf(" create_files_below_dev Create files below /dev\n"); printf(" exec_ls execve() the program ls\n"); - printf(" (used by user_mgmt_binaries below)\n"); + printf(" (used by user_mgmt_binaries, db_program_spawn_process)\n"); printf(" user_mgmt_binaries Become the program \"vipw\", which triggers\n"); printf(" rules related to user management programs\n"); printf(" exfiltration Read /etc/shadow and send it via udp to a\n"); @@ -230,9 +232,14 @@ void spawn_shell() { } } +void spawn_shell_under_httpd() { + printf("Becoming the program \"httpd\" and then spawning a shell\n"); + respawn("./httpd", "spawn_shell", "0"); +} + void db_program_spawn_process() { - printf("Becoming the program \"mysql\" and then spawning a shell\n"); - respawn("./mysqld", "spawn_shell", "0"); + printf("Becoming the program \"mysql\" and then running ls\n"); + respawn("./mysqld", "exec_ls", "0"); } void modify_binary_dirs() { @@ -360,6 +367,7 @@ map defined_actions = {{"write_binary_dir", write_binary_dir}, {"read_sensitive_file_after_startup", read_sensitive_file_after_startup}, {"write_rpm_database", write_rpm_database}, {"spawn_shell", spawn_shell}, + {"spawn_shell_under_httpd", spawn_shell_under_httpd}, {"db_program_spawn_process", db_program_spawn_process}, {"modify_binary_dirs", modify_binary_dirs}, {"mkdir_binary_dirs", mkdir_binary_dirs}, @@ -375,7 +383,7 @@ map defined_actions = {{"write_binary_dir", write_binary_dir}, // Some actions don't directly result in suspicious behavior. These // actions are excluded from the ones run with -a all. -set exclude_from_all_actions = {"exec_ls", "network_activity"}; +set exclude_from_all_actions = {"spawn_shell", "exec_ls", "network_activity"}; void create_symlinks(const char *program) { diff --git a/examples/nodejs-bad-rest-api/demo.yml b/examples/nodejs-bad-rest-api/demo.yml index a1f94809..00b45d2c 100644 --- a/examples/nodejs-bad-rest-api/demo.yml +++ b/examples/nodejs-bad-rest-api/demo.yml @@ -1,9 +1,7 @@ -# Owned by software vendor, serving install-software.sh. express_server: container_name: express_server image: node:latest - working_dir: /usr/src/app - command: bash -c "npm install && node server.js" + command: bash -c "apt-get -y update && apt-get -y install runit && npm install && runsv /usr/src/app" ports: - "8181:8181" volumes: diff --git a/examples/nodejs-bad-rest-api/run b/examples/nodejs-bad-rest-api/run new file mode 100755 index 00000000..efc63234 --- /dev/null +++ b/examples/nodejs-bad-rest-api/run @@ -0,0 +1,2 @@ +#!/bin/sh +node server.js diff --git a/rules/falco_rules.yaml b/rules/falco_rules.yaml index 6ce68f39..5c35fd6b 100644 --- a/rules/falco_rules.yaml +++ b/rules/falco_rules.yaml @@ -41,10 +41,10 @@ - macro: bin_dir_mkdir condition: > - evt.arg[0] startswith /bin/ or - evt.arg[0] startswith /sbin/ or - evt.arg[0] startswith /usr/bin/ or - evt.arg[0] startswith /usr/sbin/ + (evt.arg[1] startswith /bin/ or + evt.arg[1] startswith /sbin/ or + evt.arg[1] startswith /usr/bin/ or + evt.arg[1] startswith /usr/sbin/) - macro: bin_dir_rename condition: > @@ -156,10 +156,10 @@ items: [chef-client] - list: http_server_binaries - items: [nginx, httpd, httpd-foregroun, lighttpd] + items: [nginx, httpd, httpd-foregroun, lighttpd, apache, apache2] - list: db_server_binaries - items: [mysqld] + items: [mysqld, postgres, sqlplus] - list: mysql_mgmt_binaries items: [mysql_install_d, mysql_ssl_rsa_s] @@ -170,6 +170,9 @@ - list: db_mgmt_binaries items: [mysql_mgmt_binaries, postgres_mgmt_binaries] +- list: nosql_server_binaries + items: [couchdb, memcached, redis-server, rabbitmq-server, mongod] + - list: gitlab_binaries items: [gitlab-shell, gitlab-mon, gitlab-runner-b, git] @@ -199,6 +202,9 @@ - macro: package_mgmt_procs condition: proc.name in (package_mgmt_binaries) +- macro: run_by_package_mgmt_binaries + condition: proc.aname in (package_mgmt_binaries, needrestart) + - list: ssl_mgmt_binaries items: [ca-certificates] @@ -562,9 +568,6 @@ - macro: parent_java_running_confluence condition: (proc.pname=java and proc.pcmdline contains "-classpath /opt/atlassian/confluence") -- macro: parent_java_running_tomcat - condition: (proc.pname=java and proc.pcmdline contains "-classpath /usr/local/tomcat") - - macro: parent_java_running_install4j condition: (proc.pname=java and proc.pcmdline contains "-classpath i4jruntime.jar") @@ -989,66 +992,104 @@ mysql_upgrade, opkg-cl, vmtoolsd, confd ] -- rule: Run shell untrusted - desc: an attempt to spawn a shell by a non-shell program. Exceptions are made for trusted binaries. +# The binaries in this list and their descendents are *not* allowed +# spawn shells. This includes the binaries spawning shells directly as +# well as indirectly. For example, apache -> php/perl for +# mod_{php,perl} -> some shell is also not allowed, because the shell +# has apache as an ancestor. + +- list: protected_shell_spawning_binaries + items: [ + http_server_binaries, db_server_binaries, nosql_server_binaries, mail_binaries, + fluentd, flanneld, splunkd, consul, runsv + ] + +- macro: parent_java_running_zookeeper + condition: (proc.pname=java and proc.pcmdline contains org.apache.zookeeper.server) + +- macro: parent_java_running_kafka + condition: (proc.pname=java and proc.pcmdline contains kafka.Kafka) + +- macro: parent_java_running_elasticsearch + condition: (proc.pname=java and proc.pcmdline contains org.elasticsearch.bootstrap.Elasticsearch) + +- macro: parent_java_running_activemq + condition: (proc.pname=java and proc.pcmdline contains activemq.jar) + +- macro: parent_java_running_cassandra + condition: (proc.pname=java and proc.pcmdline contains org.apache.cassandra.service.CassandraDaemon) + +- macro: parent_java_running_jboss_wildfly + condition: (proc.pname=java and proc.pcmdline contains org.jboss) + +- macro: parent_java_running_glassfish + condition: (proc.pname=java and proc.pcmdline contains com.sun.enterprise.glassfish) + +- macro: parent_java_running_hadoop + condition: (proc.pname=java and proc.pcmdline contains org.apache.hadoop) + +- macro: parent_java_running_datastax + condition: (proc.pname=java and proc.pcmdline contains com.datastax) + +- macro: parent_java_running_sumologic + condition: (proc.pname=java and proc.pcmdline contains com.sumologic) + +- macro: nginx_starting_nginx + condition: (proc.pname=nginx and proc.cmdline contains "/usr/sbin/nginx -c /etc/nginx/nginx.conf") + +- macro: consul_running_curl + condition: (proc.pname=consul and proc.cmdline startswith "sh -c curl") + +- macro: serf_script + condition: (proc.cmdline startswith "sh -c serf") + +- macro: check_process_status + condition: (proc.cmdline startswith "sh -c kill -0 ") + +- macro: protected_shell_spawner condition: > - spawned_process and not container + (proc.aname in (protected_shell_spawning_binaries) + or parent_java_running_zookeeper + or parent_java_running_kafka + or parent_java_running_elasticsearch + or parent_java_running_activemq + or parent_java_running_cassandra + or parent_java_running_jboss_wildfly + or parent_java_running_glassfish + or parent_java_running_hadoop + or parent_java_running_datastax) + +# Note that runsv is both in protected_shell_spawner and the +# exclusions by pname. This means that runsv can itself spawn shells +# (the ./run and ./finish scripts), but the processes runsv can not +# spawn shells. +- rule: Run shell untrusted + desc: an attempt to spawn a shell below a non-shell application. Specific applications are monitored. + condition: > + spawned_process and shell_procs and proc.pname exists - and not proc.pname in (cron_binaries, shell_binaries, make_binaries, known_shell_spawn_binaries, docker_binaries, - k8s_binaries, package_mgmt_binaries, aide_wrapper_binaries, nids_binaries, - monitoring_binaries, gitlab_binaries, mesos_slave_binaries, - keepalived_binaries, - needrestart_binaries, phusion_passenger_binaries, chef_binaries, nomachine_binaries, - x2go_binaries, db_mgmt_binaries, plesk_binaries) - and not parent_ansible_running_python - and not parent_bro_running_python - and not parent_python_running_denyhosts - and not parent_python_running_sdchecks - and not parent_linux_image_upgrade_script - and not parent_java_running_jenkins + and protected_shell_spawner + and not proc.pname in (shell_binaries, gitlab_binaries, cron_binaries, + erl_child_setup, exechealthz, + PM2, PassengerWatchd, c_rehash, svlogd, logrotate, hhvm, serf, + lb-controller, nvidia-installe, runsv, statsite) and not proc.cmdline in (known_shell_spawn_cmdlines) - and not jenkins_scripts - and not parent_java_running_echo - and not parent_scripting_running_builds - and not makefile_perl - and not parent_Xvfb_running_xkbcomp - and not parent_nginx_running_serf - and not parent_node_running_npm - and not parent_npm_running_node - and not parent_java_running_sbt - and not parent_beam_running_python - and not parent_strongswan_running_starter - and not run_by_chef - and not run_by_puppet - and not run_by_adclient - and not run_by_centrify - and not parent_dovecot_running_auth + and not proc.aname in (unicorn_launche) + and not consul_running_curl + and not nginx_starting_nginx + and not run_by_package_mgmt_binaries + and not serf_script + and not check_process_status and not run_by_foreman - and not run_by_openshift - and not parent_java_running_tomcat - and not parent_java_running_install4j - and not parent_java_running_endeca - and not parent_running_datastax - and not parent_java_running_appdynamics - and not parent_cpanm_running_perl - and not parent_ruby_running_discourse - and not parent_ruby_running_pups - and not assemble_running_php - and not node_running_bitnami - and not node_running_threatstack - and not parent_python_running_localstack - and not parent_python_running_zookeeper - and not parent_python_running_airflow - and not perl_running_plesk - and not plesk_autoinstaller - and not parent_perl_running_openresty + and not python_mesos_marathon_scripting + and not user_shell_container_exclusions output: > Shell spawned by untrusted binary (user=%user.name shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline pcmdline=%proc.pcmdline gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4] ggggparent=%proc.aname[5]) priority: DEBUG - tags: [host, shell] + tags: [shell] - macro: trusted_containers condition: (container.image startswith sysdig/agent or @@ -1122,7 +1163,7 @@ # when we lose events and lose track of state. - macro: container_entrypoint - condition: (not proc.pname exists or proc.pname in (runc:[0:PARENT], runc:[1:CHILD], docker-runc)) + condition: (not proc.pname exists or proc.pname in (runc:[0:PARENT], runc:[1:CHILD], docker-runc, exe)) - rule: Launch Sensitive Mount Container desc: > @@ -1171,11 +1212,11 @@ tags: [users] - rule: Terminal shell in container - desc: A shell was spawned by a program in a container with an attached terminal. + desc: A shell was used as the entrypoint/exec point into a container with an attached terminal. condition: > spawned_process and container and shell_procs and proc.tty != 0 - and not proc.cmdline in (known_shell_spawn_cmdlines) + and container_entrypoint output: > A shell was spawned in a container with an attached terminal (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty) @@ -1262,84 +1303,6 @@ (proc.pname=node and (proc.pcmdline contains /var/www/edi/process.js or proc.pcmdline contains "sh -c /var/www/edi/bin/sftp.sh")) -- rule: Run shell in container - desc: a shell was spawned by a non-shell program in a container. Container entrypoints are excluded. - condition: > - spawned_process and container - and shell_procs - and not container_entrypoint - and not proc.pname in (shell_binaries, make_binaries, docker_binaries, k8s_binaries, package_mgmt_binaries, - lxd_binaries, mesos_slave_binaries, aide_wrapper_binaries, nids_binaries, - cron_binaries, - user_known_container_shell_spawn_binaries, - needrestart_binaries, - phusion_passenger_binaries, - chef_binaries, - nomachine_binaries, - x2go_binaries, - db_mgmt_binaries, - plesk_binaries, - monitoring_binaries, gitlab_binaries, initdb, awk, falco, cron, - erl_child_setup, erlexec, ceph, PM2, pycompile, py3compile, hhvm, npm, serf, - runsv, supervisord, varnishd, crond, logrotate, timeout, tini, - xrdb, xfce4-session, weave, logdna-agent, bundle, configure, luajit, nginx, - beam.smp, paster, postfix-local, hawkular-metric, fluentd, x2gormforward, - "[celeryd:", flock, nsrun, consul, migrate-databas, airflow, bootstrap-qmf-l, - build-qmf-artif, colormake.pl, doxygen, Cypress, lb-controller, vmtoolsd, - haproxy_reload., curator, consul-template, xargs, scl, find, awstats_updatea, - sa-update, mysql_upgrade, opkg-cl, peer-finder, confd, aws) - and not trusted_containers - and not shell_spawning_containers - and not parent_java_running_echo - and not parent_scripting_running_builds - and not makefile_perl - and not parent_Xvfb_running_xkbcomp - and not mysql_image_running_healthcheck - and not parent_nginx_running_serf - and not proc.cmdline in (known_container_shell_spawn_cmdlines) - and not parent_node_running_npm - and not parent_npm_running_node - and not user_shell_container_exclusions - and not node_running_edi_dynamodb - and not run_by_h2o - and not run_by_passenger_agent - and not parent_java_running_jenkins - and not parent_java_running_maven - and not parent_java_running_appdynamics - and not parent_java_running_sbt - and not python_running_es_curator - and not parent_beam_running_python - and not jenkins_scripts - and not bundle_running_ruby - and not parent_dovecot_running_auth - and not parent_strongswan_running_starter - and not parent_phusion_passenger_my_init - and not parent_java_running_confluence - and not parent_java_running_tomcat - and not parent_java_running_install4j - and not parent_running_datastax - and not ics_running_java - and not parent_ruby_running_discourse - and not parent_ruby_running_pups - and not assemble_running_php - and not node_running_bitnami - and not node_running_threatstack - and not parent_python_running_localstack - and not parent_python_running_zookeeper - and not parent_python_running_airflow - and not parent_docker_start_script - and not parent_java_running_endeca - and not python_mesos_healthcheck - and not python_mesos_marathon_scripting - and not perl_running_plesk - and not parent_rancher_running_healthcheck - and not parent_perl_running_openresty - output: > - Shell spawned in a container other than entrypoint (user=%user.name %container.info image=%container.image - shell=%proc.name pcmdline=%proc.pcmdline cmdline=%proc.cmdline parent=%proc.pname gparent=%proc.aname[2] ggparent=%proc.aname[3]) - priority: DEBUG - tags: [container, shell] - - macro: login_doing_dns_lookup condition: (proc.name=login and fd.l4proto=udp and fd.sport=53) diff --git a/test/falco_tests.yaml b/test/falco_tests.yaml index 485e87fb..f176a9b9 100644 --- a/test/falco_tests.yaml +++ b/test/falco_tests.yaml @@ -319,7 +319,7 @@ trace_files: !mux detect_counts: - "Write below binary dir": 1 - "Read sensitive file untrusted": 3 - - "Run shell in container": 1 + - "Run shell untrusted": 1 - "Write below rpm database": 1 - "Write below etc": 1 - "System procs network activity": 1 diff --git a/test/falco_traces.yaml.in b/test/falco_traces.yaml.in index 10d0c84b..1245160c 100644 --- a/test/falco_traces.yaml.in +++ b/test/falco_traces.yaml.in @@ -43,11 +43,11 @@ traces: !mux falco-event-generator: trace_file: traces-positive/falco-event-generator.scap detect: True - detect_level: [ERROR, WARNING, INFO, NOTICE] + detect_level: [ERROR, WARNING, INFO, NOTICE, DEBUG] detect_counts: - "Write below binary dir": 1 - "Read sensitive file untrusted": 3 - - "Run shell in container": 1 + - "Run shell untrusted": 1 - "Write below rpm database": 1 - "Write below etc": 1 - "System procs network activity": 1 @@ -146,13 +146,6 @@ traces: !mux detect_counts: - "Run shell untrusted": 1 - shell-in-container: - trace_file: traces-positive/shell-in-container.scap - detect: True - detect_level: DEBUG - detect_counts: - - "Run shell in container": 1 - system-binaries-network-activity: trace_file: traces-positive/system-binaries-network-activity.scap detect: True