kubeshark/tap/tlstapper/bpf/common.c
M. Mert Yıldıran 2ad4838cf5
Add Go crypto/tls eBPF tracer for TLS connections (#1120)
* Run `go generate tls_tapper.go`

* Add `golang_uprobes.c`

* Add Golang hooks and offsets

* Add `golangConnection` struct and implement `pollGolangReadWrite` method

* Upgrade `github.com/cilium/ebpf` version to `v0.8.1`

* Fix the linter error

* Move map related stuff to `maps.h` and run `go generate tls_tapper.go`

* Remove unused parameter

* Add an environment variable to test Golang locally

* Replace `Libssl` occurrences with `Ssllib` for consistency

* Fix exe path finding

* Temporarily disable OpenSSL

* Fix the mixed offsets and dissection preparation

* Change the read symbol from `net/http.(*persistConn).Read` to `crypto/tls.(*Conn).Read`

* Remove `len` and `cap` fields

* Fix the indent

* Fix the read data address

* Make `golang_dial_writes` key `__u64` and include the PID

* Fix the read data address one more time

* Temporarily disable the PCAP capture

* Add a uprobe for `net/http.(*gzipReader).Read` to read chunked HTTP response body

* Cancel `golang_crypto_tls_read_uprobe` if it's a gzip read

* Make hash map names more meaningful

* Pass the connection address from `write` to `gzip` through a common address between `gzip` and `dial`

* Fix the probed line number links

* Add `golangReader` struct and implement its `Read` method

* Have a single counter pair and request response matcher per Golang connection

* Add `MIZU_GLOBAL_GOLANG_PATH` environment variable

* `NULL` terminate the bytes with `unix.ByteSliceToString`

* Temporarily reject the gzip chunks

* Add malformed TODOs

* Revert "`NULL` terminate the bytes with `unix.ByteSliceToString`"

This reverts commit 7ee7ef7e44.

* Bring back `len` and `cap` fields

* Set `len` and `cap` in `golang_net_http_gzipreader_read_uprobe` as well

* Remove two `TODO`s

* Fix the `key_gzip` offsets

* Compress if it's gzip chunk (probably wrong!)

* Revert "Compress if it's gzip chunk (probably wrong!)"

This reverts commit 094a7c3da4.

* Remove `golang_net_http_gzipreader_read_uprobe`

* Read constant 4KiB

* Use constant read length

* Get the correct len of bytes (saw the second entry)

* Set all buffer sizes to `CHUNK_SIZE`

* Remove a `TODO`

* Revert "Temporarily disable the PCAP capture"

This reverts commit a2da15ef2d.

* Update `golang_crypto_tls_read_uprobe`

* Set the `reader` field of `tlsStream` to fix a `nil pointer dereference` error

* Don't export any fields of `golangConnection`

* Close the reader when we drop the connection

* Add a tracepoint for `sys_enter_close` to detect socket closes

* Rename `socket` struct to `golang_socket`

* Call `should_tap` in Golang uprobes

* Add `log_error` calls

* Revert "Temporarily disable OpenSSL"

This reverts commit f54d9a453f.

* Fix linter

* Revert "Revert "Temporarily disable OpenSSL""

This reverts commit 2433d867af.

* Change `golang_read_writes` map type from `BPF_RINGBUF` to `BPF_PERF_OUTPUT`

* Rename `golang_read_write` to `golang_event`

* Define an error

* Add comments

* Revert "Revert "Revert "Temporarily disable OpenSSL"""

This reverts commit e5a1de9c71.

* Fix `pollGolang`

* Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""

This reverts commit 6e1bd5d4f3.

* Fix `panic: send on closed channel`

* Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""

This reverts commit 57d0584655.

* Use `findLibraryByPid`

* Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""""

This reverts commit 46f3d290b0.

* Revert "Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""""

This reverts commit 775c833c06.

* Log tapping Golang

* Fix `Poll`

* Refactor `golang_net_http_dialconn_uprobe`

* Remove an excess error check

* Fix `can only use path@version syntax with 'go get' and 'go install' in module-aware mode` error in `tap/tlstapper/bpf-builder/build.sh`

* Unify Golang and OpenSSL under a single perf event buffer and `tls_chunk` struct

* Generate `tlsTapperChunkType` type (enum) as well

* Use kernel page size for the `sys_closes` perf buffer

* Fix the linter error

* Fix `MIZU_GLOBAL_GOLANG_PID` environment variable's functionality

* Rely on tracepoints for file descriptor retrieval in Golang implementation

* Remove the unnecessary changes

* Move common functions into `common.c`

* Declare `lookup_ssl_info` function to reduce duplication

* Fix linter

* Add comments and TODOs

* Remove `MIZU_GLOBAL_GOLANG_PATH` environment variable

* Update the object files

* Fix indentation

* Update object files

* Add `go_abi_internal.h`

* Fix `lookup_ssl_info`

* Convert indentation to spaces

* Add header guard comment

* Add more comments

* Find the `ret` instructions using Capstone Engine and `uprobe` the `return` statements

* Implement `get_fd_from_tcp_conn` function

* Separate SSL contexts to OpenSSL and Go

* Move `get_count_bytes` from `common.c` to `openssl_uprobes.c`

* Rename everything contains Golang to Go

* Reduce duplication in `go_uprobes.c`

* Update the comments

* Install Capstone in CI and Docker native builds

* Update `devops/install-capstone.sh`

* Add Capstone to AArch64 cross-compilation target

* Fix some of the issues on ARM64

* Delete the map element in `_ex_urpobe`

* Remove an unsued `LOG_` macro

* Rename `aquynh` to `capstone-engine`

* Add comment

* Revert "Fix some of the issues on ARM64"

This reverts commit 0b3eceddf4.

* Revert "Revert "Fix some of the issues on ARM64""

This reverts commit 681534ada1.

* Update object files

* Remove unnecessary return

* Increase timeout

* #run_acceptance_tests

* #run_acceptance_tests

* Fix the `arm64v8` sourced builds

* #run_acceptance_tests
2022-06-12 14:14:41 +03:00

146 lines
4.4 KiB
C

/*
Note: This file is licenced differently from the rest of the project
SPDX-License-Identifier: GPL-2.0
Copyright (C) UP9 Inc.
*/
#include "include/headers.h"
#include "include/util.h"
#include "include/maps.h"
#include "include/log.h"
#include "include/logger_messages.h"
#include "include/common.h"
static __always_inline int add_address_to_chunk(struct pt_regs *ctx, struct tls_chunk* chunk, __u64 id, __u32 fd) {
__u32 pid = id >> 32;
__u64 key = (__u64) pid << 32 | fd;
struct fd_info *fdinfo = bpf_map_lookup_elem(&file_descriptor_to_ipv4, &key);
if (fdinfo == NULL) {
return 0;
}
int err = bpf_probe_read(chunk->address, sizeof(chunk->address), fdinfo->ipv4_addr);
chunk->flags |= (fdinfo->flags & FLAGS_IS_CLIENT_BIT);
if (err != 0) {
log_error(ctx, LOG_ERROR_READING_FD_ADDRESS, id, err, 0l);
return 0;
}
return 1;
}
static __always_inline void send_chunk_part(struct pt_regs *ctx, __u8* buffer, __u64 id,
struct tls_chunk* chunk, int start, int end) {
size_t recorded = MIN(end - start, sizeof(chunk->data));
if (recorded <= 0) {
return;
}
chunk->recorded = recorded;
chunk->start = start;
// This ugly trick is for the ebpf verifier happiness
//
long err = 0;
if (chunk->recorded == sizeof(chunk->data)) {
err = bpf_probe_read(chunk->data, sizeof(chunk->data), buffer + start);
} else {
recorded &= (sizeof(chunk->data) - 1); // Buffer must be N^2
err = bpf_probe_read(chunk->data, recorded, buffer + start);
}
if (err != 0) {
log_error(ctx, LOG_ERROR_READING_FROM_SSL_BUFFER, id, err, 0l);
return;
}
bpf_perf_event_output(ctx, &chunks_buffer, BPF_F_CURRENT_CPU, chunk, sizeof(struct tls_chunk));
}
static __always_inline void send_chunk(struct pt_regs *ctx, __u8* buffer, __u64 id, struct tls_chunk* chunk) {
// ebpf loops must be bounded at compile time, we can't use (i < chunk->len / CHUNK_SIZE)
//
// https://lwn.net/Articles/794934/
//
// However we want to run in kernel older than 5.3, hence we use "#pragma unroll" anyway
//
#pragma unroll
for (int i = 0; i < MAX_CHUNKS_PER_OPERATION; i++) {
if (chunk->len <= (CHUNK_SIZE * i)) {
break;
}
send_chunk_part(ctx, buffer, id, chunk, CHUNK_SIZE * i, chunk->len);
}
}
static __always_inline void output_ssl_chunk(struct pt_regs *ctx, struct ssl_info* info, int count_bytes, __u64 id, __u32 flags) {
if (count_bytes <= 0) {
return;
}
if (count_bytes > (CHUNK_SIZE * MAX_CHUNKS_PER_OPERATION)) {
log_error(ctx, LOG_ERROR_BUFFER_TOO_BIG, id, count_bytes, 0l);
return;
}
struct tls_chunk* chunk;
int zero = 0;
// If other thread, running on the same CPU get to this point at the same time like us (context switch)
// the data will be corrupted - protection may be added in the future
//
chunk = bpf_map_lookup_elem(&heap, &zero);
if (!chunk) {
log_error(ctx, LOG_ERROR_ALLOCATING_CHUNK, id, 0l, 0l);
return;
}
chunk->flags = flags;
chunk->pid = id >> 32;
chunk->tgid = id;
chunk->len = count_bytes;
chunk->fd = info->fd;
if (!add_address_to_chunk(ctx, chunk, id, chunk->fd)) {
// Without an address, we drop the chunk because there is not much to do with it in Go
//
return;
}
send_chunk(ctx, info->buffer, id, chunk);
}
static __always_inline struct ssl_info new_ssl_info() {
struct ssl_info info = { .fd = invalid_fd, .created_at_nano = bpf_ktime_get_ns() };
return info;
}
static __always_inline struct ssl_info lookup_ssl_info(struct pt_regs *ctx, struct bpf_map_def* map_fd, __u64 pid_tgid) {
struct ssl_info *infoPtr = bpf_map_lookup_elem(map_fd, &pid_tgid);
struct ssl_info info = new_ssl_info();
if (infoPtr != NULL) {
long err = bpf_probe_read(&info, sizeof(struct ssl_info), infoPtr);
if (err != 0) {
log_error(ctx, LOG_ERROR_READING_SSL_CONTEXT, pid_tgid, err, ORIGIN_SSL_UPROBE_CODE);
}
if ((bpf_ktime_get_ns() - info.created_at_nano) > SSL_INFO_MAX_TTL_NANO) {
// If the ssl info is too old, we don't want to use its info because it may be incorrect.
//
info.fd = invalid_fd;
info.created_at_nano = bpf_ktime_get_ns();
}
}
return info;
}