Commit Graph

11 Commits

Author SHA1 Message Date
M. Mert Yıldıran
52c9251c00
Add ABI0 support to Go crypto/tls eBPF tracer (#1169)
* Determine the Go ABI and get `goid` offset from DWARF

* Add `ABI` enum and morph the function according to the detected ABI

* Pass `goid` offset to an eBPF map to retrieve it in eBPF context

* Add `vmlinux.h` and implement `get_goid_from_thread_local_storage`

* Fix BPF verifier errors

* Update the comments

* Add `go_abi_0.h` and implement `ABI0` specific reads for `arm64`

* Upgrade `github.com/cilium/ebpf` to `v0.9.0`

* Add a comment

* Add macros for x86 specific parts

* Update `x86.o`

* Fix the map key type

* Add `user_pt_regs`

* Update arm64 object file

* Fix the version detection logic

* Add `getGStructOffset` method

* Define `goid_offsets`, `goid_offsets_map` structs and pass the offsets correctly

* Fix the `net.TCPConn` and buffer addresses for `ABI0`

* Remove comment

* Fix the issues for arm64 build

* Update x86.o

* Revert "Fix the issues for arm64 build"

This reverts commit 48b041b1b6.

* Revert `user_pt_regs`

* Add `vmlinux` directory

* Fix the `build.sh` and `Dockerfile`

* Add vmlinux_arm64.h

* Disable `get_goid_from_thread_local_storage` on ARM64 with a macro

* Update x86.o

* Update arm64.o

* x86

* arm64

* Fix the cross-compilation issue from x86 to arm64

* Fix the same thing for x86

* Use `BPF_CORE_READ` macro instead of `bpf_ringbuf_reserve` to support kernel versions older than 5.8

Also;
Add legacy version of thread_struct: thread_struct___v46
Build an additional object file for the kernel versions older than or equal to 4.6 and load them accordingly.
Add github.com/moby/moby

* Make #define directives more definitive

* Select the x86 and arm64 versions of `vmlinux.h` using macros

* Put `goid` offsets into the map before installing `uprobe`(s)

* arm64

* #run_acceptance_tests

* Remove a forgotten `fmt.Printf`

* Log the detected Linux kernel version
2022-07-05 14:35:30 +03:00
David Levanon
48619b3e1c
fix small regression with openssl (#1176) 2022-06-30 17:39:03 +03:00
M. Mert Yıldıran
cf91f3dab4
Fix Go crypto/tls eBPF tracer on ARM64 (#1162)
* Handle the Go `ABIInternal` differences on ARM64

* Log the Capstone version, arch and mode

* Upgrade Capstone from `4.0.2` to `5.0-rc2` to have ARM64 instructions fix

* Use the correct register on ARM64 for reading buffer length

* Fix the addresses on ARM64

* #run_acceptance_tests

* Update `x86.o`

* Update `arm64.o`

* #run_acceptance_tests
2022-06-26 14:11:04 +03:00
M. Mert Yıldıran
f9a9c05f48
Add ARM64 support to eBPF (#1151)
* Define and use `BPF_CFLAGS` environment variable

* Add eBPF dependencies to `builder-from-amd64-to-arm64v8` and `builder-native-base`

* Add eBPF dependencies to `builder-from-arm64v8-to-amd64`

* Only compile x86 arch of Capstone for x86 target

* Build and install `libbpf` from source

* Fix `builder-from-arm64v8-to-amd64`

* Add `BPF_TARGET` environment variable

* Fix the eBPF verifier error on ARM64

* Fix `go_crypto_tls_ex_uprobe`

* Fix the check

* #run_acceptance_tests

* Fix the build script

* Include ARM64 files

* Bring back `x86.o`

* Generate both endianness

* Fix Dockerfile

* #run_acceptance_tests

* Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests

* Revert "Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests"

This reverts commit a2c83c6040.

* Remove big-endian files #run_acceptance_tests

* Fix Dockerfile #run_acceptance_tests

Co-authored-by: Ubuntu <ubuntu@ip-172-31-33-233.eu-central-1.compute.internal>
2022-06-20 16:16:56 +03:00
M. Mert Yıldıran
2ad4838cf5
Add Go crypto/tls eBPF tracer for TLS connections (#1120)
* Run `go generate tls_tapper.go`

* Add `golang_uprobes.c`

* Add Golang hooks and offsets

* Add `golangConnection` struct and implement `pollGolangReadWrite` method

* Upgrade `github.com/cilium/ebpf` version to `v0.8.1`

* Fix the linter error

* Move map related stuff to `maps.h` and run `go generate tls_tapper.go`

* Remove unused parameter

* Add an environment variable to test Golang locally

* Replace `Libssl` occurrences with `Ssllib` for consistency

* Fix exe path finding

* Temporarily disable OpenSSL

* Fix the mixed offsets and dissection preparation

* Change the read symbol from `net/http.(*persistConn).Read` to `crypto/tls.(*Conn).Read`

* Remove `len` and `cap` fields

* Fix the indent

* Fix the read data address

* Make `golang_dial_writes` key `__u64` and include the PID

* Fix the read data address one more time

* Temporarily disable the PCAP capture

* Add a uprobe for `net/http.(*gzipReader).Read` to read chunked HTTP response body

* Cancel `golang_crypto_tls_read_uprobe` if it's a gzip read

* Make hash map names more meaningful

* Pass the connection address from `write` to `gzip` through a common address between `gzip` and `dial`

* Fix the probed line number links

* Add `golangReader` struct and implement its `Read` method

* Have a single counter pair and request response matcher per Golang connection

* Add `MIZU_GLOBAL_GOLANG_PATH` environment variable

* `NULL` terminate the bytes with `unix.ByteSliceToString`

* Temporarily reject the gzip chunks

* Add malformed TODOs

* Revert "`NULL` terminate the bytes with `unix.ByteSliceToString`"

This reverts commit 7ee7ef7e44.

* Bring back `len` and `cap` fields

* Set `len` and `cap` in `golang_net_http_gzipreader_read_uprobe` as well

* Remove two `TODO`s

* Fix the `key_gzip` offsets

* Compress if it's gzip chunk (probably wrong!)

* Revert "Compress if it's gzip chunk (probably wrong!)"

This reverts commit 094a7c3da4.

* Remove `golang_net_http_gzipreader_read_uprobe`

* Read constant 4KiB

* Use constant read length

* Get the correct len of bytes (saw the second entry)

* Set all buffer sizes to `CHUNK_SIZE`

* Remove a `TODO`

* Revert "Temporarily disable the PCAP capture"

This reverts commit a2da15ef2d.

* Update `golang_crypto_tls_read_uprobe`

* Set the `reader` field of `tlsStream` to fix a `nil pointer dereference` error

* Don't export any fields of `golangConnection`

* Close the reader when we drop the connection

* Add a tracepoint for `sys_enter_close` to detect socket closes

* Rename `socket` struct to `golang_socket`

* Call `should_tap` in Golang uprobes

* Add `log_error` calls

* Revert "Temporarily disable OpenSSL"

This reverts commit f54d9a453f.

* Fix linter

* Revert "Revert "Temporarily disable OpenSSL""

This reverts commit 2433d867af.

* Change `golang_read_writes` map type from `BPF_RINGBUF` to `BPF_PERF_OUTPUT`

* Rename `golang_read_write` to `golang_event`

* Define an error

* Add comments

* Revert "Revert "Revert "Temporarily disable OpenSSL"""

This reverts commit e5a1de9c71.

* Fix `pollGolang`

* Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""

This reverts commit 6e1bd5d4f3.

* Fix `panic: send on closed channel`

* Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""

This reverts commit 57d0584655.

* Use `findLibraryByPid`

* Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""""

This reverts commit 46f3d290b0.

* Revert "Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""""

This reverts commit 775c833c06.

* Log tapping Golang

* Fix `Poll`

* Refactor `golang_net_http_dialconn_uprobe`

* Remove an excess error check

* Fix `can only use path@version syntax with 'go get' and 'go install' in module-aware mode` error in `tap/tlstapper/bpf-builder/build.sh`

* Unify Golang and OpenSSL under a single perf event buffer and `tls_chunk` struct

* Generate `tlsTapperChunkType` type (enum) as well

* Use kernel page size for the `sys_closes` perf buffer

* Fix the linter error

* Fix `MIZU_GLOBAL_GOLANG_PID` environment variable's functionality

* Rely on tracepoints for file descriptor retrieval in Golang implementation

* Remove the unnecessary changes

* Move common functions into `common.c`

* Declare `lookup_ssl_info` function to reduce duplication

* Fix linter

* Add comments and TODOs

* Remove `MIZU_GLOBAL_GOLANG_PATH` environment variable

* Update the object files

* Fix indentation

* Update object files

* Add `go_abi_internal.h`

* Fix `lookup_ssl_info`

* Convert indentation to spaces

* Add header guard comment

* Add more comments

* Find the `ret` instructions using Capstone Engine and `uprobe` the `return` statements

* Implement `get_fd_from_tcp_conn` function

* Separate SSL contexts to OpenSSL and Go

* Move `get_count_bytes` from `common.c` to `openssl_uprobes.c`

* Rename everything contains Golang to Go

* Reduce duplication in `go_uprobes.c`

* Update the comments

* Install Capstone in CI and Docker native builds

* Update `devops/install-capstone.sh`

* Add Capstone to AArch64 cross-compilation target

* Fix some of the issues on ARM64

* Delete the map element in `_ex_urpobe`

* Remove an unsued `LOG_` macro

* Rename `aquynh` to `capstone-engine`

* Add comment

* Revert "Fix some of the issues on ARM64"

This reverts commit 0b3eceddf4.

* Revert "Revert "Fix some of the issues on ARM64""

This reverts commit 681534ada1.

* Update object files

* Remove unnecessary return

* Increase timeout

* #run_acceptance_tests

* #run_acceptance_tests

* Fix the `arm64v8` sourced builds

* #run_acceptance_tests
2022-06-12 14:14:41 +03:00
David Levanon
5c012641a5
Feature/limit fd to address map (#1078)
* limit fd to address map

* limit fd to address map

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-05-16 12:32:26 +03:00
David Levanon
57f8a8dca9
Feature/fix tls not listening (#1046)
* avoid chunks with invalid address

* tls tapper should distict between pids

* prettfy tls verbose log and tls key

* support tls from multi threads + duplicate calls to the same target

* introduce fdCache and user address pair as tls key

* remove unused comment

* fix merge conflicts

* use lru for fdcache

* pr fixes - renaming

* fix conflict issue
2022-05-02 21:33:26 +03:00
David Levanon
590fa08c81
EBPF error handling 2022-03-28 14:19:06 +03:00
David Levanon
12f46da5c6
Support TLS big buffers (#893) 2022-03-23 11:27:06 +02:00
David Levanon
6979441422
tls missing addresses (#825)
* stream seen file descriptors from ebpf

* re-generate bpf object files

* fixing pr comments
2022-03-14 15:40:27 +02:00
David Levanon
87ef469e25
Add tls tapper (#683)
* initial tls tapper commit

* add tls flag to mizu cli

* support ssl_read_ex/ssl_write_ex

* use hostproc to find libssl

* auto discover tls processes

* support libssl1.0

* recompile ebpf with old clang/llvm

* Update tap/passive_tapper.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* Update tap/tlstapper/tls_poller.go

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>

* upgrade ebpf go lib

* handling big tls messages

* fixing max buffer size in ebpf

* remove unused import

* fix linter issues

* minor pr fixes

* compile with old clang

* fix cgroup file format

* pr fixes + cgroup extract enhance

* fix linter

* adding indirect ebpf dep to agent go.mod

* adding ebpf docker builder

* minor pr fixes

* add req resp matcher to dissect

* rename ssl hooks to ssl hooks structs

* move to alpine, use local copy of mizu instead of git, add readme

* use global req resp mather for tls

Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-02-16 15:34:51 +02:00