Fetch source and destination addresses with bpf from tcp kprobes, similar to how it is done for openssl lib.
Chunk contains both source address and destination address.
FD is no longer used to obtain addresses.
* Determine the Go ABI and get `goid` offset from DWARF
* Add `ABI` enum and morph the function according to the detected ABI
* Pass `goid` offset to an eBPF map to retrieve it in eBPF context
* Add `vmlinux.h` and implement `get_goid_from_thread_local_storage`
* Fix BPF verifier errors
* Update the comments
* Add `go_abi_0.h` and implement `ABI0` specific reads for `arm64`
* Upgrade `github.com/cilium/ebpf` to `v0.9.0`
* Add a comment
* Add macros for x86 specific parts
* Update `x86.o`
* Fix the map key type
* Add `user_pt_regs`
* Update arm64 object file
* Fix the version detection logic
* Add `getGStructOffset` method
* Define `goid_offsets`, `goid_offsets_map` structs and pass the offsets correctly
* Fix the `net.TCPConn` and buffer addresses for `ABI0`
* Remove comment
* Fix the issues for arm64 build
* Update x86.o
* Revert "Fix the issues for arm64 build"
This reverts commit 48b041b1b6.
* Revert `user_pt_regs`
* Add `vmlinux` directory
* Fix the `build.sh` and `Dockerfile`
* Add vmlinux_arm64.h
* Disable `get_goid_from_thread_local_storage` on ARM64 with a macro
* Update x86.o
* Update arm64.o
* x86
* arm64
* Fix the cross-compilation issue from x86 to arm64
* Fix the same thing for x86
* Use `BPF_CORE_READ` macro instead of `bpf_ringbuf_reserve` to support kernel versions older than 5.8
Also;
Add legacy version of thread_struct: thread_struct___v46
Build an additional object file for the kernel versions older than or equal to 4.6 and load them accordingly.
Add github.com/moby/moby
* Make #define directives more definitive
* Select the x86 and arm64 versions of `vmlinux.h` using macros
* Put `goid` offsets into the map before installing `uprobe`(s)
* arm64
* #run_acceptance_tests
* Remove a forgotten `fmt.Printf`
* Log the detected Linux kernel version
* Define and use `BPF_CFLAGS` environment variable
* Add eBPF dependencies to `builder-from-amd64-to-arm64v8` and `builder-native-base`
* Add eBPF dependencies to `builder-from-arm64v8-to-amd64`
* Only compile x86 arch of Capstone for x86 target
* Build and install `libbpf` from source
* Fix `builder-from-arm64v8-to-amd64`
* Add `BPF_TARGET` environment variable
* Fix the eBPF verifier error on ARM64
* Fix `go_crypto_tls_ex_uprobe`
* Fix the check
* #run_acceptance_tests
* Fix the build script
* Include ARM64 files
* Bring back `x86.o`
* Generate both endianness
* Fix Dockerfile
* #run_acceptance_tests
* Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests
* Revert "Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests"
This reverts commit a2c83c6040.
* Remove big-endian files #run_acceptance_tests
* Fix Dockerfile #run_acceptance_tests
Co-authored-by: Ubuntu <ubuntu@ip-172-31-33-233.eu-central-1.compute.internal>
* Run `go generate tls_tapper.go`
* Add `golang_uprobes.c`
* Add Golang hooks and offsets
* Add `golangConnection` struct and implement `pollGolangReadWrite` method
* Upgrade `github.com/cilium/ebpf` version to `v0.8.1`
* Fix the linter error
* Move map related stuff to `maps.h` and run `go generate tls_tapper.go`
* Remove unused parameter
* Add an environment variable to test Golang locally
* Replace `Libssl` occurrences with `Ssllib` for consistency
* Fix exe path finding
* Temporarily disable OpenSSL
* Fix the mixed offsets and dissection preparation
* Change the read symbol from `net/http.(*persistConn).Read` to `crypto/tls.(*Conn).Read`
* Remove `len` and `cap` fields
* Fix the indent
* Fix the read data address
* Make `golang_dial_writes` key `__u64` and include the PID
* Fix the read data address one more time
* Temporarily disable the PCAP capture
* Add a uprobe for `net/http.(*gzipReader).Read` to read chunked HTTP response body
* Cancel `golang_crypto_tls_read_uprobe` if it's a gzip read
* Make hash map names more meaningful
* Pass the connection address from `write` to `gzip` through a common address between `gzip` and `dial`
* Fix the probed line number links
* Add `golangReader` struct and implement its `Read` method
* Have a single counter pair and request response matcher per Golang connection
* Add `MIZU_GLOBAL_GOLANG_PATH` environment variable
* `NULL` terminate the bytes with `unix.ByteSliceToString`
* Temporarily reject the gzip chunks
* Add malformed TODOs
* Revert "`NULL` terminate the bytes with `unix.ByteSliceToString`"
This reverts commit 7ee7ef7e44.
* Bring back `len` and `cap` fields
* Set `len` and `cap` in `golang_net_http_gzipreader_read_uprobe` as well
* Remove two `TODO`s
* Fix the `key_gzip` offsets
* Compress if it's gzip chunk (probably wrong!)
* Revert "Compress if it's gzip chunk (probably wrong!)"
This reverts commit 094a7c3da4.
* Remove `golang_net_http_gzipreader_read_uprobe`
* Read constant 4KiB
* Use constant read length
* Get the correct len of bytes (saw the second entry)
* Set all buffer sizes to `CHUNK_SIZE`
* Remove a `TODO`
* Revert "Temporarily disable the PCAP capture"
This reverts commit a2da15ef2d.
* Update `golang_crypto_tls_read_uprobe`
* Set the `reader` field of `tlsStream` to fix a `nil pointer dereference` error
* Don't export any fields of `golangConnection`
* Close the reader when we drop the connection
* Add a tracepoint for `sys_enter_close` to detect socket closes
* Rename `socket` struct to `golang_socket`
* Call `should_tap` in Golang uprobes
* Add `log_error` calls
* Revert "Temporarily disable OpenSSL"
This reverts commit f54d9a453f.
* Fix linter
* Revert "Revert "Temporarily disable OpenSSL""
This reverts commit 2433d867af.
* Change `golang_read_writes` map type from `BPF_RINGBUF` to `BPF_PERF_OUTPUT`
* Rename `golang_read_write` to `golang_event`
* Define an error
* Add comments
* Revert "Revert "Revert "Temporarily disable OpenSSL"""
This reverts commit e5a1de9c71.
* Fix `pollGolang`
* Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""
This reverts commit 6e1bd5d4f3.
* Fix `panic: send on closed channel`
* Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""
This reverts commit 57d0584655.
* Use `findLibraryByPid`
* Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""""
This reverts commit 46f3d290b0.
* Revert "Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""""
This reverts commit 775c833c06.
* Log tapping Golang
* Fix `Poll`
* Refactor `golang_net_http_dialconn_uprobe`
* Remove an excess error check
* Fix `can only use path@version syntax with 'go get' and 'go install' in module-aware mode` error in `tap/tlstapper/bpf-builder/build.sh`
* Unify Golang and OpenSSL under a single perf event buffer and `tls_chunk` struct
* Generate `tlsTapperChunkType` type (enum) as well
* Use kernel page size for the `sys_closes` perf buffer
* Fix the linter error
* Fix `MIZU_GLOBAL_GOLANG_PID` environment variable's functionality
* Rely on tracepoints for file descriptor retrieval in Golang implementation
* Remove the unnecessary changes
* Move common functions into `common.c`
* Declare `lookup_ssl_info` function to reduce duplication
* Fix linter
* Add comments and TODOs
* Remove `MIZU_GLOBAL_GOLANG_PATH` environment variable
* Update the object files
* Fix indentation
* Update object files
* Add `go_abi_internal.h`
* Fix `lookup_ssl_info`
* Convert indentation to spaces
* Add header guard comment
* Add more comments
* Find the `ret` instructions using Capstone Engine and `uprobe` the `return` statements
* Implement `get_fd_from_tcp_conn` function
* Separate SSL contexts to OpenSSL and Go
* Move `get_count_bytes` from `common.c` to `openssl_uprobes.c`
* Rename everything contains Golang to Go
* Reduce duplication in `go_uprobes.c`
* Update the comments
* Install Capstone in CI and Docker native builds
* Update `devops/install-capstone.sh`
* Add Capstone to AArch64 cross-compilation target
* Fix some of the issues on ARM64
* Delete the map element in `_ex_urpobe`
* Remove an unsued `LOG_` macro
* Rename `aquynh` to `capstone-engine`
* Add comment
* Revert "Fix some of the issues on ARM64"
This reverts commit 0b3eceddf4.
* Revert "Revert "Fix some of the issues on ARM64""
This reverts commit 681534ada1.
* Update object files
* Remove unnecessary return
* Increase timeout
* #run_acceptance_tests
* #run_acceptance_tests
* Fix the `arm64v8` sourced builds
* #run_acceptance_tests
* avoid chunks with invalid address
* tls tapper should distict between pids
* prettfy tls verbose log and tls key
* support tls from multi threads + duplicate calls to the same target
* introduce fdCache and user address pair as tls key
* remove unused comment
* fix merge conflicts
* use lru for fdcache
* pr fixes - renaming
* fix conflict issue
* Fix `panic: interface conversion: api.RequestResponseMatcher is nil, not *http.requestResponseMatcher` error
Also fix the request-response matcher maps iteration in `clean()` method.
* Fix the mocks in the unit tests
* Remove unnecessary fields from `tlsPoller` and implement `SetProtocol` method
* Use concrete types in `tap` package
* Share the streams map with the TLS tapper
* Check interface conversion error
* initial tls tapper commit
* add tls flag to mizu cli
* support ssl_read_ex/ssl_write_ex
* use hostproc to find libssl
* auto discover tls processes
* support libssl1.0
* recompile ebpf with old clang/llvm
* Update tap/passive_tapper.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* Update tap/tlstapper/tls_poller.go
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
* upgrade ebpf go lib
* handling big tls messages
* fixing max buffer size in ebpf
* remove unused import
* fix linter issues
* minor pr fixes
* compile with old clang
* fix cgroup file format
* pr fixes + cgroup extract enhance
* fix linter
* adding indirect ebpf dep to agent go.mod
* adding ebpf docker builder
* minor pr fixes
* add req resp matcher to dissect
* rename ssl hooks to ssl hooks structs
* move to alpine, use local copy of mizu instead of git, add readme
* use global req resp mather for tls
Co-authored-by: M. Mert Yıldıran <mehmet@up9.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
