Commit Graph

1189 Commits

Author SHA1 Message Date
Roee Gadot
d7b5de9ae1 no message 2022-07-07 17:38:49 +03:00
Roee Gadot
9e3c76fdd4 change a little bit 2022-07-07 13:32:20 +03:00
Roee Gadot
dadb617861 first iteration - maybe working 2022-07-07 13:23:45 +03:00
AmitUp9
274fbeb34a
warning cleaning from console (#1187)
* warning cleaning from console

* code cleaning
2022-07-06 13:13:04 +03:00
leon-up9
38c05a6634
UI/feature flag for replay modal (#1186)
* context added

* import added

* chnages

* ui enabled

* moved to Consts

* changes to recoil

* change

* new useEffect

Co-authored-by: Leon <>
2022-07-06 11:19:46 +03:00
leon-up9
d857935889
move icon to right side (#1185)
Co-authored-by: Leon <>
2022-07-05 16:35:03 +03:00
gadotroee
ec11b21b51
Add available protocols to the stats endpoint (colors for methods are coming from server) (#1184)
* add protocols array to the endpoint

* no message

* no message

* fix tests and small fix for the iteration

* fix the color of the protocol

* Get protocols list and method colors from server

* fix tests

* cr fixes

Co-authored-by: Amit Fainholts <amit@up9.com>
2022-07-05 15:30:39 +03:00
M. Mert Yıldıran
52c9251c00
Add ABI0 support to Go crypto/tls eBPF tracer (#1169)
* Determine the Go ABI and get `goid` offset from DWARF

* Add `ABI` enum and morph the function according to the detected ABI

* Pass `goid` offset to an eBPF map to retrieve it in eBPF context

* Add `vmlinux.h` and implement `get_goid_from_thread_local_storage`

* Fix BPF verifier errors

* Update the comments

* Add `go_abi_0.h` and implement `ABI0` specific reads for `arm64`

* Upgrade `github.com/cilium/ebpf` to `v0.9.0`

* Add a comment

* Add macros for x86 specific parts

* Update `x86.o`

* Fix the map key type

* Add `user_pt_regs`

* Update arm64 object file

* Fix the version detection logic

* Add `getGStructOffset` method

* Define `goid_offsets`, `goid_offsets_map` structs and pass the offsets correctly

* Fix the `net.TCPConn` and buffer addresses for `ABI0`

* Remove comment

* Fix the issues for arm64 build

* Update x86.o

* Revert "Fix the issues for arm64 build"

This reverts commit 48b041b1b6.

* Revert `user_pt_regs`

* Add `vmlinux` directory

* Fix the `build.sh` and `Dockerfile`

* Add vmlinux_arm64.h

* Disable `get_goid_from_thread_local_storage` on ARM64 with a macro

* Update x86.o

* Update arm64.o

* x86

* arm64

* Fix the cross-compilation issue from x86 to arm64

* Fix the same thing for x86

* Use `BPF_CORE_READ` macro instead of `bpf_ringbuf_reserve` to support kernel versions older than 5.8

Also;
Add legacy version of thread_struct: thread_struct___v46
Build an additional object file for the kernel versions older than or equal to 4.6 and load them accordingly.
Add github.com/moby/moby

* Make #define directives more definitive

* Select the x86 and arm64 versions of `vmlinux.h` using macros

* Put `goid` offsets into the map before installing `uprobe`(s)

* arm64

* #run_acceptance_tests

* Remove a forgotten `fmt.Printf`

* Log the detected Linux kernel version
2022-07-05 14:35:30 +03:00
leon-up9
f3a6b3a9d4
with loading HOC (#1181)
* withLoading

* optional props

* LoadingWrapper

* pr comments

* changes

Co-authored-by: Leon <>
2022-07-05 12:23:47 +03:00
leon-up9
5f73c2d50a
allow skipping formating error (#1183)
Co-authored-by: Leon <>
2022-07-04 16:24:23 +03:00
gadotroee
d6944d467c
Merge traffic stats endpoints to one and add auto interval logic (#1174) 2022-07-04 10:01:49 +03:00
leon-up9
57078517a4
Ui/replay demo notes (#1180)
* close ws on open

* chech if json before parsing

* setting defualt tab reponse and missing dep

* remove redundant

* space

* PR fixes

* remove redundant

* changed order

* Revert "remove redundant"

This reverts commit 2f0bef5d33.

* revert order change

* changes

* change

* changes

Co-authored-by: Leon <>
2022-07-03 15:36:16 +03:00
AmitUp9
b4bc09637c
fix select in traffic stats ui (#1179) 2022-07-03 11:48:38 +03:00
lirazyehezkel
302333b4ae
TRA-4622 Remove rules feature UI (#1178)
* Removed policy rules (validation rules) feature

* updated test pcap

* Remove rules

* fix replay in rules

Co-authored-by: Roy Island <roy@up9.com>
Co-authored-by: RoyUP9 <87927115+RoyUP9@users.noreply.github.com>
Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2022-07-03 11:32:23 +03:00
leon-up9
13ed8eb58a
Ui/TRA-4607 - replay mizu requests (#1165)
* modal & keyValueTable added

* added pulse animation
KeyValueTable Behavior improved
CodeEditor addded

* style changed

* codeEditor styling
support query params

* send request data

* finally stop loading

* select width

* methods and requesr format

* icon changed & moved near request tab

* accordions added and response presented

* 2 way params biding

* remove redundant

* host path fixed

* fix path input

* icon styles

* fallback for format body

* refresh button

* changes

* remove redundant

* closing tag

* capitilized character

* PR comments

* removed props

* small changes

* color added to reponse data

Co-authored-by: Leon <>
2022-07-03 09:48:56 +03:00
David Levanon
48619b3e1c
fix small regression with openssl (#1176) 2022-06-30 17:39:03 +03:00
AmitUp9
3b0b311e1e
TRA_4623 - methods view on timeline stats with coordination to the pie stats (#1175)
* Add select protocol → when selected, the view will be on commands of that exact protocol

* CR fixes

* added const instead of free string

* remove redundant sass file
2022-06-30 13:35:08 +03:00
gadotroee
3a9236a381
Fix replay to be like the test (#1173)
Represent should get entry that Marshaled and UnMarshaled
2022-06-29 11:54:16 +03:00
AmitUp9
2e7fd34210
Move general functions from timeline stats to helpers utils (#1170)
* no message

* format document
2022-06-29 11:15:22 +03:00
gadotroee
01af6aa19c
Add reply endpoint for http (#1168) 2022-06-28 18:39:23 +03:00
David Levanon
2bfae1baae
allow to configure max live streams from mizu cli (#1172)
* allow to configure max live streams from mizu cli

* Update cli/cmd/tap.go

Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>

Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>
2022-06-28 14:41:47 +03:00
RoyUP9
2df9fb49db
Changed entry protocol field from struct to unique string (#1167) 2022-06-26 17:08:09 +03:00
M. Mert Yıldıran
c1b2cda468
Change the log level from ERROR to INFO for cases; not a Go binary or symbol table is stripped (#1164) 2022-06-26 15:25:26 +03:00
M. Mert Yıldıran
cf91f3dab4
Fix Go crypto/tls eBPF tracer on ARM64 (#1162)
* Handle the Go `ABIInternal` differences on ARM64

* Log the Capstone version, arch and mode

* Upgrade Capstone from `4.0.2` to `5.0-rc2` to have ARM64 instructions fix

* Use the correct register on ARM64 for reading buffer length

* Fix the addresses on ARM64

* #run_acceptance_tests

* Update `x86.o`

* Update `arm64.o`

* #run_acceptance_tests
2022-06-26 14:11:04 +03:00
RoyUP9
1ebc51b45e
Remove OAS contract monitoring feature (#1163) 2022-06-26 12:22:25 +03:00
gadotroee
a96072623f
Fix problems that golang inspection reported (#1158) 2022-06-23 16:49:56 +03:00
gadotroee
8c187179b0
Adding tests to the bucket statistics (#1160) 2022-06-23 11:04:12 +03:00
lirazyehezkel
d7d802830f
TRA-4274 fix ui common linter issues (#1161)
* fix lint issues

* fix all ui-common line issues

* max warnings 0
2022-06-22 09:56:27 +03:00
AmitUp9
4d64dd4b04
TRA-4602_timeline bar charts to traffic statistics (#1159)
* pie chart for protocols and methods by requests and volume

* protocols legend

* timeline bar chart component created

* timeline can view requests and volume

* sorting the bra charts by timestemp

* disable view of <1% pieces in pie

* space added to the end of the file

* package.json update

* cr fixes

* remove spave

* remove unnecessary react fragment

Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
2022-06-21 14:54:56 +03:00
gadotroee
9a40895e9c
Add timing endpoint (timeline data) for stats (#1157) 2022-06-21 14:33:04 +03:00
M. Mert Yıldıran
f9a9c05f48
Add ARM64 support to eBPF (#1151)
* Define and use `BPF_CFLAGS` environment variable

* Add eBPF dependencies to `builder-from-amd64-to-arm64v8` and `builder-native-base`

* Add eBPF dependencies to `builder-from-arm64v8-to-amd64`

* Only compile x86 arch of Capstone for x86 target

* Build and install `libbpf` from source

* Fix `builder-from-arm64v8-to-amd64`

* Add `BPF_TARGET` environment variable

* Fix the eBPF verifier error on ARM64

* Fix `go_crypto_tls_ex_uprobe`

* Fix the check

* #run_acceptance_tests

* Fix the build script

* Include ARM64 files

* Bring back `x86.o`

* Generate both endianness

* Fix Dockerfile

* #run_acceptance_tests

* Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests

* Revert "Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests"

This reverts commit a2c83c6040.

* Remove big-endian files #run_acceptance_tests

* Fix Dockerfile #run_acceptance_tests

Co-authored-by: Ubuntu <ubuntu@ip-172-31-33-233.eu-central-1.compute.internal>
2022-06-20 16:16:56 +03:00
lirazyehezkel
c9d4f88de8
TRA-4602 Traffic statistics - Pie Chart (#1153)
* pie chart for protocols and methods by requests and volume

* protocols legend

* methods legend

* pie chart component

* code refactor

* no message

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-06-19 17:19:06 +03:00
gadotroee
6f117d0a84
Accumulative stats for protocol and methods (#1144)
only data for pie chart currently
2022-06-19 16:59:35 +03:00
David Levanon
99cb0b4f44
Close gopacket connection immediately, basic throttling and assembler mutex removal
* close gopacket conn immediately

* increase last ack threshold to 3 seconds

* remove empty condition

* add periodic stats

* protect connections from concurrent updates

* implement basic throttling base on live streams count

* remove assembler mutex

* pr fixes

* change max conns default to 500

* create connectionId type

* fix linter
2022-06-19 16:47:03 +03:00
M. Mert Yıldıran
167bbe3741
Fix a slice bounds out of range panic in Go offsets calculation (#1155)
* Fix a `slice bounds out of range` error in Go offsets calculation

* Change `Errorf` to `Warningf`
2022-06-19 12:19:32 +03:00
Andrey Pokhilko
7a823e89f1
Add option to limit example len (#1145)
Do not save OAS example response if it is over the limit.
SpecGen.MaxExampleLen is measured in bytes.
Limit=0 means "don't record any examples", while limit=-1 means "record all".

Limit is configured via the config file / --set: --set oas.max-example-len=100.
The default is 10KB.

This PR breaks the previous config. Moved oas bool (enable/disable OAS) to oas.enable.
Users who wish to disable OAS (or to explicitly enable it) should now to so with --set oas.enable=false instead of --set oas=false.

Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>
2022-06-16 14:18:42 +03:00
gadotroee
be98d85cb8
Recreate package-lock to remove "old" packages (#1152) 2022-06-16 13:04:38 +03:00
AmitUp9
e743ab7f7a
TRA_4579 - fix acceptance test of right panel body checkboxes (#1150)
* #run_acceptance_tests

* added data-cy to checkboxes in body section, removed unnecessary tests and fix decoded text testing

* #run_acceptance_tests
2022-06-16 10:27:31 +03:00
AmitUp9
78081a4a16
Node sass npm package migrate to sass npm package (#1143)
* remove node-sass from project and install sass instead

* fix ui-common path
2022-06-14 16:53:34 +03:00
lirazyehezkel
0ef6a7d2c4
Fix images imports (#1139)
* Fix images imports

* fix image name
2022-06-13 16:19:50 +03:00
Igor Gov
ba361df7e7
Mizu install should be namespace aware (#1141) 2022-06-13 15:26:00 +03:00
M. Mert Yıldıran
1f70eab0a4
Update Basenine hashes (#1138) 2022-06-12 16:36:33 +03:00
M. Mert Yıldıran
ea4991905b
Upgrade Basenine to v0.8.3 (#1137)
* Upgrade Basenine to `v0.8.3`

* #run_acceptance_tests
2022-06-12 16:15:33 +03:00
M. Mert Yıldıran
2ad4838cf5
Add Go crypto/tls eBPF tracer for TLS connections (#1120)
* Run `go generate tls_tapper.go`

* Add `golang_uprobes.c`

* Add Golang hooks and offsets

* Add `golangConnection` struct and implement `pollGolangReadWrite` method

* Upgrade `github.com/cilium/ebpf` version to `v0.8.1`

* Fix the linter error

* Move map related stuff to `maps.h` and run `go generate tls_tapper.go`

* Remove unused parameter

* Add an environment variable to test Golang locally

* Replace `Libssl` occurrences with `Ssllib` for consistency

* Fix exe path finding

* Temporarily disable OpenSSL

* Fix the mixed offsets and dissection preparation

* Change the read symbol from `net/http.(*persistConn).Read` to `crypto/tls.(*Conn).Read`

* Remove `len` and `cap` fields

* Fix the indent

* Fix the read data address

* Make `golang_dial_writes` key `__u64` and include the PID

* Fix the read data address one more time

* Temporarily disable the PCAP capture

* Add a uprobe for `net/http.(*gzipReader).Read` to read chunked HTTP response body

* Cancel `golang_crypto_tls_read_uprobe` if it's a gzip read

* Make hash map names more meaningful

* Pass the connection address from `write` to `gzip` through a common address between `gzip` and `dial`

* Fix the probed line number links

* Add `golangReader` struct and implement its `Read` method

* Have a single counter pair and request response matcher per Golang connection

* Add `MIZU_GLOBAL_GOLANG_PATH` environment variable

* `NULL` terminate the bytes with `unix.ByteSliceToString`

* Temporarily reject the gzip chunks

* Add malformed TODOs

* Revert "`NULL` terminate the bytes with `unix.ByteSliceToString`"

This reverts commit 7ee7ef7e44.

* Bring back `len` and `cap` fields

* Set `len` and `cap` in `golang_net_http_gzipreader_read_uprobe` as well

* Remove two `TODO`s

* Fix the `key_gzip` offsets

* Compress if it's gzip chunk (probably wrong!)

* Revert "Compress if it's gzip chunk (probably wrong!)"

This reverts commit 094a7c3da4.

* Remove `golang_net_http_gzipreader_read_uprobe`

* Read constant 4KiB

* Use constant read length

* Get the correct len of bytes (saw the second entry)

* Set all buffer sizes to `CHUNK_SIZE`

* Remove a `TODO`

* Revert "Temporarily disable the PCAP capture"

This reverts commit a2da15ef2d.

* Update `golang_crypto_tls_read_uprobe`

* Set the `reader` field of `tlsStream` to fix a `nil pointer dereference` error

* Don't export any fields of `golangConnection`

* Close the reader when we drop the connection

* Add a tracepoint for `sys_enter_close` to detect socket closes

* Rename `socket` struct to `golang_socket`

* Call `should_tap` in Golang uprobes

* Add `log_error` calls

* Revert "Temporarily disable OpenSSL"

This reverts commit f54d9a453f.

* Fix linter

* Revert "Revert "Temporarily disable OpenSSL""

This reverts commit 2433d867af.

* Change `golang_read_writes` map type from `BPF_RINGBUF` to `BPF_PERF_OUTPUT`

* Rename `golang_read_write` to `golang_event`

* Define an error

* Add comments

* Revert "Revert "Revert "Temporarily disable OpenSSL"""

This reverts commit e5a1de9c71.

* Fix `pollGolang`

* Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""

This reverts commit 6e1bd5d4f3.

* Fix `panic: send on closed channel`

* Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""

This reverts commit 57d0584655.

* Use `findLibraryByPid`

* Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""""

This reverts commit 46f3d290b0.

* Revert "Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""""

This reverts commit 775c833c06.

* Log tapping Golang

* Fix `Poll`

* Refactor `golang_net_http_dialconn_uprobe`

* Remove an excess error check

* Fix `can only use path@version syntax with 'go get' and 'go install' in module-aware mode` error in `tap/tlstapper/bpf-builder/build.sh`

* Unify Golang and OpenSSL under a single perf event buffer and `tls_chunk` struct

* Generate `tlsTapperChunkType` type (enum) as well

* Use kernel page size for the `sys_closes` perf buffer

* Fix the linter error

* Fix `MIZU_GLOBAL_GOLANG_PID` environment variable's functionality

* Rely on tracepoints for file descriptor retrieval in Golang implementation

* Remove the unnecessary changes

* Move common functions into `common.c`

* Declare `lookup_ssl_info` function to reduce duplication

* Fix linter

* Add comments and TODOs

* Remove `MIZU_GLOBAL_GOLANG_PATH` environment variable

* Update the object files

* Fix indentation

* Update object files

* Add `go_abi_internal.h`

* Fix `lookup_ssl_info`

* Convert indentation to spaces

* Add header guard comment

* Add more comments

* Find the `ret` instructions using Capstone Engine and `uprobe` the `return` statements

* Implement `get_fd_from_tcp_conn` function

* Separate SSL contexts to OpenSSL and Go

* Move `get_count_bytes` from `common.c` to `openssl_uprobes.c`

* Rename everything contains Golang to Go

* Reduce duplication in `go_uprobes.c`

* Update the comments

* Install Capstone in CI and Docker native builds

* Update `devops/install-capstone.sh`

* Add Capstone to AArch64 cross-compilation target

* Fix some of the issues on ARM64

* Delete the map element in `_ex_urpobe`

* Remove an unsued `LOG_` macro

* Rename `aquynh` to `capstone-engine`

* Add comment

* Revert "Fix some of the issues on ARM64"

This reverts commit 0b3eceddf4.

* Revert "Revert "Fix some of the issues on ARM64""

This reverts commit 681534ada1.

* Update object files

* Remove unnecessary return

* Increase timeout

* #run_acceptance_tests

* #run_acceptance_tests

* Fix the `arm64v8` sourced builds

* #run_acceptance_tests
2022-06-12 14:14:41 +03:00
lirazyehezkel
e41488ef3e
TRA-4276 Folder structure refactor (#1131)
* mizu-common folder structure refactor

* no message

* change images import from absolute to relative

* relative sass imports

* no message

* remove tgz

* spinner style
2022-06-12 11:20:47 +03:00
AmitUp9
533fb71bf4
TRA-4596_Update material UI to v5 (#1132)
* install @mui v5 and change imports

* upgrade MUI v5 in mizu community

* package-lock of mizu community and ui-common

* fix ui-common path

* #run_acceptance_tests

* remove comment

* cr fixes
2022-06-08 15:32:34 +03:00
RoyUP9
6f8aad83e6
Minikube start with more memory&cpu (#1130) 2022-06-06 17:53:12 +03:00
leon-up9
6e6bcec77e
Ui/TRA-4586_add-insertion-filter-to-settings-page (#1127)
* working query input

* removed prop

* splited to diffrent events

* export alias

* PR comments

Co-authored-by: Leon <>
2022-06-06 16:40:23 +03:00
leon-up9
71db792a4e
list takes 100% from parent (#1129)
Co-authored-by: Leon <>
2022-06-06 16:26:07 +03:00
gadotroee
f7f61c1217
Fix cypress files to cypress version 10 (#1124) 2022-06-06 11:13:20 +03:00