Commit Graph

1170 Commits

Author SHA1 Message Date
gadotroee
01af6aa19c
Add reply endpoint for http (#1168) 2022-06-28 18:39:23 +03:00
David Levanon
2bfae1baae
allow to configure max live streams from mizu cli (#1172)
* allow to configure max live streams from mizu cli

* Update cli/cmd/tap.go

Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>

Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>
2022-06-28 14:41:47 +03:00
RoyUP9
2df9fb49db
Changed entry protocol field from struct to unique string (#1167) 2022-06-26 17:08:09 +03:00
M. Mert Yıldıran
c1b2cda468
Change the log level from ERROR to INFO for cases; not a Go binary or symbol table is stripped (#1164) 2022-06-26 15:25:26 +03:00
M. Mert Yıldıran
cf91f3dab4
Fix Go crypto/tls eBPF tracer on ARM64 (#1162)
* Handle the Go `ABIInternal` differences on ARM64

* Log the Capstone version, arch and mode

* Upgrade Capstone from `4.0.2` to `5.0-rc2` to have ARM64 instructions fix

* Use the correct register on ARM64 for reading buffer length

* Fix the addresses on ARM64

* #run_acceptance_tests

* Update `x86.o`

* Update `arm64.o`

* #run_acceptance_tests
2022-06-26 14:11:04 +03:00
RoyUP9
1ebc51b45e
Remove OAS contract monitoring feature (#1163) 2022-06-26 12:22:25 +03:00
gadotroee
a96072623f
Fix problems that golang inspection reported (#1158) 2022-06-23 16:49:56 +03:00
gadotroee
8c187179b0
Adding tests to the bucket statistics (#1160) 2022-06-23 11:04:12 +03:00
lirazyehezkel
d7d802830f
TRA-4274 fix ui common linter issues (#1161)
* fix lint issues

* fix all ui-common line issues

* max warnings 0
2022-06-22 09:56:27 +03:00
AmitUp9
4d64dd4b04
TRA-4602_timeline bar charts to traffic statistics (#1159)
* pie chart for protocols and methods by requests and volume

* protocols legend

* timeline bar chart component created

* timeline can view requests and volume

* sorting the bra charts by timestemp

* disable view of <1% pieces in pie

* space added to the end of the file

* package.json update

* cr fixes

* remove spave

* remove unnecessary react fragment

Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
2022-06-21 14:54:56 +03:00
gadotroee
9a40895e9c
Add timing endpoint (timeline data) for stats (#1157) 2022-06-21 14:33:04 +03:00
M. Mert Yıldıran
f9a9c05f48
Add ARM64 support to eBPF (#1151)
* Define and use `BPF_CFLAGS` environment variable

* Add eBPF dependencies to `builder-from-amd64-to-arm64v8` and `builder-native-base`

* Add eBPF dependencies to `builder-from-arm64v8-to-amd64`

* Only compile x86 arch of Capstone for x86 target

* Build and install `libbpf` from source

* Fix `builder-from-arm64v8-to-amd64`

* Add `BPF_TARGET` environment variable

* Fix the eBPF verifier error on ARM64

* Fix `go_crypto_tls_ex_uprobe`

* Fix the check

* #run_acceptance_tests

* Fix the build script

* Include ARM64 files

* Bring back `x86.o`

* Generate both endianness

* Fix Dockerfile

* #run_acceptance_tests

* Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests

* Revert "Determine the endianness on runtime if it's possible in Go (default little-endian) #run_acceptance_tests"

This reverts commit a2c83c6040.

* Remove big-endian files #run_acceptance_tests

* Fix Dockerfile #run_acceptance_tests

Co-authored-by: Ubuntu <ubuntu@ip-172-31-33-233.eu-central-1.compute.internal>
2022-06-20 16:16:56 +03:00
lirazyehezkel
c9d4f88de8
TRA-4602 Traffic statistics - Pie Chart (#1153)
* pie chart for protocols and methods by requests and volume

* protocols legend

* methods legend

* pie chart component

* code refactor

* no message

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-06-19 17:19:06 +03:00
gadotroee
6f117d0a84
Accumulative stats for protocol and methods (#1144)
only data for pie chart currently
2022-06-19 16:59:35 +03:00
David Levanon
99cb0b4f44
Close gopacket connection immediately, basic throttling and assembler mutex removal
* close gopacket conn immediately

* increase last ack threshold to 3 seconds

* remove empty condition

* add periodic stats

* protect connections from concurrent updates

* implement basic throttling base on live streams count

* remove assembler mutex

* pr fixes

* change max conns default to 500

* create connectionId type

* fix linter
2022-06-19 16:47:03 +03:00
M. Mert Yıldıran
167bbe3741
Fix a slice bounds out of range panic in Go offsets calculation (#1155)
* Fix a `slice bounds out of range` error in Go offsets calculation

* Change `Errorf` to `Warningf`
2022-06-19 12:19:32 +03:00
Andrey Pokhilko
7a823e89f1
Add option to limit example len (#1145)
Do not save OAS example response if it is over the limit.
SpecGen.MaxExampleLen is measured in bytes.
Limit=0 means "don't record any examples", while limit=-1 means "record all".

Limit is configured via the config file / --set: --set oas.max-example-len=100.
The default is 10KB.

This PR breaks the previous config. Moved oas bool (enable/disable OAS) to oas.enable.
Users who wish to disable OAS (or to explicitly enable it) should now to so with --set oas.enable=false instead of --set oas=false.

Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com>
2022-06-16 14:18:42 +03:00
gadotroee
be98d85cb8
Recreate package-lock to remove "old" packages (#1152) 2022-06-16 13:04:38 +03:00
AmitUp9
e743ab7f7a
TRA_4579 - fix acceptance test of right panel body checkboxes (#1150)
* #run_acceptance_tests

* added data-cy to checkboxes in body section, removed unnecessary tests and fix decoded text testing

* #run_acceptance_tests
2022-06-16 10:27:31 +03:00
AmitUp9
78081a4a16
Node sass npm package migrate to sass npm package (#1143)
* remove node-sass from project and install sass instead

* fix ui-common path
2022-06-14 16:53:34 +03:00
lirazyehezkel
0ef6a7d2c4
Fix images imports (#1139)
* Fix images imports

* fix image name
2022-06-13 16:19:50 +03:00
Igor Gov
ba361df7e7
Mizu install should be namespace aware (#1141) 2022-06-13 15:26:00 +03:00
M. Mert Yıldıran
1f70eab0a4
Update Basenine hashes (#1138) 2022-06-12 16:36:33 +03:00
M. Mert Yıldıran
ea4991905b
Upgrade Basenine to v0.8.3 (#1137)
* Upgrade Basenine to `v0.8.3`

* #run_acceptance_tests
2022-06-12 16:15:33 +03:00
M. Mert Yıldıran
2ad4838cf5
Add Go crypto/tls eBPF tracer for TLS connections (#1120)
* Run `go generate tls_tapper.go`

* Add `golang_uprobes.c`

* Add Golang hooks and offsets

* Add `golangConnection` struct and implement `pollGolangReadWrite` method

* Upgrade `github.com/cilium/ebpf` version to `v0.8.1`

* Fix the linter error

* Move map related stuff to `maps.h` and run `go generate tls_tapper.go`

* Remove unused parameter

* Add an environment variable to test Golang locally

* Replace `Libssl` occurrences with `Ssllib` for consistency

* Fix exe path finding

* Temporarily disable OpenSSL

* Fix the mixed offsets and dissection preparation

* Change the read symbol from `net/http.(*persistConn).Read` to `crypto/tls.(*Conn).Read`

* Remove `len` and `cap` fields

* Fix the indent

* Fix the read data address

* Make `golang_dial_writes` key `__u64` and include the PID

* Fix the read data address one more time

* Temporarily disable the PCAP capture

* Add a uprobe for `net/http.(*gzipReader).Read` to read chunked HTTP response body

* Cancel `golang_crypto_tls_read_uprobe` if it's a gzip read

* Make hash map names more meaningful

* Pass the connection address from `write` to `gzip` through a common address between `gzip` and `dial`

* Fix the probed line number links

* Add `golangReader` struct and implement its `Read` method

* Have a single counter pair and request response matcher per Golang connection

* Add `MIZU_GLOBAL_GOLANG_PATH` environment variable

* `NULL` terminate the bytes with `unix.ByteSliceToString`

* Temporarily reject the gzip chunks

* Add malformed TODOs

* Revert "`NULL` terminate the bytes with `unix.ByteSliceToString`"

This reverts commit 7ee7ef7e44.

* Bring back `len` and `cap` fields

* Set `len` and `cap` in `golang_net_http_gzipreader_read_uprobe` as well

* Remove two `TODO`s

* Fix the `key_gzip` offsets

* Compress if it's gzip chunk (probably wrong!)

* Revert "Compress if it's gzip chunk (probably wrong!)"

This reverts commit 094a7c3da4.

* Remove `golang_net_http_gzipreader_read_uprobe`

* Read constant 4KiB

* Use constant read length

* Get the correct len of bytes (saw the second entry)

* Set all buffer sizes to `CHUNK_SIZE`

* Remove a `TODO`

* Revert "Temporarily disable the PCAP capture"

This reverts commit a2da15ef2d.

* Update `golang_crypto_tls_read_uprobe`

* Set the `reader` field of `tlsStream` to fix a `nil pointer dereference` error

* Don't export any fields of `golangConnection`

* Close the reader when we drop the connection

* Add a tracepoint for `sys_enter_close` to detect socket closes

* Rename `socket` struct to `golang_socket`

* Call `should_tap` in Golang uprobes

* Add `log_error` calls

* Revert "Temporarily disable OpenSSL"

This reverts commit f54d9a453f.

* Fix linter

* Revert "Revert "Temporarily disable OpenSSL""

This reverts commit 2433d867af.

* Change `golang_read_writes` map type from `BPF_RINGBUF` to `BPF_PERF_OUTPUT`

* Rename `golang_read_write` to `golang_event`

* Define an error

* Add comments

* Revert "Revert "Revert "Temporarily disable OpenSSL"""

This reverts commit e5a1de9c71.

* Fix `pollGolang`

* Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""

This reverts commit 6e1bd5d4f3.

* Fix `panic: send on closed channel`

* Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""

This reverts commit 57d0584655.

* Use `findLibraryByPid`

* Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL""""""

This reverts commit 46f3d290b0.

* Revert "Revert "Revert "Revert "Revert "Revert "Revert "Temporarily disable OpenSSL"""""""

This reverts commit 775c833c06.

* Log tapping Golang

* Fix `Poll`

* Refactor `golang_net_http_dialconn_uprobe`

* Remove an excess error check

* Fix `can only use path@version syntax with 'go get' and 'go install' in module-aware mode` error in `tap/tlstapper/bpf-builder/build.sh`

* Unify Golang and OpenSSL under a single perf event buffer and `tls_chunk` struct

* Generate `tlsTapperChunkType` type (enum) as well

* Use kernel page size for the `sys_closes` perf buffer

* Fix the linter error

* Fix `MIZU_GLOBAL_GOLANG_PID` environment variable's functionality

* Rely on tracepoints for file descriptor retrieval in Golang implementation

* Remove the unnecessary changes

* Move common functions into `common.c`

* Declare `lookup_ssl_info` function to reduce duplication

* Fix linter

* Add comments and TODOs

* Remove `MIZU_GLOBAL_GOLANG_PATH` environment variable

* Update the object files

* Fix indentation

* Update object files

* Add `go_abi_internal.h`

* Fix `lookup_ssl_info`

* Convert indentation to spaces

* Add header guard comment

* Add more comments

* Find the `ret` instructions using Capstone Engine and `uprobe` the `return` statements

* Implement `get_fd_from_tcp_conn` function

* Separate SSL contexts to OpenSSL and Go

* Move `get_count_bytes` from `common.c` to `openssl_uprobes.c`

* Rename everything contains Golang to Go

* Reduce duplication in `go_uprobes.c`

* Update the comments

* Install Capstone in CI and Docker native builds

* Update `devops/install-capstone.sh`

* Add Capstone to AArch64 cross-compilation target

* Fix some of the issues on ARM64

* Delete the map element in `_ex_urpobe`

* Remove an unsued `LOG_` macro

* Rename `aquynh` to `capstone-engine`

* Add comment

* Revert "Fix some of the issues on ARM64"

This reverts commit 0b3eceddf4.

* Revert "Revert "Fix some of the issues on ARM64""

This reverts commit 681534ada1.

* Update object files

* Remove unnecessary return

* Increase timeout

* #run_acceptance_tests

* #run_acceptance_tests

* Fix the `arm64v8` sourced builds

* #run_acceptance_tests
2022-06-12 14:14:41 +03:00
lirazyehezkel
e41488ef3e
TRA-4276 Folder structure refactor (#1131)
* mizu-common folder structure refactor

* no message

* change images import from absolute to relative

* relative sass imports

* no message

* remove tgz

* spinner style
2022-06-12 11:20:47 +03:00
AmitUp9
533fb71bf4
TRA-4596_Update material UI to v5 (#1132)
* install @mui v5 and change imports

* upgrade MUI v5 in mizu community

* package-lock of mizu community and ui-common

* fix ui-common path

* #run_acceptance_tests

* remove comment

* cr fixes
2022-06-08 15:32:34 +03:00
RoyUP9
6f8aad83e6
Minikube start with more memory&cpu (#1130) 2022-06-06 17:53:12 +03:00
leon-up9
6e6bcec77e
Ui/TRA-4586_add-insertion-filter-to-settings-page (#1127)
* working query input

* removed prop

* splited to diffrent events

* export alias

* PR comments

Co-authored-by: Leon <>
2022-06-06 16:40:23 +03:00
leon-up9
71db792a4e
list takes 100% from parent (#1129)
Co-authored-by: Leon <>
2022-06-06 16:26:07 +03:00
gadotroee
f7f61c1217
Fix cypress files to cypress version 10 (#1124) 2022-06-06 11:13:20 +03:00
AmitUp9
696501fa11
remove poweredBy by display none the css calss (#1126) 2022-06-02 14:31:34 +03:00
gadotroee
415b5e08fd
Use Specific cypress version in the acceptance tests (#1125) 2022-06-02 13:21:44 +03:00
AmitUp9
7810f6defb
TRA_4565 - npm packages major updates (#1123)
* Major npm packages update

* npm i command added to dockerfile

* globally install npm new version

* remove the packages lock file copy

* package-lock commit and dockerfile revert changes

* space removal

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2022-06-02 11:22:37 +03:00
gadotroee
2aeac6c9e6
Remove analyze feature (#1121) 2022-06-01 10:17:14 +03:00
AmitUp9
dc241218bf
minor changes updated (#1122) 2022-05-31 15:08:15 +03:00
leon-up9
02b3672e09
added map object (#1119)
Co-authored-by: Leon <>
2022-05-30 13:13:07 +03:00
RoyUP9
45b368b33e
Updated the install command UX (#1118) 2022-05-30 11:18:46 +03:00
Igor Gov
8f64fdaa61
Alert to slack if release action fails (#1117)
* Trigger mizu ent stg deployment on release
2022-05-29 15:16:45 +03:00
AmitUp9
7edb0b153b
added condition that if no selector wouldn't be tooltip also (#1114) 2022-05-26 15:19:00 +03:00
AmitUp9
569a687fdf
Line numbers counter func added to utils (#1112)
* #run_acceptance_tests

* added new line to the end of the file

* #run_acceptance_tests

* linter fix

* more linter issues fix
2022-05-25 13:44:50 +03:00
AmitUp9
11e8b5eb65
TRA_4437- disable line numbers checkbox when only one line in content (#1109)
* number of lines state added

* added useEffect to update to showLineNubers state dynamically

* small cr fixes
2022-05-24 15:07:40 +03:00
AmitUp9
3901f3f3fe
change icon style css to move the icon so it can be clicked (#1111) 2022-05-24 14:16:27 +03:00
Nimrod Gilboa Markevich
2f1cc21fcb
Change redact to opt in (#1104) 2022-05-24 14:12:37 +03:00
leon-up9
433253a27b
font & padding change (#1108)
Co-authored-by: Leon <>
2022-05-24 12:09:13 +03:00
RoyUP9
00cc94fbe5
Changed OAS generator to get entries by push (#1103) 2022-05-22 14:53:48 +03:00
Nimrod Gilboa Markevich
8feef78ab1
Ignore mizu traffic in performance tests (#1102) 2022-05-22 12:08:22 +03:00
Nimrod Gilboa Markevich
992abc99bc
Disable redaction in performance tests (#1101) 2022-05-22 12:05:24 +03:00
Nimrod Gilboa Markevich
486d0b1088
Support tapper profiling on macOS (#1098) 2022-05-19 15:11:46 +03:00
David Levanon
f61a02d288
fix heap and goroutines metrics (#1099) 2022-05-19 14:55:13 +03:00