Commit Graph

1122 Commits

Author SHA1 Message Date
David Levanon
092bd5d22d stop tapping self tapper traffic 2022-05-16 15:46:25 +03:00
Nimrod Gilboa Markevich
97691d6279 Fix rss plot units (MB instead of bytes) 2022-05-15 15:00:22 +03:00
David Levanon
7515d18f88 create tapper modes for debugging using env vars 2022-05-15 14:42:21 +03:00
Nimrod Gilboa Markevich
f6f06b4b27 Add tap.cpuprofile flag. Change memprofile flag to tap.memprofile 2022-05-15 13:52:44 +03:00
Nimrod Gilboa Markevich
8f30c9a1c7 Add newline at end of file 2022-05-15 13:27:43 +03:00
Nimrod Gilboa Markevich
8c788742f8 Temporary: Marked with comments where to disable code for experiments 2022-05-15 13:26:41 +03:00
Nimrod Gilboa Markevich
b27a87a168 Allow grouping based on filenames 2022-05-15 09:16:46 +03:00
Nimrod Gilboa Markevich
4355c58e4a Reduce code repetition by adding function for plotting 2022-05-15 08:47:42 +03:00
Nimrod Gilboa Markevich
2179e8db7a Fix rss units 2022-05-15 08:46:19 +03:00
Nimrod Gilboa Markevich
de623488ab Make COLORMAP const 2022-05-15 08:42:01 +03:00
Nimrod Gilboa Markevich
567343a28d Plot in turbo colorscheme 2022-05-14 23:28:19 +03:00
Nimrod Gilboa Markevich
71b4f1e5d4 Concat to DataFrame 2022-05-14 23:08:42 +03:00
Nimrod Gilboa Markevich
7086a47858 Plot packetsCount in analysis 2022-05-14 22:33:31 +03:00
Nimrod Gilboa Markevich
a5d28bac9d Add scripts to plot performance 2022-05-13 00:02:46 +03:00
David Levanon
2a0ade6c62 measure cpu of current pid instead of globaly on the system 2022-05-12 19:36:11 +03:00
Nimrod Gilboa Markevich
25b180d6ff Print cpu usage percentage 2022-05-12 18:50:04 +03:00
Nimrod Gilboa Markevich
deb18618cd Fix error message 2022-05-12 16:55:02 +03:00
Nimrod Gilboa Markevich
267a9343d9 Add --profiler flag to cli 2022-05-12 16:55:02 +03:00
Nimrod Gilboa Markevich
413ebc3b9d Run pprof server on agent with --profiler flag 2022-05-12 16:55:02 +03:00
Nimrod Gilboa Markevich
ece673bd25 Add gin-contrib/pprof dependency 2022-05-12 16:55:02 +03:00
leon-up9
414e5cfe5a
match selectlist ui (#1077)
* insert filter and header to selectlist

* handle single select

* rename search var

* font size changed
2022-05-11 19:51:12 +03:00
RoyUP9
2fac0009ea
Fixed oas query (#1076) 2022-05-11 15:36:46 +03:00
M. Mert Yıldıran
36d59ede07
Fix acceptance tests failures caused by fetch 50 and a race in the tests (#1074)
* #run_acceptance_tests

* Wait 4 seconds after filter submission #run_acceptance_tests

* Revert "Wait 4 seconds after filter submission #run_acceptance_tests"

This reverts commit 3c20ccfcec.

* Upgrade Basenine version to `v0.8.2` #run_acceptance_tests

* #run_acceptance_tests

* Revert "Revert "Wait 4 seconds after filter submission #run_acceptance_tests""

This reverts commit f0c595150d.

* Reduce the delay to 2 seconds #run_acceptance_tests

* Use `cy.wait()` instead of `setTimeout`

* #run_acceptance_tests

* Increase it to 4 seconds #run_acceptance_tests

* #run_acceptance_tests

* Wait for a second and pause the stream to preserve the DOM #run_acceptance_tests

* Wait in a better place #run_acceptance_tests

* Wait `pause-icon` to disappear #run_acceptance_tests

* Wait one more second #run_acceptance_tests

* Fix `setup.sh`

* #run_acceptance_tests

* Fix the place of `npm i` call

* #run_acceptance_tests

* Fix the other workflow as well #run_acceptance_tests

* Fix the workflows once more

* #run_acceptance_tests

* Don't ignore `*/cypress/support` #run_acceptance_tests

* Do `npm i` in `setup.sh` #run_acceptance_tests

* Wait for at least 4 seconds #run_acceptance_tests

* #run_acceptance_tests

* Fix #run_acceptance_tests

* Comment #run_acceptance_tests

* Fix `{alt+enter}` to `{ctrl+enter}` and update the `checkFilter` and `deeperCheck` method signatures #run_acceptance_tests

* #run_acceptance_tests

* #run_acceptance_tests

* Extract it to a function #run_acceptance_tests

* Remove `cypress-wait-until`

* Revert `.gitignore` #run_acceptance_tests
2022-05-11 13:14:01 +03:00
AmitUp9
ac53508ad7
TRA-4513_grpc heading overlap request size (#1075)
* screen resolution changes

* fix window width to not overlap between headlines
2022-05-11 09:42:47 +03:00
leon-up9
e24c18254c
UI/servicemap-dynamic-protocol-filters (#1072)
* filters visble on click

* servicemap layout

* headers added
Icon changed

* flex removed , we have max size for filter

* selection only from edges

* caption changed

* removed duplicated code

* changes

Co-authored-by: Leon <>
2022-05-10 13:49:46 +03:00
leon-up9
b2830f133f
UI/Service-map-GUI-improvements (#1055)
* filters visble on click

* servicemap layout

* headers added
Icon changed

* flex removed , we have max size for filter

* added mixins

* search input caption

* PR commemts

* changed colors to variables

* lint fix
small buttom style fix

Co-authored-by: Leon <>
2022-05-10 13:27:41 +03:00
M. Mert Yıldıran
eef0ee8023
Fetch N number of records in M milliseconds timeout before streaming the records (#1056)
* Fetch N number of records in M milliseconds timeout before streaming the records

* Implement the functionality inside socket data streamer

* Reverse the `fetchData` slice

* #run_acceptance_tests

* Trying to fix the tests.
#run_acceptance_tests

* javascript compilation error.

* #run_acceptance_tests

* Name the method better

* Upgrade Basenine version to `v0.8.0`

* Fix some issues related to `Fetch`

* Upgrade the Basenine version in `Dockerfile` as well

* Remove underscore from the parameter name

* Parameterize fetch timeout ms

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2022-05-09 19:11:27 +03:00
RoyUP9
4c0aeb8146
Fixed default mime type to empty string instead of text/html (#1071) 2022-05-08 16:52:50 +03:00
lirazyehezkel
0dc0459dff
Disable tapping status when streaming is paused (#1070) 2022-05-08 15:45:33 +03:00
lirazyehezkel
81f06003d5
Outgoing filter remove redundant code (#1069) 2022-05-08 14:58:13 +03:00
lirazyehezkel
c7d068748a
TRA-4553 Fix outgoing filter (#1068)
* Fix outgoing filter

* fix labels
2022-05-08 13:44:56 +03:00
gadotroee
8102c49138
Improve acceptance tests setup script (#887) 2022-05-04 11:13:18 +03:00
AmitUp9
65fb2a4fe4
update close icon in oas and service map modals (#1066)
* changed icon

* close icon postion

* linter fixes

* make onSelectedOASService as callback function

* removing lint warning

* small fix

Co-authored-by: Leon <>
2022-05-03 13:59:28 +03:00
David Levanon
57f8a8dca9
Feature/fix tls not listening (#1046)
* avoid chunks with invalid address

* tls tapper should distict between pids

* prettfy tls verbose log and tls key

* support tls from multi threads + duplicate calls to the same target

* introduce fdCache and user address pair as tls key

* remove unused comment

* fix merge conflicts

* use lru for fdcache

* pr fixes - renaming

* fix conflict issue
2022-05-02 21:33:26 +03:00
M. Mert Yıldıran
aead6cbc19
Bring back debug.FreeOSMemory() call to prevent OOMKilled(s) (#1063)
* Revert "Revert "Bring back `debug.FreeOSMemory()` call""

This reverts commit 69538cb7e3.

* Revert "Revert "Revert "Remove `GOGC` environment variable from tapper"""

This reverts commit 3d9f37ab4b.

* Revert "Revert "Revert "Revert "Remove `GOGC` environment variable from tapper""""

This reverts commit 58ff0607ff.

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-05-02 14:13:04 +03:00
gadotroee
2f89690da6
Remove go-cache unused dependency (#1065) 2022-05-02 13:24:28 +03:00
M. Mert Yıldıran
3e47abf208
Add from ARM64 to AMD64 cross-compilation support to the agent image (#910)
* Add from ARM64 to AMD64 cross-compilation support to the agent image

* Replace `RUN`s with `&&`

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2022-05-02 11:16:05 +03:00
M. Mert Yıldıran
3cbccccb8b
Add GraphQL detection (#980)
* Add GraphQL detection

* Change the abbreviation to `GQL`

* Fix the `TestMacros` test case

* Try to fix the security issues reported by Snyk

* Run `go mod tidy` in `agent` directory

* Upgrade `golang.org/x/crypto`

* Downgrade `golang.org/x/crypto`

* Downgrade `golang.org/x/crypto`

* Downgrade `golang.org/x/crypto`

* Upgrade to `github.com/vektah/gqlparser/v2 v2.4.2`

* Run `go mod tidy`

* Replace `github.com/vektah/gqlparser/v2` with its fork `github.com/mertyildiran/gqlparser/v2` to fix the vulns

* Upgrade the `github.com/mertyildiran/gqlparser/v2 v2.4.6`

* Don't upgrade `golang.org/x/net`
2022-05-02 11:06:01 +03:00
gadotroee
08ae2bf6d7
Move "es Lint" to different step and add go lint on logger module (#1064) 2022-05-02 10:01:41 +03:00
M. Mert Yıldıran
684c51686f
Remove the obsolete replace statements that are artifacts of a merge conflict (#1061) 2022-05-01 16:51:43 +03:00
M. Mert Yıldıran
1de50b0572
Fix the request-response matcher maps iteration in clean() method and share the streams map with the TLS tapper (#1059)
* Fix `panic: interface conversion: api.RequestResponseMatcher is nil, not *http.requestResponseMatcher` error

Also fix the request-response matcher maps iteration in `clean()` method.

* Fix the mocks in the unit tests

* Remove unnecessary fields from `tlsPoller` and implement `SetProtocol` method

* Use concrete types in `tap` package

* Share the streams map with the TLS tapper

* Check interface conversion error
2022-05-01 16:16:22 +03:00
AmitUp9
0881dad17f
update craco config to resolve material ui instances (#1060) 2022-05-01 16:05:53 +03:00
David Levanon
cade960b9b
Fix tls + creating tls_stream (#1058) 2022-05-01 14:46:31 +03:00
M. Mert Yıldıran
d3e6a69d82
Refactor tap module to achieve synchronously closing other protocol dissectors upon identification (#1026)
* Remove `tcpStreamWrapper` struct

* Refactor `tap` module and move some of the code to `tap/api` module

* Move `TrafficFilteringOptions` struct to `shared` module

* Change the `Dissect` method signature to have `*TcpReader` as an argument

* Add `CloseOtherProtocolDissectors` method and use it to synchronously close the other protocol dissectors

* Run `go mod tidy` in `cli` module

* Rename `SuperIdentifier` struct to `ProtoIdentifier`

* Remove `SuperTimer` struct

* Bring back `CloseTimedoutTcpStreamChannels` method

* Run `go mod tidy` everywhere

* Remove `GOGC` environment variable from tapper

* Fix the tests

* Bring back `debug.FreeOSMemory()` call

* Make `CloseOtherProtocolDissectors` method mutexed

* Revert "Remove `GOGC` environment variable from tapper"

This reverts commit cfc2484bbb.

* Bring back the removed `checksum`, `nooptcheck` and `ignorefsmerr` flags

* Define a bunch of interfaces and don't export any new structs from `tap/api`

* Keep the interfaces in `tap/api` but move the structs to `tap/tcp`

* Fix the unit tests by depending on `github.com/up9inc/mizu/tap`

* Use the modified `tlsEmitter`

* Define `TlsChunk` interface and make `tlsReader` implement `TcpReader`

* Remove unused fields in `tlsReader`

* Define `ReassemblyStream` interface and separate `gopacket` specififc fields to `tcpReassemblyStream` struct

Such that make `tap/api` don't depend on `gopacket`

* Remove the unused fields

* Make `tlsPoller` implement `TcpStream` interface and remove the call to `NewTcpStreamDummy` method

* Remove unused fields from `tlsPoller`

* Remove almost all of the setter methods in `TcpReader` and `TcpStream` interface and remove `TlsChunk` interface

* Revert "Revert "Remove `GOGC` environment variable from tapper""

This reverts commit ab2b9a803b.

* Revert "Bring back `debug.FreeOSMemory()` call"

This reverts commit 1cce863bbb.

* Remove excess comment

* Fix acceptance tests (`logger` module) #run_acceptance_tests

* Bring back `github.com/patrickmn/go-cache`

* Fix `NewTcpStream` method signature

* Put `tcpReader` and `tcpStream` mocks into protocol dissectors to remove `github.com/up9inc/mizu/tap` dependency

* Fix AMQP tests

* Revert 960ba644cd

* Revert `go.mod` and `go.sum` files in protocol dissectors

* Fix the comment position

* Revert `AppStatsInst` change

* Fix indent

* Fix CLI build

* Fix linter error

* Fix error msg

* Revert some of the changes in `chunk.go`
2022-04-28 17:19:14 +03:00
AmitUp9
ed9e162af0
validation that grpc error render only when needed (#1051) 2022-04-28 15:18:08 +03:00
Igor Gov
4e22e77597
Fix: remove agent unused go dependency (#1050) 2022-04-28 12:08:59 +03:00
AmitUp9
3978ace4ef
searchable dropdown added to oas modal (#1044)
* searchable dropdown added to oas modal

* remove unnecessary attribute from autocomplete

* move css to sass file
2022-04-28 11:59:13 +03:00
Igor Gov
e71a12d399
Introducing eslint (#1048)
* Introducing eslint
2022-04-28 11:46:00 +03:00
M. Mert Yıldıran
90c54f9505
Fix acceptance tests (logger module) (#1049) 2022-04-27 23:59:16 +03:00
M. Mert Yıldıran
e1ad302c29
Make logger a separate module such that don't depend on shared module as a whole for logging (#1047)
* Make `logger` a separate module such that don't depend on `shared` module as a whole for logging

* Update `Dockerfile`
2022-04-27 22:26:27 +03:00