mirror of
https://github.com/kubeshark/kubeshark.git
synced 2025-08-06 19:14:50 +00:00
codegen-placeholders
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Nimrod Gilboa Markevich
|
ab38f4c011
|
Add profiling tools (#1087)
* Add gin-contrib/pprof dependency * Run pprof server on agent with --profiler flag * Add --profiler flag to cli * Fix error message * Print cpu usage percentage * measure cpu of current pid instead of globaly on the system * Add scripts to plot performance * Plot packetsCount in analysis * Concat to DataFrame * Plot in turbo colorscheme * Make COLORMAP const * Fix rss units * Reduce code repetition by adding function for plotting * Allow grouping based on filenames * Temporary: Marked with comments where to disable code for experiments * Add newline at end of file * Add tap.cpuprofile flag. Change memprofile flag to tap.memprofile * create tapper modes for debugging using env vars * Fix rss plot units (MB instead of bytes) * Remove comment * Add info to plot script * Remove tap.cpumemprofile. Rename tap.memprofile to memprofile * Remove unused import * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Remove whitespaces Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> * Rename debug env vars * Create package for debug env vars, read each env var once * Run go mod tidy * Increment MatchedPairs before emitting * Only count cores once * Count virtual and physical cores * Add dbgctl replace in cli * Fix lint: Check return values * Add tap/dbgctl to test-lint make rule * Replace tap/dbgctl in all modules * #run_acceptance_tests * Copy dbgctl module to docker image * Debug/profile tapper benchmark (#1093) * add mizu debug env to avoid all extensions * add readme + run_tapper_benchmark.sh * temporary change branch name * fix readme * fix MIZU_BENCHMARK_CLIENTS_COUNT env * change tap target to tcp stream * track live tcp streams * pr fixes * rename tapperPacketsCount to ignored_packets_count * change mizu tapper to mizu debugg Co-authored-by: David Levanon <dvdlevanon@gmail.com> Co-authored-by: M. Mert Yıldıran <mehmet@up9.com> |
||
M. Mert Yıldıran
|
bfa834e840
|
Spawn only two Goroutines per TCP stream (#1062)
* Spawn only two Goroutines per TCP stream * Fix the linter error * Use `isProtocolIdentified` method instead * Fix the `Read` method of `tcpReader` * Remove unnecessary `append` * Copy to buffer only a message is received * Remove `exhaustBuffer` field and add `rewind` function * Rename `buffer` field to `pastData` * Update tap/tcp_reader.go Co-authored-by: Nimrod Gilboa Markevich <59927337+nimrod-up9@users.noreply.github.com> * Use `copy` instead of assignment * No lint * #run_acceptance_tests * Fix `rewind` #run_acceptance_tests * Fix the buffering algorithm #run_acceptance_tests * Add `TODO` * Fix the problems in AMQP and Kafka #run_acceptance_tests * Use `*bytes.Buffer` instead of `[]api.TcpReaderDataMsg` #run_acceptance_tests * Have a single `*bytes.Buffer` * Revert "Have a single `*bytes.Buffer`" This reverts commit |
||
|
M. Mert Yıldıran
|
aead6cbc19
|
Bring back debug.FreeOSMemory() call to prevent OOMKilled(s) (#1063)
* Revert "Revert "Bring back `debug.FreeOSMemory()` call"" This reverts commit |
||
|
M. Mert Yıldıran
|
1de50b0572
|
Fix the request-response matcher maps iteration in clean() method and share the streams map with the TLS tapper (#1059)
* Fix `panic: interface conversion: api.RequestResponseMatcher is nil, not *http.requestResponseMatcher` error Also fix the request-response matcher maps iteration in `clean()` method. * Fix the mocks in the unit tests * Remove unnecessary fields from `tlsPoller` and implement `SetProtocol` method * Use concrete types in `tap` package * Share the streams map with the TLS tapper * Check interface conversion error |
||
|
M. Mert Yıldıran
|
d3e6a69d82
|
Refactor tap module to achieve synchronously closing other protocol dissectors upon identification (#1026)
* Remove `tcpStreamWrapper` struct * Refactor `tap` module and move some of the code to `tap/api` module * Move `TrafficFilteringOptions` struct to `shared` module * Change the `Dissect` method signature to have `*TcpReader` as an argument * Add `CloseOtherProtocolDissectors` method and use it to synchronously close the other protocol dissectors * Run `go mod tidy` in `cli` module * Rename `SuperIdentifier` struct to `ProtoIdentifier` * Remove `SuperTimer` struct * Bring back `CloseTimedoutTcpStreamChannels` method * Run `go mod tidy` everywhere * Remove `GOGC` environment variable from tapper * Fix the tests * Bring back `debug.FreeOSMemory()` call * Make `CloseOtherProtocolDissectors` method mutexed * Revert "Remove `GOGC` environment variable from tapper" This reverts commit |
||
|
M. Mert Yıldıran
|
e1ad302c29
|
Make logger a separate module such that don't depend on shared module as a whole for logging (#1047)
* Make `logger` a separate module such that don't depend on `shared` module as a whole for logging * Update `Dockerfile` |
||
|
M. Mert Yıldıran
|
1a5378b64b
|
Increase the OOM risk in exchange of less idle CPU usage (#979)
* Increase the OOM risk in exchange of less idle CPU usage * Read the interval from an environment variable named `CLOSE_TIMEDOUT_TCP_STREAM_CHANNELS_INTERVAL_MS` * Log the `getCloseTimedoutTcpChannelsInterval` return value |
||
RoyUP9
|
763f72a640
|
remove newline in logs, fixed logs time format (#469) | ||
David Levanon
|
e696851261
|
Tapper Refactor (#396)
* introduce tcp_assembler and tcp_packet_source - the motivation is to … (#380) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * fix a typo * remove unused pid param Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * extract stats functions out of the main tapping function (#381) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Feature/tapper refactor i/internal tapper stats (#384) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Feature/tapper refactor i/diagnose package (#386) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats * move errors map + app stats + internal stats + periodic tasks to diagnose package * initialize tapper internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move tcp packet source to its packet (#387) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats * move errors map + app stats + internal stats + periodic tasks to diagnose package * move tcp packet source to its packet * initialize tapper internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Fix coding style * Remove `tap/internal_stats.go` * make channel between input and assembler blocking - to preserve the same behaviour we have before the refactor Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> Co-authored-by: M. Mert Yildiran <mehmet@up9.com> |
||
|
David Levanon
|
2944493e2d
|
passive-tapper refactor - first phase
* add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> |