mirror of
https://github.com/kubeshark/kubeshark.git
synced 2025-08-15 07:05:27 +00:00
feature/multiarch_build_static
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
M. Mert Yıldıran
|
e358aa4c8f
|
Remove TCP related logging to reduce the amount of logs on DEBUG level (#553)
|
||
David Levanon
|
e696851261
|
Tapper Refactor (#396)
* introduce tcp_assembler and tcp_packet_source - the motivation is to … (#380) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * fix a typo * remove unused pid param Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * extract stats functions out of the main tapping function (#381) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Feature/tapper refactor i/internal tapper stats (#384) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Feature/tapper refactor i/diagnose package (#386) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats * move errors map + app stats + internal stats + periodic tasks to diagnose package * initialize tapper internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move tcp packet source to its packet (#387) * add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue * introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads * make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation * extract stats functions out of the main tapping function * move context to tcp_assembly + fix error check of tcp source packet * use param instead of gloab flag for ipdefrag * introduce internal tapper stats * minor fixes for errors map and internal stats * move errors map + app stats + internal stats + periodic tasks to diagnose package * move tcp packet source to its packet * initialize tapper internal stats Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * Fix coding style * Remove `tap/internal_stats.go` * make channel between input and assembler blocking - to preserve the same behaviour we have before the refactor Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> Co-authored-by: M. Mert Yildiran <mehmet@up9.com> |
||
|
David Levanon
|
2944493e2d
|
passive-tapper refactor - first phase
* add passive-tapper main tester (#353) * add passive-tapper main tester * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * rename main to tester * build extenssions as part of the tester launch * add a README to the tester * solving go.mod and .sum conflicts with addition of go-errors * trivial warning fixes (#354) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * disable host mode for tester - to avoid filterAuthorities * tcp streams map (#355) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * change rlog to mizu logger * errors map (#356) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * change int to uint - errorsmap * change from int to uint * Change errorsMap.nErrors to uint. * change errors map to mizu logger instead of rlog * init mizu logger in tester + fix errormap declaration Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> * move own ips to tcp stream factory (#358) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * Feature/tapper refactor i/move own ips to tcp stream factory (#379) * add passive-tapper main tester * trivial warning fixes * add errors to go.sum of mizu agent * tcp streams map * disable host mode for tester - to avoid filterAuthorities * set tcp streams map for tcp stream factory * errors map * move own ips to tcp stream factory * fix ownips compilation issue Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com> |
||
|
M. Mert Yıldıran
|
b7ff076571
|
Set the default log level for Agent to INFO and raise it to DEBUG if dump-logs=true is provided (#373)
* Set the default log level for Agent to `INFO` and change it to `DEBUG` if `dump-logs=true` is provided
* Remove `Trace` method and replace its calls with `Debug`
* Export logging levels from `logger` by defining functions
* Revert "Export logging levels from `logger` by defining functions"
This reverts commit
|
||
|
M. Mert Yıldıran
|
7dca1ad889
|
Move stats_tracker.go into the extension API and increment MatchedPairs from inside the Emit method (#272)
* Move `stats_tracker.go` into the extension API and increment `MatchedPairs` from inside the `Emit` method * Replace multiple `sync.Mutex`(es) with low-level atomic memory primitives |
||
|
M. Mert Yıldıran
|
858a64687d
|
Stop the hanging Goroutines by dropping the old, unidentified TCP streams (#260)
* Close the hanging TCP message channels after a dynamically aligned timeout (base `10000` milliseconds) * Bring back `source.Lazy` * Add a one more `sync.Map.Delete` call * Improve the formula by taking base Goroutine count into account * Reduce duplication * Include the dropped TCP streams count into the stats tracker and print a debug log whenever it happens * Add `superIdentifier` field to `tcpStream` to check if it has identified Also stop the other protocol dissectors if a TCP stream identified by a protocol. * Take one step forward in fixing the channel closing issue (WIP) Add `sync.Mutex` to `tcpReader` and make the loops reference based. * Fix the channel closing issue * Improve the accuracy of the formula, log better and multiply `baseStreamChannelTimeoutMs` by 100 * Remove `fmt.Printf` * Replace `runtime.Gosched()` with `time.Sleep(1 * time.Millisecond)` * Close the channels of other protocols in case of an identification * Simplify the logic * Replace the formula with hard timeout 5000 milliseconds and 4000 maximum number of Goroutines |
||
|
M. Mert Yıldıran
|
4e0ff74944
|
Fix body size, receive (elapsed time) and timestamps (#258)
* Fix the HTTP body size (it's not applicable to AMQP and Kafka) * Fix the elapsed time * Change JSON fields from snake_case to camelCase |
||
|
M. Mert Yıldıran
|
366c1d0c6c
|
Refactor Mizu, define an extension API and add new protocols: AMQP, Kafka (#224)
* Separate HTTP related code into `extensions/http` as a Go plugin * Move `extensions` folder into `tap` folder * Move HTTP files into `tap/extensions/lib` for now * Replace `orcaman/concurrent-map` with `sync.Map` * Remove `grpc_assembler.go` * Remove `github.com/up9inc/mizu/tap/extensions/http/lib` * Add a build script to automatically build extensions from a known path and load them * Start to define the extension API * Implement the `run()` function for the TCP stream * Add support of defining multiple ports to the extension API * Set the extension name inside the extension * Declare the `Dissect` function in the extension API * Dissect HTTP request from inside the HTTP extension * Make the distinction of outbound and inbound ports * Dissect HTTP response from inside the HTTP extension * Bring back the HTTP request-response pair matcher * Return a `*api.RequestResponsePair` from the dissection * Bring back the gRPC-HTTP/2 parser * Fix the issues in `handleHTTP1ClientStream` and `handleHTTP1ServerStream` * Call a function pointer to emit dissected data back to the `tap` package * roee changes - trying to fix agent to work with the "api" object) - ***still not working*** * small mistake in the conflicts * Fix the issues that are introduced by the merge conflict * Add `Emitter` interface to the API and send `OutputChannelItem`(s) to `OutputChannel` * Fix the `HTTP1` handlers * Set `ConnectionInfo` in HTTP handlers * Fix the `Dockerfile` to build the extensions * remove some unwanted code * no message * Re-enable `getStreamProps` function * Migrate back from `gopacket/tcpassembly` to `gopacket/reassembly` * Introduce `HTTPPayload` struct and `HTTPPayloader` interface to `MarshalJSON()` all the data structures that are returned by the HTTP protocol * Read `socketHarOutChannel` instead of `filteredHarChannel` * Connect `OutputChannelItem` to the last WebSocket means that finally the web UI started to work again * Add `.env.example` to React app * Marshal and unmarshal `*http.Request`, `*http.Response` pairs * Move `loadExtensions` into `main.go` and map extensions into `extensionsMap` * Add `Summarize()` method to the `Dissector` interface * Add `Analyze` method to the `Dissector` interface and `MizuEntry` to the extension API * Add `Protocol` struct and make it effect the UI * Refactor `BaseEntryDetails` struct and display the source and destination ports in the UI * Display the protocol name inside the details layout * Add `Represent` method to the `Dissector` interface and manipulate the UI through this method * Make the protocol color affect the details layout color and write protocol abbreviation vertically * Remove everything HTTP related from the `tap` package and make the extension system fully functional * Fix the TypeScript warnings * Bring in the files related AMQP into `amqp` directory * Add `--nodefrag` flag to the tapper and bring in the main AMQP code * Implement the AMQP `BasicPublish` and fix some issues in the UI when the response payload is missing * Implement `representBasicPublish` method * Fix several minor issues * Implement the AMQP `BasicDeliver` * Implement the AMQP `QueueDeclare` * Implement the AMQP `ExchangeDeclare` * Implement the AMQP `ConnectionStart` * Implement the AMQP `ConnectionClose` * Implement the AMQP `QueueBind` * Implement the AMQP `BasicConsume` * Fix an issue in `ConnectionStart` * Fix a linter error * Bring in the files related Kafka into `kafka` directory * Fix the build errors in Kafka Go files * Implement `Dissect` method of Kafka and adapt request-response pair matcher to asynchronous client-server stream * Do the "Is reversed?" checked inside `getStreamProps` and fix an issue in Kafka `Dissect` method * Implement `Analyze`, `Summarize` methods of Kafka * Implement the representations for Kafka `Metadata`, `RequestHeader` and `ResponseHeader` * Refactor the AMQP and Kafka implementations to create the summary string only inside the `Analyze` method * Implement the representations for Kafka `ApiVersions` * Implement the representations for Kafka `Produce` * Implement the representations for Kafka `Fetch` * Implement the representations for Kafka `ListOffsets`, `CreateTopics` and `DeleteTopics` * Fix the encoding of AMQP `BasicPublish` and `BasicDeliver` body * Remove the unnecessary logging * Remove more logging * Introduce `Version` field to `Protocol` struct for dynamically switching the HTTP protocol to HTTP/2 * Fix the issues in analysis and representation of HTTP/2 (gRPC) protocol * Fix the issues in summary section of details layout for HTTP/2 (gRPC) protocol * Fix the read errors that freezes the sniffer in HTTP and Kafka * Fix the issues in HTTP POST data * Fix one more issue in HTTP POST data * Fix an infinite loop in Kafka * Fix another freezing issue in Kafka * Revert "UI Infra - Support multiple entry types + refactoring (#211)" This reverts commit |
||
Nimrod Gilboa Markevich
|
5d5c11c37c
|
Add to periodic stats print in tapper (#220) | ||
gadotroee
|
f03df50def
|
Change logging to use "rlog" library (#99)
* change logging to use the nice rlog library * no message * review fixes * no message |
||
|
nimrod-up9
|
135b1a5e1e
|
TRA-3212 Passive-Tapper and Mizu share code (#70)
* Use log in tap package instead of fmt. * Moved api/pkg/tap to root. * Added go.mod and go.sum for tap. * Added replace for shared. * api uses tap module instead of tap package. * Removed dependency of tap in shared by moving env var out of tap. * Fixed compilation bugs. * Fixed: Forgot to export struct field HostMode. * Removed unused flag. * Close har output channel when done. * Moved websocket out of mizu and into passive-tapper. * Send connection details over har output channel. * Fixed compilation errors. * Removed unused info from request response cache. * Renamed connection -> connectionID. * Fixed rename bug. * Export setters and getters for filter ips and ports. * Added tap dependency to Dockerfile. * Uncomment error messages. * Renamed `filterIpAddresses` -> `filterAuthorities`. * Renamed ConnectionID -> ConnectionInfo. * Fixed: Missed one replace. |