* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically
* Remove `MAX_NUMBER_OF_GOROUTINES` environment variable
* Change the line
* Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`
* Write the client and integrate to the new real-time database
* Refactor the WebSocket implementaiton for `/ws`
* Adapt the UI to the new filtering system
* Fix the rest of the issues in the UI
* Increase the buffer of the scanner
* Implement accessing single records
* Increase the buffer of another scanner
* Populate `Request` and `Response` fields of `MizuEntry`
* Add syntax highlighting for the query
* Add database to `Dockerfile`
* Fix some issues
* Update the `realtime_dbms` Git module commit hash
* Upgrade Gin version and print the query string
* Revert "Upgrade Gin version and print the query string"
This reverts commit aa09f904ee.
* Use WebSocket's itself to query instead of the query string
* Fix some errors related to conversion to HAR
* Fix the issues caused by the latest merge
* Fix the build error
* Fix PR validation GitHub workflow
* Replace the git submodule with latest Basenine version `0.1.0`
Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.
* Move Basenine host and port constants to `shared` module
* Reliably execute and wait for Basenine to become available
* Upgrade Basenine version
* Properly close WebSocket and data channel
* Fix the issues caused by the recent merge commit
* Clean up the TypeScript code
* Update `.gitignore`
* Limit the database size
* Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions
* Run `go mod tidy` on `agent`
* Upgrade `github.com/up9inc/basenine/client/go` version
* Implement a mechanism to update the query using click events in the UI and use it for protocol macros
* Update the query on click to timestamps
* Fix some issues in the WebSocket and channel handling
* Update the query on clicks to status code
* Update the query on clicks to method, path and service
* Update the query on clicks to is outgoing, source and destination ports
* Add an API endpoint to validate the query against syntax errors
* Move the query background color state into `TrafficPage`
* Fix the logic in `setQuery`
* Display a toast message in case of a syntax error in the query
* Remove a call to `fmt.Printf`
* Upgrade Basenine version to `0.1.3`
* Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable
* Have the `path` key in request details, in HTTP
* Rearrange the HTTP headers for the querying
* Do the same thing for `cookies` and `queryString`
* Update the query on click to table elements
Add the selectors for `TABLE` type representations in HTTP extension.
* Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`
* Add the selectors for `TABLE` type representations in AMQP extension
* Add the selectors for `TABLE` type representations in Kafka extension
* Add the selectors for `TABLE` type representations in Redis extension
* Define a struct in `tap/api.go` for the section representation data
* Add the selectors for `BODY` type representations
* Add `request.path` to the HTTP request details
* Change the summary string's field name from `path` to `summary`
* Introduce `queryable` CSS class for queryable UI elements and underline them on hover
* Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values
Upgrade Basenine version to `0.2.0`.
* Verify the sha256sum of Basenine executable inside `Dockerfile`
* Pass the start time to web UI through WebSocket and always show the `EntriesList` footer
* Pipe the `stderr` of Basenine as well
* Fix the layout issues related to `CodeEditor` in the UI
* Use the correct `shasum` command in `Dockerfile`
* Upgrade Basenine version to `0.2.1`
* Limit the height of `CodeEditor` container
* Remove `Paused` enum `ConnectionStatus` in UI
* Fix the issue caused by the recent merge
* Add the filtering guide (cheatsheet)
* Update open cheatsheet button's title
* Update cheatsheet content
* Remove the old SQLite code, adapt the `--analyze` related code to Basenine
* Change the method signature of `NewEntry`
* Change the method signature of `Represent`
* Introduce `HTTPPair` field in `MizuEntry` specific to HTTP
* Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`
Also remove the `getEstimatedEntrySizeBytes` method.
* Remove `gorm.io/gorm` dependency
* Remove unused `sensitiveDataFiltering` folder
* Increase the left margin of open cheatsheet button
* Add `overflow: auto` to the cheatsheet `Modal`
* Fix `GetEntry` method
* Fix the macro for gRPC
* Fix an interface conversion in case of AMQP
* Fix two more interface conversion errors in AMQP
* Make the `syncEntriesImpl` method blocking
* Fix a grammar mistake in the cheatsheet
* Adapt to the changes in the recent merge commit
* Improve the cheatsheet text
* Always display the timestamp in `en-US`
* Upgrade Basenine version to `0.2.2`
* Fix the order of closing Basenine connections and channels
* Don't close the Basenine channels at all
* Upgrade Basenine version to `0.2.3`
* Set the initial filter to `rlimit(100)`
* Make Basenine persistent
* Upgrade Basenine version to `0.2.4`
* Update `debug.Dockerfile`
* Fix a failing test
* Upgrade Basenine version to `0.2.5`
* Revert "Do not show play icon when disconnected (#428)"
This reverts commit 8af2e562f8.
* Upgrade Basenine version to `0.2.6`
* Make all non-informative things informative
* Make `100` a constant
* Use `===` in JavaScript no matter what
* Remove a forgotten `console.log`
* Add a comment and update the `query` in `syncEntriesImpl`
* Don't call `panic` in `GetEntry`
* Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`
* Remove unnecessary `\n` characters in the logs
* multiple netns listen - initial commit
* multiple netns listen - actual work
* remove redundant log line
* map /proc of host to tapper
* changing kubernetes provider again after big conflict
* revert node-sass version back to 5.0.0
* Rename host_source to hostSource
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
* PR fixes - adding comment + typos + naming conventions
* go fmt + making procfs read only
* setns back to the original value after packet source initialized
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
* WIP
* WIP
* WIP
* WIP
* WIP
* Update tapRunner.go and k8sTapManager.go
* Update cleanRunner.go, common.go, and 8 more files...
* Update common.go, tapConfig.go, and 2 more files...
* Update config.go, config.go, and 5 more files...
* Update tapRunner.go, config.go, and 7 more files...
* Update cleanRunner.go, logs.go, and 2 more files...
* Update k8sTapManager.go, provider.go, and watch.go
* Update go.sum, go.mod, and go.sum
* Update go.mod and go.sum
* Update go.mod, go.sum, and 2 more files...
* Revert "Update go.mod, go.sum, and 2 more files..."
This reverts commit 8140311349.
* Update funcWrappers.go, tapRunner.go, and 4 more files...
* Update main.go, tapRunner.go, and mizuTapperSyncer.go
* introduce tcp_assembler and tcp_packet_source - the motivation is to … (#380)
* add passive-tapper main tester (#353)
* add passive-tapper main tester
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* rename main to tester
* build extenssions as part of the tester launch
* add a README to the tester
* solving go.mod and .sum conflicts with addition of go-errors
* trivial warning fixes (#354)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* tcp streams map (#355)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* change rlog to mizu logger
* errors map (#356)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* change int to uint - errorsmap
* change from int to uint
* Change errorsMap.nErrors to uint.
* change errors map to mizu logger instead of rlog
* init mizu logger in tester + fix errormap declaration
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move own ips to tcp stream factory (#358)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* Feature/tapper refactor i/move own ips to tcp stream factory (#379)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* fix ownips compilation issue
* introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
* make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
* move context to tcp_assembly + fix error check of tcp source packet
* use param instead of gloab flag for ipdefrag
* fix a typo
* remove unused pid param
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* extract stats functions out of the main tapping function (#381)
* add passive-tapper main tester (#353)
* add passive-tapper main tester
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* rename main to tester
* build extenssions as part of the tester launch
* add a README to the tester
* solving go.mod and .sum conflicts with addition of go-errors
* trivial warning fixes (#354)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* tcp streams map (#355)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* change rlog to mizu logger
* errors map (#356)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* change int to uint - errorsmap
* change from int to uint
* Change errorsMap.nErrors to uint.
* change errors map to mizu logger instead of rlog
* init mizu logger in tester + fix errormap declaration
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move own ips to tcp stream factory (#358)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* Feature/tapper refactor i/move own ips to tcp stream factory (#379)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* fix ownips compilation issue
* introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
* make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
* extract stats functions out of the main tapping function
* move context to tcp_assembly + fix error check of tcp source packet
* use param instead of gloab flag for ipdefrag
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* Feature/tapper refactor i/internal tapper stats (#384)
* add passive-tapper main tester (#353)
* add passive-tapper main tester
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* rename main to tester
* build extenssions as part of the tester launch
* add a README to the tester
* solving go.mod and .sum conflicts with addition of go-errors
* trivial warning fixes (#354)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* tcp streams map (#355)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* change rlog to mizu logger
* errors map (#356)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* change int to uint - errorsmap
* change from int to uint
* Change errorsMap.nErrors to uint.
* change errors map to mizu logger instead of rlog
* init mizu logger in tester + fix errormap declaration
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move own ips to tcp stream factory (#358)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* Feature/tapper refactor i/move own ips to tcp stream factory (#379)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* fix ownips compilation issue
* introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
* make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
* extract stats functions out of the main tapping function
* move context to tcp_assembly + fix error check of tcp source packet
* use param instead of gloab flag for ipdefrag
* introduce internal tapper stats
* minor fixes for errors map and internal stats
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* Feature/tapper refactor i/diagnose package (#386)
* add passive-tapper main tester (#353)
* add passive-tapper main tester
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* rename main to tester
* build extenssions as part of the tester launch
* add a README to the tester
* solving go.mod and .sum conflicts with addition of go-errors
* trivial warning fixes (#354)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* tcp streams map (#355)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* change rlog to mizu logger
* errors map (#356)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* change int to uint - errorsmap
* change from int to uint
* Change errorsMap.nErrors to uint.
* change errors map to mizu logger instead of rlog
* init mizu logger in tester + fix errormap declaration
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move own ips to tcp stream factory (#358)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* Feature/tapper refactor i/move own ips to tcp stream factory (#379)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* fix ownips compilation issue
* introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
* make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
* extract stats functions out of the main tapping function
* move context to tcp_assembly + fix error check of tcp source packet
* use param instead of gloab flag for ipdefrag
* introduce internal tapper stats
* minor fixes for errors map and internal stats
* move errors map + app stats + internal stats + periodic tasks to diagnose package
* initialize tapper internal stats
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move tcp packet source to its packet (#387)
* add passive-tapper main tester (#353)
* add passive-tapper main tester
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* rename main to tester
* build extenssions as part of the tester launch
* add a README to the tester
* solving go.mod and .sum conflicts with addition of go-errors
* trivial warning fixes (#354)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* disable host mode for tester - to avoid filterAuthorities
* tcp streams map (#355)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* change rlog to mizu logger
* errors map (#356)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* change int to uint - errorsmap
* change from int to uint
* Change errorsMap.nErrors to uint.
* change errors map to mizu logger instead of rlog
* init mizu logger in tester + fix errormap declaration
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* move own ips to tcp stream factory (#358)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* Feature/tapper refactor i/move own ips to tcp stream factory (#379)
* add passive-tapper main tester
* trivial warning fixes
* add errors to go.sum of mizu agent
* tcp streams map
* disable host mode for tester - to avoid filterAuthorities
* set tcp streams map for tcp stream factory
* errors map
* move own ips to tcp stream factory
* fix ownips compilation issue
* introduce tcp_assembler and tcp_packet_source - the motivation is to split the actual packet sources from the assembler, so we can have a single thread for the assembly which is separated from packet source threads
* make struts private at this point - planning to move some packages to their own package so we can utilize encapsulation
* extract stats functions out of the main tapping function
* move context to tcp_assembly + fix error check of tcp source packet
* use param instead of gloab flag for ipdefrag
* introduce internal tapper stats
* minor fixes for errors map and internal stats
* move errors map + app stats + internal stats + periodic tasks to diagnose package
* move tcp packet source to its packet
* initialize tapper internal stats
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
* Fix coding style
* Remove `tap/internal_stats.go`
* make channel between input and assembler blocking - to preserve the same behaviour we have before the refactor
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
Co-authored-by: M. Mert Yildiran <mehmet@up9.com>
