Commit Graph

184 Commits

Author SHA1 Message Date
M. Mert Yıldıran
4e17ac5654
Remove unnecessary fields and split service into src.name and dst.name (#525)
* Remove unnecessary fields and split `service` into `src.name` and `dst.name`

* Don't fall back to IP address but instead display `[Unresolved]` text

* Fix the CSS issues in the plus icon position and replace the separator `->` text with `SwapHorizIcon`
2021-12-14 11:36:02 +03:00
M. Mert Yıldıran
d274db2d87
Fix the CSS issues in queryable vertical protocol element (#526) 2021-12-12 19:38:14 +03:00
M. Mert Yıldıran
0a2aacfb02
Include milliseconds information into the timestamps in the UI (#524)
* Include milliseconds information into the timestamps in the UI

* Upgrade Basenine version from `0.2.16` to `0.2.17`

* Increase the `width` of timestamp
2021-12-10 18:03:17 +03:00
M. Mert Yıldıran
9f1e311689
TRA-4017 Bring back getOldEntries method using fetch API and always start streaming from now (#518)
* Bring back `getOldEntries` method using fetch API

* Determine no more data on top based on `leftOff` value

* Remove `entriesBuffer` state

* Always open WebSocket with some `leftOff` value

* Rename `leftOff` state to `leftOffBottom`

* Don't set the `focusedEntryId` through WebSocket if the WebSocket is closed

* Call `setQueriedCurrent` with addition

* Close WebSocket upon reaching to top

* Open WebSocket upon snapping to bottom

* Close the WebSocket on snap broken event instead

* Set queried current value to zero upon filter submit

* Upgrade `react-scrollable-feed-virtualized` version and use `scrollToIndex` function

* Change the footer text format

* Improve no more data top logic

* Fix `closeWebSocket()` call logic in `onSnapBrokenEvent` and handle `data.meta` being `null` in `getOldEntries`

* Fix the issues around fetching old records

* Clean up `EntriesList.module.sass`

* Decrement initial `leftOffTop` value by `2`

* Fix the order of `incomingEntries` in `getOldEntries`

* Request `leftOffTop - 1` from `fetchEntries`

* Limit the front-end total entries fetched through WebSocket count to `10000`

* Lose the UI performance gain that's provided by #452

* Revert "Fix the selected entry behavior by propagating the `focusedEntryId` through WebSocket (before #452) TRA-3983 (#513)"

This reverts commit 873f252544.

* Fix the issues caused by 09371f141f

* Upgrade Basenine version from `0.2.13` to `0.2.14`

* Upgrade Basenine version from `0.2.14` to `0.2.15`

* Fix the condition of "Fetch old records" button visibility

* Upgrade Basenine version from `0.2.15` to `0.2.16` and fix the UI code related to fetching old records

* Make `newEntries` constant
2021-12-08 15:19:35 +03:00
M. Mert Yıldıran
9aaf18842b
Fix the CSS issue in Queryable inside EntryViewLine (#521) 2021-12-07 14:15:49 +03:00
M. Mert Yıldıran
880842c39f
Fix the styling of Queryable under StatusCode and Summary components (#519) 2021-12-04 20:25:22 +03:00
M. Mert Yıldıran
af557f7052
Add Queryable component to show a green add circle icon for the queryable UI elements (#512)
* Add `Queryable` component to show a green circle and use it in `EntryViewLine`

* Refactor `Queryable` component

* Use the `Queryable` component `EntryDetailed`

* Use the `Queryable` component `Summary`

* Instead of passing the style to `Queryable`, pass the children components directly

* Make `useTooltip = true` by default in `Queryable`

* Refactor a lot of styling to achieve using `Queryable` in `Protocol` component

* Migrate the last queryable elements in `EntryListItem` to `Queryable` component

* Fix some of the styling issues

* Make horizontal `Protocol` `Queryable` too

* Remove unnecessary child constants

* Revert some of the changes in 2a93f365f5

* Fix rest of the styling issues

* Fix one more styling issue

* Update the screenshots and text in the cheatsheet according to the change

* Use `let` not `var`

* Add missing dependencies to the React hook
2021-11-30 17:52:21 +03:00
M. Mert Yıldıran
b745f65971
Handle unexpected socket close and replace the default rlimit(100) filter with leftOff(-1) filter (#508)
* Handle unexpected socket close and replace the default `rlimit(100)` filter with `leftOff(-1)` filter

* Rename `dontClear` parameter to `resetEntriesBuffer` and remove negation
2021-11-30 16:30:18 +03:00
M. Mert Yıldıran
873f252544
Fix the selected entry behavior by propagating the focusedEntryId through WebSocket (before #452) TRA-3983 (#513)
* Revert the select entry behavior into its original state RACING! (before #452) [TRA-3983 alternative 3]

* Remove the remaining `forceSelect`(s)

* Add a missing `focusedEntryId` prop

* Fix the race condition

* Propagate the `focusedEntryId` through WebSocket to prevent racing
2021-11-30 15:27:10 +03:00
M. Mert Yıldıran
9696ad9bad
Show the EntryItem as EntrySummary in EntryDetailed (#506) 2021-11-28 10:59:40 +03:00
M. Mert Yıldıran
a1bda0a6c3
Hide Encoding field if it's undefined or empty in the UI (#511) 2021-11-26 09:40:44 +03:00
M. Mert Yıldıran
12ca3d8779
Make the gRPC and HTTP/2 distinction (#492)
* Remove the extra negation on `nodefrag` flag's value

* Support IPv4 fragmentation and IPv6 at the same time

* Set `Method` and `StatusCode` fields correctly for `HTTP/2`

* Replace unnecessary `grpc` naming with `http2`

* Make the `gRPC` and `HTTP/2` distinction

* Fix the macros of `http` extension

* Fix the macros of other protocol extensions

* Update the method signature of `Represent`

* Fix the `HTTP/2` support

* Fix some minor issues

* Upgrade Basenine version from `0.2.10` to `0.2.11`

Sorts macros before expanding them and prioritize the long macros.

* Don't regex split the gRPC method name

* Re-enable `nodefrag` flag
2021-11-22 17:46:35 +03:00
M. Mert Yıldıran
8118569460
Show the source and destination IP in the entry feed (#485) 2021-11-18 20:21:51 +03:00
M. Mert Yıldıran
dd53a36d5f
Prevent the crash on client-side in case of text being undefined in FancyTextDisplay (#481)
* Prevent the crash on client-side in case of `text` being undefined in `FancyTextDisplay`

* Use `String(text)` instead
2021-11-17 18:50:09 +03:00
M. Mert Yıldıran
ad78f1dcd7
Clear focusedEntryId state in case of a filter is applied (#482) 2021-11-17 18:20:23 +03:00
M. Mert Yıldıran
bb85312b9f
Don't omit the key-value pair if the value is false in EntryTableSection (#478) 2021-11-17 15:02:23 +03:00
M. Mert Yıldıran
7f265dc4c5
Return 404 instead of 500 if the entry could not be found and display a toast message (#464) 2021-11-16 17:13:07 +03:00
M. Mert Yıldıran
618cb3a409
Optimize UI entry feed performance (#452)
* Optimize the React code for feeding the entries

By building `EntryItem` only once and updating the `entries` state on meta query messages.

* Upgrade `react-scrollable-feed-virtualized` version from `1.4.3` to `1.4.8`

* Fix the `isSelected` state

* Set the query text before deciding the background to prevent lags while typing

* Upgrade Basenine version from `0.2.6` to `0.2.7`

* Set the query background color only if the query is same after the HTTP request and use `useEffect` instead

* Upgrade Basenine version from `0.2.7` to `0.2.8`

* Use `CancelToken` of `axios` instead of trying to check the query state

* Turn `updateQuery` function into a state hook

* Update the macro for `http`

* Do the `source.cancel()` call in `axios.CancelToken`

* Reduce client-side logging
2021-11-15 17:32:05 +03:00
M. Mert Yıldıran
2582b7a65c
Ignore SNYK-JS-JSONSCHEMA-1920922 (#462)
Dependency tree:
`node-sass@5.0.0 > node-gyp@7.1.2 > request@2.88.2 > http-signature@1.2.0 > jsprim@1.4.1 > json-schema@0.2.3`

`node-sass` should fix it first.
2021-11-15 17:29:20 +03:00
M. Mert Yıldıran
5382a52025
Fix the CSS issues in the cheatsheet modal (#448)
* Fix the CSS issues in the cheatsheet modal

* Change the Sass variable names
2021-11-11 11:26:20 +03:00
M. Mert Yıldıran
eb61831a2c
Fix the Analysis button's style into its original state (#447)
* Fix the `Analysis` button's style into its original state

* Fix the MUI button style into its original state
2021-11-10 17:52:51 +03:00
M. Mert Yıldıran
b970640ebc
Remove the Reconnect button (#444) 2021-11-10 17:06:41 +03:00
M. Mert Yıldıran
d2fe3f6620
Migrate from SQLite to Basenine and introduce a new filtering syntax (#279)
* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically

* Remove `MAX_NUMBER_OF_GOROUTINES` environment variable

* Change the line

* Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`

* Write the client and integrate to the new real-time database

* Refactor the WebSocket implementaiton for `/ws`

* Adapt the UI to the new filtering system

* Fix the rest of the issues in the UI

* Increase the buffer of the scanner

* Implement accessing single records

* Increase the buffer of another scanner

* Populate `Request` and `Response` fields of `MizuEntry`

* Add syntax highlighting for the query

* Add database to `Dockerfile`

* Fix some issues

* Update the `realtime_dbms` Git module commit hash

* Upgrade Gin version and print the query string

* Revert "Upgrade Gin version and print the query string"

This reverts commit aa09f904ee.

* Use WebSocket's itself to query instead of the query string

* Fix some errors related to conversion to HAR

* Fix the issues caused by the latest merge

* Fix the build error

* Fix PR validation GitHub workflow

* Replace the git submodule with latest Basenine version `0.1.0`

Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.

* Move Basenine host and port constants to `shared` module

* Reliably execute and wait for Basenine to become available

* Upgrade Basenine version

* Properly close WebSocket and data channel

* Fix the issues caused by the recent merge commit

* Clean up the TypeScript code

* Update `.gitignore`

* Limit the database size

* Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions

* Run `go mod tidy` on `agent`

* Upgrade `github.com/up9inc/basenine/client/go` version

* Implement a mechanism to update the query using click events in the UI and use it for protocol macros

* Update the query on click to timestamps

* Fix some issues in the WebSocket and channel handling

* Update the query on clicks to status code

* Update the query on clicks to method, path and service

* Update the query on clicks to is outgoing, source and destination ports

* Add an API endpoint to validate the query against syntax errors

* Move the query background color state into `TrafficPage`

* Fix the logic in `setQuery`

* Display a toast message in case of a syntax error in the query

* Remove a call to `fmt.Printf`

* Upgrade Basenine version to `0.1.3`

* Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable

* Have the `path` key in request details, in HTTP

* Rearrange the HTTP headers for the querying

* Do the same thing for `cookies` and `queryString`

* Update the query on click to table elements

Add the selectors for `TABLE` type representations in HTTP extension.

* Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`

* Add the selectors for `TABLE` type representations in AMQP extension

* Add the selectors for `TABLE` type representations in Kafka extension

* Add the selectors for `TABLE` type representations in Redis extension

* Define a struct in `tap/api.go` for the section representation data

* Add the selectors for `BODY` type representations

* Add `request.path` to the HTTP request details

* Change the summary string's field name from `path` to `summary`

* Introduce `queryable` CSS class for queryable UI elements and underline them on hover

* Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values

Upgrade Basenine version to `0.2.0`.

* Verify the sha256sum of Basenine executable inside `Dockerfile`

* Pass the start time to web UI through WebSocket and always show the `EntriesList` footer

* Pipe the `stderr` of Basenine as well

* Fix the layout issues related to `CodeEditor` in the UI

* Use the correct `shasum` command in `Dockerfile`

* Upgrade Basenine version to `0.2.1`

* Limit the height of `CodeEditor` container

* Remove `Paused` enum `ConnectionStatus` in UI

* Fix the issue caused by the recent merge

* Add the filtering guide (cheatsheet)

* Update open cheatsheet button's title

* Update cheatsheet content

* Remove the old SQLite code, adapt the `--analyze` related code to Basenine

* Change the method signature of `NewEntry`

* Change the method signature of `Represent`

* Introduce `HTTPPair` field in `MizuEntry` specific to HTTP

* Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`

Also remove the `getEstimatedEntrySizeBytes` method.

* Remove `gorm.io/gorm` dependency

* Remove unused `sensitiveDataFiltering` folder

* Increase the left margin of open cheatsheet button

* Add `overflow: auto` to the cheatsheet `Modal`

* Fix `GetEntry` method

* Fix the macro for gRPC

* Fix an interface conversion in case of AMQP

* Fix two more interface conversion errors in AMQP

* Make the `syncEntriesImpl` method blocking

* Fix a grammar mistake in the cheatsheet

* Adapt to the changes in the recent merge commit

* Improve the cheatsheet text

* Always display the timestamp in `en-US`

* Upgrade Basenine version to `0.2.2`

* Fix the order of closing Basenine connections and channels

* Don't close the Basenine channels at all

* Upgrade Basenine version to `0.2.3`

* Set the initial filter to `rlimit(100)`

* Make Basenine persistent

* Upgrade Basenine version to `0.2.4`

* Update `debug.Dockerfile`

* Fix a failing test

* Upgrade Basenine version to `0.2.5`

* Revert "Do not show play icon when disconnected (#428)"

This reverts commit 8af2e562f8.

* Upgrade Basenine version to `0.2.6`

* Make all non-informative things informative

* Make `100` a constant

* Use `===` in JavaScript no matter what

* Remove a forgotten `console.log`

* Add a comment and update the `query` in `syncEntriesImpl`

* Don't call `panic` in `GetEntry`

* Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`

* Remove unnecessary `\n` characters in the logs
2021-11-09 19:54:48 +03:00
RamiBerm
4cf3c9c6d3
TRA-3913 support mizu via expose service (#440)
* Update README.md, tapRunner.go, and 4 more files...

* Update testsUtils.go

* Update proxy.go

* Update README.md, testsUtils.go, and 3 more files...

* Update testsUtils.go and provider.go
2021-11-08 11:23:03 +02:00
lirazyehezkel
8af2e562f8
Do not show play icon when disconnected (#428) 2021-11-03 15:34:54 +02:00
lirazyehezkel
63f6877924
Broken icons when mizu disconnect (#419) 2021-11-02 18:33:40 +02:00
Igor Gov
2c29903910
Run snyk on all project with ignore current issues (#415) 2021-10-28 21:59:42 +03:00
RoyUP9
4f6da91d74
fixed naming of latency to response time (#388) 2021-10-21 12:45:17 +03:00
M. Mert Yıldıran
145e7cda01
Add OAS contract monitoring support (#325)
* Add OAS contract monitoring support

* Pass the contract failure reason to UI

* Fix the issues related to contract validation

* Fix rest of the issues in the UI

* Add documentation related to contract monitoring feature

* Fix a typo in the docs

* Unmarshal to `HTTPRequestResponsePair` only if the OAS validation is enabled

* Fix an issue caused by the merge commit

* Slightly change the logic in the `validateOAS` method

Change the `contractText` value to `No Breaches` or `Breach` and make the text `white-space: nowrap`.

* Retrieve and display the failure reason for both request and response

Also display the content of the contract/OAS file in the UI.

* Display the OAS under `CONTRACT` tab with syntax highlighting

Also fix the styling in the entry feed.

* Remove `EnforcePolicyFileDeprecated` constant

* Log the other errors as well

* Get context from caller instead

* Define a type for the contract status and make its values enum-like

* Remove an unnecessary `if` statement

* Validate OAS in the CLI before passing it to Agent

* Get rid of the `github.com/ghodss/yaml` dependency in `loadOAS` by using `LoadFromData`

* Fix an artifact from the merge conflict
2021-10-19 14:24:22 +03:00
M. Mert Yıldıran
9d179c7227
Ignore an eslint error (#351)
* Ignore an eslint error

* Change the fix
2021-10-17 15:28:03 +03:00
RoyUP9
26788bb3a6
organize routes (#348) 2021-10-13 17:31:15 +03:00
lirazyehezkel
d308468f1b
Feature/UI/mizu analysis with up9 auth (#346)
* analysis button layout

* get auth status api

* status auth state

* css
2021-10-12 17:47:24 +03:00
M. Mert Yıldıran
10e695d7a0
Fix expanded button color (#343) 2021-10-12 14:25:44 +03:00
M. Mert Yıldıran
61f24320b8
Fix the issues in the Tabs React component (#335)
* Fix the issues in the `Tabs` React component

* Update the boolean expression as well
2021-10-09 13:16:08 +03:00
lirazyehezkel
15021daa2e
service path filter (#327) 2021-10-07 10:51:30 +03:00
lirazyehezkel
aa3510e936
service filter (#324) 2021-10-06 16:22:08 +03:00
Selton Fiuza
c26eb843e3
[refactor/TRA-3693] type:latency to slo and latency field to response-time (#282)
* type:latency to slo and latency field to response-time

* remove comment from import

* Friendly message on ignored rules and format

* formatting

* change conditional to catch negative values and ignore it

* Fix Bug Alon Reported

* sliceUtils to shared
2021-09-29 11:51:03 -03:00
lirazyehezkel
51fc3307be
Mizu rules font (#306) 2021-09-27 14:18:10 +03:00
M. Mert Yıldıran
cdf1c39a52
Omit the RULES tab if the policy rules feature is inactive (#303)
* Omit the `RULES` tab if the policy rules feature is inactive (WIP)

* Propagate the boolean value `isRulesEnabled` from file read error to UI

* Remove the debug log
2021-09-25 18:15:54 +03:00
M. Mert Yıldıran
db1f7d34cf
Omit the RESPONSE tab and elapsedTime if the response is empty (#298)
* Omit the `RESPONSE` tab and `elapsedTime` if the `response` is empty

* Use the `hidden` attribute instead
2021-09-24 13:49:20 +03:00
M. Mert Yıldıran
e8ea93cb64
Upgrade react-scrollable-feed-virtualized version from 1.4.2 to 1.4.3 (#299) 2021-09-23 08:36:44 +03:00
Selton Fiuza
6337b75f0e
[TRA-3659] Fix rules (#271)
* Fix rules

* Not reay, error on running

* Empty dissector Rules()

* almost working

* Finally, fixed

* undo changes on agent/pkg/utils/har.go

* fix not showing service on rules detail

* Update tap/api/api.go

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>

* Update agent/pkg/controllers/entries_controller.go

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>

* Update agent/pkg/controllers/entries_controller.go

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>

* unwrap Data

* Fix bug off using more than one latency rule that always get the first.

* fix json type, decoding base64 before unmarshal

* Run `go mod tidy` on `cli/go.sum`

* Fix the linting issues

* Remove a `FIXME` comment

* Remove a CSS rule

* Adapt `ruleNumberText` CSS class to the design language of the UI

* Fix an issue in the UI related to `rule.Latency` slipping out

* Removed unecessary codes.

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: M. Mert Yildiran <mehmet@up9.com>
2021-09-18 14:02:18 -03:00
M. Mert Yıldıran
616eccb2cf
Make ScrollableFeed virtualized by replacing react-scrollable-feed with react-scrollable-feed-virtualized (#268)
* Make `ScrollableFeed` virtualized by replacing `react-scrollable-feed` with `react-scrollable-feed-virtualized`

* fix get new entries button

* Fix the not populated `Protocol` struct in case of `GetEntries` endpoint is called

Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
2021-09-13 16:18:43 +03:00
M. Mert Yıldıran
30f07479cb
Fix the JSON style to camelCase and rename CONTRIBUTE.md to CONTRIBUTING.md (#274)
* Fix the JSON style to camelCase and rename `CONTRIBUTE.md` to `CONTRIBUTING.md`

* Move `CONTRIBUTING.md` to `.github/` directory
2021-09-13 10:55:46 +03:00
M. Mert Yıldıran
eba7a3b476
Temporarily fix the filtering into its original state (#266) 2021-09-07 09:50:33 +03:00
gadotroee
c8705822b3
TRA-3658 - Fix analysis feature (#261) 2021-09-06 09:46:49 +03:00
M. Mert Yıldıran
4e0ff74944
Fix body size, receive (elapsed time) and timestamps (#258)
* Fix the HTTP body size (it's not applicable to AMQP and Kafka)

* Fix the elapsed time

* Change JSON fields from snake_case to camelCase
2021-09-04 17:15:39 +03:00
M. Mert Yıldıran
366c1d0c6c
Refactor Mizu, define an extension API and add new protocols: AMQP, Kafka (#224)
* Separate HTTP related code into `extensions/http` as a Go plugin

* Move `extensions` folder into `tap` folder

* Move HTTP files into `tap/extensions/lib` for now

* Replace `orcaman/concurrent-map` with `sync.Map`

* Remove `grpc_assembler.go`

* Remove `github.com/up9inc/mizu/tap/extensions/http/lib`

* Add a build script to automatically build extensions from a known path and load them

* Start to define the extension API

* Implement the `run()` function for the TCP stream

* Add support of defining multiple ports to the extension API

* Set the extension name inside the extension

* Declare the `Dissect` function in the extension API

* Dissect HTTP request from inside the HTTP extension

* Make the distinction of outbound and inbound ports

* Dissect HTTP response from inside the HTTP extension

* Bring back the HTTP request-response pair matcher

* Return a `*api.RequestResponsePair` from the dissection

* Bring back the gRPC-HTTP/2 parser

* Fix the issues in `handleHTTP1ClientStream` and `handleHTTP1ServerStream`

* Call a function pointer to emit dissected data back to the `tap` package

* roee changes -
trying to fix agent to work with the "api" object) - ***still not working***

* small mistake in the conflicts

* Fix the issues that are introduced by the merge conflict

* Add `Emitter` interface to the API and send `OutputChannelItem`(s) to `OutputChannel`

* Fix the `HTTP1` handlers

* Set `ConnectionInfo` in HTTP handlers

* Fix the `Dockerfile` to build the extensions

* remove some unwanted code

* no message

* Re-enable `getStreamProps` function

* Migrate back from `gopacket/tcpassembly` to `gopacket/reassembly`

* Introduce `HTTPPayload` struct and `HTTPPayloader` interface to `MarshalJSON()` all the data structures that are returned by the HTTP protocol

* Read `socketHarOutChannel` instead of `filteredHarChannel`

* Connect `OutputChannelItem` to the last WebSocket means that finally the web UI started to work again

* Add `.env.example` to React app

* Marshal and unmarshal `*http.Request`, `*http.Response` pairs

* Move `loadExtensions` into `main.go` and map extensions into `extensionsMap`

* Add `Summarize()` method to the `Dissector` interface

* Add `Analyze` method to the `Dissector` interface and `MizuEntry` to the extension API

* Add `Protocol` struct and make it effect the UI

* Refactor `BaseEntryDetails` struct and display the source and destination ports in the UI

* Display the protocol name inside the details layout

* Add `Represent` method to the `Dissector` interface and manipulate the UI through this method

* Make the protocol color affect the details layout color and write protocol abbreviation vertically

* Remove everything HTTP related from the `tap` package and make the extension system fully functional

* Fix the TypeScript warnings

* Bring in the files related AMQP into `amqp` directory

* Add `--nodefrag` flag to the tapper and bring in the main AMQP code

* Implement the AMQP `BasicPublish` and fix some issues in the UI when the response payload is missing

* Implement `representBasicPublish` method

* Fix several minor issues

* Implement the AMQP `BasicDeliver`

* Implement the AMQP `QueueDeclare`

* Implement the AMQP `ExchangeDeclare`

* Implement the AMQP `ConnectionStart`

* Implement the AMQP `ConnectionClose`

* Implement the AMQP `QueueBind`

* Implement the AMQP `BasicConsume`

* Fix an issue in `ConnectionStart`

* Fix a linter error

* Bring in the files related Kafka into `kafka` directory

* Fix the build errors in Kafka Go files

* Implement `Dissect` method of Kafka and adapt request-response pair matcher to asynchronous client-server stream

* Do the "Is reversed?" checked inside `getStreamProps` and fix an issue in Kafka `Dissect` method

* Implement `Analyze`, `Summarize` methods of Kafka

* Implement the representations for Kafka `Metadata`, `RequestHeader` and `ResponseHeader`

* Refactor the AMQP and Kafka implementations to create the summary string only inside the `Analyze` method

* Implement the representations for Kafka `ApiVersions`

* Implement the representations for Kafka `Produce`

* Implement the representations for Kafka `Fetch`

* Implement the representations for Kafka `ListOffsets`, `CreateTopics` and `DeleteTopics`

* Fix the encoding of AMQP `BasicPublish` and `BasicDeliver` body

* Remove the unnecessary logging

* Remove more logging

* Introduce `Version` field to `Protocol` struct for dynamically switching the HTTP protocol to HTTP/2

* Fix the issues in analysis and representation of HTTP/2 (gRPC) protocol

* Fix the issues in summary section of details layout for HTTP/2 (gRPC) protocol

* Fix the read errors that freezes the sniffer in HTTP and Kafka

* Fix the issues in HTTP POST data

* Fix one more issue in HTTP POST data

* Fix an infinite loop in Kafka

* Fix another freezing issue in Kafka

* Revert "UI Infra - Support multiple entry types + refactoring (#211)"

This reverts commit f74a52d4dc.

* Fix more issues that are introduced by the merge

* Fix the status code in the summary section

* adding the cleaner again (why we removed it?).
add TODO: on the extension loop .

* fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists

* fix GetEntrties ("/entries" endpoint) - working with "tapApi.BaseEntryDetail" (moved from shared)

* Fix an issue in the UI summary section

* Refactor the protocol payload structs

* Fix a log message in the passive tapper

* Adapt `APP_PORTS` environment variable to the new extension system and change its format to `APP_PORTS='{"http": ["8001"]}' `

* Revert "fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists"

This reverts commit 4f514ae1f4.

* Bring in the necessary changes from f74a52d4dc

* Open the API server URL in the web browser as soon as Mizu is ready

* Make the TCP reader consists of a single Go routine (instead of two) and try to dissect in both client and server mode by rewinding

* Swap `TcpID` without overwriting it

* Sort extension by priority

* Try to dissect with looping through all the extensions

* fix getStreamProps function.
(it should be passed from CLI as it was before).

* Turn TCP reader back into two Goroutines (client and server)

* typo

* Learn `isClient` from the TCP stream

* Set `viewer` style `overflow: "auto"`

* Fix the memory leaks in AMQP and Kafka dissectors

* Revert some of the changes in be7c65eb6d

* Remove `allExtensionPorts` since it's no longer needed

* Remove `APP_PORTS` since it's no longer needed

* Fix all of the minor issues in the React code

* Check Kafka header size and fail-fast

* Break the dissectors loop upon a successful dissection

* Don't break the dissector loop. Protocols might collide

* Improve the HTTP request-response counter (still not perfect)

* Make the HTTP request-response counter perfect

* Revert "Revert some of the changes in be7c65eb6d3fb657a059707da3ca559937e59739"

This reverts commit 08e7d786d8.

* Bring back `filterItems` and `isHealthCheckByUserAgent` functions

* Remove some development artifacts

* remove unused and commented lines that are not relevant

* Fix the performance in TCP stream factory. Make it create two `tcpReader`(s) per extension

* Change a log to debug

* Make `*api.CounterPair` a field of `tcpReader`

* Set `isTapTarget` to always `true` again since `filterAuthorities` implementation has problems

* Remove a variable that's only used for logging even though not introduced by this branch

* Bring back the `NumberOfRules` field of `ApplicableRules` struct

* Remove the unused `NewEntry` function

* Move `k8sResolver == nil` check to a more appropriate place

* default healthChecksUserAgentHeaders should be empty array (like the default config value)

* remove spam console.log

* Rules button cause app to crash (access the service via incorrect property)

* Ignore all .env* files in docker build.

* Better caching in dockerfile: only copy go.mod before go mod download.

* Check for errors while loading an extension

* Add a comment about why `Protocol` is not a pointer

* Bring back the call to `deleteOlderThan`

* Remove the `nil` check

* Reduce the maximum allowed AMQP message from 128MB to 1MB

* Fix an error that only occurs when a Kafka broker is initiating

* Revert the change in b2abd7b990

* Fix the service name resolution in all protocols

* Remove the `anydirection` flag and fix the issue in `filterAuthorities`

* Pass `sync.Map` by reference to `deleteOlderThan` method

* Fix the packet capture issue in standalone mode that's introduced by the removal of `anydirection`

* Temporarily resolve the memory exhaustion in AMQP

* Fix a nil pointer dereference error

* Fix the CLI build error

* Fix a memory leak that's identified by `pprof`

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
2021-09-02 14:34:06 +03:00
Neim Elezi
3644fdb533
Feature/tra 3533 ssl connection pop up (#223)
* pop-up message for HTTPS domains is modified

* scroll added on hover of the TLS pop-up

* domains that were for testing are removed

* height of the pop-up is decreased

* condition for return is changed
2021-09-01 13:39:02 +03:00
Selton Fiuza
e6e7d8d58b
Fix TRA-3590 TRA-3589 (#229) 2021-08-18 22:28:13 +03:00
lirazyehezkel
f74a52d4dc
UI Infra - Support multiple entry types + refactoring (#211)
* no message

* change local api path

* generic entry list item + rename files and vars

* entry detailed generic

* fix api file

* clean warnings

* switch

* empty lines

* fix scroll to end feature

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2021-08-15 12:09:56 +03:00
Neim Elezi
6d2e9af5d7
Feature/tra 3475 scroll to end (#206)
* configuration changed

* testing scroll with button

* back to scroll button feature is done

* scroll to the end of entries feature is done

* config of docker image is reverted back

* path of docker image is changed in configStruct.go
2021-08-15 10:58:16 +03:00
Selton Fiuza
8c9b8d3217
Redesign test rules entry component (#174) 2021-08-10 16:20:16 +03:00
Selton Fiuza
dea223bfe1
Feature/tra 3349 validation rules merged with develop (#148)
* Implemented validation rules, based on: https://up9.atlassian.net/browse/TRA-3349

* Color on Entry based on rules

* Background red/green based on rules

* Change flag --validation-rules to --test-rules

* rules tab UI updated

* rules tab font and background-color is changed for objects

* Merged with develop

* Fixed compilation issues.

* Renamed fullEntry -> harEntry where appropriate.

* Change green/red logic

* Update models.go

* Fix latency bug and alignment

* Merge Conflicts fix

* Working after merge

* Working on Nimrod comments

* Resolving conflicts

* Resolving conflicts

* Resolving conflicts

* Nimrod Comments pt.3

* Log Error on configmap creation if the user doesn't have permission.

* Checking configmap permission to ignore --test-rules

* Revert time for mizu to get ready

* Nimrod comments pt 4 && merge develop pt3

* Nimrod comments pt 4 && merge develop pt3

* Const rulePolicyPath and filename

Co-authored-by: Neim <elezin9@gmail.com>
Co-authored-by: nimrod-up9 <nimrod@up9.com>
2021-08-04 09:21:36 -03:00
RamiBerm
ceb8d714e3
TRA-3420 inform user of tls traffic (#149)
* Update passive_tapper.go and tls_utils.go

* Update go.mod, go.sum, and 18 more files...

* go fmt

* Update http_reader.go, passive_tapper.go, and 3 more files...

* Update status_controller.go and status_provider.go

Co-authored-by: RamiBerm <rami.berman@up9.com>
2021-08-01 14:57:43 +03:00
lirazyehezkel
a2150b4a78
UI api helper (#131)
* Api helper

* cr fixes
2021-07-22 15:41:09 +03:00
gadotroee
6e279bfca5
TRA-3440 - add confirmation message and info on hover (tooltip) chagne analyze to analysis (#109) 2021-07-14 18:37:59 +03:00
lirazyehezkel
115692dbfc
Change logo and favicon (#106)
* Change logo and favicon

* remove warning
2021-07-13 15:52:44 +03:00
RamiBerm
09702697ad WIP 2021-07-10 18:56:07 +03:00
gadotroee
fc03ba2eda
Fix styles (#91) 2021-07-01 23:04:35 +03:00
gadotroee
3662fbcdf6
Mizu analyze improvements (#90) 2021-07-01 16:18:02 +03:00
nimrod-up9
b762e3c194
New icons for direction (#88)
* Replaced direction icons with designed icons. Match color to status code.

* Took care of padding. Added seperator line.

* Removed 1 div level and unnecessary properties in sass.

* Removed ;

* Changed to fixed svgs.
2021-06-30 12:18:29 +03:00
nimrod-up9
6f47ad862e
TRA-3317 Tap and show outbound traffic (#83)
* Tap outgoing: If --anydirection flag is passed with HOST_MODE, tap by source IP.

* Moved ConnectionInfo from http_matcher to http_reader.

* Generalized shouldTap in stream factory to get more properties.

* tap reports IsOutgoing property of tcp connection.

* gofmt.

* CLI instructs tapper to tap outgoing connections.

* API saves IsOutgoing to DB and passes it to UI.

* Add a visual marker in the HAR list for outgoing messages.

* Fixed: Swapped src and dst.

* Resolver keeps a list of all ClusterIP services.

* Do not save HARs with destination ClusterIP services.

* CLI accepts flag that controls traffic direction.

* Indicate incoming/outgoing with icon instead of with border color.

* Fixed: Didn't filter messages to services in aggregator.

* Clearer syntax around the direction icon. Added title text.

* Fixed width around direction icon.

* Less repetition.

* Removed TODO.

* Renamed incoming -> ingoing.

* More verbose title text to image.

* Switched routine order for readability.
2021-06-24 15:10:11 +03:00
lirazyehezkel
fc5d6b2d0a
Show pod name and namespace (#61) 2021-05-27 13:48:37 +03:00
RamiBerm
6fa7bfb6f9 Update HarPage.tsx 2021-05-25 11:02:36 +03:00
RamiBerm
1d6f59df99 Update HarPage.tsx 2021-05-25 10:19:52 +03:00
RamiBerm
22576a35a3 Update socket_server_handlers.go, consts.go, and HarPage.tsx 2021-05-24 17:13:40 +03:00
RamiBerm
5e6b6544de Update controlSocket.go, mizuRunner.go, and 5 more files... 2021-05-24 17:06:30 +03:00
RamiBerm
fbeb7fe9cd Update main.go, socket_server_handlers.go, and 4 more files... 2021-05-24 17:06:03 +03:00
gadotroee
5cbb5a011e
More complex message (preparation for knowing status of tapped items in UI) (#49)
* no message
* no message
2021-05-23 13:11:59 +03:00
nimrod-up9
1ddc7f2f6b
Decode gRPC body (#48)
* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.
2021-05-12 18:42:42 +03:00
lirazyehezkel
d975e76555
Handle nullable vars (#47) 2021-05-12 12:00:35 +03:00
lirazyehezkel
129acae58e
Feature/UI/light theme (#44)
* light theme

* css polish

* unused code

* css

* text shadow

* footer style
2021-05-11 17:35:21 +03:00
lirazyehezkel
c7a20ed9c0
Feature/UI/filters (#32)
* UI filters

* refactor

* Revert "refactor"

This reverts commit 70e7d4b6ac.

* remove recursive func
2021-05-06 17:40:35 +03:00
nimrod-up9
938ca26cd4
Show gRPC as ASCII (#31)
* Moved try-catch up one block.

* Display grpc as ASCII.
2021-05-05 17:23:10 +03:00
lirazyehezkel
377fc79315
Infinite scroll (#28)
* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
2021-05-04 18:23:58 +03:00
Liraz Yehezkel
f23f50373b meta information 2021-04-29 18:01:30 +03:00
Liraz Yehezkel
a8c72ceb69 Live traffic web socket 2021-04-28 16:15:13 +03:00
Liraz Yehezkel
5f71fcfb9f stating with web socket 2021-04-28 13:49:07 +03:00
Liraz Yehezkel
b7ea99a9f5 Clean warnings 2021-04-27 17:14:20 +03:00
Liraz Yehezkel
8447d11f24 css 2021-04-27 15:10:39 +03:00
Liraz Yehezkel
ef5b68a0c5 cleaning 2021-04-27 15:05:01 +03:00
Liraz Yehezkel
0e0b51b42c Traffic viewer 2021-04-27 14:48:22 +03:00
Alex Haiut
f0859b9277 folder structure 2021-04-19 13:53:48 +03:00