Commit Graph

92 Commits

Author SHA1 Message Date
M. Mert Yildiran
8b52a950d5 Merge branch 'develop' into feature/multiarch_build 2022-01-23 21:46:46 +03:00
M. Mert Yildiran
4ddcb7cb92
Add MizuAgentImageRepo constant and use it as default AgentImage value 2022-01-23 19:31:43 +03:00
RoyUP9
569f8ae143
Added post install check (#630) 2022-01-23 16:52:58 +02:00
M. Mert Yildiran
20ffe2a2de
Upgrade Basenine version to v0.4.12 2022-01-23 16:41:29 +03:00
M. Mert Yildiran
4fde6e9dac
Merge branch 'develop' into feature/multiarch_build 2022-01-20 21:28:06 +03:00
gadotroee
6bab381280
Make kratos image configurable (#670) 2022-01-20 13:48:02 +02:00
gadotroee
27dee4e09b
TRA-4193 - Try port forward if proxy is not available (#662) 2022-01-20 11:33:00 +02:00
M. Mert Yildiran
1374a7982d
Fix Dockerfile 2022-01-18 01:52:44 +03:00
M. Mert Yildiran
0f7bd9ea02
Upgrade the Basenine version from v0.4.6 to v0.4.10 2022-01-18 00:32:01 +03:00
M. Mert Yildiran
c39576b559
Switch BasenineImageRepo to Docker Hub 2022-01-17 23:35:02 +03:00
M. Mert Yildiran
e8d7cd0751
Upgrade the Basenine version from v0.3.0 to v0.4.6 2022-01-17 22:44:02 +03:00
RoyUP9
26a9c31d1e
Extracted agent status to consistent volume (#628) 2022-01-12 16:03:50 +02:00
Igor Gov
bfbbc27e62
Adding experimental feature flags (#627) 2022-01-12 09:33:41 +02:00
RoyUP9
b96542a8ed
Refactor to agent status (#622) 2022-01-11 20:01:39 +02:00
M. Mert Yıldıran
80e881fee2
Upgrade Basenine to 0.3.0, do a refactor to enable redact helper and update the cheatsheet (#614)
* Upgrade Basenine version from `0.2.26` to `0.3.0`

* Remove `Summarize` method from `Dissector` interface and refactor the data structures in `tap/api/api.go`

* Rename `MizuEntry` to `Entry` and `BaseEntryDetails` to `BaseEntry`

* Populate `ContractStatus` field as well

* Update the cheatsheet

* Upgrade the Basenine version in the helm chart as well

* Remove a forgoten `console.log` call
2022-01-11 12:51:30 +03:00
M. Mert Yıldıran
9771d689ca
Fix the acceptance tests and a typo in CONFIGURATION.md (#610)
* Enable acceptance tests

* Fix the acceptance tests and a typo in `CONFIGURATION.md`

* Include the container name into the log fetching function

* Duplicate the fix for the logs test

* Revert "Enable acceptance tests"

This reverts commit c10a67c293.
2022-01-09 17:38:41 +03:00
Nimrod Gilboa Markevich
5a044875d3
Rename Istio to service mesh (#605)
- Rename --istio flag to the more general --service-mesh
- Rename internal variables, consts and structures to reflect this conceptual change
- Update the docs accordingly
2022-01-09 13:21:14 +02:00
M. Mert Yıldıran
e3e9681110
Move Basenine binary into a separate container (#603)
* Move Basenine binary into a separate container

* Set `WorkingDir` to `shared.DataDirPath` in the `basenine` container

* Use `consts.go` to set the Basenine image and port

* Bring back the `net-wait-go` usage to prevent startup failures
2022-01-09 11:18:34 +03:00
RoyUP9
b88bdb90f6
Fixed standalone config, small refactor (#589) 2022-01-06 12:04:58 +02:00
RamiBerm
833d08bb40
TRA-4075 integrate kratos user management (#583)
* WIP

* WIP

* WIP

* WIP

* Update App.tsx and Header.tsx

* Update createResources.go, provider.go, and 2 more files...

* WIP

* fix eof newlines

* Fix ts imports, add readiness probe to kratos to prevent mizu being used while kratos isnt ready

* cleaned code

* fix install create namespace

* Update package-lock.json

* Update provider.go

* Update provider.go

* Update provider.go

* Update install_controller.go

* Update kratos.yml

* Update start.sh

* Update provider.go

* Update provider.go

* Update main.go, socket_routes.go, and 8 more files...

* Update App.tsx

* Update installRunner.go

* Update App.tsx
2022-01-05 17:50:27 +02:00
RoyUP9
8e20ca797b
Added endpoint for getting tapped namespaces (#587) 2022-01-05 11:15:42 +02:00
RoyUP9
02b2cbaa03
Added update config route for install mode (#581) 2022-01-04 17:27:14 +02:00
Nimrod Gilboa Markevich
2834ae1e85
Support custom RBAC resources (#572)
Motivation: Allow users to change the default RBAC resources (ServiceAccount, ClusterRole, ClusterRoleBinding, Role and RoleBinding) without having Mizu delete them every run.

Adds app.kubernetes.io/created-by and app.kubernetes.io/managed-by labels to all resources.
The value of app.kubernetes.io/created-by is either mizu-cli or mizu-agent.
The value of app.kubernetes.io/managed-by is mizu.
When Mizu cleans resources (ctrl-c in tap cmd or mizu clean cmd) it removes all RBAC resources that have managed-by=mizu, and only those.

A user may have a ClusterRole named mizu-clusterrole. If it doesn't have the label app.kubernetes.io/managed-by=mizu, then Mizu won't overwrite it and won't delete it.
Other resources (deployments, services etc.) are always removed, regardless of their labels.
2022-01-04 09:48:22 +02:00
RoyUP9
482036182b
Added setting ui mode - enterprise or normal (#575) 2021-12-30 14:58:53 +02:00
RoyUP9
de046c65b0
Mizu install mode (#566) 2021-12-28 15:44:37 +02:00
Igor Gov
b039c2abad
Fix: [EKS] server could not find the requested resourceevents.events.k8s.io (#565) 2021-12-28 13:24:51 +02:00
RoyUP9
15895d5947
Changed sync tappers to start only when sync tappers config is true (#562) 2021-12-27 17:19:00 +02:00
RamiBerm
cfe9e863b7
TRA-4065 support inflight tap target update (#556)
* WIP

* WIP

* Update main.go

* Update main.go and passive_tapper.go

* Update passive_tapper.go

* Update passive_tapper.go

* Update passive_tapper.go

* Update passive_tapper.go
2021-12-27 11:50:34 +02:00
Igor Gov
cba0c682e5
Report pods "isTapped" to FE (#535) 2021-12-19 13:03:53 +02:00
David Levanon
ab029f4394
Send pod info to tapper (#532) 2021-12-16 10:51:03 +02:00
Igor Gov
f266f32fea
Decreased API server boot time (#536) 2021-12-16 10:36:03 +02:00
Igor Gov
90c210452d
API server stores tappers status (#531) 2021-12-15 14:52:49 +02:00
M. Mert Yıldıran
9f1e311689
TRA-4017 Bring back getOldEntries method using fetch API and always start streaming from now (#518)
* Bring back `getOldEntries` method using fetch API

* Determine no more data on top based on `leftOff` value

* Remove `entriesBuffer` state

* Always open WebSocket with some `leftOff` value

* Rename `leftOff` state to `leftOffBottom`

* Don't set the `focusedEntryId` through WebSocket if the WebSocket is closed

* Call `setQueriedCurrent` with addition

* Close WebSocket upon reaching to top

* Open WebSocket upon snapping to bottom

* Close the WebSocket on snap broken event instead

* Set queried current value to zero upon filter submit

* Upgrade `react-scrollable-feed-virtualized` version and use `scrollToIndex` function

* Change the footer text format

* Improve no more data top logic

* Fix `closeWebSocket()` call logic in `onSnapBrokenEvent` and handle `data.meta` being `null` in `getOldEntries`

* Fix the issues around fetching old records

* Clean up `EntriesList.module.sass`

* Decrement initial `leftOffTop` value by `2`

* Fix the order of `incomingEntries` in `getOldEntries`

* Request `leftOffTop - 1` from `fetchEntries`

* Limit the front-end total entries fetched through WebSocket count to `10000`

* Lose the UI performance gain that's provided by #452

* Revert "Fix the selected entry behavior by propagating the `focusedEntryId` through WebSocket (before #452) TRA-3983 (#513)"

This reverts commit 873f252544.

* Fix the issues caused by 09371f141f

* Upgrade Basenine version from `0.2.13` to `0.2.14`

* Upgrade Basenine version from `0.2.14` to `0.2.15`

* Fix the condition of "Fetch old records" button visibility

* Upgrade Basenine version from `0.2.15` to `0.2.16` and fix the UI code related to fetching old records

* Make `newEntries` constant
2021-12-08 15:19:35 +03:00
David Levanon
296e1bb667
Replace privileged with specific CAPABILITIES requests (#514) 2021-12-02 11:41:13 +02:00
M. Mert Yıldıran
873f252544
Fix the selected entry behavior by propagating the focusedEntryId through WebSocket (before #452) TRA-3983 (#513)
* Revert the select entry behavior into its original state RACING! (before #452) [TRA-3983 alternative 3]

* Remove the remaining `forceSelect`(s)

* Add a missing `focusedEntryId` prop

* Fix the race condition

* Propagate the `focusedEntryId` through WebSocket to prevent racing
2021-11-30 15:27:10 +03:00
Nimrod Gilboa Markevich
1c18eb1b84
Use one channel for events instead of three (#495)
Use one channel for events instead of three separate channels by event type
2021-11-23 15:06:27 +02:00
David Levanon
01d6005a7b
minor logging changes (#499)
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2021-11-23 14:21:53 +02:00
M. Mert Yıldıran
02a125bb86
Disable IPv4 defragmentation and support IPv6 (#487)
* Remove the extra negation on `nodefrag` flag's value

* Support IPv4 fragmentation and IPv6 at the same time

* Re-enable `nodefrag` flag
2021-11-22 17:35:17 +03:00
Nimrod Gilboa Markevich
b1ad2efb96
Warn pods not starting (#493)
Print warning event related to mizu k8s resources.
In non-daemon print to CLI. In Daemon print to API-Server logs.
2021-11-22 15:30:10 +02:00
David Levanon
6caa94f08f
Add support to auto discover envoy processes (#459)
* discover envoy pids using cluster ips

* add istio flag to cli + rename mtls flag to istio

* add istio.md to docs

* Fixing typos

* Fix minor typos and grammer in docs

Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
2021-11-21 15:45:07 +02:00
Nimrod Gilboa Markevich
2e75834dd0
Refactor watch pods to allow reusing watch wrapper (#470)
Currently shared/kubernetes/watch.go:FilteredWatch only watches pods.
This PR makes it reusable for other types of resources.
This is done in preparation for watching k8s events.
2021-11-18 11:53:11 +02:00
RamiBerm
b7f7daa05c
TRA-3903 fix daemon mode in permission restricted configs (#473)
* Update tapRunner.go, permissions-all-namespaces-daemon.yaml, and 2 more files...

* Update tapRunner.go

* Update tapRunner.go and permissions-ns-daemon.yaml

* Update tapRunner.go

* Update tapRunner.go

* Update tapRunner.go
2021-11-17 11:14:43 +02:00
RamiBerm
89836d8d75
TRA-3903 better health endpoint for daemon mode (#471)
* Update main.go, status_controller.go, and 2 more files...

* Update status_controller.go and mizuTapperSyncer.go
2021-11-16 15:44:27 +02:00
RoyUP9
763f72a640
remove newline in logs, fixed logs time format (#469) 2021-11-16 12:07:48 +02:00
RoyUP9
3e30815fb4
changes log format to be more readable (#463) 2021-11-16 11:01:40 +02:00
RoyUP9
14a5fe11e7
changed logger debug mode to log level (#456) 2021-11-14 12:21:48 +02:00
M. Mert Yıldıran
d2fe3f6620
Migrate from SQLite to Basenine and introduce a new filtering syntax (#279)
* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically

* Remove `MAX_NUMBER_OF_GOROUTINES` environment variable

* Change the line

* Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`

* Write the client and integrate to the new real-time database

* Refactor the WebSocket implementaiton for `/ws`

* Adapt the UI to the new filtering system

* Fix the rest of the issues in the UI

* Increase the buffer of the scanner

* Implement accessing single records

* Increase the buffer of another scanner

* Populate `Request` and `Response` fields of `MizuEntry`

* Add syntax highlighting for the query

* Add database to `Dockerfile`

* Fix some issues

* Update the `realtime_dbms` Git module commit hash

* Upgrade Gin version and print the query string

* Revert "Upgrade Gin version and print the query string"

This reverts commit aa09f904ee.

* Use WebSocket's itself to query instead of the query string

* Fix some errors related to conversion to HAR

* Fix the issues caused by the latest merge

* Fix the build error

* Fix PR validation GitHub workflow

* Replace the git submodule with latest Basenine version `0.1.0`

Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.

* Move Basenine host and port constants to `shared` module

* Reliably execute and wait for Basenine to become available

* Upgrade Basenine version

* Properly close WebSocket and data channel

* Fix the issues caused by the recent merge commit

* Clean up the TypeScript code

* Update `.gitignore`

* Limit the database size

* Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions

* Run `go mod tidy` on `agent`

* Upgrade `github.com/up9inc/basenine/client/go` version

* Implement a mechanism to update the query using click events in the UI and use it for protocol macros

* Update the query on click to timestamps

* Fix some issues in the WebSocket and channel handling

* Update the query on clicks to status code

* Update the query on clicks to method, path and service

* Update the query on clicks to is outgoing, source and destination ports

* Add an API endpoint to validate the query against syntax errors

* Move the query background color state into `TrafficPage`

* Fix the logic in `setQuery`

* Display a toast message in case of a syntax error in the query

* Remove a call to `fmt.Printf`

* Upgrade Basenine version to `0.1.3`

* Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable

* Have the `path` key in request details, in HTTP

* Rearrange the HTTP headers for the querying

* Do the same thing for `cookies` and `queryString`

* Update the query on click to table elements

Add the selectors for `TABLE` type representations in HTTP extension.

* Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`

* Add the selectors for `TABLE` type representations in AMQP extension

* Add the selectors for `TABLE` type representations in Kafka extension

* Add the selectors for `TABLE` type representations in Redis extension

* Define a struct in `tap/api.go` for the section representation data

* Add the selectors for `BODY` type representations

* Add `request.path` to the HTTP request details

* Change the summary string's field name from `path` to `summary`

* Introduce `queryable` CSS class for queryable UI elements and underline them on hover

* Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values

Upgrade Basenine version to `0.2.0`.

* Verify the sha256sum of Basenine executable inside `Dockerfile`

* Pass the start time to web UI through WebSocket and always show the `EntriesList` footer

* Pipe the `stderr` of Basenine as well

* Fix the layout issues related to `CodeEditor` in the UI

* Use the correct `shasum` command in `Dockerfile`

* Upgrade Basenine version to `0.2.1`

* Limit the height of `CodeEditor` container

* Remove `Paused` enum `ConnectionStatus` in UI

* Fix the issue caused by the recent merge

* Add the filtering guide (cheatsheet)

* Update open cheatsheet button's title

* Update cheatsheet content

* Remove the old SQLite code, adapt the `--analyze` related code to Basenine

* Change the method signature of `NewEntry`

* Change the method signature of `Represent`

* Introduce `HTTPPair` field in `MizuEntry` specific to HTTP

* Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`

Also remove the `getEstimatedEntrySizeBytes` method.

* Remove `gorm.io/gorm` dependency

* Remove unused `sensitiveDataFiltering` folder

* Increase the left margin of open cheatsheet button

* Add `overflow: auto` to the cheatsheet `Modal`

* Fix `GetEntry` method

* Fix the macro for gRPC

* Fix an interface conversion in case of AMQP

* Fix two more interface conversion errors in AMQP

* Make the `syncEntriesImpl` method blocking

* Fix a grammar mistake in the cheatsheet

* Adapt to the changes in the recent merge commit

* Improve the cheatsheet text

* Always display the timestamp in `en-US`

* Upgrade Basenine version to `0.2.2`

* Fix the order of closing Basenine connections and channels

* Don't close the Basenine channels at all

* Upgrade Basenine version to `0.2.3`

* Set the initial filter to `rlimit(100)`

* Make Basenine persistent

* Upgrade Basenine version to `0.2.4`

* Update `debug.Dockerfile`

* Fix a failing test

* Upgrade Basenine version to `0.2.5`

* Revert "Do not show play icon when disconnected (#428)"

This reverts commit 8af2e562f8.

* Upgrade Basenine version to `0.2.6`

* Make all non-informative things informative

* Make `100` a constant

* Use `===` in JavaScript no matter what

* Remove a forgotten `console.log`

* Add a comment and update the `query` in `syncEntriesImpl`

* Don't call `panic` in `GetEntry`

* Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`

* Remove unnecessary `\n` characters in the logs
2021-11-09 19:54:48 +03:00
RamiBerm
4cf3c9c6d3
TRA-3913 support mizu via expose service (#440)
* Update README.md, tapRunner.go, and 4 more files...

* Update testsUtils.go

* Update proxy.go

* Update README.md, testsUtils.go, and 3 more files...

* Update testsUtils.go and provider.go
2021-11-08 11:23:03 +02:00
David Levanon
8a90f02161
Add support of listening to multiple netns (#418)
* multiple netns listen - initial commit

* multiple netns listen - actual work

* remove redundant log line

* map /proc of host to tapper

* changing kubernetes provider again after big conflict

* revert node-sass version back to 5.0.0

* Rename host_source to hostSource

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>

* PR fixes - adding comment + typos + naming conventions

* go fmt + making procfs read only

* setns back to the original value after packet source initialized

Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
2021-11-07 16:00:59 +02:00
RamiBerm
a3ec5d147e
TRA-3842 daemon mode (#427)
* Update config.go, tapConfig.go, and models.go

* WIP

* Update go.sum

* Update tapRunner.go

* Update tap.go

* WIP

* WIP

* Update Dockerfile, main.go, and 2 more files...

* WIP

* Update utils.go, tapClusterResourceManagement.go, and utils.go

* Merge branch 'develop'

* Update metadata_controller.go, utils.go, and 2 more files...

* Update main.go, utils.go, and tapRunner.go

* Update tapRunner.go

* Update config.go, config.go, and models.go

* Update main.go, main.go, and stats_provider_test.go

* Update provider.go

* bug fixes

* Update main.go, metadata_controller.go, and 13 more files...

* Update metadata_controller.go, status_controller.go, and 4 more files...

* Update main.go, config.go, and 3 more files...

* Update tapRunner.go

* Update config.go, stats_provider_test.go, and consts.go
2021-11-04 11:46:45 +02:00