Commit Graph

87 Commits

Author SHA1 Message Date
RoyUP9
779257b864
added alert and error when there is a proxy before k8b server (#402) 2021-10-26 16:57:56 +03:00
M. Mert Yıldıran
145e7cda01
Add OAS contract monitoring support (#325)
* Add OAS contract monitoring support

* Pass the contract failure reason to UI

* Fix the issues related to contract validation

* Fix rest of the issues in the UI

* Add documentation related to contract monitoring feature

* Fix a typo in the docs

* Unmarshal to `HTTPRequestResponsePair` only if the OAS validation is enabled

* Fix an issue caused by the merge commit

* Slightly change the logic in the `validateOAS` method

Change the `contractText` value to `No Breaches` or `Breach` and make the text `white-space: nowrap`.

* Retrieve and display the failure reason for both request and response

Also display the content of the contract/OAS file in the UI.

* Display the OAS under `CONTRACT` tab with syntax highlighting

Also fix the styling in the entry feed.

* Remove `EnforcePolicyFileDeprecated` constant

* Log the other errors as well

* Get context from caller instead

* Define a type for the contract status and make its values enum-like

* Remove an unnecessary `if` statement

* Validate OAS in the CLI before passing it to Agent

* Get rid of the `github.com/ghodss/yaml` dependency in `loadOAS` by using `LoadFromData`

* Fix an artifact from the merge conflict
2021-10-19 14:24:22 +03:00
M. Mert Yıldıran
b30b62ef77
Move cli/logger to shared, and refactor all its usages in cli (#349)
* Move `cli/logger` to `shared`, and refactor all its usages in `cli`

* Remove indirect for `op/go-logging` in `shared`
2021-10-14 10:18:01 +03:00
RoyUP9
04c0f8cbcd
tap to workspace (#315) 2021-10-11 15:42:41 +03:00
RoyUP9
9d378ed75b
refactor login (#339) 2021-10-11 11:31:12 +03:00
RoyUP9
8b8c4609ce
renamed upload entries to sync entries (#330) 2021-10-07 18:33:14 +03:00
RoyUP9
14b616a856
Connecting Mizu to the application (#317) 2021-10-07 17:28:28 +03:00
RoyUP9
f1a2ee7fb4
removed enforce policy file deprecated flag (#328) 2021-10-07 11:08:48 +03:00
Igor Gov
fd48cc6d87
Renaming ignored user agents var (#320) 2021-10-06 13:52:30 +03:00
RoyUP9
9c98a4c2b1
Revert "Connecting Mizu to the application (#313)" (#316) 2021-10-06 10:41:23 +03:00
RoyUP9
d2d4ed5aee
Connecting Mizu to the application (#313) 2021-10-05 16:35:16 +03:00
Igor Gov
30fce5d765
Supporting Mizu view from given url (#312)
* Supporting Mizu view from given url
2021-10-05 12:24:50 +03:00
Selton Fiuza
c26eb843e3
[refactor/TRA-3693] type:latency to slo and latency field to response-time (#282)
* type:latency to slo and latency field to response-time

* remove comment from import

* Friendly message on ignored rules and format

* formatting

* change conditional to catch negative values and ignore it

* Fix Bug Alon Reported

* sliceUtils to shared
2021-09-29 11:51:03 -03:00
RoyUP9
f164e54fee
changed test rules config to readonly (#301) 2021-09-23 09:13:17 +03:00
RoyUP9
5dacd41ba9
renamed traffic-validation to traffic-validation-file (#296) 2021-09-22 11:21:43 +03:00
Selton Fiuza
b50eced489
[Refactor/TRA-3692] rename test rules to traffic validation (#281) 2021-09-19 14:47:19 +03:00
gadotroee
073b0b72d3
Remove fetch command (and direction) (#264) 2021-09-06 10:16:04 +03:00
M. Mert Yıldıran
366c1d0c6c
Refactor Mizu, define an extension API and add new protocols: AMQP, Kafka (#224)
* Separate HTTP related code into `extensions/http` as a Go plugin

* Move `extensions` folder into `tap` folder

* Move HTTP files into `tap/extensions/lib` for now

* Replace `orcaman/concurrent-map` with `sync.Map`

* Remove `grpc_assembler.go`

* Remove `github.com/up9inc/mizu/tap/extensions/http/lib`

* Add a build script to automatically build extensions from a known path and load them

* Start to define the extension API

* Implement the `run()` function for the TCP stream

* Add support of defining multiple ports to the extension API

* Set the extension name inside the extension

* Declare the `Dissect` function in the extension API

* Dissect HTTP request from inside the HTTP extension

* Make the distinction of outbound and inbound ports

* Dissect HTTP response from inside the HTTP extension

* Bring back the HTTP request-response pair matcher

* Return a `*api.RequestResponsePair` from the dissection

* Bring back the gRPC-HTTP/2 parser

* Fix the issues in `handleHTTP1ClientStream` and `handleHTTP1ServerStream`

* Call a function pointer to emit dissected data back to the `tap` package

* roee changes -
trying to fix agent to work with the "api" object) - ***still not working***

* small mistake in the conflicts

* Fix the issues that are introduced by the merge conflict

* Add `Emitter` interface to the API and send `OutputChannelItem`(s) to `OutputChannel`

* Fix the `HTTP1` handlers

* Set `ConnectionInfo` in HTTP handlers

* Fix the `Dockerfile` to build the extensions

* remove some unwanted code

* no message

* Re-enable `getStreamProps` function

* Migrate back from `gopacket/tcpassembly` to `gopacket/reassembly`

* Introduce `HTTPPayload` struct and `HTTPPayloader` interface to `MarshalJSON()` all the data structures that are returned by the HTTP protocol

* Read `socketHarOutChannel` instead of `filteredHarChannel`

* Connect `OutputChannelItem` to the last WebSocket means that finally the web UI started to work again

* Add `.env.example` to React app

* Marshal and unmarshal `*http.Request`, `*http.Response` pairs

* Move `loadExtensions` into `main.go` and map extensions into `extensionsMap`

* Add `Summarize()` method to the `Dissector` interface

* Add `Analyze` method to the `Dissector` interface and `MizuEntry` to the extension API

* Add `Protocol` struct and make it effect the UI

* Refactor `BaseEntryDetails` struct and display the source and destination ports in the UI

* Display the protocol name inside the details layout

* Add `Represent` method to the `Dissector` interface and manipulate the UI through this method

* Make the protocol color affect the details layout color and write protocol abbreviation vertically

* Remove everything HTTP related from the `tap` package and make the extension system fully functional

* Fix the TypeScript warnings

* Bring in the files related AMQP into `amqp` directory

* Add `--nodefrag` flag to the tapper and bring in the main AMQP code

* Implement the AMQP `BasicPublish` and fix some issues in the UI when the response payload is missing

* Implement `representBasicPublish` method

* Fix several minor issues

* Implement the AMQP `BasicDeliver`

* Implement the AMQP `QueueDeclare`

* Implement the AMQP `ExchangeDeclare`

* Implement the AMQP `ConnectionStart`

* Implement the AMQP `ConnectionClose`

* Implement the AMQP `QueueBind`

* Implement the AMQP `BasicConsume`

* Fix an issue in `ConnectionStart`

* Fix a linter error

* Bring in the files related Kafka into `kafka` directory

* Fix the build errors in Kafka Go files

* Implement `Dissect` method of Kafka and adapt request-response pair matcher to asynchronous client-server stream

* Do the "Is reversed?" checked inside `getStreamProps` and fix an issue in Kafka `Dissect` method

* Implement `Analyze`, `Summarize` methods of Kafka

* Implement the representations for Kafka `Metadata`, `RequestHeader` and `ResponseHeader`

* Refactor the AMQP and Kafka implementations to create the summary string only inside the `Analyze` method

* Implement the representations for Kafka `ApiVersions`

* Implement the representations for Kafka `Produce`

* Implement the representations for Kafka `Fetch`

* Implement the representations for Kafka `ListOffsets`, `CreateTopics` and `DeleteTopics`

* Fix the encoding of AMQP `BasicPublish` and `BasicDeliver` body

* Remove the unnecessary logging

* Remove more logging

* Introduce `Version` field to `Protocol` struct for dynamically switching the HTTP protocol to HTTP/2

* Fix the issues in analysis and representation of HTTP/2 (gRPC) protocol

* Fix the issues in summary section of details layout for HTTP/2 (gRPC) protocol

* Fix the read errors that freezes the sniffer in HTTP and Kafka

* Fix the issues in HTTP POST data

* Fix one more issue in HTTP POST data

* Fix an infinite loop in Kafka

* Fix another freezing issue in Kafka

* Revert "UI Infra - Support multiple entry types + refactoring (#211)"

This reverts commit f74a52d4dc.

* Fix more issues that are introduced by the merge

* Fix the status code in the summary section

* adding the cleaner again (why we removed it?).
add TODO: on the extension loop .

* fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists

* fix GetEntrties ("/entries" endpoint) - working with "tapApi.BaseEntryDetail" (moved from shared)

* Fix an issue in the UI summary section

* Refactor the protocol payload structs

* Fix a log message in the passive tapper

* Adapt `APP_PORTS` environment variable to the new extension system and change its format to `APP_PORTS='{"http": ["8001"]}' `

* Revert "fix dockerfile (remove deleting .env file) - it is found in dockerignore and fails to build if the file not exists"

This reverts commit 4f514ae1f4.

* Bring in the necessary changes from f74a52d4dc

* Open the API server URL in the web browser as soon as Mizu is ready

* Make the TCP reader consists of a single Go routine (instead of two) and try to dissect in both client and server mode by rewinding

* Swap `TcpID` without overwriting it

* Sort extension by priority

* Try to dissect with looping through all the extensions

* fix getStreamProps function.
(it should be passed from CLI as it was before).

* Turn TCP reader back into two Goroutines (client and server)

* typo

* Learn `isClient` from the TCP stream

* Set `viewer` style `overflow: "auto"`

* Fix the memory leaks in AMQP and Kafka dissectors

* Revert some of the changes in be7c65eb6d

* Remove `allExtensionPorts` since it's no longer needed

* Remove `APP_PORTS` since it's no longer needed

* Fix all of the minor issues in the React code

* Check Kafka header size and fail-fast

* Break the dissectors loop upon a successful dissection

* Don't break the dissector loop. Protocols might collide

* Improve the HTTP request-response counter (still not perfect)

* Make the HTTP request-response counter perfect

* Revert "Revert some of the changes in be7c65eb6d3fb657a059707da3ca559937e59739"

This reverts commit 08e7d786d8.

* Bring back `filterItems` and `isHealthCheckByUserAgent` functions

* Remove some development artifacts

* remove unused and commented lines that are not relevant

* Fix the performance in TCP stream factory. Make it create two `tcpReader`(s) per extension

* Change a log to debug

* Make `*api.CounterPair` a field of `tcpReader`

* Set `isTapTarget` to always `true` again since `filterAuthorities` implementation has problems

* Remove a variable that's only used for logging even though not introduced by this branch

* Bring back the `NumberOfRules` field of `ApplicableRules` struct

* Remove the unused `NewEntry` function

* Move `k8sResolver == nil` check to a more appropriate place

* default healthChecksUserAgentHeaders should be empty array (like the default config value)

* remove spam console.log

* Rules button cause app to crash (access the service via incorrect property)

* Ignore all .env* files in docker build.

* Better caching in dockerfile: only copy go.mod before go mod download.

* Check for errors while loading an extension

* Add a comment about why `Protocol` is not a pointer

* Bring back the call to `deleteOlderThan`

* Remove the `nil` check

* Reduce the maximum allowed AMQP message from 128MB to 1MB

* Fix an error that only occurs when a Kafka broker is initiating

* Revert the change in b2abd7b990

* Fix the service name resolution in all protocols

* Remove the `anydirection` flag and fix the issue in `filterAuthorities`

* Pass `sync.Map` by reference to `deleteOlderThan` method

* Fix the packet capture issue in standalone mode that's introduced by the removal of `anydirection`

* Temporarily resolve the memory exhaustion in AMQP

* Fix a nil pointer dereference error

* Fix the CLI build error

* Fix a memory leak that's identified by `pprof`

Co-authored-by: Roee Gadot <roee.gadot@up9.com>
Co-authored-by: Nimrod Gilboa Markevich <nimrod@up9.com>
2021-09-02 14:34:06 +03:00
RoyUP9
17fa163ee3
added proxy logs, added events logs (#254) 2021-09-01 15:30:37 +03:00
RoyUP9
e25e7925b6
fixed version blocking (#251) 2021-08-30 15:11:14 +03:00
RoyUP9
80237c8090
fixed error on invalid config path (#250) 2021-08-30 11:43:44 +03:00
RoyUP9
a9e92b60f5
added custom config path option (#247) 2021-08-26 13:50:41 +03:00
RoyUP9
667f0dc87d
fixed namespace restricted validation (#235) 2021-08-19 11:33:48 +03:00
RoyUP9
7f9fd82c0e
fixed panic when using invalid kube config path (#231) 2021-08-19 10:59:31 +03:00
RoyUP9
bf27e94003
fixed version check, removed duplicate kube config, fix flags warning, fixed log of invalid config (#227) 2021-08-18 18:10:47 +03:00
RoyUP9
db1f4458c5
Introducing acceptance test (#222) 2021-08-18 10:22:45 +03:00
RoyUP9
a427534605
tests refactor (#216) 2021-08-15 12:30:34 +03:00
RoyUP9
f9677dbaa1
added resources to config (#208) 2021-08-12 16:33:32 +03:00
RoyUP9
0afab6c068
added set hierarchy, removed allowed set flags (#205) 2021-08-12 16:01:33 +03:00
Igor Gov
7b73004e85
Cli pkg refactor (2) (#200) 2021-08-11 09:56:03 +03:00
gadotroee
2f33f9229a
TRA-3234 Fetch command (#54)
* preparation to fetch command

* get har as zip from server

* no message

* no message
2021-05-24 19:29:46 +03:00
nimrod-up9
ee63247888
Removed flags without implementation (#38) 2021-05-09 14:09:39 +03:00
nimrod-up9
506e36fbdc
CLI cleanup 2 (#34)
* Renamed dashboard -> GUI/web interface.

* Commented out --quiet, removed unused config variables.

* Quiter output when calling unimplemented subcommands.
2021-05-06 18:30:01 +03:00
nimrod-up9
6ceaa56474
CLI cleanup (#33)
* Moved cli root command to tap subcommand.

* tap subcommand works.

* Added view and fetch placeholders.

* Updated descriptions.

* Fixed indentation.

* Added versio subcommand.

* Removed version flag.

* gofmt.

* Changed pod from flag to arg.

* Commented out "all namespaces" flag.
2021-05-06 17:43:22 +03:00
nimrod-up9
38146a644d
Select node by pod (#18)
* Select node by pod.

* Removed watch pod by regex. Irrelevant for now.

* Changed default image to develop:latest.
2021-04-29 15:46:18 +03:00
RamiBerm
9531f7a14b Update Dockerfile, multi-runner.sh, and 31 more files... 2021-04-28 17:32:06 +03:00
up9-github
baa33427ff Update README.md, root.go, and 4 more files... 2021-04-21 17:35:08 +03:00