Compare commits

..

22 Commits

Author SHA1 Message Date
RoyUP9
c3995009ee Hotfix - ignore not allowed set flags (#192)
#patch
2021-08-10 14:21:16 +03:00
RoyUP9
6e9fe2986e removed duplicate har page header (#187) 2021-08-09 13:31:53 +03:00
RoyUP9
603240fedb temp fix - ignore agent image in config command (#185) 2021-08-09 11:55:45 +03:00
Igor Gov
e61871a68e Merge pull request #182 from up9inc/develop
Release 2021-08-08
2021-08-08 14:50:30 +03:00
Nimrod Gilboa Markevich
02e02718d2 Fixed fetch not using from/to options (#179) 2021-08-08 14:36:24 +03:00
Alex Haiut
1a0517f46b TRA-3547 separated permissions section into separate file (#181) 2021-08-08 14:21:33 +03:00
Alex Haiut
efbb432df9 TRA-3547 separated permissions section into separate file (#181) 2021-08-08 14:19:49 +03:00
RoyUP9
dfea8884d4 Adding 'configuration' section in readme (#180) 2021-08-08 14:05:15 +03:00
Igor Gov
d34dacbbe2 View command - moving version check after proxy creation (#177) 2021-08-08 12:26:58 +03:00
Nimrod Gilboa Markevich
0595df8b87 Adds Namespace-Restricted Mode to README. (#178) 2021-08-08 12:23:11 +03:00
nimrod-up9
379af59f07 Merge pull request #121 from up9inc/develop
Missing request body (#120)
2021-07-19 13:53:49 +03:00
gadotroee
ef9afe31a4 Merge pull request #119 from up9inc/develop
Mizu release
2021-07-18 16:54:31 +03:00
gadotroee
dca636b0fd Merge pull request #94 from up9inc/develop
Mizu release
2021-07-06 21:05:40 +03:00
Roee Gadot
9b72cc7aa6 Merge branch 'develop' into main
# Conflicts:
#	README.md
#	api/main.go
#	api/pkg/api/main.go
#	api/pkg/models/models.go
#	api/pkg/resolver/resolver.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/tapRunner.go
#	tap/http_matcher.go
#	tap/http_reader.go
#	tap/tcp_stream_factory.go
2021-06-29 11:16:47 +03:00
Alex Haiut
d3c023b3ba mizu release 2021-06-21 (#79)
* Show pod name and namespace (#61)

* WIP

* Update main.go, consts.go, and 2 more files...

* Update messageSensitiveDataCleaner.go

* Update consts.go and messageSensitiveDataCleaner.go

* Update messageSensitiveDataCleaner.go

* Update main.go, consts.go, and 3 more files...

* WIP

* Update main.go, messageSensitiveDataCleaner.go, and 6 more files...

* Update main.go, messageSensitiveDataCleaner.go, and 3 more files...

* Update consts.go, messageSensitiveDataCleaner.go, and tap.go

* Update provider.go

* Update serializableRegexp.go

* Update tap.go

* TRA-3234 fetch with _source + no hard limit (#64)

* remove the HARD limit of 5000

* TRA-3299 Reduce footprint and Add Tolerances(#65)

* Use lib const for DNSClusterFirstWithHostNet.

* Whitespace.

* Break lines.

* Added affinity to pod names.

* Added tolerations to NoExecute and NoSchedule taints.

* Implementation of Mizu view command

* .

* .

* Update main.go and messageSensitiveDataCleaner.go

* Update main.go

* String and not pointers (#68)

* TRA-3318 - Cookies not null and fix har file names  (#69)

* no message

* TRA-3212 Passive-Tapper and Mizu share code (#70)

* Use log in tap package instead of fmt.

* Moved api/pkg/tap to root.

* Added go.mod and go.sum for tap.

* Added replace for shared.

* api uses tap module instead of tap package.

* Removed dependency of tap in shared by moving env var out of tap.

* Fixed compilation bugs.

* Fixed: Forgot to export struct field HostMode.

* Removed unused flag.

* Close har output channel when done.

* Moved websocket out of mizu and into passive-tapper.

* Send connection details over har output channel.

* Fixed compilation errors.

* Removed unused info from request response cache.

* Renamed connection -> connectionID.

* Fixed rename bug.

* Export setters and getters for filter ips and ports.

* Added tap dependency to Dockerfile.

* Uncomment error messages.

* Renamed `filterIpAddresses` -> `filterAuthorities`.

* Renamed ConnectionID -> ConnectionInfo.

* Fixed: Missed one replace.

* TRA-3342 Mizu/tap dump to har directory fails on Linux (#71)

* Instead of saving incomplete temp har files in a temp dir, save them in the output dir with a *.har.tmp suffix.

* API only loads har from *.har files (by extension).

* Add export entries endpoint for better up9 connect funcionality  (#72)

* no message
* no message
* no message

* Filter 'cookie' header

* Release action  (#73)

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* trying new approach

* no message

* yaml error

* no message

* no message

* no message

* missing )

* no message

* no message

* remove main.yml and fix branches

* Create tag-temp.yaml

* Update tag-temp.yaml

* Update tag-temp.yaml

* no message

* no message

* no message

* no message

* no message

* no message

* no message

* #minor

* no message

* no message

* added checksum calc to CLI makefile

* fixed build error - created bin directory upfront

* using markdown for release text

* use separate checksum files

* fixed release readme

* #minor

* readme updated

Co-authored-by: Alex Haiut <alex@up9.com>

* TRA-3360 Fix: Mizu ignores -n namespace flag and records traffic from all pods (#75)

Do not tap pods in namespaces which were not requested.

* added apple/m1 binary, updated readme (#77)

Co-authored-by: Alex Haiut <alex@up9.com>

* Update README.md (#78)

Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: gadotroee <55343099+gadotroee@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: Igor Gov <igor.govorov1@gmail.com>
Co-authored-by: Alex Haiut <alex@up9.com>
2021-06-21 15:17:31 +03:00
Roee Gadot
5f2a4deb19 remove file 2021-05-26 18:08:37 +03:00
Roee Gadot
91f290987e Merge branch 'develop' into main
# Conflicts:
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/kubernetes/provider.go
#	cli/mizu/consts.go
#	cli/mizu/mizuRunner.go
#	debug.Dockerfile
#	ui/src/components/HarPage.tsx
2021-05-26 17:58:17 +03:00
gadotroee
2f3215b71a Fix mizu image parameter (#53) 2021-05-23 13:34:32 +03:00
Alex Haiut
2e87a01346 end of week - develop to master (#50)
* Provide cli version as git hash from makefile

* Update Makefile, version.go, and 3 more files...

* Update mizuRunner.go

* Update README.md, resolver.go, and 2 more files...

* Update provider.go

* Feature/UI/light theme (#44)

* light theme

* css polish

* unused code

* css

* text shadow

* footer style

* Update mizuRunner.go

* Handle nullable vars (#47)

* Decode gRPC body (#48)

* Decode grpc.

* Better variable names.

* Added protobuf-decoder dependency.

* Updated protobuf-decoder's version.

Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
2021-05-13 20:29:31 +03:00
gadotroee
453003bf14 remove leftovers (#43) 2021-05-10 17:35:59 +03:00
Roee Gadot
80ca377668 Merge branch 'develop' into main
# Conflicts:
#	Dockerfile
#	Makefile
#	api/go.mod
#	api/go.sum
#	api/main.go
#	api/pkg/controllers/entries_controller.go
#	api/pkg/inserter/main.go
#	api/pkg/models/models.go
#	api/pkg/tap/grpc_assembler.go
#	api/pkg/tap/har_writer.go
#	api/pkg/tap/http_matcher.go
#	api/pkg/tap/http_reader.go
#	api/pkg/tap/passive_tapper.go
#	api/pkg/utils/utils.go
#	cli/Makefile
#	cli/cmd/tap.go
#	cli/cmd/version.go
#	cli/config/config.go
#	cli/kubernetes/provider.go
#	cli/mizu/mizuRunner.go
2021-05-10 17:27:32 +03:00
gadotroee
d21297bc9c 0.9 (#37)
* Update .gitignore

* WIP

* WIP

* Update README.md, root.go, and 4 more files...

* Update README.md

* Update README.md

* Update root.go

* Update provider.go

* Update provider.go

* Update root.go, go.mod, and go.sum

* Update mizu.go

* Update go.sum and provider.go

* Update portForward.go, watch.go, and mizu.go

* Update README.md

* Update watch.go

* Update mizu.go

* Update mizu.go

* no message

* no message

* remove unused things and use external for object id (instead of copy)

* no message

* Update mizu.go

* Update go.mod, go.sum, and 2 more files...

* no message

* Update README.md, go.mod, and resolver.go

* Update README.md

* Update go.mod

* Update loader.go

* some refactor

* Update loader.go

* no message

* status to statusCode

* return data directly

* Traffic viewer

* cleaning

* css

* no message

* Clean warnings

* Makefile - first draft

* Update Makefile

* Update Makefile

* Update Makefile, README.md, and 4 more files...

* Add api build and clean to makefile (files restructure) (#9)

* no message
* add clean api command

* no message

* stating with web socket

* Add tap as a separate executable (#10)

* Added tap.

* Ignore build directories.

* Added tapper build to Makefile.

* Improvements  (#12)

* no message

* no message

* Feature/makefile (#11)

* minor fixes

* makefile fixes - docker build

* minor fix in Makefile
Co-authored-by: Alex Haiut <alex@up9.com>

* Update Dockerfile, multi-runner.sh, and 31 more files...

* Update multi-runner.sh

* no message

* Update .dockerignore, Dockerfile, and 30 more files...

* Update cleaner.go, grpc_assembler.go, and 2 more files...

* start the pod with host network and privileged

* fix multi runner passive tapper command

* add HOST_MODE env var

* do not return true in the should tap function

* remove line in the end

* default value in api is input
fix description and pass the parameter in the multi runner script

* missing flag.parse

* no message

* fix image

* Create main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Update main.yml

* Small fixes - permission + har writing exception (#17)

* Select node by pod (#18)

* Select node by pod.

* Removed watch pod by regex. Irrelevant for now.

* Changed default image to develop:latest.

* Features/clifix (#19)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

Co-authored-by: Alex Haiut <alex@up9.com>

* meta information

* Only record traffic of the requested pod. Filtered by pod IP. (#21)

* fixed readme and reduced batch size to 5 (#22)

Co-authored-by: Alex Haiut <alex@up9.com>

* API and TAP in single process  (#24)

* no message
* no message

* CLI make --pod required flag and faster api image build (#25)

* makefile fixes - docker build

* readme update, CLI usage fix

* added chmod

* typo

* run example incorreect in makefile

* no message

* no message

* no message

Co-authored-by: Alex Haiut <alex@up9.com>

* Reduce delay between tap and UI - Skip dump to file (#26)

* Pass HARs between tap and api via channel.

* Fixed make docker commad.

* Various fixes.

* Added .DS_Store to .gitignore.

* Parse flags in Mizu main instead of in tap_output.go.

* Use channel to pass HAR by default instead of files.

* Infinite scroll (#28)

* no message

* infinite scroll + new ws implementation

* no message

* scrolling top

* fetch button

* more Backend changes

* fix go mod and sum

* mire fixes against develop

* unused code

* small ui refactor

Co-authored-by: Roee Gadot <roee.gadot@up9.com>

* Fix gRPC crash, display gRPC as base64, display gRPC URL and status code (#27)

* Added Method (POST) and URL (emtpy) to gRPC requests.

* Removed quickfix that skips writing HTTP/2 to HAR.

* Use HTTP/2 body to fill out http.Request and htt.Response.

* Make sure that in HARs request.postData.mimeType and response.content.mimeType are application/grpc in case of grpc.

* Comment.

* Add URL and status code for gRPC.

* Don't assume http scheme.

* Use http.Header.Set instead of manually acccessing the underlaying map.

* General stats api fix  (#29)

* refactor and validation

* Show gRPC as ASCII (#31)

* Moved try-catch up one block.

* Display grpc as ASCII.

* Better code in entries fetch endpoint (#30)

* no message
* no message

* Feature/UI/filters (#32)

* UI filters

* refactor

* Revert "refactor"

This reverts commit 70e7d4b6ac.

* remove recursive func

* CLI cleanup (#33)

* Moved cli root command to tap subcommand.

* tap subcommand works.

* Added view and fetch placeholders.

* Updated descriptions.

* Fixed indentation.

* Added versio subcommand.

* Removed version flag.

* gofmt.

* Changed pod from flag to arg.

* Commented out "all namespaces" flag.

* CLI cleanup 2 (#34)

* Renamed dashboard -> GUI/web interface.

* Commented out --quiet, removed unused config variables.

* Quiter output when calling unimplemented subcommands.

* Leftovers from PR #30 (#36)

Co-authored-by: up9-github <info@up9.com>
Co-authored-by: RamiBerm <54766858+RamiBerm@users.noreply.github.com>
Co-authored-by: Liraz Yehezkel <lirazy@up9.com>
Co-authored-by: Alex Haiut <alex@testr.io>
Co-authored-by: lirazyehezkel <61656597+lirazyehezkel@users.noreply.github.com>
Co-authored-by: Alex Haiut <alex@up9.com>
Co-authored-by: nimrod-up9 <59927337+nimrod-up9@users.noreply.github.com>
Co-authored-by: RamiBerm <rami.berman@up9.com>
Co-authored-by: Alex Haiut <alex.haiut@gmail.com>
2021-05-09 11:45:39 +03:00
16 changed files with 447 additions and 385 deletions

328
PERMISSIONS.md Normal file
View File

@@ -0,0 +1,328 @@
![Mizu: The API Traffic Viewer for Kubernetes](assets/mizu-logo.svg)
# Kubernetes permissions for MIZU
This document describes in details all permissions required for full and correct operation of Mizu
We broke down this list into few categories:
- Required - what is needed for `mizu` to run properly on your k8s cluster
- Optional - permissions needed for proper name resolving for service & pod IPs
- addition required for policy validation
# Required permissions
Mizu needs following permissions on your Kubernetes cluster to run properly
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- create
- patch
- delete
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
```
## Permissions required for service / pod name resolving (opt)
Optionally, for proper resolving of IP addresses to Kubernetes service name, Mizu needs below permissions:
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- create
- patch
- delete
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterroles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterrolebindings
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
verbs:
- get
- create
- delete
- apiGroups:
- apps
- extensions
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- apps
- extensions
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
- apps
- extensions
resources:
- endpoints
verbs:
- get
- list
- watch
```
## Permissions for Policy rules validation feature (opt)
Optionally, in order to use the policy rules validation feature, Mizu requires the following additional permissions:
```yaml
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- create
- delete
```
- - -
## Namespace-Restricted mode
Alternatively, in order to restrict Mizu to one namespace only (by setting `agent.namespace` in the config file), Mizu needs the following permissions in that namespace:
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- create
- patch
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
```
### Name resolving in Namespace-Restricted mode (opt)
To restrict Mizu to one namespace while also resolving IPs, Mizu needs the following permissions in that namespace:
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- create
- patch
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
verbs:
- get
- create
- delete
- apiGroups:
- apps
- extensions
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- apps
- extensions
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
- apps
- extensions
resources:
- endpoints
verbs:
- get
- list
- watch
```

365
README.md
View File

@@ -14,9 +14,9 @@ A simple-yet-powerful API traffic viewer for Kubernetes to help you troubleshoot
## Download
Download `mizu` for your platform and operating system
Download Mizu for your platform and operating system
### Latest stable release
### Latest Stable Release
* for MacOS - Intel
```
@@ -34,322 +34,20 @@ https://github.com/up9inc/mizu/releases/latest/download/mizu_linux_amd64 \
SHA256 checksums are available on the [Releases](https://github.com/up9inc/mizu/releases) page.
### Development (unstable) build
### Development (unstable) Build
Pick one from the [Releases](https://github.com/up9inc/mizu/releases) page.
## Prerequisites
1. Set `KUBECONFIG` environment variable to your kubernetes configuration. If this is not set, mizu assumes that configuration is at `${HOME}/.kube/config`
2. mizu needs following permissions on your kubernetes cluster to run
1. Set `KUBECONFIG` environment variable to your Kubernetes configuration. If this is not set, Mizu assumes that configuration is at `${HOME}/.kube/config`
2. `mizu` assumes user running the command has permissions to create resources (such as pods, services, namespaces) on your Kubernetes cluster (no worries - `mizu` resources are cleaned up upon termination)
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- create
- patch
- delete
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
```
For detailed list of k8s permissions see [PERMISSIONS](PERMISSIONS.md) document
3. Optionally, for resolving traffic IP to kubernetes service name, mizu needs below permissions
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- create
- patch
- delete
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterroles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterrolebindings
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
verbs:
- get
- create
- delete
- apiGroups:
- apps
- extensions
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- apps
- extensions
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
- apps
- extensions
resources:
- endpoints
verbs:
- get
- list
- watch
```
4. Optionally, in order to use the policy rules validation feature, mizu requires the following additional permissions:
```yaml
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- create
- delete
```
5. Alternatively, in order to restrict mizu to one namespace only (by setting `agent.namespace` in the config file), mizu needs the following permissions in that namespace:
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- create
- patch
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
```
6. To restrict mizu to one namespace while also resolving IPs, mizu needs the following permissions in that namespace:
```yaml
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- apps
resources:
- daemonsets
verbs:
- get
- create
- patch
- delete
- apiGroups:
- ""
resources:
- services/proxy
verbs:
- get
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
verbs:
- get
- create
- delete
- apiGroups:
- apps
- extensions
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- apps
- extensions
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
- apps
- extensions
resources:
- endpoints
verbs:
- get
- list
- watch
```
See `examples/roles` for example `clusterroles`.
## How to run
## How to Run
1. Find pods you'd like to tap to in your Kubernetes cluster
2. Run `mizu tap PODNAME` or `mizu tap REGEX`
2. Run `mizu tap` or `mizu tap PODNAME`
3. Open browser on `http://localhost:8899/mizu` **or** as instructed in the CLI ..
4. Watch the API traffic flowing ..
5. Type ^C to stop
@@ -358,6 +56,23 @@ See `examples/roles` for example `clusterroles`.
Run `mizu help` for usage options
To tap all pods in current namespace -
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
carts-66c77f5fbb-fq65r 2/2 Running 0 20m
catalogue-5f4cb7cf5-7zrmn 2/2 Running 0 20m
front-end-649fc5fd6-kqbtn 2/2 Running 0 20m
..
$ mizu tap
+carts-66c77f5fbb-fq65r
+catalogue-5f4cb7cf5-7zrmn
+front-end-649fc5fd6-kqbtn
Web interface is now available at http://localhost:8899
^C
```
To tap specific pod -
```
@@ -388,3 +103,33 @@ To tap multiple pods using regex -
^C
```
## Configuration
Mizu can work with config file which should be stored in ${HOME}/.mizu/config.yaml (macOS: ~/.mizu/config.yaml) <br />
In case no config file found, defaults will be used. <br />
In case of partial configuration defined, all other fields will be used with defaults. <br />
You can always override the defaults or config file with CLI flags.
To get the default config params run `mizu config` <br />
To generate a new config file with default values use `mizu config -r`
Mizu has several undocumented flags which can be set by using --set flag (e.g., `mizu tap --set dump-logs=true`)
* **mizu-resources-namespace**: Type - String, See [Namespace-Restricted Mode](#namespace-restricted-mode)
* **telemetry**: Type - Boolean, Reports telemetry
* **dump-logs**: Type - Boolean, At the end of the execution it creates a zip file with logs (in .mizu folder)
* **kube-config-path**: Type - String, Setting the path to kube config (which isn't in standard path)
## Advanced Usage
### Namespace-Restricted Mode
Some users have permission to only manage resources in one particular namespace assigned to them.
By default `mizu tap` creates a new namespace `mizu` for all of its Kubernetes resources. In order to instead install
Mizu in an existing namespace, set the `mizu-resources-namespace` config option.
If `mizu-resources-namespace` is set to a value other than the default `mizu`, Mizu will operate in a
Namespace-Restricted mode. It will only tap pods in `mizu-resources-namespace`. This way Mizu only requires permissions
to the namespace set by `mizu-resources-namespace`. The user must set the tapped namespace to the same namespace by
using the `--namespace` flag or by setting `tap.namespaces` in the config file.
Setting `mizu-resources-namespace=mizu` resets Mizu to its default behavior.

View File

@@ -57,7 +57,7 @@ func GetEntries(c *gin.Context) {
}
func GetHARs(c *gin.Context) {
entriesFilter := &models.HarFetchRequestBody{}
entriesFilter := &models.HarFetchRequestQuery{}
order := database.OrderDesc
if err := c.BindQuery(entriesFilter); err != nil {
c.JSON(http.StatusBadRequest, err)
@@ -146,12 +146,12 @@ func GetHARs(c *gin.Context) {
func UploadEntries(c *gin.Context) {
rlog.Infof("Upload entries - started\n")
uploadRequestBody := &models.UploadEntriesRequestBody{}
if err := c.BindQuery(uploadRequestBody); err != nil {
uploadParams := &models.UploadEntriesRequestQuery{}
if err := c.BindQuery(uploadParams); err != nil {
c.JSON(http.StatusBadRequest, err)
return
}
if err := validation.Validate(uploadRequestBody); err != nil {
if err := validation.Validate(uploadParams); err != nil {
c.JSON(http.StatusBadRequest, err)
return
}
@@ -160,19 +160,19 @@ func UploadEntries(c *gin.Context) {
return
}
rlog.Infof("Upload entries - creating token. dest %s\n", uploadRequestBody.Dest)
token, err := up9.CreateAnonymousToken(uploadRequestBody.Dest)
rlog.Infof("Upload entries - creating token. dest %s\n", uploadParams.Dest)
token, err := up9.CreateAnonymousToken(uploadParams.Dest)
if err != nil {
c.String(http.StatusServiceUnavailable, "Cannot analyze, mizu is already analyzing")
return
}
rlog.Infof("Upload entries - uploading. token: %s model: %s\n", token.Token, token.Model)
go up9.UploadEntriesImpl(token.Token, token.Model, uploadRequestBody.Dest, uploadRequestBody.SleepIntervalSec)
go up9.UploadEntriesImpl(token.Token, token.Model, uploadParams.Dest, uploadParams.SleepIntervalSec)
c.String(http.StatusOK, "OK")
}
func GetFullEntries(c *gin.Context) {
entriesFilter := &models.HarFetchRequestBody{}
entriesFilter := &models.HarFetchRequestQuery{}
if err := c.BindQuery(entriesFilter); err != nil {
c.JSON(http.StatusBadRequest, err)
}

View File

@@ -119,19 +119,19 @@ func (fedex *FullEntryDetailsExtra) UnmarshalData(entry *MizuEntry) error {
}
type EntriesFilter struct {
Limit int `query:"limit" validate:"required,min=1,max=200"`
Operator string `query:"operator" validate:"required,oneof='lt' 'gt'"`
Timestamp int64 `query:"timestamp" validate:"required,min=1"`
Limit int `form:"limit" validate:"required,min=1,max=200"`
Operator string `form:"operator" validate:"required,oneof='lt' 'gt'"`
Timestamp int64 `form:"timestamp" validate:"required,min=1"`
}
type UploadEntriesRequestBody struct {
type UploadEntriesRequestQuery struct {
Dest string `form:"dest"`
SleepIntervalSec int `form:"interval"`
}
type HarFetchRequestBody struct {
From int64 `query:"from"`
To int64 `query:"to"`
type HarFetchRequestQuery struct {
From int64 `form:"from"`
To int64 `form:"to"`
}
type WebSocketEntryMessage struct {

View File

@@ -1,26 +0,0 @@
# mizu CLI
## Usage
`./mizu {pod_name_regex}`
### Optional Flags
| flag | default | purpose |
|----------------------|------------------|--------------------------------------------------------------------------------------------------------------|
| `--no-gui` | `false` | Don't host the web interface (not applicable at the moment) |
| `--gui-port` | `8899` | local port that web interface will be forwarded to |
| `--namespace` | | use namespace different than the one found in kubeconfig |
| `--kubeconfig` | | Path to custom kubeconfig file |
There are some extra flags defined in code that will show up in `./mizu --help`, these are non functional stubs for now
## Installation
Make sure your go version is at least 1.11
1. cd to `mizu/cli`
2. Run `go mod download` (may take a moment)
3. Run `go build mizu.go`
Alternatively, you can build+run directly using `go run mizu.go {pod_name_regex}`
## Known issues
* mid-flight port forwarding failures are not detected and no indication will be shown when this occurs

View File

@@ -12,7 +12,7 @@ var fetchCmd = &cobra.Command{
Short: "Download recorded traffic to files",
RunE: func(cmd *cobra.Command, args []string) error {
go mizu.ReportRun("fetch", mizu.Config.Fetch)
if isCompatible, err := mizu.CheckVersionCompatibility(mizu.Config.Fetch.MizuPort); err != nil {
if isCompatible, err := mizu.CheckVersionCompatibility(mizu.Config.Fetch.GuiPort); err != nil {
return err
} else if !isCompatible {
return nil
@@ -31,5 +31,5 @@ func init() {
fetchCmd.Flags().StringP(configStructs.DirectoryFetchName, "d", defaultFetchConfig.Directory, "Provide a custom directory for fetched entries")
fetchCmd.Flags().Int(configStructs.FromTimestampFetchName, defaultFetchConfig.FromTimestamp, "Custom start timestamp for fetched entries")
fetchCmd.Flags().Int(configStructs.ToTimestampFetchName, defaultFetchConfig.ToTimestamp, "Custom end timestamp fetched entries")
fetchCmd.Flags().Uint16P(configStructs.MizuPortFetchName, "p", defaultFetchConfig.MizuPort, "Custom port for mizu")
fetchCmd.Flags().Uint16P(configStructs.GuiPortFetchName, "p", defaultFetchConfig.GuiPort, "Provide a custom port for the web interface webserver")
}

View File

@@ -16,7 +16,7 @@ import (
)
func RunMizuFetch() {
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Fetch.MizuPort)
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Fetch.GuiPort)
resp, err := http.Get(fmt.Sprintf("http://%s/api/har?from=%v&to=%v", mizuProxiedUrl, mizu.Config.Fetch.FromTimestamp, mizu.Config.Fetch.ToTimestamp))
if err != nil {
log.Fatal(err)

View File

@@ -64,9 +64,9 @@ func init() {
tapCmd.Flags().Bool(configStructs.AnalysisTapName, defaultTapConfig.Analysis, "Uploads traffic to UP9 for further analysis (Beta)")
tapCmd.Flags().BoolP(configStructs.AllNamespacesTapName, "A", defaultTapConfig.AllNamespaces, "Tap all namespaces")
tapCmd.Flags().StringArrayP(configStructs.PlainTextFilterRegexesTapName, "r", defaultTapConfig.PlainTextFilterRegexes, "List of regex expressions that are used to filter matching values from text/plain http bodies")
tapCmd.Flags().Bool(configStructs.HideHealthChecksTapName, defaultTapConfig.HideHealthChecks, "hides requests with kube-probe or prometheus user-agent headers")
tapCmd.Flags().Bool(configStructs.HideHealthChecksTapName, defaultTapConfig.HideHealthChecks, "Hides requests with kube-probe or prometheus user-agent headers")
tapCmd.Flags().Bool(configStructs.DisableRedactionTapName, defaultTapConfig.DisableRedaction, "Disables redaction of potentially sensitive request/response headers and body values")
tapCmd.Flags().String(configStructs.HumanMaxEntriesDBSizeTapName, defaultTapConfig.HumanMaxEntriesDBSize, "override the default max entries db size of 200mb")
tapCmd.Flags().String(configStructs.HumanMaxEntriesDBSizeTapName, defaultTapConfig.HumanMaxEntriesDBSize, "Override the default max entries db size")
tapCmd.Flags().String(configStructs.DirectionTapName, defaultTapConfig.Direction, "Record traffic that goes in this direction (relative to the tapped pod): in/any")
tapCmd.Flags().Bool(configStructs.DryRunTapName, defaultTapConfig.DryRun, "Preview of all pods matching the regex, without tapping them")
tapCmd.Flags().String(configStructs.EnforcePolicyFile, defaultTapConfig.EnforcePolicyFile, "Yaml file with policy rules")

View File

@@ -329,7 +329,7 @@ func waitUntilNamespaceDeleted(ctx context.Context, cancel context.CancelFunc, k
}
func reportTappedPods() {
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Fetch.MizuPort)
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Tap.GuiPort)
tappedPodsUrl := fmt.Sprintf("http://%s/status/tappedPods", mizuProxiedUrl)
podInfos := make([]shared.PodInfo, 0)
@@ -498,17 +498,8 @@ func createProxyToApiServerPod(ctx context.Context, kubernetesProvider *kubernet
mizu.Log.Debugf("Watching API Server pod loop, modified: %v", modifiedPod.Status.Phase)
if modifiedPod.Status.Phase == core.PodRunning && !isPodReady {
isPodReady = true
go func() {
err := kubernetes.StartProxy(kubernetesProvider, mizu.Config.Tap.GuiPort, mizu.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
mizu.Log.Errorf(uiUtils.Error, fmt.Sprintf("Error occured while running k8s proxy %v\n"+
"Try setting different port by using --%s", errormessage.FormatError(err), configStructs.GuiPortTapName))
cancel()
}
}()
mizuProxiedUrl := kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Tap.GuiPort)
mizu.Log.Infof("Mizu is available at http://%s\n", mizuProxiedUrl)
go startProxyReportErrorIfAny(kubernetesProvider, cancel)
mizu.Log.Infof("Mizu is available at http://%s\n", kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.Tap.GuiPort))
time.Sleep(time.Second * 5) // Waiting to be sure the proxy is ready
requestForAnalysis()
reportTappedPods()
@@ -525,6 +516,15 @@ func createProxyToApiServerPod(ctx context.Context, kubernetesProvider *kubernet
}
}
func startProxyReportErrorIfAny(kubernetesProvider *kubernetes.Provider, cancel context.CancelFunc) {
err := kubernetes.StartProxy(kubernetesProvider, mizu.Config.Tap.GuiPort, mizu.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
mizu.Log.Errorf(uiUtils.Error, fmt.Sprintf("Error occured while running k8s proxy %v\n"+
"Try setting different port by using --%s", errormessage.FormatError(err), configStructs.GuiPortTapName))
cancel()
}
}
func requestForAnalysis() {
if !mizu.Config.Tap.Analysis {
return

View File

@@ -12,11 +12,6 @@ var viewCmd = &cobra.Command{
Short: "Open GUI in browser",
RunE: func(cmd *cobra.Command, args []string) error {
go mizu.ReportRun("view", mizu.Config.View)
if isCompatible, err := mizu.CheckVersionCompatibility(mizu.Config.View.GuiPort); err != nil {
return err
} else if !isCompatible {
return nil
}
runMizuView()
return nil
},

View File

@@ -20,10 +20,13 @@ func runMizuView() {
exists, err := kubernetesProvider.DoesServicesExist(ctx, mizu.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
panic(err)
mizu.Log.Errorf("Failed to found mizu service %v", err)
cancel()
return
}
if !exists {
mizu.Log.Infof("The %s service not found", mizu.ApiServerPodName)
mizu.Log.Infof("%s service not found, you should run `mizu tap` command first", mizu.ApiServerPodName)
cancel()
return
}
@@ -33,11 +36,19 @@ func runMizuView() {
mizu.Log.Infof("Found a running service %s and open port %d", mizu.ApiServerPodName, mizu.Config.View.GuiPort)
return
}
mizu.Log.Infof("Found service %s, creating k8s proxy", mizu.ApiServerPodName)
mizu.Log.Debugf("Found service %s, creating k8s proxy", mizu.ApiServerPodName)
go startProxyReportErrorIfAny(kubernetesProvider, cancel)
mizu.Log.Infof("Mizu is available at http://%s\n", kubernetes.GetMizuApiServerProxiedHostAndPath(mizu.Config.View.GuiPort))
err = kubernetes.StartProxy(kubernetesProvider, mizu.Config.View.GuiPort, mizu.Config.MizuResourcesNamespace, mizu.ApiServerPodName)
if err != nil {
mizu.Log.Errorf("Error occurred while running k8s proxy %v", err)
if isCompatible, err := mizu.CheckVersionCompatibility(mizu.Config.View.GuiPort); err != nil {
mizu.Log.Errorf("Failed to check versions compatibility %v", err)
cancel()
return
} else if !isCompatible {
cancel()
return
}
waitForFinish(ctx, cancel)
}

View File

@@ -17,8 +17,12 @@ func FormatError(err error) error {
var errorNew error
if k8serrors.IsForbidden(err) {
errorNew = fmt.Errorf("insufficient permissions: %w. "+
"supply the required permission or control Mizu's access to namespaces by setting MizuResourcesNamespace "+
"in the config file or setting the tapped namespace with --%s %s=<NAMEPSACE>", err, mizu.SetCommandName, mizu.MizuResourcesNamespaceConfigName)
"supply the required permission or control Mizu's access to namespaces by setting %s "+
"in the config file or setting the tapped namespace with --%s %s=<NAMEPSACE>",
err,
mizu.MizuResourcesNamespaceConfigName,
mizu.SetCommandName,
mizu.MizuResourcesNamespaceConfigName)
} else if syntaxError, isSyntaxError := asRegexSyntaxError(err); isSyntaxError {
errorNew = fmt.Errorf("regex %s is invalid: %w", syntaxError.Expr, err)
} else {

View File

@@ -69,6 +69,10 @@ func GetConfigWithDefaults() (string, error) {
if err := defaults.Set(&defaultConf); err != nil {
return "", err
}
// TODO: change to generic solution
defaultConf.AgentImage = ""
return uiUtils.PrettyYaml(defaultConf)
}
@@ -129,6 +133,7 @@ func mergeSetFlag(setValues []string) {
if !Contains(allowedSetFlags, argumentKey) {
Log.Warningf(uiUtils.Warning, fmt.Sprintf("Ignoring set argument %s, flag name must be one of the following: \"%s\"", setValue, strings.Join(allowedSetFlags, "\", \"")))
continue
}
mergeFlagValue(configElem, argumentKey, argumentValue)

View File

@@ -19,7 +19,7 @@ type ConfigStruct struct {
Fetch configStructs.FetchConfig `yaml:"fetch"`
Version configStructs.VersionConfig `yaml:"version"`
View configStructs.ViewConfig `yaml:"view"`
AgentImage string `yaml:"agent-image"`
AgentImage string `yaml:"agent-image,omitempty"`
MizuResourcesNamespace string `yaml:"mizu-resources-namespace" default:"mizu"`
Telemetry bool `yaml:"telemetry" default:"true"`
DumpLogs bool `yaml:"dump-logs" default:"false"`

View File

@@ -4,12 +4,12 @@ const (
DirectoryFetchName = "directory"
FromTimestampFetchName = "from"
ToTimestampFetchName = "to"
MizuPortFetchName = "port"
GuiPortFetchName = "gui-port"
)
type FetchConfig struct {
Directory string `yaml:"directory" default:"."`
FromTimestamp int `yaml:"from" default:"0"`
ToTimestamp int `yaml:"to" default:"0"`
MizuPort uint16 `yaml:"port" default:"8899"`
GuiPort uint16 `yaml:"gui-port" default:"8899"`
}

View File

@@ -46,7 +46,7 @@ func CheckVersionCompatibility(port uint16) (bool, error) {
return true, nil
}
Log.Infof(uiUtils.Red, fmt.Sprintf("cli version (%s) is not compatible with api version (%s)", SemVer, apiSemVer))
Log.Errorf(uiUtils.Red, fmt.Sprintf("cli version (%s) is not compatible with api version (%s)", SemVer, apiSemVer))
return false, nil
}