* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically * Remove `MAX_NUMBER_OF_GOROUTINES` environment variable * Change the line * Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000` * Write the client and integrate to the new real-time database * Refactor the WebSocket implementaiton for `/ws` * Adapt the UI to the new filtering system * Fix the rest of the issues in the UI * Increase the buffer of the scanner * Implement accessing single records * Increase the buffer of another scanner * Populate `Request` and `Response` fields of `MizuEntry` * Add syntax highlighting for the query * Add database to `Dockerfile` * Fix some issues * Update the `realtime_dbms` Git module commit hash * Upgrade Gin version and print the query string * Revert "Upgrade Gin version and print the query string" This reverts commitaa09f904ee
. * Use WebSocket's itself to query instead of the query string * Fix some errors related to conversion to HAR * Fix the issues caused by the latest merge * Fix the build error * Fix PR validation GitHub workflow * Replace the git submodule with latest Basenine version `0.1.0` Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead. * Move Basenine host and port constants to `shared` module * Reliably execute and wait for Basenine to become available * Upgrade Basenine version * Properly close WebSocket and data channel * Fix the issues caused by the recent merge commit * Clean up the TypeScript code * Update `.gitignore` * Limit the database size * Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions * Run `go mod tidy` on `agent` * Upgrade `github.com/up9inc/basenine/client/go` version * Implement a mechanism to update the query using click events in the UI and use it for protocol macros * Update the query on click to timestamps * Fix some issues in the WebSocket and channel handling * Update the query on clicks to status code * Update the query on clicks to method, path and service * Update the query on clicks to is outgoing, source and destination ports * Add an API endpoint to validate the query against syntax errors * Move the query background color state into `TrafficPage` * Fix the logic in `setQuery` * Display a toast message in case of a syntax error in the query * Remove a call to `fmt.Printf` * Upgrade Basenine version to `0.1.3` * Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable * Have the `path` key in request details, in HTTP * Rearrange the HTTP headers for the querying * Do the same thing for `cookies` and `queryString` * Update the query on click to table elements Add the selectors for `TABLE` type representations in HTTP extension. * Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle` * Add the selectors for `TABLE` type representations in AMQP extension * Add the selectors for `TABLE` type representations in Kafka extension * Add the selectors for `TABLE` type representations in Redis extension * Define a struct in `tap/api.go` for the section representation data * Add the selectors for `BODY` type representations * Add `request.path` to the HTTP request details * Change the summary string's field name from `path` to `summary` * Introduce `queryable` CSS class for queryable UI elements and underline them on hover * Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values Upgrade Basenine version to `0.2.0`. * Verify the sha256sum of Basenine executable inside `Dockerfile` * Pass the start time to web UI through WebSocket and always show the `EntriesList` footer * Pipe the `stderr` of Basenine as well * Fix the layout issues related to `CodeEditor` in the UI * Use the correct `shasum` command in `Dockerfile` * Upgrade Basenine version to `0.2.1` * Limit the height of `CodeEditor` container * Remove `Paused` enum `ConnectionStatus` in UI * Fix the issue caused by the recent merge * Add the filtering guide (cheatsheet) * Update open cheatsheet button's title * Update cheatsheet content * Remove the old SQLite code, adapt the `--analyze` related code to Basenine * Change the method signature of `NewEntry` * Change the method signature of `Represent` * Introduce `HTTPPair` field in `MizuEntry` specific to HTTP * Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry` Also remove the `getEstimatedEntrySizeBytes` method. * Remove `gorm.io/gorm` dependency * Remove unused `sensitiveDataFiltering` folder * Increase the left margin of open cheatsheet button * Add `overflow: auto` to the cheatsheet `Modal` * Fix `GetEntry` method * Fix the macro for gRPC * Fix an interface conversion in case of AMQP * Fix two more interface conversion errors in AMQP * Make the `syncEntriesImpl` method blocking * Fix a grammar mistake in the cheatsheet * Adapt to the changes in the recent merge commit * Improve the cheatsheet text * Always display the timestamp in `en-US` * Upgrade Basenine version to `0.2.2` * Fix the order of closing Basenine connections and channels * Don't close the Basenine channels at all * Upgrade Basenine version to `0.2.3` * Set the initial filter to `rlimit(100)` * Make Basenine persistent * Upgrade Basenine version to `0.2.4` * Update `debug.Dockerfile` * Fix a failing test * Upgrade Basenine version to `0.2.5` * Revert "Do not show play icon when disconnected (#428)" This reverts commit8af2e562f8
. * Upgrade Basenine version to `0.2.6` * Make all non-informative things informative * Make `100` a constant * Use `===` in JavaScript no matter what * Remove a forgotten `console.log` * Add a comment and update the `query` in `syncEntriesImpl` * Don't call `panic` in `GetEntry` * Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf` * Remove unnecessary `\n` characters in the logs
The API Traffic Viewer for Kubernetes
A simple-yet-powerful API traffic viewer for Kubernetes enabling you to view all API communication between microservices to help your debug and troubleshoot regressions.
Think TCPDump and Chrome Dev Tools combined.
Features
- Simple and powerful CLI
- Real-time view of all HTTP requests, REST and gRPC API calls
- No installation or code instrumentation
- Works completely on premises
Requirements
A Kubernetes server version of 1.16.0 or higher is required.
Download
Download Mizu for your platform and operating system
Latest Stable Release
- for MacOS - Intel
curl -Lo mizu \
https://github.com/up9inc/mizu/releases/latest/download/mizu_darwin_amd64 \
&& chmod 755 mizu
- for Linux - Intel 64bit
curl -Lo mizu \
https://github.com/up9inc/mizu/releases/latest/download/mizu_linux_amd64 \
&& chmod 755 mizu
SHA256 checksums are available on the Releases page
Development (unstable) Build
Pick one from the Releases page
Kubeconfig & Permissions
While mizu
most often works out of the box, you can influence its behavior:
- [OPTIONAL] Set
KUBECONFIG
environment variable to your Kubernetes configuration. If this is not set, Mizu assumes that configuration is at${HOME}/.kube/config
mizu
assumes user running the command has permissions to create resources (such as pods, services, namespaces) on your Kubernetes cluster (no worries -mizu
resources are cleaned up upon termination)
For detailed list of k8s permissions see PERMISSIONS document
How to Run
- Find pods you'd like to tap to in your Kubernetes cluster
- Run
mizu tap
ormizu tap PODNAME
- Open browser on
http://localhost:8899/mizu
or as instructed in the CLI - Watch the API traffic flowing
- Type ^C to stop
Examples
Run mizu help
for usage options
To tap all pods in current namespace -
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
carts-66c77f5fbb-fq65r 2/2 Running 0 20m
catalogue-5f4cb7cf5-7zrmn 2/2 Running 0 20m
front-end-649fc5fd6-kqbtn 2/2 Running 0 20m
..
$ mizu tap
+carts-66c77f5fbb-fq65r
+catalogue-5f4cb7cf5-7zrmn
+front-end-649fc5fd6-kqbtn
Web interface is now available at http://localhost:8899
^C
To tap specific pod
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
front-end-649fc5fd6-kqbtn 2/2 Running 0 7m
..
$ mizu tap front-end-649fc5fd6-kqbtn
+front-end-649fc5fd6-kqbtn
Web interface is now available at http://localhost:8899
^C
To tap multiple pods using regex
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
carts-66c77f5fbb-fq65r 2/2 Running 0 20m
catalogue-5f4cb7cf5-7zrmn 2/2 Running 0 20m
front-end-649fc5fd6-kqbtn 2/2 Running 0 20m
..
$ mizu tap "^ca.*"
+carts-66c77f5fbb-fq65r
+catalogue-5f4cb7cf5-7zrmn
Web interface is now available at http://localhost:8899
^C
To run mizu mizu daemon mode (detached from cli)
$ mizu tap "^ca.*" --daemon
Mizu will store up to 200MB of traffic, old traffic will be cleared once the limit is reached.
Tapping pods in namespaces "sock-shop"
Waiting for mizu to be ready... (may take a few minutes)
+carts-66c77f5fbb-fq65r
+catalogue-5f4cb7cf5-7zrmn
..
$ mizu view
Establishing connection to k8s cluster...
Mizu is available at http://localhost:8899
^C
..
$ mizu clean # mizu will continue running in cluster until clean is executed
Removing mizu resources
To run mizu daemon mode with LoadBalancer kubernetes service
$ mizu tap "^ca.*" --daemon
Mizu will store up to 200MB of traffic, old traffic will be cleared once the limit is reached.
Tapping pods in namespaces "sock-shop"
Waiting for mizu to be ready... (may take a few minutes)
..
$ kubectl expose deployment -n mizu --port 80 --target-port 8899 mizu-api-server --type=LoadBalancer --name=mizu-lb
service/mizu-lb exposed
..
$ kubectl get services -n mizu
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mizu-api-server ClusterIP 10.107.200.100 <none> 80/TCP 5m5s
mizu-lb LoadBalancer 10.107.200.101 34.77.120.116 80:30141/TCP 76s
Note that LoadBalancer
services only work on supported clusters (usually cloud providers) and might incur extra costs
If you changed the mizu-resources-namespace
value, make sure the -n mizu
flag of the kubectl expose
command is changed to the value of mizu-resources-namespace
mizu will now be available both by running mizu view
or by accessing the EXTERNAL-IP
of the mizu-lb
service through your browser.
Configuration
Mizu can work with config file which should be stored in ${HOME}/.mizu/config.yaml (macOS: ~/.mizu/config.yaml)
In case no config file found, defaults will be used
In case of partial configuration defined, all other fields will be used with defaults
You can always override the defaults or config file with CLI flags
To get the default config params run mizu config
To generate a new config file with default values use mizu config -r
Telemetry
By default, mizu reports usage telemetry. It can be disabled by adding a line of telemetry: false
in the ${HOME}/.mizu/config.yaml
file
Advanced Usage
Namespace-Restricted Mode
Some users have permission to only manage resources in one particular namespace assigned to them
By default mizu tap
creates a new namespace mizu
for all of its Kubernetes resources. In order to instead install
Mizu in an existing namespace, set the mizu-resources-namespace
config option
If mizu-resources-namespace
is set to a value other than the default mizu
, Mizu will operate in a
Namespace-Restricted mode. It will only tap pods in mizu-resources-namespace
. This way Mizu only requires permissions
to the namespace set by mizu-resources-namespace
. The user must set the tapped namespace to the same namespace by
using the --namespace
flag or by setting tap.namespaces
in the config file
Setting mizu-resources-namespace=mizu
resets Mizu to its default behavior
User agent filtering
User-agent filtering (like health checks) - can be configured using command-line options:
$ mizu tap "^ca.*" --set tap.ignored-user-agents=kube-probe --set tap.ignored-user-agents=prometheus
+carts-66c77f5fbb-fq65r
+catalogue-5f4cb7cf5-7zrmn
Web interface is now available at http://localhost:8899
^C
Any request that contains User-Agent
header with one of the specified values (kube-probe
or prometheus
) will not be captured
Traffic validation rules
This feature allows you to define set of simple rules, and test the traffic against them. Such validation may test response for specific JSON fields, headers, etc.
Please see TRAFFIC RULES page for more details and syntax.
OpenAPI Specification (OAS) Contract Monitoring
An OAS/Swagger file can contain schemas under parameters
and responses
fields. With --contract catalogue.yaml
CLI option, you can pass your API description to Mizu and the traffic will automatically be validated
against the contracts.
Please see CONTRACT MONITORING page for more details and syntax.
Configure proxy host
By default, mizu will be accessible via local host: 'http://localhost:8899/mizu/', it is possible to change the host,
for instance, to '0.0.0.0' which can grant access via machine IP address.
This setting can be changed via command line flag --set tap.proxy-host=<value>
or via config file:
tap
proxy-host: 0.0.0.0
and when changed it will support accessing by IP
Run in daemon mode
Mizu can be ran detached from the cli using the daemon flag: mizu tap --daemon
. This type of mizu instance will run indefinitely in the cluster.
Please note that daemon mode requires you to have RBAC creation permissions, see the permissions doc for more details.
In order to access a daemon mizu you will have to run mizu view
after running the tap --daemon
command.
To stop the detached mizu instance and clean all cluster side resources, run mizu clean
How to Run local UI
-
run from mizu/agent
go run main.go --hars-read --hars-dir <folder>
-
copy Har files into the folder from last command
-
change
MizuWebsocketURL
andapiURL
inapi.js
file -
run from mizu/ui -
npm run start
-
open browser on
localhost:3000