From d2fe3f6620de63c91f425058b3d2b47c28855b51 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?M=2E=20Mert=20Y=C4=B1ld=C4=B1ran?= Date: Tue, 9 Nov 2021 19:54:48 +0300 Subject: [PATCH] Migrate from SQLite to Basenine and introduce a new filtering syntax (#279) * Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically * Remove `MAX_NUMBER_OF_GOROUTINES` environment variable * Change the line * Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000` * Write the client and integrate to the new real-time database * Refactor the WebSocket implementaiton for `/ws` * Adapt the UI to the new filtering system * Fix the rest of the issues in the UI * Increase the buffer of the scanner * Implement accessing single records * Increase the buffer of another scanner * Populate `Request` and `Response` fields of `MizuEntry` * Add syntax highlighting for the query * Add database to `Dockerfile` * Fix some issues * Update the `realtime_dbms` Git module commit hash * Upgrade Gin version and print the query string * Revert "Upgrade Gin version and print the query string" This reverts commit aa09f904ee34c9906d1035d4f192c315ff563f12. * Use WebSocket's itself to query instead of the query string * Fix some errors related to conversion to HAR * Fix the issues caused by the latest merge * Fix the build error * Fix PR validation GitHub workflow * Replace the git submodule with latest Basenine version `0.1.0` Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead. * Move Basenine host and port constants to `shared` module * Reliably execute and wait for Basenine to become available * Upgrade Basenine version * Properly close WebSocket and data channel * Fix the issues caused by the recent merge commit * Clean up the TypeScript code * Update `.gitignore` * Limit the database size * Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions * Run `go mod tidy` on `agent` * Upgrade `github.com/up9inc/basenine/client/go` version * Implement a mechanism to update the query using click events in the UI and use it for protocol macros * Update the query on click to timestamps * Fix some issues in the WebSocket and channel handling * Update the query on clicks to status code * Update the query on clicks to method, path and service * Update the query on clicks to is outgoing, source and destination ports * Add an API endpoint to validate the query against syntax errors * Move the query background color state into `TrafficPage` * Fix the logic in `setQuery` * Display a toast message in case of a syntax error in the query * Remove a call to `fmt.Printf` * Upgrade Basenine version to `0.1.3` * Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable * Have the `path` key in request details, in HTTP * Rearrange the HTTP headers for the querying * Do the same thing for `cookies` and `queryString` * Update the query on click to table elements Add the selectors for `TABLE` type representations in HTTP extension. * Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle` * Add the selectors for `TABLE` type representations in AMQP extension * Add the selectors for `TABLE` type representations in Kafka extension * Add the selectors for `TABLE` type representations in Redis extension * Define a struct in `tap/api.go` for the section representation data * Add the selectors for `BODY` type representations * Add `request.path` to the HTTP request details * Change the summary string's field name from `path` to `summary` * Introduce `queryable` CSS class for queryable UI elements and underline them on hover * Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values Upgrade Basenine version to `0.2.0`. * Verify the sha256sum of Basenine executable inside `Dockerfile` * Pass the start time to web UI through WebSocket and always show the `EntriesList` footer * Pipe the `stderr` of Basenine as well * Fix the layout issues related to `CodeEditor` in the UI * Use the correct `shasum` command in `Dockerfile` * Upgrade Basenine version to `0.2.1` * Limit the height of `CodeEditor` container * Remove `Paused` enum `ConnectionStatus` in UI * Fix the issue caused by the recent merge * Add the filtering guide (cheatsheet) * Update open cheatsheet button's title * Update cheatsheet content * Remove the old SQLite code, adapt the `--analyze` related code to Basenine * Change the method signature of `NewEntry` * Change the method signature of `Represent` * Introduce `HTTPPair` field in `MizuEntry` specific to HTTP * Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry` Also remove the `getEstimatedEntrySizeBytes` method. * Remove `gorm.io/gorm` dependency * Remove unused `sensitiveDataFiltering` folder * Increase the left margin of open cheatsheet button * Add `overflow: auto` to the cheatsheet `Modal` * Fix `GetEntry` method * Fix the macro for gRPC * Fix an interface conversion in case of AMQP * Fix two more interface conversion errors in AMQP * Make the `syncEntriesImpl` method blocking * Fix a grammar mistake in the cheatsheet * Adapt to the changes in the recent merge commit * Improve the cheatsheet text * Always display the timestamp in `en-US` * Upgrade Basenine version to `0.2.2` * Fix the order of closing Basenine connections and channels * Don't close the Basenine channels at all * Upgrade Basenine version to `0.2.3` * Set the initial filter to `rlimit(100)` * Make Basenine persistent * Upgrade Basenine version to `0.2.4` * Update `debug.Dockerfile` * Fix a failing test * Upgrade Basenine version to `0.2.5` * Revert "Do not show play icon when disconnected (#428)" This reverts commit 8af2e562f8f5fd384731612ed6861e956937d03f. * Upgrade Basenine version to `0.2.6` * Make all non-informative things informative * Make `100` a constant * Use `===` in JavaScript no matter what * Remove a forgotten `console.log` * Add a comment and update the `query` in `syncEntriesImpl` * Don't call `panic` in `GetEntry` * Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf` * Remove unnecessary `\n` characters in the logs --- .gitignore | 3 + Dockerfile | 12 +- acceptanceTests/tap_test.go | 6 +- agent/go.mod | 7 +- agent/go.sum | 76 +- agent/main.go | 73 +- agent/pkg/api/main.go | 48 +- agent/pkg/api/socket_routes.go | 101 +- agent/pkg/controllers/entries_controller.go | 89 +- agent/pkg/controllers/query_controller.go | 31 + agent/pkg/database/main.go | 78 -- agent/pkg/database/size_enforcer.go | 102 -- agent/pkg/models/models.go | 65 +- agent/pkg/providers/stats_provider_test.go | 6 - agent/pkg/routes/entries_routes.go | 6 +- agent/pkg/routes/query_routes.go | 13 + agent/pkg/sensitiveDataFiltering/consts.go | 10 - agent/pkg/up9/main.go | 117 ++- agent/pkg/utils/har.go | 56 +- agent/pkg/utils/truncating_logger.go | 60 -- cli/config/config.go | 7 +- debug.Dockerfile | 12 +- shared/consts.go | 2 + shared/models.go | 3 + tap/api/api.go | 108 ++- tap/extensions/amqp/helpers.go | 591 ++++++------ tap/extensions/amqp/main.go | 111 ++- tap/extensions/amqp/spec091.go | 98 +- tap/extensions/amqp/types.go | 26 +- tap/extensions/http/helpers.go | 35 + tap/extensions/http/main.go | 263 +++--- tap/extensions/kafka/helpers.go | 617 ++++++------ tap/extensions/kafka/main.go | 135 +-- tap/extensions/kafka/request.go | 14 +- tap/extensions/kafka/response.go | 8 +- tap/extensions/kafka/structs.go | 894 +++++++++--------- tap/extensions/redis/helpers.go | 38 +- tap/extensions/redis/main.go | 86 +- ui/package-lock.json | 350 +++++++ ui/package.json | 3 + ui/src/components/EntriesList.tsx | 121 +-- ui/src/components/EntryDetailed.tsx | 57 +- .../EntryDetailed/EntrySections.tsx | 42 +- .../components/EntryDetailed/EntryViewer.tsx | 16 +- .../EntryListItem/EntryListItem.tsx | 79 +- ui/src/components/Filters.tsx | 398 +++++--- ui/src/components/TrafficPage.tsx | 153 ++- ui/src/components/UI/EndpointPath.tsx | 15 - ui/src/components/UI/FilterSelect.tsx | 28 - ui/src/components/UI/Protocol.tsx | 35 +- ui/src/components/UI/StatusCode.tsx | 11 +- ui/src/components/UI/Summary.tsx | 32 + .../UI/SyntaxHighlighter/highlighterStyle.ts | 2 +- .../components/UI/style/Protocol.module.sass | 1 - ...ntPath.module.sass => Summary.module.sass} | 4 +- .../components/assets/filter-ui-example-1.png | Bin 0 -> 41498 bytes .../components/assets/filter-ui-example-2.png | Bin 0 -> 16773 bytes .../components/style/EntriesList.module.sass | 9 - ui/src/components/style/Filters.module.sass | 7 +- ui/src/components/style/TrafficPage.sass | 5 + ui/src/helpers/api.js | 16 +- ui/src/index.sass | 13 +- 62 files changed, 3077 insertions(+), 2327 deletions(-) create mode 100644 agent/pkg/controllers/query_controller.go delete mode 100644 agent/pkg/database/main.go delete mode 100644 agent/pkg/database/size_enforcer.go create mode 100644 agent/pkg/routes/query_routes.go delete mode 100644 agent/pkg/sensitiveDataFiltering/consts.go delete mode 100644 agent/pkg/utils/truncating_logger.go create mode 100644 tap/extensions/http/helpers.go delete mode 100644 ui/src/components/UI/EndpointPath.tsx delete mode 100644 ui/src/components/UI/FilterSelect.tsx create mode 100644 ui/src/components/UI/Summary.tsx rename ui/src/components/UI/style/{EndpointPath.module.sass => Summary.module.sass} (75%) create mode 100644 ui/src/components/assets/filter-ui-example-1.png create mode 100644 ui/src/components/assets/filter-ui-example-2.png diff --git a/.gitignore b/.gitignore index 4d5c55b7e..e3efed6b5 100644 --- a/.gitignore +++ b/.gitignore @@ -29,3 +29,6 @@ build # pprof pprof/* + +# Database Files +*.bin diff --git a/Dockerfile b/Dockerfile index 65b7b46ea..9345df30c 100644 --- a/Dockerfile +++ b/Dockerfile @@ -13,7 +13,7 @@ FROM golang:1.16-alpine AS builder # Set necessary environment variables needed for our image. ENV CGO_ENABLED=1 GOOS=linux GOARCH=amd64 -RUN apk add libpcap-dev gcc g++ make bash +RUN apk add libpcap-dev gcc g++ make bash perl-utils # Move to agent working directory (/agent-build). WORKDIR /app/agent-build @@ -24,7 +24,7 @@ COPY tap/go.mod tap/go.mod ../tap/ COPY tap/api/go.* ../tap/api/ RUN go mod download # cheap trick to make the build faster (As long as go.mod wasn't changes) -RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' -e 'sqlite' | xargs go get +RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' | xargs go get ARG COMMIT_HASH ARG GIT_BRANCH @@ -41,16 +41,24 @@ RUN go build -ldflags="-s -w \ -X 'mizuserver/pkg/version.BuildTimestamp=${BUILD_TIMESTAMP}' \ -X 'mizuserver/pkg/version.SemVer=${SEM_VER}'" -o mizuagent . +# Download Basenine executable, verify the sha1sum and move it to a directory in $PATH +ADD https://github.com/up9inc/basenine/releases/download/v0.2.6/basenine_linux_amd64 ./basenine_linux_amd64 +ADD https://github.com/up9inc/basenine/releases/download/v0.2.6/basenine_linux_amd64.sha256 ./basenine_linux_amd64.sha256 +RUN shasum -a 256 -c basenine_linux_amd64.sha256 +RUN chmod +x ./basenine_linux_amd64 + COPY devops/build_extensions.sh .. RUN cd .. && /bin/bash build_extensions.sh FROM alpine:3.14 RUN apk add bash libpcap-dev tcpdump + WORKDIR /app # Copy binary and config files from /build to root folder of scratch container. COPY --from=builder ["/app/agent-build/mizuagent", "."] +COPY --from=builder ["/app/agent-build/basenine_linux_amd64", "/usr/local/bin/basenine"] COPY --from=builder ["/app/agent/build/extensions", "extensions"] COPY --from=site-build ["/app/ui-build/build", "site"] RUN mkdir /app/data/ diff --git a/acceptanceTests/tap_test.go b/acceptanceTests/tap_test.go index 02bca771f..ebee856de 100644 --- a/acceptanceTests/tap_test.go +++ b/acceptanceTests/tap_test.go @@ -472,7 +472,7 @@ func TestTapRedact(t *testing.T) { entryPayload := entryRequest["payload"].(map[string]interface{}) entryDetails := entryPayload["details"].(map[string]interface{}) - headers := entryDetails["headers"].([]interface{}) + headers := entryDetails["_headers"].([]interface{}) for _, headerInterface := range headers { header := headerInterface.(map[string]interface{}) if header["name"].(string) != "User-Agent" { @@ -587,7 +587,7 @@ func TestTapNoRedact(t *testing.T) { entryPayload := entryRequest["payload"].(map[string]interface{}) entryDetails := entryPayload["details"].(map[string]interface{}) - headers := entryDetails["headers"].([]interface{}) + headers := entryDetails["_headers"].([]interface{}) for _, headerInterface := range headers { header := headerInterface.(map[string]interface{}) if header["name"].(string) != "User-Agent" { @@ -808,7 +808,7 @@ func TestTapIgnoredUserAgents(t *testing.T) { entryPayload := entryRequest["payload"].(map[string]interface{}) entryDetails := entryPayload["details"].(map[string]interface{}) - entryHeaders := entryDetails["headers"].([]interface{}) + entryHeaders := entryDetails["_headers"].([]interface{}) for _, headerInterface := range entryHeaders { header := headerInterface.(map[string]interface{}) if header["name"].(string) != ignoredUserAgentCustomHeader { diff --git a/agent/go.mod b/agent/go.mod index 905c80f0b..f1af6b2fb 100644 --- a/agent/go.mod +++ b/agent/go.mod @@ -3,8 +3,8 @@ module mizuserver go 1.16 require ( + github.com/antelman107/net-wait-go v0.0.0-20210623112055-cf684aebda7b github.com/djherbis/atime v1.0.0 - github.com/fsnotify/fsnotify v1.4.9 github.com/getkin/kin-openapi v0.76.0 github.com/gin-contrib/static v0.0.1 github.com/gin-gonic/gin v1.7.2 @@ -16,13 +16,12 @@ require ( github.com/op/go-logging v0.0.0-20160315200505-970db520ece7 github.com/orcaman/concurrent-map v0.0.0-20210106121528-16402b402231 github.com/patrickmn/go-cache v2.1.0+incompatible + github.com/up9inc/basenine/client/go v0.0.0-20211106180626-0193408db715 github.com/up9inc/mizu/shared v0.0.0 github.com/up9inc/mizu/tap v0.0.0 github.com/up9inc/mizu/tap/api v0.0.0 github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0 - go.mongodb.org/mongo-driver v1.7.1 - gorm.io/driver/sqlite v1.1.4 - gorm.io/gorm v1.21.8 + golang.org/x/text v0.3.5 // indirect k8s.io/api v0.21.2 k8s.io/apimachinery v0.21.2 k8s.io/client-go v0.21.2 diff --git a/agent/go.sum b/agent/go.sum index 52d06e485..fe9b06095 100644 --- a/agent/go.sum +++ b/agent/go.sum @@ -52,6 +52,8 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= +github.com/antelman107/net-wait-go v0.0.0-20210623112055-cf684aebda7b h1:8m+eVxVVDDyJFidv7Ck1OwqnDaQR6pTSRGlCC2Dnw0A= +github.com/antelman107/net-wait-go v0.0.0-20210623112055-cf684aebda7b/go.mod h1:+tQQjzrp2501Nd6JXrb9s/XsNvFK3ZbxOnCdQl/vDRo= github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= @@ -94,7 +96,6 @@ github.com/djherbis/atime v1.0.0 h1:ySLvBAM0EvOGaX7TI4dAM5lWj+RdJUCKtGSEHN8SGBg= github.com/djherbis/atime v1.0.0/go.mod h1:5W+KBIuTwVGcqjIfaTwt+KSYX1o6uep8dtevevQP/f8= github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= -github.com/docker/go-units v0.4.0 h1:3uh0PgVws3nIA0Q+MwDC8yjEPf9zjRfZZWXZYDct3Tw= github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= @@ -111,7 +112,6 @@ github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5Kwzbycv github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTgWiPvpYe4Xau31I0PRk= github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fvbommel/sortorder v1.0.1/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= github.com/getkin/kin-openapi v0.76.0 h1:j77zg3Ec+k+r+GA3d8hBoXpAc6KX9TbBPrwQGBIy2sY= @@ -195,31 +195,7 @@ github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn github.com/go-playground/validator/v10 v10.5.0 h1:X9rflw/KmpACwT8zdrm1upefpvdy6ur8d1kWyq6sg3E= github.com/go-playground/validator/v10 v10.5.0/go.mod h1:xm76BBt941f7yWdGnI2DVPFFg1UK3YY04qifoXU3lOk= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/gobuffalo/attrs v0.0.0-20190224210810-a9411de4debd/go.mod h1:4duuawTqi2wkkpB4ePgWMaai6/Kc6WEz83bhFwpHzj0= -github.com/gobuffalo/depgen v0.0.0-20190329151759-d478694a28d3/go.mod h1:3STtPUQYuzV0gBVOY3vy6CfMm/ljR4pABfrTeHNLHUY= -github.com/gobuffalo/depgen v0.1.0/go.mod h1:+ifsuy7fhi15RWncXQQKjWS9JPkdah5sZvtHc2RXGlg= -github.com/gobuffalo/envy v1.6.15/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI= -github.com/gobuffalo/envy v1.7.0/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI= -github.com/gobuffalo/flect v0.1.0/go.mod h1:d2ehjJqGOH/Kjqcoz+F7jHTBbmDb38yXA598Hb50EGs= -github.com/gobuffalo/flect v0.1.1/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI= -github.com/gobuffalo/flect v0.1.3/go.mod h1:8JCgGVbRjJhVgD6399mQr4fx5rRfGKVzFjbj6RE/9UI= -github.com/gobuffalo/genny v0.0.0-20190329151137-27723ad26ef9/go.mod h1:rWs4Z12d1Zbf19rlsn0nurr75KqhYp52EAGGxTbBhNk= -github.com/gobuffalo/genny v0.0.0-20190403191548-3ca520ef0d9e/go.mod h1:80lIj3kVJWwOrXWWMRzzdhW3DsrdjILVil/SFKBzF28= -github.com/gobuffalo/genny v0.1.0/go.mod h1:XidbUqzak3lHdS//TPu2OgiFB+51Ur5f7CSnXZ/JDvo= -github.com/gobuffalo/genny v0.1.1/go.mod h1:5TExbEyY48pfunL4QSXxlDOmdsD44RRq4mVZ0Ex28Xk= -github.com/gobuffalo/gitgen v0.0.0-20190315122116-cc086187d211/go.mod h1:vEHJk/E9DmhejeLeNt7UVvlSGv3ziL+djtTr3yyzcOw= -github.com/gobuffalo/gogen v0.0.0-20190315121717-8f38393713f5/go.mod h1:V9QVDIxsgKNZs6L2IYiGR8datgMhB577vzTDqypH360= -github.com/gobuffalo/gogen v0.1.0/go.mod h1:8NTelM5qd8RZ15VjQTFkAW6qOMx5wBbW4dSCS3BY8gg= -github.com/gobuffalo/gogen v0.1.1/go.mod h1:y8iBtmHmGc4qa3urIyo1shvOD8JftTtfcKi+71xfDNE= github.com/gobuffalo/here v0.6.0/go.mod h1:wAG085dHOYqUpf+Ap+WOdrPTp5IYcDAs/x7PLa8Y5fM= -github.com/gobuffalo/logger v0.0.0-20190315122211-86e12af44bc2/go.mod h1:QdxcLw541hSGtBnhUc4gaNIXRjiDppFGaDqzbrBd3v8= -github.com/gobuffalo/mapi v1.0.1/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc= -github.com/gobuffalo/mapi v1.0.2/go.mod h1:4VAGh89y6rVOvm5A8fKFxYG+wIW6LO1FMTG9hnKStFc= -github.com/gobuffalo/packd v0.0.0-20190315124812-a385830c7fc0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4= -github.com/gobuffalo/packd v0.1.0/go.mod h1:M2Juc+hhDXf/PnmBANFCqx4DM3wRbgDvnVWeG2RIxq4= -github.com/gobuffalo/packr/v2 v2.0.9/go.mod h1:emmyGweYTm6Kdper+iywB6YK5YzuKchGtJQZ0Odn4pQ= -github.com/gobuffalo/packr/v2 v2.2.0/go.mod h1:CaAwI0GPIAv+5wKLtv8Afwl+Cm78K/I/VCm/3ptBN+0= -github.com/gobuffalo/syncx v0.0.0-20190224160051-33c29581e754/go.mod h1:HhnNqWY95UYwwW3uSASeV7vtgYkT2t16hJgV3AEPUpw= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= @@ -251,7 +227,6 @@ github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= -github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golangplus/testing v0.0.0-20180327235837-af21d9c3145e/go.mod h1:0AA//k/eakGydO4jKRoRL2j92ZKSzTgj9tclaCrvXHk= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= @@ -321,12 +296,6 @@ github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1: github.com/imdario/mergo v0.3.5 h1:JboBksRwiiAJWvIYJVo46AfV+IAIKZpfrSzVKj42R4Q= github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= -github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E= -github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= -github.com/jinzhu/now v1.1.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= -github.com/jinzhu/now v1.1.2 h1:eVKgfIdy9b6zbWBMgFpfDPoAMifwSZagU9HmEU6zgiI= -github.com/jinzhu/now v1.1.2/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= -github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= @@ -336,14 +305,10 @@ github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/karrick/godirwalk v1.8.0/go.mod h1:H5KPZjojv4lE+QYImBI8xVtrBRgYrIVsaRPx4tDPEn4= -github.com/karrick/godirwalk v1.10.3/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA= github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.9.5/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= @@ -364,16 +329,12 @@ github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM= github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs= -github.com/markbates/oncer v0.0.0-20181203154359-bf2de49a0be2/go.mod h1:Ld9puTsIW75CHf65OeIOkyKbteujpZVXDpWK6YGZbxE= github.com/markbates/pkger v0.17.1/go.mod h1:0JoVlrol20BSywW79rN3kdFFsE5xYM+rSCQDXbLhiuI= -github.com/markbates/safe v1.0.1/go.mod h1:nAqgmRi7cY2nqMc92/bSEeQA+R4OheNU2T1kNSCBdG0= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY= github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= -github.com/mattn/go-sqlite3 v1.14.5 h1:1IdxlwTNazvbKJQSxoJ5/9ECbEeaTTyeU7sEAZ5KKTQ= -github.com/mattn/go-sqlite3 v1.14.5/go.mod h1:WVKg1VTActs4Qso6iwGbiFih2UIHo0ENGwNd0Lj+XmI= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= @@ -395,7 +356,6 @@ github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lN github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= -github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc= github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus= @@ -419,7 +379,6 @@ github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaR github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ= github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= -github.com/pelletier/go-toml v1.7.0/go.mod h1:vwGMzjaWMwyfHwgIBhI2YUM4fB6nL6lVAvS1LBMMhTE= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= @@ -447,8 +406,6 @@ github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= -github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= -github.com/rogpeppe/go-internal v1.2.2/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= @@ -458,8 +415,6 @@ github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAm github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= @@ -468,7 +423,6 @@ github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasO github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= -github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE= github.com/spf13/cobra v1.1.1/go.mod h1:WnodtKOvamDL/PwE2M4iKs8aMDBZ5Q5klgD3qfVJQMI= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= @@ -485,8 +439,9 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= -github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk= github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= @@ -495,26 +450,22 @@ github.com/ugorji/go v1.1.7 h1:/68gy2h+1mWMrwZFeD1kQialdSzAb432dtpeJ42ovdo= github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw= github.com/ugorji/go/codec v1.1.7 h1:2SvQaVZ1ouYrrKKwoSk2pzd4A9evlKJb9oTL+OaLUSs= github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY= +github.com/up9inc/basenine/client/go v0.0.0-20211106180626-0193408db715 h1:3RNTMQZO/4g5gRn4R98cPwCjCrsMklmcOS0g+QwCh5c= +github.com/up9inc/basenine/client/go v0.0.0-20211106180626-0193408db715/go.mod h1:SvJGPoa/6erhUQV7kvHBwM/0x5LyO6XaG2lUaCaKiUI= github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw= github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f h1:p4VB7kIXpOQvVn1ZaTIVp+3vuYAXFe3OJEvjbUYJLaA= github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0= -github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI= -github.com/xdg-go/scram v1.0.2/go.mod h1:1WAq6h33pAW+iRreB34OORO2Nf7qel3VV3fjBj+hCSs= -github.com/xdg-go/stringprep v1.0.2/go.mod h1:8F9zXuvzgwmyT5DUm4GUfZGDdT3W+LCvS6+da4O5kxM= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca/go.mod h1:ce1O1j6UtZfjr22oyGxGLbauSBp2YVXpARAosm7dHBg= github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0 h1:6fRhSjgLCkTD3JnJxvaJ4Sj+TYblw757bqYgZaOq5ZY= github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0/go.mod h1:/LWChgwKmvncFJFHJ7Gvn9wZArjbV5/FppcK2fKk/tI= -github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.mongodb.org/mongo-driver v1.0.3/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= go.mongodb.org/mongo-driver v1.1.1/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= go.mongodb.org/mongo-driver v1.1.2/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= -go.mongodb.org/mongo-driver v1.7.1 h1:jwqTeEM3x6L9xDXrCxN0Hbg7vdGfPBOTIkr0+/LYZDA= -go.mongodb.org/mongo-driver v1.7.1/go.mod h1:Q4oFMbo1+MSNqICAdYMlC/zSTrwCogR4R8NzkI+yfU8= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= @@ -527,13 +478,11 @@ golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnf golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190422162423-af44ce270edf/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190617133340-57b3e21c3d56/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g= @@ -611,7 +560,6 @@ golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190412183630-56d357773e84/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -625,13 +573,10 @@ golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5h golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190419153524-e8e3143a4f4a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190531175056-4c3a928424d2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -685,13 +630,9 @@ golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3 golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190329151228-23e29df326fe/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190416151739-9c9e1878f421/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190420181800-aa740d480789/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190531172133-b3315ee88b7d/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190617190820-da514acc4774/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= @@ -805,11 +746,6 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gorm.io/driver/sqlite v1.1.4 h1:PDzwYE+sI6De2+mxAneV9Xs11+ZyKV6oxD3wDGkaNvM= -gorm.io/driver/sqlite v1.1.4/go.mod h1:mJCeTFr7+crvS+TRnWc5Z3UvwxUN1BGBLMrf5LA9DYw= -gorm.io/gorm v1.20.7/go.mod h1:0HFTzE/SqkGTzK6TlDPPQbAYCluiVvhzoA1+aVyzenw= -gorm.io/gorm v1.21.8 h1:2CEwZSzogdhsKPlJ9OvBKTdlWIpELXb6HbfLfMNhSYI= -gorm.io/gorm v1.21.8/go.mod h1:F+OptMscr0P2F2qU97WT1WimdH9GaQPoDW7AYd5i2Y0= gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/agent/main.go b/agent/main.go index 51ffc9a41..907920203 100644 --- a/agent/main.go +++ b/agent/main.go @@ -6,13 +6,10 @@ import ( "errors" "flag" "fmt" - "github.com/up9inc/mizu/shared/kubernetes" "io/ioutil" - v1 "k8s.io/api/core/v1" "mizuserver/pkg/api" "mizuserver/pkg/config" "mizuserver/pkg/controllers" - "mizuserver/pkg/database" "mizuserver/pkg/models" "mizuserver/pkg/providers" "mizuserver/pkg/routes" @@ -20,6 +17,7 @@ import ( "mizuserver/pkg/utils" "net/http" "os" + "os/exec" "os/signal" "path" "path/filepath" @@ -28,10 +26,15 @@ import ( "syscall" "time" + "github.com/up9inc/mizu/shared/kubernetes" + v1 "k8s.io/api/core/v1" + + "github.com/antelman107/net-wait-go/wait" "github.com/gin-contrib/static" "github.com/gin-gonic/gin" "github.com/gorilla/websocket" "github.com/op/go-logging" + basenine "github.com/up9inc/basenine/client/go" "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/shared/logger" "github.com/up9inc/mizu/tap" @@ -49,10 +52,12 @@ var harsDir = flag.String("hars-dir", "", "Directory to read hars from") var extensions []*tapApi.Extension // global var extensionsMap map[string]*tapApi.Extension // global +var startTime int64 + const ( - socketConnectionRetries = 10 + socketConnectionRetries = 10 socketConnectionRetryDelay = time.Second * 2 - socketHandshakeTimeout = time.Second * 2 + socketHandshakeTimeout = time.Second * 2 ) func main() { @@ -109,7 +114,8 @@ func main() { go pipeTapChannelToSocket(socketConnection, filteredOutputItemsChannel) } else if *apiServerMode { - database.InitDataBase(config.Config.AgentDatabasePath) + startBasenineServer(shared.BasenineHost, shared.BaseninePort) + startTime = time.Now().UnixNano() / int64(time.Millisecond) api.StartResolving(*namespace) outputItemsChannel := make(chan *tapApi.OutputChannelItem) @@ -142,6 +148,53 @@ func main() { logger.Log.Info("Exiting") } +func startBasenineServer(host string, port string) { + cmd := exec.Command("basenine", "-addr", host, "-port", port, "-persistent") + cmd.Dir = config.Config.AgentDatabasePath + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + err := cmd.Start() + if err != nil { + logger.Log.Panicf("Failed starting Basenine: %v", err) + } + + if !wait.New( + wait.WithProto("tcp"), + wait.WithWait(200*time.Millisecond), + wait.WithBreak(50*time.Millisecond), + wait.WithDeadline(5*time.Second), + wait.WithDebug(true), + ).Do([]string{fmt.Sprintf("%s:%s", host, port)}) { + logger.Log.Panicf("Basenine is not available: %v", err) + } + + // Make a channel to gracefully exit Basenine. + channel := make(chan os.Signal) + signal.Notify(channel, os.Interrupt, syscall.SIGTERM) + + // Handle the channel. + go func() { + <-channel + cmd.Process.Signal(syscall.SIGTERM) + }() + + // Limit the database size to default 200MB + err = basenine.Limit(host, port, config.Config.MaxDBSizeBytes) + if err != nil { + logger.Log.Panicf("Error while limiting database size: %v", err) + } + + for _, extension := range extensions { + macros := extension.Dissector.Macros() + for macro, expanded := range macros { + err = basenine.Macro(host, port, macro, expanded) + if err != nil { + logger.Log.Panicf("Error while adding a macro: %v", err) + } + } + } +} + func loadExtensions() { dir, _ := filepath.Abs(filepath.Dir(os.Args[0])) extensionsDir := path.Join(dir, "./extensions/") @@ -200,7 +253,8 @@ func hostApi(socketHarOutputChannel chan<- *tapApi.OutputChannelItem) { app.Use(static.ServeRoot("/", "./site")) app.Use(CORSMiddleware()) // This has to be called after the static middleware, does not work if its called before - api.WebSocketRoutes(app, &eventHandlers) + api.WebSocketRoutes(app, &eventHandlers, startTime) + routes.QueryRoutes(app) routes.EntriesRoutes(app) routes.MetadataRoutes(app) routes.StatusRoutes(app) @@ -361,7 +415,7 @@ func dialSocketWithRetry(socketAddress string, retryAmount int, retryDelay time. socketConnection, _, err := dialer.Dial(socketAddress, nil) if err != nil { if i < retryAmount { - logger.Log.Infof("socket connection to %s failed: %v, retrying %d out of %d in %d seconds...", socketAddress, err, i, retryAmount, retryDelay / time.Second) + logger.Log.Infof("socket connection to %s failed: %v, retrying %d out of %d in %d seconds...", socketAddress, err, i, retryAmount, retryDelay/time.Second) time.Sleep(retryDelay) } } else { @@ -371,8 +425,7 @@ func dialSocketWithRetry(socketAddress string, retryAmount int, retryDelay time. return nil, lastErr } - -func startMizuTapperSyncer(ctx context.Context) (*kubernetes.MizuTapperSyncer, error){ +func startMizuTapperSyncer(ctx context.Context) (*kubernetes.MizuTapperSyncer, error) { provider, err := kubernetes.NewProviderInCluster() if err != nil { return nil, err diff --git a/agent/pkg/api/main.go b/agent/pkg/api/main.go index 56968ff3c..8bfc65787 100644 --- a/agent/pkg/api/main.go +++ b/agent/pkg/api/main.go @@ -5,7 +5,6 @@ import ( "context" "encoding/json" "fmt" - "mizuserver/pkg/database" "mizuserver/pkg/holder" "mizuserver/pkg/providers" "os" @@ -14,15 +13,16 @@ import ( "strings" "time" - "go.mongodb.org/mongo-driver/bson/primitive" - "github.com/google/martian/har" + "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/shared/logger" tapApi "github.com/up9inc/mizu/tap/api" "mizuserver/pkg/models" "mizuserver/pkg/resolver" "mizuserver/pkg/utils" + + basenine "github.com/up9inc/basenine/client/go" ) var k8sResolver *resolver.Resolver @@ -99,6 +99,12 @@ func startReadingChannel(outputItems <-chan *tapApi.OutputChannelItem, extension panic("Channel of captured messages is nil") } + connection, err := basenine.NewConnection(shared.BasenineHost, shared.BaseninePort) + if err != nil { + panic(err) + } + connection.InsertMode() + disableOASValidation := false ctx := context.Background() doc, contractContent, router, err := loadOAS(ctx) @@ -112,13 +118,13 @@ func startReadingChannel(outputItems <-chan *tapApi.OutputChannelItem, extension extension := extensionsMap[item.Protocol.Name] resolvedSource, resolvedDestionation := resolveIP(item.ConnectionInfo) - mizuEntry := extension.Dissector.Analyze(item, primitive.NewObjectID().Hex(), resolvedSource, resolvedDestionation) + mizuEntry := extension.Dissector.Analyze(item, resolvedSource, resolvedDestionation) baseEntry := extension.Dissector.Summarize(mizuEntry) - mizuEntry.EstimatedSizeBytes = getEstimatedEntrySizeBytes(mizuEntry) + mizuEntry.Base = baseEntry if extension.Protocol.Name == "http" { if !disableOASValidation { var httpPair tapApi.HTTPRequestResponsePair - json.Unmarshal([]byte(mizuEntry.Entry), &httpPair) + json.Unmarshal([]byte(mizuEntry.HTTPPair), &httpPair) contract := handleOAS(ctx, doc, router, httpPair.Request.Payload.RawRequest, httpPair.Response.Payload.RawResponse, contractContent) baseEntry.ContractStatus = contract.Status @@ -128,18 +134,18 @@ func startReadingChannel(outputItems <-chan *tapApi.OutputChannelItem, extension mizuEntry.ContractContent = contract.Content } - var pair tapApi.RequestResponsePair - json.Unmarshal([]byte(mizuEntry.Entry), &pair) - harEntry, err := utils.NewEntry(&pair) + harEntry, err := utils.NewEntry(mizuEntry.Request, mizuEntry.Response, mizuEntry.StartTime, mizuEntry.ElapsedTime) if err == nil { rules, _, _ := models.RunValidationRulesState(*harEntry, mizuEntry.Service) baseEntry.Rules = rules } } - database.CreateEntry(mizuEntry) - baseEntryBytes, _ := models.CreateBaseEntryWebSocketMessage(baseEntry) - BroadcastToBrowserClients(baseEntryBytes) + data, err := json.Marshal(mizuEntry) + if err != nil { + panic(err) + } + connection.SendText(string(data)) } } @@ -171,21 +177,3 @@ func CheckIsServiceIP(address string) bool { } return k8sResolver.CheckIsServiceIP(address) } - -// gives a rough estimate of the size this will take up in the db, good enough for maintaining db size limit accurately -func getEstimatedEntrySizeBytes(mizuEntry *tapApi.MizuEntry) int { - sizeBytes := len(mizuEntry.Entry) - sizeBytes += len(mizuEntry.EntryId) - sizeBytes += len(mizuEntry.Service) - sizeBytes += len(mizuEntry.Url) - sizeBytes += len(mizuEntry.Method) - sizeBytes += len(mizuEntry.RequestSenderIp) - sizeBytes += len(mizuEntry.ResolvedDestination) - sizeBytes += len(mizuEntry.ResolvedSource) - sizeBytes += 8 // Status bytes (sqlite integer is always 8 bytes) - sizeBytes += 8 // Timestamp bytes - sizeBytes += 8 // SizeBytes bytes - sizeBytes += 1 // IsOutgoing bytes - - return sizeBytes -} diff --git a/agent/pkg/api/socket_routes.go b/agent/pkg/api/socket_routes.go index 7e765268c..dff86157f 100644 --- a/agent/pkg/api/socket_routes.go +++ b/agent/pkg/api/socket_routes.go @@ -1,13 +1,18 @@ package api import ( + "encoding/json" "errors" + "fmt" + "mizuserver/pkg/models" "net/http" "sync" "time" "github.com/gin-gonic/gin" "github.com/gorilla/websocket" + basenine "github.com/up9inc/basenine/client/go" + "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/shared/debounce" "github.com/up9inc/mizu/shared/logger" ) @@ -39,17 +44,17 @@ func init() { connectedWebsockets = make(map[int]*SocketConnection, 0) } -func WebSocketRoutes(app *gin.Engine, eventHandlers EventHandlers) { +func WebSocketRoutes(app *gin.Engine, eventHandlers EventHandlers, startTime int64) { app.GET("/ws", func(c *gin.Context) { - websocketHandler(c.Writer, c.Request, eventHandlers, false) + websocketHandler(c.Writer, c.Request, eventHandlers, false, startTime) }) app.GET("/wsTapper", func(c *gin.Context) { - websocketHandler(c.Writer, c.Request, eventHandlers, true) + websocketHandler(c.Writer, c.Request, eventHandlers, true, startTime) }) } -func websocketHandler(w http.ResponseWriter, r *http.Request, eventHandlers EventHandlers, isTapper bool) { - conn, err := websocketUpgrader.Upgrade(w, r, nil) +func websocketHandler(w http.ResponseWriter, r *http.Request, eventHandlers EventHandlers, isTapper bool, startTime int64) { + ws, err := websocketUpgrader.Upgrade(w, r, nil) if err != nil { logger.Log.Errorf("Failed to set websocket upgrade: %v", err) return @@ -59,23 +64,103 @@ func websocketHandler(w http.ResponseWriter, r *http.Request, eventHandlers Even connectedWebsocketIdCounter++ socketId := connectedWebsocketIdCounter - connectedWebsockets[socketId] = &SocketConnection{connection: conn, lock: &sync.Mutex{}, eventHandlers: eventHandlers, isTapper: isTapper} + connectedWebsockets[socketId] = &SocketConnection{connection: ws, lock: &sync.Mutex{}, eventHandlers: eventHandlers, isTapper: isTapper} websocketIdsLock.Unlock() + var connection *basenine.Connection + var isQuerySet bool + + // `!isTapper` means it's a connection from the web UI + if !isTapper { + connection, err = basenine.NewConnection(shared.BasenineHost, shared.BaseninePort) + if err != nil { + panic(err) + } + } + + data := make(chan []byte) + meta := make(chan []byte) + defer func() { + data <- []byte(basenine.CloseChannel) + meta <- []byte(basenine.CloseChannel) + connection.Close() socketCleanup(socketId, connectedWebsockets[socketId]) }() eventHandlers.WebSocketConnect(socketId, isTapper) + startTimeBytes, _ := models.CreateWebsocketStartTimeMessage(startTime) + BroadcastToBrowserClients(startTimeBytes) + for { - _, msg, err := conn.ReadMessage() + _, msg, err := ws.ReadMessage() if err != nil { logger.Log.Errorf("Error reading message, socket id: %d, error: %v", socketId, err) break } - eventHandlers.WebSocketMessage(socketId, msg) + + if !isTapper && !isQuerySet { + query := string(msg) + err = basenine.Validate(shared.BasenineHost, shared.BaseninePort, query) + if err != nil { + toastBytes, _ := models.CreateWebsocketToastMessage(&models.ToastMessage{ + Type: "error", + AutoClose: 5000, + Text: fmt.Sprintf("Syntax error: %s", err.Error()), + }) + BroadcastToBrowserClients(toastBytes) + break + } + + isQuerySet = true + + handleDataChannel := func(c *basenine.Connection, data chan []byte) { + for { + bytes := <-data + + if string(bytes) == basenine.CloseChannel { + return + } + + var dataMap map[string]interface{} + err = json.Unmarshal(bytes, &dataMap) + + base := dataMap["base"].(map[string]interface{}) + base["id"] = uint(dataMap["id"].(float64)) + + baseEntryBytes, _ := models.CreateBaseEntryWebSocketMessage(base) + BroadcastToBrowserClients(baseEntryBytes) + } + } + + handleMetaChannel := func(c *basenine.Connection, meta chan []byte) { + for { + bytes := <-meta + + if string(bytes) == basenine.CloseChannel { + return + } + + var metadata *basenine.Metadata + err = json.Unmarshal(bytes, &metadata) + if err != nil { + logger.Log.Debugf("Error recieving metadata: %v\n", err.Error()) + } + + metadataBytes, _ := models.CreateWebsocketQueryMetadataMessage(metadata) + BroadcastToBrowserClients(metadataBytes) + } + } + + go handleDataChannel(connection, data) + go handleMetaChannel(connection, meta) + + connection.Query(query, data, meta) + } else { + eventHandlers.WebSocketMessage(socketId, msg) + } } } diff --git a/agent/pkg/controllers/entries_controller.go b/agent/pkg/controllers/entries_controller.go index 32422d5e4..8b900b23d 100644 --- a/agent/pkg/controllers/entries_controller.go +++ b/agent/pkg/controllers/entries_controller.go @@ -2,14 +2,17 @@ package controllers import ( "encoding/json" - "fmt" - "github.com/gin-gonic/gin" - tapApi "github.com/up9inc/mizu/tap/api" - "mizuserver/pkg/database" "mizuserver/pkg/models" "mizuserver/pkg/utils" - "mizuserver/pkg/validation" "net/http" + "strconv" + + "github.com/gin-gonic/gin" + + basenine "github.com/up9inc/basenine/client/go" + "github.com/up9inc/mizu/shared" + "github.com/up9inc/mizu/shared/logger" + tapApi "github.com/up9inc/mizu/tap/api" ) var extensionsMap map[string]*tapApi.Extension // global @@ -18,68 +21,36 @@ func InitExtensionsMap(ref map[string]*tapApi.Extension) { extensionsMap = ref } -func GetEntries(c *gin.Context) { - entriesFilter := &models.EntriesFilter{} - - if err := c.BindQuery(entriesFilter); err != nil { - c.JSON(http.StatusBadRequest, err) - } - err := validation.Validate(entriesFilter) +func Error(c *gin.Context, err error) bool { if err != nil { - c.JSON(http.StatusBadRequest, err) + logger.Log.Errorf("Error getting entry: %v", err) + c.Error(err) + c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": true, "msg": err.Error()}) + return true // signal that there was an error and the caller should return } - - order := database.OperatorToOrderMapping[entriesFilter.Operator] - operatorSymbol := database.OperatorToSymbolMapping[entriesFilter.Operator] - var entries []tapApi.MizuEntry - database.GetEntriesTable(). - Order(fmt.Sprintf("timestamp %s", order)). - Where(fmt.Sprintf("timestamp %s %v", operatorSymbol, entriesFilter.Timestamp)). - Limit(entriesFilter.Limit). - Find(&entries) - - if len(entries) > 0 && order == database.OrderDesc { - // the entries always order from oldest to newest - we should reverse - utils.ReverseSlice(entries) - } - - baseEntries := make([]tapApi.BaseEntryDetails, 0) - for _, entry := range entries { - baseEntryDetails := tapApi.BaseEntryDetails{} - if err := models.GetEntry(&entry, &baseEntryDetails); err != nil { - continue - } - - var pair tapApi.RequestResponsePair - json.Unmarshal([]byte(entry.Entry), &pair) - harEntry, err := utils.NewEntry(&pair) - if err == nil { - rules, _, _ := models.RunValidationRulesState(*harEntry, entry.Service) - baseEntryDetails.Rules = rules - } - - baseEntries = append(baseEntries, baseEntryDetails) - } - - c.JSON(http.StatusOK, baseEntries) + return false // no error, can continue } func GetEntry(c *gin.Context) { - var entryData tapApi.MizuEntry - database.GetEntriesTable(). - Where(map[string]string{"entryId": c.Param("entryId")}). - First(&entryData) + id, _ := strconv.Atoi(c.Param("id")) + var entry tapApi.MizuEntry + bytes, err := basenine.Single(shared.BasenineHost, shared.BaseninePort, id) + if Error(c, err) { + return // exit + } + err = json.Unmarshal(bytes, &entry) + if Error(c, err) { + return // exit + } - extension := extensionsMap[entryData.ProtocolName] - protocol, representation, bodySize, _ := extension.Dissector.Represent(&entryData) + extension := extensionsMap[entry.Protocol.Name] + protocol, representation, bodySize, _ := extension.Dissector.Represent(entry.Protocol, entry.Request, entry.Response) var rules []map[string]interface{} var isRulesEnabled bool - if entryData.ProtocolName == "http" { - var pair tapApi.RequestResponsePair - json.Unmarshal([]byte(entryData.Entry), &pair) - harEntry, _ := utils.NewEntry(&pair) - _, rulesMatched, _isRulesEnabled := models.RunValidationRulesState(*harEntry, entryData.Service) + if entry.Protocol.Name == "http" { + harEntry, _ := utils.NewEntry(entry.Request, entry.Response, entry.StartTime, entry.ElapsedTime) + _, rulesMatched, _isRulesEnabled := models.RunValidationRulesState(*harEntry, entry.Service) isRulesEnabled = _isRulesEnabled inrec, _ := json.Marshal(rulesMatched) json.Unmarshal(inrec, &rules) @@ -89,7 +60,7 @@ func GetEntry(c *gin.Context) { Protocol: protocol, Representation: string(representation), BodySize: bodySize, - Data: entryData, + Data: entry, Rules: rules, IsRulesEnabled: isRulesEnabled, }) diff --git a/agent/pkg/controllers/query_controller.go b/agent/pkg/controllers/query_controller.go new file mode 100644 index 000000000..f74230f8b --- /dev/null +++ b/agent/pkg/controllers/query_controller.go @@ -0,0 +1,31 @@ +package controllers + +import ( + "net/http" + + "github.com/gin-gonic/gin" + basenine "github.com/up9inc/basenine/client/go" + "github.com/up9inc/mizu/shared" +) + +type ValidateResponse struct { + Valid bool `json:"valid"` + Message string `json:"message"` +} + +func PostValidate(c *gin.Context) { + query := c.PostForm("query") + valid := true + message := "" + + err := basenine.Validate(shared.BasenineHost, shared.BaseninePort, query) + if err != nil { + valid = false + message = err.Error() + } + + c.JSON(http.StatusOK, ValidateResponse{ + Valid: valid, + Message: message, + }) +} diff --git a/agent/pkg/database/main.go b/agent/pkg/database/main.go deleted file mode 100644 index 6d199823c..000000000 --- a/agent/pkg/database/main.go +++ /dev/null @@ -1,78 +0,0 @@ -package database - -import ( - "fmt" - "mizuserver/pkg/utils" - "time" - - "gorm.io/driver/sqlite" - "gorm.io/gorm" - "gorm.io/gorm/logger" - - tapApi "github.com/up9inc/mizu/tap/api" -) - -const ( - OrderDesc = "desc" - OrderAsc = "asc" - LT = "lt" - GT = "gt" - TimeFormat = "2006-01-02 15:04:05.000000000" -) - -var ( - DB *gorm.DB - IsDBLocked = false - OperatorToSymbolMapping = map[string]string{ - LT: "<", - GT: ">", - } - OperatorToOrderMapping = map[string]string{ - LT: OrderDesc, - GT: OrderAsc, - } -) - -var DBPath string - -func GetEntriesTable() *gorm.DB { - return DB.Table("mizu_entries") -} - -func CreateEntry(entry *tapApi.MizuEntry) { - if IsDBLocked { - return - } - GetEntriesTable().Create(entry) -} - -func InitDataBase(databasePath string) *gorm.DB { - DBPath = databasePath - DB, _ = gorm.Open(sqlite.Open(databasePath), &gorm.Config{ - Logger: &utils.TruncatingLogger{LogLevel: logger.Warn, SlowThreshold: 500 * time.Millisecond}, - }) - _ = DB.AutoMigrate(&tapApi.MizuEntry{}) // this will ensure table is created - go StartEnforcingDatabaseSize() - return DB -} - -func GetEntriesFromDb(timeFrom time.Time, timeTo time.Time, protocolName *string) []tapApi.MizuEntry { - order := OrderDesc - protocolNameCondition := "1 = 1" - if protocolName != nil { - protocolNameCondition = fmt.Sprintf("protocolName = '%s'", *protocolName) - } - - var entries []tapApi.MizuEntry - GetEntriesTable(). - Where(protocolNameCondition). - Where(fmt.Sprintf("created_at BETWEEN '%s' AND '%s'", timeFrom.Format(TimeFormat), timeTo.Format(TimeFormat))). - Order(fmt.Sprintf("timestamp %s", order)). - Find(&entries) - - if len(entries) > 0 { - // the entries always order from oldest to newest so we should revers - utils.ReverseSlice(entries) - } - return entries -} diff --git a/agent/pkg/database/size_enforcer.go b/agent/pkg/database/size_enforcer.go deleted file mode 100644 index 7e06da7fe..000000000 --- a/agent/pkg/database/size_enforcer.go +++ /dev/null @@ -1,102 +0,0 @@ -package database - -import ( - "mizuserver/pkg/config" - "os" - "time" - - "github.com/fsnotify/fsnotify" - "github.com/up9inc/mizu/shared/debounce" - "github.com/up9inc/mizu/shared/logger" - "github.com/up9inc/mizu/shared/units" - tapApi "github.com/up9inc/mizu/tap/api" -) - -const percentageOfMaxSizeBytesToPrune = 15 - -func StartEnforcingDatabaseSize() { - watcher, err := fsnotify.NewWatcher() - if err != nil { - logger.Log.Fatalf("Error creating filesystem watcher for db size enforcement: %v\n", err) - return - } - - checkFileSizeDebouncer := debounce.NewDebouncer(5*time.Second, func() { - checkFileSize(config.Config.MaxDBSizeBytes) - }) - - go func() { - for { - select { - case event, ok := <-watcher.Events: - if !ok { - return // closed channel - } - if event.Op == fsnotify.Write { - checkFileSizeDebouncer.SetOn() - } - case err, ok := <-watcher.Errors: - if !ok { - return // closed channel - } - logger.Log.Errorf("filesystem watcher encountered error:%v", err) - } - } - }() - - err = watcher.Add(DBPath) - if err != nil { - logger.Log.Fatalf("Error adding %s to filesystem watcher for db size enforcement: %v\n", DBPath, err) - } -} - -func checkFileSize(maxSizeBytes int64) { - fileStat, err := os.Stat(DBPath) - if err != nil { - logger.Log.Errorf("Error checking %s file size: %v", DBPath, err) - } else { - if fileStat.Size() > maxSizeBytes { - pruneOldEntries(fileStat.Size()) - } - } -} - -func pruneOldEntries(currentFileSize int64) { - // sqlite locks the database while delete or VACUUM are running and sqlite is terrible at handling its own db lock while a lot of inserts are attempted, we prevent a significant bottleneck by handling the db lock ourselves here - IsDBLocked = true - defer func() { IsDBLocked = false }() - - amountOfBytesToTrim := currentFileSize / (100 / percentageOfMaxSizeBytesToPrune) - - rows, err := GetEntriesTable().Limit(10000).Order("id").Rows() - if err != nil { - logger.Log.Errorf("Error getting 10000 first db rows: %v", err) - return - } - - entryIdsToRemove := make([]uint, 0) - bytesToBeRemoved := int64(0) - for rows.Next() { - if bytesToBeRemoved >= amountOfBytesToTrim { - break - } - var entry tapApi.MizuEntry - err = DB.ScanRows(rows, &entry) - if err != nil { - logger.Log.Errorf("Error scanning db row: %v", err) - continue - } - - entryIdsToRemove = append(entryIdsToRemove, entry.ID) - bytesToBeRemoved += int64(entry.EstimatedSizeBytes) - } - - if len(entryIdsToRemove) > 0 { - GetEntriesTable().Where(entryIdsToRemove).Delete(tapApi.MizuEntry{}) - // VACUUM causes sqlite to shrink the db file after rows have been deleted, the db file will not shrink without this - DB.Exec("VACUUM") - logger.Log.Errorf("Removed %d rows and cleared %s", len(entryIdsToRemove), units.BytesToHumanReadable(bytesToBeRemoved)) - } else { - logger.Log.Error("Found no rows to remove when pruning") - } -} diff --git a/agent/pkg/models/models.go b/agent/pkg/models/models.go index 0209f3675..6add39eeb 100644 --- a/agent/pkg/models/models.go +++ b/agent/pkg/models/models.go @@ -2,12 +2,12 @@ package models import ( "encoding/json" + "mizuserver/pkg/rules" tapApi "github.com/up9inc/mizu/tap/api" - "mizuserver/pkg/rules" - "github.com/google/martian/har" + basenine "github.com/up9inc/basenine/client/go" "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/tap" ) @@ -16,15 +16,9 @@ func GetEntry(r *tapApi.MizuEntry, v tapApi.DataUnmarshaler) error { return v.UnmarshalData(r) } -type EntriesFilter struct { - Limit int `form:"limit" validate:"required,min=1,max=200"` - Operator string `form:"operator" validate:"required,oneof='lt' 'gt'"` - Timestamp int64 `form:"timestamp" validate:"required,min=1"` -} - type WebSocketEntryMessage struct { *shared.WebSocketMessageMetadata - Data *tapApi.BaseEntryDetails `json:"data,omitempty"` + Data map[string]interface{} `json:"data,omitempty"` } type WebSocketTappedEntryMessage struct { @@ -42,7 +36,28 @@ type AuthStatus struct { Model string `json:"model"` } -func CreateBaseEntryWebSocketMessage(base *tapApi.BaseEntryDetails) ([]byte, error) { +type ToastMessage struct { + Type string `json:"type"` + AutoClose uint `json:"autoClose"` + Text string `json:"text"` +} + +type WebSocketToastMessage struct { + *shared.WebSocketMessageMetadata + Data *ToastMessage `json:"data,omitempty"` +} + +type WebSocketQueryMetadataMessage struct { + *shared.WebSocketMessageMetadata + Data *basenine.Metadata `json:"data,omitempty"` +} + +type WebSocketStartTimeMessage struct { + *shared.WebSocketMessageMetadata + Data int64 `json:"data"` +} + +func CreateBaseEntryWebSocketMessage(base map[string]interface{}) ([]byte, error) { message := &WebSocketEntryMessage{ WebSocketMessageMetadata: &shared.WebSocketMessageMetadata{ MessageType: shared.WebSocketMessageTypeEntry, @@ -72,6 +87,36 @@ func CreateWebsocketOutboundLinkMessage(base *tap.OutboundLink) ([]byte, error) return json.Marshal(message) } +func CreateWebsocketToastMessage(base *ToastMessage) ([]byte, error) { + message := &WebSocketToastMessage{ + WebSocketMessageMetadata: &shared.WebSocketMessageMetadata{ + MessageType: shared.WebSocketMessageTypeToast, + }, + Data: base, + } + return json.Marshal(message) +} + +func CreateWebsocketQueryMetadataMessage(base *basenine.Metadata) ([]byte, error) { + message := &WebSocketQueryMetadataMessage{ + WebSocketMessageMetadata: &shared.WebSocketMessageMetadata{ + MessageType: shared.WebSocketMessageTypeQueryMetadata, + }, + Data: base, + } + return json.Marshal(message) +} + +func CreateWebsocketStartTimeMessage(base int64) ([]byte, error) { + message := &WebSocketStartTimeMessage{ + WebSocketMessageMetadata: &shared.WebSocketMessageMetadata{ + MessageType: shared.WebSocketMessageTypeStartTime, + }, + Data: base, + } + return json.Marshal(message) +} + // ExtendedHAR is the top level object of a HAR log. type ExtendedHAR struct { Log *ExtendedLog `json:"log"` diff --git a/agent/pkg/providers/stats_provider_test.go b/agent/pkg/providers/stats_provider_test.go index 4e0d3ff62..13acfece9 100644 --- a/agent/pkg/providers/stats_provider_test.go +++ b/agent/pkg/providers/stats_provider_test.go @@ -2,16 +2,10 @@ package providers_test import ( "fmt" - "mizuserver/pkg/config" - "mizuserver/pkg/database" "mizuserver/pkg/providers" "testing" ) -func init() { - database.InitDataBase(config.DefaultDatabasePath) -} - func TestNoEntryAddedCount(t *testing.T) { entriesStats := providers.GetGeneralStats() diff --git a/agent/pkg/routes/entries_routes.go b/agent/pkg/routes/entries_routes.go index 8e578fea4..12a597461 100644 --- a/agent/pkg/routes/entries_routes.go +++ b/agent/pkg/routes/entries_routes.go @@ -1,14 +1,14 @@ package routes import ( - "github.com/gin-gonic/gin" "mizuserver/pkg/controllers" + + "github.com/gin-gonic/gin" ) // EntriesRoutes defines the group of har entries routes. func EntriesRoutes(ginApp *gin.Engine) { routeGroup := ginApp.Group("/entries") - routeGroup.GET("/", controllers.GetEntries) // get entries (base/thin entries) - routeGroup.GET("/:entryId", controllers.GetEntry) // get single (full) entry + routeGroup.GET("/:id", controllers.GetEntry) // get single (full) entry } diff --git a/agent/pkg/routes/query_routes.go b/agent/pkg/routes/query_routes.go new file mode 100644 index 000000000..2807bd0b6 --- /dev/null +++ b/agent/pkg/routes/query_routes.go @@ -0,0 +1,13 @@ +package routes + +import ( + "mizuserver/pkg/controllers" + + "github.com/gin-gonic/gin" +) + +func QueryRoutes(ginApp *gin.Engine) { + routeGroup := ginApp.Group("/query") + + routeGroup.POST("/validate", controllers.PostValidate) +} diff --git a/agent/pkg/sensitiveDataFiltering/consts.go b/agent/pkg/sensitiveDataFiltering/consts.go deleted file mode 100644 index e5624de73..000000000 --- a/agent/pkg/sensitiveDataFiltering/consts.go +++ /dev/null @@ -1,10 +0,0 @@ -package sensitiveDataFiltering - -const maskedFieldPlaceholderValue = "[REDACTED]" - -//these values MUST be all lower case and contain no `-` or `_` characters -var personallyIdentifiableDataFields = []string{"token", "authorization", "authentication", "cookie", "userid", "password", - "username", "user", "key", "passcode", "pass", "auth", "authtoken", "jwt", - "bearer", "clientid", "clientsecret", "redirecturi", "phonenumber", - "zip", "zipcode", "address", "country", "firstname", "lastname", - "middlename", "fname", "lname", "birthdate"} diff --git a/agent/pkg/up9/main.go b/agent/pkg/up9/main.go index 6fa9304bd..49739bbac 100644 --- a/agent/pkg/up9/main.go +++ b/agent/pkg/up9/main.go @@ -7,15 +7,16 @@ import ( "encoding/json" "fmt" "io/ioutil" - "mizuserver/pkg/database" "mizuserver/pkg/utils" "net/http" "net/url" "regexp" "strings" + "sync" "time" "github.com/google/martian/har" + basenine "github.com/up9inc/basenine/client/go" "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/shared/logger" tapApi "github.com/up9inc/mizu/tap/api" @@ -23,6 +24,7 @@ import ( const ( AnalyzeCheckSleepTime = 5 * time.Second + SentCountLogInterval = 100 ) type GuestToken struct { @@ -204,44 +206,62 @@ func syncEntriesImpl(token string, model string, envPrefix string, uploadInterva analyzeInformation.AnalyzeDestination = envPrefix analyzeInformation.SentCount = 0 - sleepTime := time.Second * time.Duration(uploadIntervalSec) + // "http or grpc" filter indicates that we're only interested in HTTP and gRPC entries + query := "http or grpc" - var timeFrom time.Time - protocolFilter := "http" + logger.Log.Infof("Getting entries from the database\n") - for { - timeTo := time.Now() - logger.Log.Infof("Getting entries from %v, to %v\n", timeFrom.Format(time.RFC3339Nano), timeTo.Format(time.RFC3339Nano)) - entriesArray := database.GetEntriesFromDb(timeFrom, timeTo, &protocolFilter) + var connection *basenine.Connection + var err error + connection, err = basenine.NewConnection(shared.BasenineHost, shared.BaseninePort) + if err != nil { + panic(err) + } - if len(entriesArray) > 0 { - result := make([]har.Entry, 0) - for _, data := range entriesArray { - var pair tapApi.RequestResponsePair - if err := json.Unmarshal([]byte(data.Entry), &pair); err != nil { - continue - } - harEntry, err := utils.NewEntry(&pair) - if err != nil { - continue - } - if data.ResolvedSource != "" { - harEntry.Request.Headers = append(harEntry.Request.Headers, har.Header{Name: "x-mizu-source", Value: data.ResolvedSource}) - } - if data.ResolvedDestination != "" { - harEntry.Request.Headers = append(harEntry.Request.Headers, har.Header{Name: "x-mizu-destination", Value: data.ResolvedDestination}) - harEntry.Request.URL = utils.SetHostname(harEntry.Request.URL, data.ResolvedDestination) - } + data := make(chan []byte) + meta := make(chan []byte) - // go's default marshal behavior is to encode []byte fields to base64, python's default unmarshal behavior is to not decode []byte fields from base64 - if harEntry.Response.Content.Text, err = base64.StdEncoding.DecodeString(string(harEntry.Response.Content.Text)); err != nil { - continue - } + defer func() { + data <- []byte(basenine.CloseChannel) + meta <- []byte(basenine.CloseChannel) + connection.Close() + }() - result = append(result, *harEntry) + handleDataChannel := func(wg *sync.WaitGroup, connection *basenine.Connection, data chan []byte) { + defer wg.Done() + for { + dataBytes := <-data + + if string(dataBytes) == basenine.CloseChannel { + return } - logger.Log.Infof("About to upload %v entries\n", len(result)) + var dataMap map[string]interface{} + err = json.Unmarshal(dataBytes, &dataMap) + + result := make([]har.Entry, 0) + var entry tapApi.MizuEntry + if err := json.Unmarshal([]byte(dataBytes), &entry); err != nil { + continue + } + harEntry, err := utils.NewEntry(entry.Request, entry.Response, entry.StartTime, entry.ElapsedTime) + if err != nil { + continue + } + if entry.ResolvedSource != "" { + harEntry.Request.Headers = append(harEntry.Request.Headers, har.Header{Name: "x-mizu-source", Value: entry.ResolvedSource}) + } + if entry.ResolvedDestination != "" { + harEntry.Request.Headers = append(harEntry.Request.Headers, har.Header{Name: "x-mizu-destination", Value: entry.ResolvedDestination}) + harEntry.Request.URL = utils.SetHostname(harEntry.Request.URL, entry.ResolvedDestination) + } + + // go's default marshal behavior is to encode []byte fields to base64, python's default unmarshal behavior is to not decode []byte fields from base64 + if harEntry.Response.Content.Text, err = base64.StdEncoding.DecodeString(string(harEntry.Response.Content.Text)); err != nil { + continue + } + + result = append(result, *harEntry) body, jMarshalErr := json.Marshal(result) if jMarshalErr != nil { @@ -273,18 +293,33 @@ func syncEntriesImpl(token string, model string, envPrefix string, uploadInterva logger.Log.Info("Stopping sync entries") logger.Log.Fatal(postErr) } - analyzeInformation.SentCount += len(entriesArray) - logger.Log.Infof("Finish uploading %v entries to %s\n", len(entriesArray), GetTrafficDumpUrl(envPrefix, model)) + analyzeInformation.SentCount += 1 - logger.Log.Infof("Uploaded %v entries until now", analyzeInformation.SentCount) - } else { - logger.Log.Infof("Nothing to upload") + if analyzeInformation.SentCount%SentCountLogInterval == 0 { + logger.Log.Infof("Uploaded %v entries until now", analyzeInformation.SentCount) + } } - - logger.Log.Infof("Sleeping for %v...\n", sleepTime) - time.Sleep(sleepTime) - timeFrom = timeTo } + + handleMetaChannel := func(wg *sync.WaitGroup, connection *basenine.Connection, meta chan []byte) { + defer wg.Done() + for { + metaBytes := <-meta + + if string(metaBytes) == basenine.CloseChannel { + return + } + } + } + + var wg sync.WaitGroup + go handleDataChannel(&wg, connection, data) + go handleMetaChannel(&wg, connection, meta) + wg.Add(2) + + connection.Query(query, data, meta) + + wg.Wait() } func UpdateAnalyzeStatus(callback func(data []byte)) { diff --git a/agent/pkg/utils/har.go b/agent/pkg/utils/har.go index a1a06be78..108e91b7b 100644 --- a/agent/pkg/utils/har.go +++ b/agent/pkg/utils/har.go @@ -10,7 +10,6 @@ import ( "github.com/google/martian/har" "github.com/up9inc/mizu/shared/logger" - "github.com/up9inc/mizu/tap/api" ) // Keep it because we might want cookies in the future @@ -120,13 +119,11 @@ func BuildPostParams(rawParams []interface{}) []har.Param { return params } -func NewRequest(request *api.GenericMessage) (harRequest *har.Request, err error) { - reqDetails := request.Payload.(map[string]interface{})["details"].(map[string]interface{}) +func NewRequest(request map[string]interface{}) (harRequest *har.Request, err error) { + headers, host, scheme, authority, path, _ := BuildHeaders(request["_headers"].([]interface{})) + cookies := make([]har.Cookie, 0) // BuildCookies(request["_cookies"].([]interface{})) - headers, host, scheme, authority, path, _ := BuildHeaders(reqDetails["headers"].([]interface{})) - cookies := make([]har.Cookie, 0) // BuildCookies(reqDetails["cookies"].([]interface{})) - - postData, _ := reqDetails["postData"].(map[string]interface{}) + postData, _ := request["postData"].(map[string]interface{}) mimeType, _ := postData["mimeType"] if mimeType == nil || len(mimeType.(string)) == 0 { mimeType = "text/html" @@ -138,7 +135,7 @@ func NewRequest(request *api.GenericMessage) (harRequest *har.Request, err error } queryString := make([]har.QueryString, 0) - for _, _qs := range reqDetails["queryString"].([]interface{}) { + for _, _qs := range request["_queryString"].([]interface{}) { qs := _qs.(map[string]interface{}) queryString = append(queryString, har.QueryString{ Name: qs["name"].(string), @@ -146,7 +143,7 @@ func NewRequest(request *api.GenericMessage) (harRequest *har.Request, err error }) } - url := fmt.Sprintf("http://%s%s", host, reqDetails["url"].(string)) + url := fmt.Sprintf("http://%s%s", host, request["url"].(string)) if strings.HasPrefix(mimeType.(string), "application/grpc") { url = fmt.Sprintf("%s://%s%s", scheme, authority, path) } @@ -157,9 +154,9 @@ func NewRequest(request *api.GenericMessage) (harRequest *har.Request, err error } harRequest = &har.Request{ - Method: reqDetails["method"].(string), + Method: request["method"].(string), URL: url, - HTTPVersion: reqDetails["httpVersion"].(string), + HTTPVersion: request["httpVersion"].(string), HeadersSize: -1, BodySize: int64(bytes.NewBufferString(postDataText).Len()), QueryString: queryString, @@ -175,13 +172,11 @@ func NewRequest(request *api.GenericMessage) (harRequest *har.Request, err error return } -func NewResponse(response *api.GenericMessage) (harResponse *har.Response, err error) { - resDetails := response.Payload.(map[string]interface{})["details"].(map[string]interface{}) +func NewResponse(response map[string]interface{}) (harResponse *har.Response, err error) { + headers, _, _, _, _, _status := BuildHeaders(response["_headers"].([]interface{})) + cookies := make([]har.Cookie, 0) // BuildCookies(response["_cookies"].([]interface{})) - headers, _, _, _, _, _status := BuildHeaders(resDetails["headers"].([]interface{})) - cookies := make([]har.Cookie, 0) // BuildCookies(resDetails["cookies"].([]interface{})) - - content, _ := resDetails["content"].(map[string]interface{}) + content, _ := response["content"].(map[string]interface{}) mimeType, _ := content["mimeType"] if mimeType == nil || len(mimeType.(string)) == 0 { mimeType = "text/html" @@ -200,9 +195,11 @@ func NewResponse(response *api.GenericMessage) (harResponse *har.Response, err e Size: int64(len(bodyText)), } - status := int(resDetails["status"].(float64)) + status := int(response["status"].(float64)) if strings.HasPrefix(mimeType.(string), "application/grpc") { - status, err = strconv.Atoi(_status) + if _status != "" { + status, err = strconv.Atoi(_status) + } if err != nil { logger.Log.Errorf("Failed converting status to int %s (%v,%+v)", err, err, err) return nil, errors.New("failed converting response status to int for HAR") @@ -210,9 +207,9 @@ func NewResponse(response *api.GenericMessage) (harResponse *har.Response, err e } harResponse = &har.Response{ - HTTPVersion: resDetails["httpVersion"].(string), + HTTPVersion: response["httpVersion"].(string), Status: status, - StatusText: resDetails["statusText"].(string), + StatusText: response["statusText"].(string), HeadersSize: -1, BodySize: int64(bytes.NewBufferString(bodyText).Len()), Headers: headers, @@ -222,34 +219,33 @@ func NewResponse(response *api.GenericMessage) (harResponse *har.Response, err e return } -func NewEntry(pair *api.RequestResponsePair) (*har.Entry, error) { - harRequest, err := NewRequest(&pair.Request) +func NewEntry(request map[string]interface{}, response map[string]interface{}, startTime time.Time, elapsedTime int64) (*har.Entry, error) { + harRequest, err := NewRequest(request) if err != nil { logger.Log.Errorf("Failed converting request to HAR %s (%v,%+v)", err, err, err) return nil, errors.New("failed converting request to HAR") } - harResponse, err := NewResponse(&pair.Response) + harResponse, err := NewResponse(response) if err != nil { logger.Log.Errorf("Failed converting response to HAR %s (%v,%+v)", err, err, err) return nil, errors.New("failed converting response to HAR") } - totalTime := pair.Response.CaptureTime.Sub(pair.Request.CaptureTime).Round(time.Millisecond).Milliseconds() - if totalTime < 1 { - totalTime = 1 + if elapsedTime < 1 { + elapsedTime = 1 } harEntry := har.Entry{ - StartedDateTime: pair.Request.CaptureTime, - Time: totalTime, + StartedDateTime: startTime, + Time: elapsedTime, Request: harRequest, Response: harResponse, Cache: &har.Cache{}, Timings: &har.Timings{ Send: -1, Wait: -1, - Receive: totalTime, + Receive: elapsedTime, }, } diff --git a/agent/pkg/utils/truncating_logger.go b/agent/pkg/utils/truncating_logger.go deleted file mode 100644 index 8302d5512..000000000 --- a/agent/pkg/utils/truncating_logger.go +++ /dev/null @@ -1,60 +0,0 @@ -package utils - -import ( - "context" - "fmt" - "time" - - loggerShared "github.com/up9inc/mizu/shared/logger" - "gorm.io/gorm/logger" - "gorm.io/gorm/utils" -) - -// TruncatingLogger implements the gorm logger.Interface interface. Its purpose is to act as gorm's logger while truncating logs to a max of 50 characters to minimise the performance impact -type TruncatingLogger struct { - LogLevel logger.LogLevel - SlowThreshold time.Duration -} - -func (truncatingLogger *TruncatingLogger) LogMode(logLevel logger.LogLevel) logger.Interface { - truncatingLogger.LogLevel = logLevel - return truncatingLogger -} - -func (truncatingLogger *TruncatingLogger) Info(_ context.Context, message string, __ ...interface{}) { - if truncatingLogger.LogLevel < logger.Info { - return - } - loggerShared.Log.Errorf("gorm info: %.150s", message) -} - -func (truncatingLogger *TruncatingLogger) Warn(_ context.Context, message string, __ ...interface{}) { - if truncatingLogger.LogLevel < logger.Warn { - return - } - loggerShared.Log.Errorf("gorm warning: %.150s", message) -} - -func (truncatingLogger *TruncatingLogger) Error(_ context.Context, message string, __ ...interface{}) { - if truncatingLogger.LogLevel < logger.Error { - return - } - loggerShared.Log.Errorf("gorm error: %.150s", message) -} - -func (truncatingLogger *TruncatingLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) { - if truncatingLogger.LogLevel == logger.Silent { - return - } - elapsed := time.Since(begin) - if err != nil { - sql, rows := fc() // copied into every condition as this is a potentially heavy operation best done only when necessary - truncatingLogger.Error(ctx, fmt.Sprintf("Error in %s: %v - elapsed: %fs affected rows: %d, sql: %s", utils.FileWithLineNum(), err, elapsed.Seconds(), rows, sql)) - } else if truncatingLogger.LogLevel >= logger.Warn && elapsed > truncatingLogger.SlowThreshold { - sql, rows := fc() - truncatingLogger.Warn(ctx, fmt.Sprintf("Slow sql query - elapse: %fs rows: %d, sql: %s", elapsed.Seconds(), rows, sql)) - } else if truncatingLogger.LogLevel >= logger.Info { - sql, rows := fc() - truncatingLogger.Info(ctx, fmt.Sprintf("Sql query - elapse: %fs rows: %d, sql: %s", elapsed.Seconds(), rows, sql)) - } -} diff --git a/cli/config/config.go b/cli/config/config.go index 386f4816a..361d50068 100644 --- a/cli/config/config.go +++ b/cli/config/config.go @@ -3,14 +3,15 @@ package config import ( "errors" "fmt" - "github.com/up9inc/mizu/tap/api" "io/ioutil" - "k8s.io/apimachinery/pkg/util/json" "os" "reflect" "strconv" "strings" + "github.com/up9inc/mizu/tap/api" + "k8s.io/apimachinery/pkg/util/json" + "github.com/up9inc/mizu/shared" "github.com/up9inc/mizu/shared/logger" @@ -396,7 +397,7 @@ func getMizuAgentConfig(targetNamespaces []string, mizuApiFilteringOptions *api. TapperResources: Config.Tap.TapperResources, MizuResourcesNamespace: Config.MizuResourcesNamespace, MizuApiFilteringOptions: *mizuApiFilteringOptions, - AgentDatabasePath: fmt.Sprintf("%s%s", shared.DataDirPath, "entries.db"), + AgentDatabasePath: shared.DataDirPath, } return &config, nil } diff --git a/debug.Dockerfile b/debug.Dockerfile index 2c91746d5..d04e6b995 100644 --- a/debug.Dockerfile +++ b/debug.Dockerfile @@ -12,7 +12,7 @@ FROM golang:1.16-alpine AS builder # Set necessary environment variables needed for our image. ENV CGO_ENABLED=1 GOOS=linux GOARCH=amd64 -RUN apk add libpcap-dev gcc g++ make bash +RUN apk add libpcap-dev gcc g++ make bash perl-utils # Move to agent working directory (/agent-build). WORKDIR /app/agent-build @@ -23,7 +23,7 @@ COPY tap/go.mod tap/go.mod ../tap/ COPY tap/api/go.* ../tap/api/ RUN go mod download # cheap trick to make the build faster (As long as go.mod wasn't changes) -RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' -e 'sqlite' | xargs go get +RUN go list -f '{{.Path}}@{{.Version}}' -m all | sed 1d | grep -e 'go-cache' | xargs go get ARG COMMIT_HASH ARG GIT_BRANCH @@ -36,6 +36,12 @@ COPY tap ../tap COPY agent . RUN go build -gcflags="all=-N -l" -o mizuagent . +# Download Basenine executable, verify the sha1sum and move it to a directory in $PATH +ADD https://github.com/up9inc/basenine/releases/download/v0.2.6/basenine_linux_amd64 ./basenine_linux_amd64 +ADD https://github.com/up9inc/basenine/releases/download/v0.2.6/basenine_linux_amd64.sha256 ./basenine_linux_amd64.sha256 +RUN shasum -a 256 -c basenine_linux_amd64.sha256 +RUN chmod +x ./basenine_linux_amd64 + COPY devops/build_extensions_debug.sh .. RUN cd .. && /bin/bash build_extensions_debug.sh @@ -43,10 +49,12 @@ RUN cd .. && /bin/bash build_extensions_debug.sh FROM golang:1.16-alpine RUN apk add bash libpcap-dev tcpdump + WORKDIR /app # Copy binary and config files from /build to root folder of scratch container. COPY --from=builder ["/app/agent-build/mizuagent", "."] +COPY --from=builder ["/app/agent-build/basenine_linux_amd64", "/usr/local/bin/basenine"] COPY --from=builder ["/app/agent/build/extensions", "extensions"] COPY --from=site-build ["/app/ui-build/build", "site"] diff --git a/shared/consts.go b/shared/consts.go index 79e4b84e5..6dc3067b5 100644 --- a/shared/consts.go +++ b/shared/consts.go @@ -14,4 +14,6 @@ const ( GoGCEnvVar = "GOGC" DefaultApiServerPort = 8899 DebugModeEnvVar = "MIZU_DEBUG" + BasenineHost = "localhost" + BaseninePort = "9099" ) diff --git a/shared/models.go b/shared/models.go index 10b47883b..82b0abaef 100644 --- a/shared/models.go +++ b/shared/models.go @@ -17,6 +17,9 @@ const ( WebSocketMessageTypeUpdateStatus WebSocketMessageType = "status" WebSocketMessageTypeAnalyzeStatus WebSocketMessageType = "analyzeStatus" WebsocketMessageTypeOutboundLink WebSocketMessageType = "outboundLink" + WebSocketMessageTypeToast WebSocketMessageType = "toast" + WebSocketMessageTypeQueryMetadata WebSocketMessageType = "queryMetadata" + WebSocketMessageTypeStartTime WebSocketMessageType = "startTime" ) type Resources struct { diff --git a/tap/api/api.go b/tap/api/api.go index 5de34befe..3d1d552ef 100644 --- a/tap/api/api.go +++ b/tap/api/api.go @@ -18,7 +18,8 @@ import ( type Protocol struct { Name string `json:"name"` LongName string `json:"longName"` - Abbreviation string `json:"abbreviation"` + Abbreviation string `json:"abbr"` + Macro string `json:"macro"` Version string `json:"version"` BackgroundColor string `json:"backgroundColor"` ForegroundColor string `json:"foregroundColor"` @@ -28,6 +29,12 @@ type Protocol struct { Priority uint8 `json:"priority"` } +type TCP struct { + IP string `json:"ip"` + Port string `json:"port"` + Name string `json:"name"` +} + type Extension struct { Protocol *Protocol Path string @@ -74,6 +81,7 @@ type OutputChannelItem struct { Timestamp int64 ConnectionInfo *ConnectionInfo Pair *RequestResponsePair + Summary *BaseEntryDetails } type SuperTimer struct { @@ -89,9 +97,10 @@ type Dissector interface { Register(*Extension) Ping() Dissect(b *bufio.Reader, isClient bool, tcpID *TcpID, counterPair *CounterPair, superTimer *SuperTimer, superIdentifier *SuperIdentifier, emitter Emitter, options *TrafficFilteringOptions) error - Analyze(item *OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *MizuEntry + Analyze(item *OutputChannelItem, resolvedSource string, resolvedDestination string) *MizuEntry Summarize(entry *MizuEntry) *BaseEntryDetails - Represent(entry *MizuEntry) (protocol Protocol, object []byte, bodySize int64, err error) + Represent(pIn Protocol, request map[string]interface{}, response map[string]interface{}) (pOut Protocol, object []byte, bodySize int64, err error) + Macros() map[string]string } type Emitting struct { @@ -109,39 +118,36 @@ func (e *Emitting) Emit(item *OutputChannelItem) { } type MizuEntry struct { - ID uint `gorm:"primarykey"` - CreatedAt time.Time - UpdatedAt time.Time - ProtocolName string `json:"protocolName" gorm:"column:protocolName"` - ProtocolLongName string `json:"protocolLongName" gorm:"column:protocolLongName"` - ProtocolAbbreviation string `json:"protocolAbbreviation" gorm:"column:protocolAbbreviation"` - ProtocolVersion string `json:"protocolVersion" gorm:"column:protocolVersion"` - ProtocolBackgroundColor string `json:"protocolBackgroundColor" gorm:"column:protocolBackgroundColor"` - ProtocolForegroundColor string `json:"protocolForegroundColor" gorm:"column:protocolForegroundColor"` - ProtocolFontSize int8 `json:"protocolFontSize" gorm:"column:protocolFontSize"` - ProtocolReferenceLink string `json:"protocolReferenceLink" gorm:"column:protocolReferenceLink"` - Entry string `json:"entry,omitempty" gorm:"column:entry"` - EntryId string `json:"entryId" gorm:"column:entryId"` - Url string `json:"url" gorm:"column:url"` - Method string `json:"method" gorm:"column:method"` - Status int `json:"status" gorm:"column:status"` - RequestSenderIp string `json:"requestSenderIp" gorm:"column:requestSenderIp"` - Service string `json:"service" gorm:"column:service"` - Timestamp int64 `json:"timestamp" gorm:"column:timestamp"` - ElapsedTime int64 `json:"elapsedTime" gorm:"column:elapsedTime"` - Path string `json:"path" gorm:"column:path"` - ResolvedSource string `json:"resolvedSource,omitempty" gorm:"column:resolvedSource"` - ResolvedDestination string `json:"resolvedDestination,omitempty" gorm:"column:resolvedDestination"` - SourceIp string `json:"sourceIp,omitempty" gorm:"column:sourceIp"` - DestinationIp string `json:"destinationIp,omitempty" gorm:"column:destinationIp"` - SourcePort string `json:"sourcePort,omitempty" gorm:"column:sourcePort"` - DestinationPort string `json:"destinationPort,omitempty" gorm:"column:destinationPort"` - IsOutgoing bool `json:"isOutgoing,omitempty" gorm:"column:isOutgoing"` - ContractStatus ContractStatus `json:"contractStatus,omitempty" gorm:"column:contractStatus"` - ContractRequestReason string `json:"contractRequestReason,omitempty" gorm:"column:contractRequestReason"` - ContractResponseReason string `json:"contractResponseReason,omitempty" gorm:"column:contractResponseReason"` - ContractContent string `json:"contractContent,omitempty" gorm:"column:contractContent"` - EstimatedSizeBytes int `json:"-" gorm:"column:estimatedSizeBytes"` + Id uint `json:"id"` + Protocol Protocol `json:"proto"` + Source *TCP `json:"src"` + Destination *TCP `json:"dst"` + Outgoing bool `json:"outgoing"` + Timestamp int64 `json:"timestamp"` + StartTime time.Time `json:"startTime"` + Request map[string]interface{} `json:"request"` + Response map[string]interface{} `json:"response"` + Base *BaseEntryDetails `json:"base"` + Summary string `json:"summary"` + Url string `json:"url"` + Method string `json:"method"` + Status int `json:"status"` + RequestSenderIp string `json:"requestSenderIp"` + Service string `json:"service"` + ElapsedTime int64 `json:"elapsedTime"` + Path string `json:"path"` + ResolvedSource string `json:"resolvedSource,omitempty"` + ResolvedDestination string `json:"resolvedDestination,omitempty"` + SourceIp string `json:"sourceIp,omitempty"` + DestinationIp string `json:"destinationIp,omitempty"` + SourcePort string `json:"sourcePort,omitempty"` + DestinationPort string `json:"destinationPort,omitempty"` + IsOutgoing bool `json:"isOutgoing,omitempty"` + ContractStatus ContractStatus `json:"contractStatus,omitempty"` + ContractRequestReason string `json:"contractRequestReason,omitempty"` + ContractResponseReason string `json:"contractResponseReason,omitempty"` + ContractContent string `json:"contractContent,omitempty"` + HTTPPair string `json:"httpPair,omitempty"` } type MizuEntryWrapper struct { @@ -154,7 +160,7 @@ type MizuEntryWrapper struct { } type BaseEntryDetails struct { - Id string `json:"id,omitempty"` + Id uint `json:"id"` Protocol Protocol `json:"protocol,omitempty"` Url string `json:"url,omitempty"` RequestSenderIp string `json:"requestSenderIp,omitempty"` @@ -194,17 +200,8 @@ type DataUnmarshaler interface { } func (bed *BaseEntryDetails) UnmarshalData(entry *MizuEntry) error { - bed.Protocol = Protocol{ - Name: entry.ProtocolName, - LongName: entry.ProtocolLongName, - Abbreviation: entry.ProtocolAbbreviation, - Version: entry.ProtocolVersion, - BackgroundColor: entry.ProtocolBackgroundColor, - ForegroundColor: entry.ProtocolForegroundColor, - FontSize: entry.ProtocolFontSize, - ReferenceLink: entry.ProtocolReferenceLink, - } - bed.Id = entry.EntryId + bed.Protocol = entry.Protocol + bed.Id = entry.Id bed.Url = entry.Url bed.RequestSenderIp = entry.RequestSenderIp bed.Service = entry.Service @@ -228,6 +225,21 @@ const ( BODY string = "body" ) +type SectionData struct { + Type string `json:"type"` + Title string `json:"title"` + Data string `json:"data"` + Encoding string `json:"encoding,omitempty"` + MimeType string `json:"mimeType,omitempty"` + Selector string `json:"selector,omitempty"` +} + +type TableData struct { + Name string `json:"name"` + Value interface{} `json:"value"` + Selector string `json:"selector"` +} + const ( TypeHttpRequest = iota TypeHttpResponse diff --git a/tap/extensions/amqp/helpers.go b/tap/extensions/amqp/helpers.go index f7a1842a7..188ae18a1 100644 --- a/tap/extensions/amqp/helpers.go +++ b/tap/extensions/amqp/helpers.go @@ -131,97 +131,109 @@ func representProperties(properties map[string]interface{}, rep []interface{}) ( userId := "" appId := "" - if properties["ContentType"] != nil { - contentType = properties["ContentType"].(string) + if properties["contentType"] != nil { + contentType = properties["contentType"].(string) } - if properties["ContentEncoding"] != nil { - contentEncoding = properties["ContentEncoding"].(string) + if properties["contentEncoding"] != nil { + contentEncoding = properties["contentEncoding"].(string) } - if properties["Delivery Mode"] != nil { - deliveryMode = fmt.Sprintf("%g", properties["DeliveryMode"].(float64)) + if properties["deliveryMode"] != nil { + deliveryMode = fmt.Sprintf("%g", properties["deliveryMode"].(float64)) } - if properties["Priority"] != nil { - priority = fmt.Sprintf("%g", properties["Priority"].(float64)) + if properties["priority"] != nil { + priority = fmt.Sprintf("%g", properties["priority"].(float64)) } - if properties["CorrelationId"] != nil { - correlationId = properties["CorrelationId"].(string) + if properties["correlationId"] != nil { + correlationId = properties["correlationId"].(string) } - if properties["ReplyTo"] != nil { - replyTo = properties["ReplyTo"].(string) + if properties["replyTo"] != nil { + replyTo = properties["replyTo"].(string) } - if properties["Expiration"] != nil { - expiration = properties["Expiration"].(string) + if properties["expiration"] != nil { + expiration = properties["expiration"].(string) } - if properties["MessageId"] != nil { - messageId = properties["MessageId"].(string) + if properties["messageId"] != nil { + messageId = properties["messageId"].(string) } - if properties["Timestamp"] != nil { - timestamp = properties["Timestamp"].(string) + if properties["timestamp"] != nil { + timestamp = properties["timestamp"].(string) } - if properties["Type"] != nil { - _type = properties["Type"].(string) + if properties["type"] != nil { + _type = properties["type"].(string) } - if properties["UserId"] != nil { - userId = properties["UserId"].(string) + if properties["userId"] != nil { + userId = properties["userId"].(string) } - if properties["AppId"] != nil { - appId = properties["AppId"].(string) + if properties["appId"] != nil { + appId = properties["appId"].(string) } - props, _ := json.Marshal([]map[string]string{ + props, _ := json.Marshal([]api.TableData{ { - "name": "Content Type", - "value": contentType, + Name: "Content Type", + Value: contentType, + Selector: `request.properties.contentType`, }, { - "name": "Content Encoding", - "value": contentEncoding, + Name: "Content Encoding", + Value: contentEncoding, + Selector: `request.properties.contentEncoding`, }, { - "name": "Delivery Mode", - "value": deliveryMode, + Name: "Delivery Mode", + Value: deliveryMode, + Selector: `request.properties.deliveryMode`, }, { - "name": "Priority", - "value": priority, + Name: "Priority", + Value: priority, + Selector: `request.properties.priority`, }, { - "name": "Correlation ID", - "value": correlationId, + Name: "Correlation ID", + Value: correlationId, + Selector: `request.properties.correlationId`, }, { - "name": "Reply To", - "value": replyTo, + Name: "Reply To", + Value: replyTo, + Selector: `request.properties.replyTo`, }, { - "name": "Expiration", - "value": expiration, + Name: "Expiration", + Value: expiration, + Selector: `request.properties.expiration`, }, { - "name": "Message ID", - "value": messageId, + Name: "Message ID", + Value: messageId, + Selector: `request.properties.messageId`, }, { - "name": "Timestamp", - "value": timestamp, + Name: "Timestamp", + Value: timestamp, + Selector: `request.properties.timestamp`, }, { - "name": "Type", - "value": _type, + Name: "Type", + Value: _type, + Selector: `request.properties.type`, }, { - "name": "User ID", - "value": userId, + Name: "User ID", + Value: userId, + Selector: `request.properties.userId`, }, { - "name": "App ID", - "value": appId, + Name: "App ID", + Value: appId, + Selector: `request.properties.appId`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Properties", - "data": string(props), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Properties", + Data: string(props), }) return rep, contentType, contentEncoding @@ -230,56 +242,62 @@ func representProperties(properties map[string]interface{}, rep []interface{}) ( func representBasicPublish(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Exchange", - "value": event["Exchange"].(string), + Name: "Exchange", + Value: event["exchange"].(string), + Selector: `request.exchange`, }, { - "name": "Routing Key", - "value": event["RoutingKey"].(string), + Name: "Routing Key", + Value: event["routingKey"].(string), + Selector: `request.routingKey`, }, { - "name": "Mandatory", - "value": strconv.FormatBool(event["Mandatory"].(bool)), + Name: "Mandatory", + Value: strconv.FormatBool(event["mandatory"].(bool)), + Selector: `request.mandatory`, }, { - "name": "Immediate", - "value": strconv.FormatBool(event["Immediate"].(bool)), + Name: "Immediate", + Value: strconv.FormatBool(event["immediate"].(bool)), + Selector: `request.immediate`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - properties := event["Properties"].(map[string]interface{}) + properties := event["properties"].(map[string]interface{}) rep, contentType, _ := representProperties(properties, rep) - if properties["Headers"] != nil { - headers := make([]map[string]string, 0) - for name, value := range properties["Headers"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if properties["headers"] != nil { + headers := make([]api.TableData, 0) + for name, value := range properties["headers"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.properties.headers["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Headers", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Headers", + Data: string(headersMarshaled), }) } - if event["Body"] != nil { - rep = append(rep, map[string]string{ - "type": api.BODY, - "title": "Body", - "encoding": "base64", - "mime_type": contentType, - "data": event["Body"].(string), + if event["body"] != nil { + rep = append(rep, api.SectionData{ + Type: api.BODY, + Title: "Body", + Encoding: "base64", + MimeType: contentType, + Data: event["body"].(string), + Selector: `request.body`, }) } @@ -293,70 +311,77 @@ func representBasicDeliver(event map[string]interface{}) []interface{} { deliveryTag := "" redelivered := "" - if event["ConsumerTag"] != nil { - consumerTag = event["ConsumerTag"].(string) + if event["consumerTag"] != nil { + consumerTag = event["consumerTag"].(string) } - if event["DeliveryTag"] != nil { - deliveryTag = fmt.Sprintf("%g", event["DeliveryTag"].(float64)) + if event["deliveryTag"] != nil { + deliveryTag = fmt.Sprintf("%g", event["deliveryTag"].(float64)) } - if event["Redelivered"] != nil { - redelivered = strconv.FormatBool(event["Redelivered"].(bool)) + if event["redelivered"] != nil { + redelivered = strconv.FormatBool(event["redelivered"].(bool)) } - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Consumer Tag", - "value": consumerTag, + Name: "Consumer Tag", + Value: consumerTag, + Selector: `request.consumerTag`, }, { - "name": "Delivery Tag", - "value": deliveryTag, + Name: "Delivery Tag", + Value: deliveryTag, + Selector: `request.deliveryTag`, }, { - "name": "Redelivered", - "value": redelivered, + Name: "Redelivered", + Value: redelivered, + Selector: `request.redelivered`, }, { - "name": "Exchange", - "value": event["Exchange"].(string), + Name: "Exchange", + Value: event["exchange"].(string), + Selector: `request.exchange`, }, { - "name": "Routing Key", - "value": event["RoutingKey"].(string), + Name: "Routing Key", + Value: event["routingKey"].(string), + Selector: `request.routingKey`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - properties := event["Properties"].(map[string]interface{}) + properties := event["properties"].(map[string]interface{}) rep, contentType, _ := representProperties(properties, rep) - if properties["Headers"] != nil { - headers := make([]map[string]string, 0) - for name, value := range properties["Headers"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if properties["headers"] != nil { + headers := make([]api.TableData, 0) + for name, value := range properties["headers"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.properties.headers["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Headers", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Headers", + Data: string(headersMarshaled), }) } - if event["Body"] != nil { - rep = append(rep, map[string]string{ - "type": api.BODY, - "title": "Body", - "encoding": "base64", - "mime_type": contentType, - "data": event["Body"].(string), + if event["body"] != nil { + rep = append(rep, api.SectionData{ + Type: api.BODY, + Title: "Body", + Encoding: "base64", + MimeType: contentType, + Data: event["body"].(string), + Selector: `request.body`, }) } @@ -366,51 +391,58 @@ func representBasicDeliver(event map[string]interface{}) []interface{} { func representQueueDeclare(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Queue", - "value": event["Queue"].(string), + Name: "Queue", + Value: event["queue"].(string), + Selector: `request.queue`, }, { - "name": "Passive", - "value": strconv.FormatBool(event["Passive"].(bool)), + Name: "Passive", + Value: strconv.FormatBool(event["passive"].(bool)), + Selector: `request.queue`, }, { - "name": "Durable", - "value": strconv.FormatBool(event["Durable"].(bool)), + Name: "Durable", + Value: strconv.FormatBool(event["durable"].(bool)), + Selector: `request.durable`, }, { - "name": "Exclusive", - "value": strconv.FormatBool(event["Exclusive"].(bool)), + Name: "Exclusive", + Value: strconv.FormatBool(event["exclusive"].(bool)), + Selector: `request.exclusive`, }, { - "name": "Auto Delete", - "value": strconv.FormatBool(event["AutoDelete"].(bool)), + Name: "Auto Delete", + Value: strconv.FormatBool(event["autoDelete"].(bool)), + Selector: `request.autoDelete`, }, { - "name": "NoWait", - "value": strconv.FormatBool(event["NoWait"].(bool)), + Name: "NoWait", + Value: strconv.FormatBool(event["noWait"].(bool)), + Selector: `request.noWait`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - if event["Arguments"] != nil { - headers := make([]map[string]string, 0) - for name, value := range event["Arguments"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if event["arguments"] != nil { + headers := make([]api.TableData, 0) + for name, value := range event["arguments"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.arguments["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Arguments", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Arguments", + Data: string(headersMarshaled), }) } @@ -420,55 +452,63 @@ func representQueueDeclare(event map[string]interface{}) []interface{} { func representExchangeDeclare(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Exchange", - "value": event["Exchange"].(string), + Name: "Exchange", + Value: event["exchange"].(string), + Selector: `request.exchange`, }, { - "name": "Type", - "value": event["Type"].(string), + Name: "Type", + Value: event["type"].(string), + Selector: `request.type`, }, { - "name": "Passive", - "value": strconv.FormatBool(event["Passive"].(bool)), + Name: "Passive", + Value: strconv.FormatBool(event["passive"].(bool)), + Selector: `request.passive`, }, { - "name": "Durable", - "value": strconv.FormatBool(event["Durable"].(bool)), + Name: "Durable", + Value: strconv.FormatBool(event["durable"].(bool)), + Selector: `request.durable`, }, { - "name": "Auto Delete", - "value": strconv.FormatBool(event["AutoDelete"].(bool)), + Name: "Auto Delete", + Value: strconv.FormatBool(event["autoDelete"].(bool)), + Selector: `request.autoDelete`, }, { - "name": "Internal", - "value": strconv.FormatBool(event["Internal"].(bool)), + Name: "Internal", + Value: strconv.FormatBool(event["internal"].(bool)), + Selector: `request.internal`, }, { - "name": "NoWait", - "value": strconv.FormatBool(event["NoWait"].(bool)), + Name: "NoWait", + Value: strconv.FormatBool(event["noWait"].(bool)), + Selector: `request.noWait`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - if event["Arguments"] != nil { - headers := make([]map[string]string, 0) - for name, value := range event["Arguments"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if event["arguments"] != nil { + headers := make([]api.TableData, 0) + for name, value := range event["arguments"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.arguments["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Arguments", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Arguments", + Data: string(headersMarshaled), }) } @@ -478,33 +518,37 @@ func representExchangeDeclare(event map[string]interface{}) []interface{} { func representConnectionStart(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Version Major", - "value": fmt.Sprintf("%g", event["VersionMajor"].(float64)), + Name: "Version Major", + Value: fmt.Sprintf("%g", event["versionMajor"].(float64)), + Selector: `request.versionMajor`, }, { - "name": "Version Minor", - "value": fmt.Sprintf("%g", event["VersionMinor"].(float64)), + Name: "Version Minor", + Value: fmt.Sprintf("%g", event["versionMinor"].(float64)), + Selector: `request.versionMinor`, }, { - "name": "Mechanisms", - "value": event["Mechanisms"].(string), + Name: "Mechanisms", + Value: event["mechanisms"].(string), + Selector: `request.mechanisms`, }, { - "name": "Locales", - "value": event["Locales"].(string), + Name: "Locales", + Value: event["locales"].(string), + Selector: `request.locales`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - if event["ServerProperties"] != nil { - headers := make([]map[string]string, 0) - for name, value := range event["ServerProperties"].(map[string]interface{}) { + if event["serverProperties"] != nil { + headers := make([]api.TableData, 0) + for name, value := range event["serverProperties"].(map[string]interface{}) { var outcome string switch value.(type) { case string: @@ -517,16 +561,17 @@ func representConnectionStart(event map[string]interface{}) []interface{} { default: panic("Unknown data type for the server property!") } - headers = append(headers, map[string]string{ - "name": name, - "value": outcome, + headers = append(headers, api.TableData{ + Name: name, + Value: outcome, + Selector: fmt.Sprintf(`request.serverProperties["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Server Properties", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Server Properties", + Data: string(headersMarshaled), }) } @@ -536,28 +581,32 @@ func representConnectionStart(event map[string]interface{}) []interface{} { func representConnectionClose(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Reply Code", - "value": fmt.Sprintf("%g", event["ReplyCode"].(float64)), + Name: "Reply Code", + Value: fmt.Sprintf("%g", event["replyCode"].(float64)), + Selector: `request.replyCode`, }, { - "name": "Reply Text", - "value": event["ReplyText"].(string), + Name: "Reply Text", + Value: event["replyText"].(string), + Selector: `request.replyText`, }, { - "name": "Class ID", - "value": fmt.Sprintf("%g", event["ClassId"].(float64)), + Name: "Class ID", + Value: fmt.Sprintf("%g", event["classId"].(float64)), + Selector: `request.classId`, }, { - "name": "Method ID", - "value": fmt.Sprintf("%g", event["MethodId"].(float64)), + Name: "Method ID", + Value: fmt.Sprintf("%g", event["methodId"].(float64)), + Selector: `request.methodId`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) return rep @@ -566,43 +615,48 @@ func representConnectionClose(event map[string]interface{}) []interface{} { func representQueueBind(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Queue", - "value": event["Queue"].(string), + Name: "Queue", + Value: event["queue"].(string), + Selector: `request.queue`, }, { - "name": "Exchange", - "value": event["Exchange"].(string), + Name: "Exchange", + Value: event["exchange"].(string), + Selector: `request.exchange`, }, { - "name": "RoutingKey", - "value": event["RoutingKey"].(string), + Name: "RoutingKey", + Value: event["routingKey"].(string), + Selector: `request.routingKey`, }, { - "name": "NoWait", - "value": strconv.FormatBool(event["NoWait"].(bool)), + Name: "NoWait", + Value: strconv.FormatBool(event["noWait"].(bool)), + Selector: `request.noWait`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - if event["Arguments"] != nil { - headers := make([]map[string]string, 0) - for name, value := range event["Arguments"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if event["arguments"] != nil { + headers := make([]api.TableData, 0) + for name, value := range event["arguments"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.arguments["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Arguments", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Arguments", + Data: string(headersMarshaled), }) } @@ -612,51 +666,58 @@ func representQueueBind(event map[string]interface{}) []interface{} { func representBasicConsume(event map[string]interface{}) []interface{} { rep := make([]interface{}, 0) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Queue", - "value": event["Queue"].(string), + Name: "Queue", + Value: event["queue"].(string), + Selector: `request.queue`, }, { - "name": "Consumer Tag", - "value": event["ConsumerTag"].(string), + Name: "Consumer Tag", + Value: event["consumerTag"].(string), + Selector: `request.consumerTag`, }, { - "name": "No Local", - "value": strconv.FormatBool(event["NoLocal"].(bool)), + Name: "No Local", + Value: strconv.FormatBool(event["noLocal"].(bool)), + Selector: `request.noLocal`, }, { - "name": "No Ack", - "value": strconv.FormatBool(event["NoAck"].(bool)), + Name: "No Ack", + Value: strconv.FormatBool(event["noAck"].(bool)), + Selector: `request.noAck`, }, { - "name": "Exclusive", - "value": strconv.FormatBool(event["Exclusive"].(bool)), + Name: "Exclusive", + Value: strconv.FormatBool(event["exclusive"].(bool)), + Selector: `request.exclusive`, }, { - "name": "NoWait", - "value": strconv.FormatBool(event["NoWait"].(bool)), + Name: "NoWait", + Value: strconv.FormatBool(event["noWait"].(bool)), + Selector: `request.noWait`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - if event["Arguments"] != nil { - headers := make([]map[string]string, 0) - for name, value := range event["Arguments"].(map[string]interface{}) { - headers = append(headers, map[string]string{ - "name": name, - "value": value.(string), + if event["arguments"] != nil { + headers := make([]api.TableData, 0) + for name, value := range event["arguments"].(map[string]interface{}) { + headers = append(headers, api.TableData{ + Name: name, + Value: value.(string), + Selector: fmt.Sprintf(`request.arguments["%s"]`, name), }) } headersMarshaled, _ := json.Marshal(headers) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Arguments", - "data": string(headersMarshaled), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Arguments", + Data: string(headersMarshaled), }) } diff --git a/tap/extensions/amqp/main.go b/tap/extensions/amqp/main.go index 133e8997a..dc2bbf9bc 100644 --- a/tap/extensions/amqp/main.go +++ b/tap/extensions/amqp/main.go @@ -16,6 +16,7 @@ var protocol api.Protocol = api.Protocol{ Name: "amqp", LongName: "Advanced Message Queuing Protocol 0-9-1", Abbreviation: "AMQP", + Macro: "amqp", Version: "0-9-1", BackgroundColor: "#ff6600", ForegroundColor: "#ffffff", @@ -222,7 +223,7 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co } } -func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry { +func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry { request := item.Pair.Request.Payload.(map[string]interface{}) reqDetails := request["details"].(map[string]interface{}) service := "amqp" @@ -235,75 +236,79 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve summary := "" switch request["method"] { case basicMethodMap[40]: - summary = reqDetails["Exchange"].(string) + summary = reqDetails["exchange"].(string) break case basicMethodMap[60]: - summary = reqDetails["Exchange"].(string) + summary = reqDetails["exchange"].(string) break case exchangeMethodMap[10]: - summary = reqDetails["Exchange"].(string) + summary = reqDetails["exchange"].(string) break case queueMethodMap[10]: - summary = reqDetails["Queue"].(string) + summary = reqDetails["queue"].(string) break case connectionMethodMap[10]: summary = fmt.Sprintf( "%s.%s", - strconv.Itoa(int(reqDetails["VersionMajor"].(float64))), - strconv.Itoa(int(reqDetails["VersionMinor"].(float64))), + strconv.Itoa(int(reqDetails["versionMajor"].(float64))), + strconv.Itoa(int(reqDetails["versionMinor"].(float64))), ) break case connectionMethodMap[50]: - summary = reqDetails["ReplyText"].(string) + summary = reqDetails["replyText"].(string) break case queueMethodMap[20]: - summary = reqDetails["Queue"].(string) + summary = reqDetails["queue"].(string) break case basicMethodMap[20]: - summary = reqDetails["Queue"].(string) + summary = reqDetails["queue"].(string) break } request["url"] = summary - entryBytes, _ := json.Marshal(item.Pair) + reqDetails["method"] = request["method"] return &api.MizuEntry{ - ProtocolName: protocol.Name, - ProtocolLongName: protocol.LongName, - ProtocolAbbreviation: protocol.Abbreviation, - ProtocolVersion: protocol.Version, - ProtocolBackgroundColor: protocol.BackgroundColor, - ProtocolForegroundColor: protocol.ForegroundColor, - ProtocolFontSize: protocol.FontSize, - ProtocolReferenceLink: protocol.ReferenceLink, - EntryId: entryId, - Entry: string(entryBytes), - Url: fmt.Sprintf("%s%s", service, summary), - Method: request["method"].(string), - Status: 0, - RequestSenderIp: item.ConnectionInfo.ClientIP, - Service: service, - Timestamp: item.Timestamp, - ElapsedTime: 0, - Path: summary, - ResolvedSource: resolvedSource, - ResolvedDestination: resolvedDestination, - SourceIp: item.ConnectionInfo.ClientIP, - DestinationIp: item.ConnectionInfo.ServerIP, - SourcePort: item.ConnectionInfo.ClientPort, - DestinationPort: item.ConnectionInfo.ServerPort, - IsOutgoing: item.ConnectionInfo.IsOutgoing, + Protocol: protocol, + Source: &api.TCP{ + Name: resolvedSource, + IP: item.ConnectionInfo.ClientIP, + Port: item.ConnectionInfo.ClientPort, + }, + Destination: &api.TCP{ + Name: resolvedDestination, + IP: item.ConnectionInfo.ServerIP, + Port: item.ConnectionInfo.ServerPort, + }, + Outgoing: item.ConnectionInfo.IsOutgoing, + Request: reqDetails, + Url: fmt.Sprintf("%s%s", service, summary), + Method: request["method"].(string), + Status: 0, + RequestSenderIp: item.ConnectionInfo.ClientIP, + Service: service, + Timestamp: item.Timestamp, + StartTime: item.Pair.Request.CaptureTime, + ElapsedTime: 0, + Summary: summary, + ResolvedSource: resolvedSource, + ResolvedDestination: resolvedDestination, + SourceIp: item.ConnectionInfo.ClientIP, + DestinationIp: item.ConnectionInfo.ServerIP, + SourcePort: item.ConnectionInfo.ClientPort, + DestinationPort: item.ConnectionInfo.ServerPort, + IsOutgoing: item.ConnectionInfo.IsOutgoing, } } func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { return &api.BaseEntryDetails{ - Id: entry.EntryId, + Id: entry.Id, Protocol: protocol, Url: entry.Url, RequestSenderIp: entry.RequestSenderIp, Service: entry.Service, - Summary: entry.Path, + Summary: entry.Summary, StatusCode: entry.Status, Method: entry.Method, Timestamp: entry.Timestamp, @@ -320,39 +325,35 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { } } -func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) { - p = protocol +func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) { + protoOut = protocol bodySize = 0 - var root map[string]interface{} - json.Unmarshal([]byte(entry.Entry), &root) representation := make(map[string]interface{}, 0) - request := root["request"].(map[string]interface{})["payload"].(map[string]interface{}) var repRequest []interface{} - details := request["details"].(map[string]interface{}) switch request["method"].(string) { case basicMethodMap[40]: - repRequest = representBasicPublish(details) + repRequest = representBasicPublish(request) break case basicMethodMap[60]: - repRequest = representBasicDeliver(details) + repRequest = representBasicDeliver(request) break case queueMethodMap[10]: - repRequest = representQueueDeclare(details) + repRequest = representQueueDeclare(request) break case exchangeMethodMap[10]: - repRequest = representExchangeDeclare(details) + repRequest = representExchangeDeclare(request) break case connectionMethodMap[10]: - repRequest = representConnectionStart(details) + repRequest = representConnectionStart(request) break case connectionMethodMap[50]: - repRequest = representConnectionClose(details) + repRequest = representConnectionClose(request) break case queueMethodMap[20]: - repRequest = representQueueBind(details) + repRequest = representQueueBind(request) break case basicMethodMap[20]: - repRequest = representBasicConsume(details) + repRequest = representBasicConsume(request) break } representation["request"] = repRequest @@ -360,4 +361,10 @@ func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []by return } +func (d dissecting) Macros() map[string]string { + return map[string]string{ + `amqp`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation), + } +} + var Dissector dissecting diff --git a/tap/extensions/amqp/spec091.go b/tap/extensions/amqp/spec091.go index 3dd3dcbd3..0af77a26c 100644 --- a/tap/extensions/amqp/spec091.go +++ b/tap/extensions/amqp/spec091.go @@ -71,11 +71,11 @@ func isSoftExceptionCode(code int) bool { } type ConnectionStart struct { - VersionMajor byte - VersionMinor byte - ServerProperties Table - Mechanisms string - Locales string + VersionMajor byte `json:"versionMajor"` + VersionMinor byte `json:"versionMinor"` + ServerProperties Table `json:"serverProperties"` + Mechanisms string `json:"mechanisms"` + Locales string `json:"locales"` } func (msg *ConnectionStart) id() (uint16, uint16) { @@ -429,10 +429,10 @@ func (msg *connectionOpenOk) read(r io.Reader) (err error) { } type ConnectionClose struct { - ReplyCode uint16 - ReplyText string - ClassId uint16 - MethodId uint16 + ReplyCode uint16 `json:"relyCode"` + ReplyText string `json:"replyText"` + ClassId uint16 `json:"classId"` + MethodId uint16 `json:"methodId"` } func (msg *ConnectionClose) id() (uint16, uint16) { @@ -767,14 +767,14 @@ func (msg *channelCloseOk) read(r io.Reader) (err error) { type ExchangeDeclare struct { reserved1 uint16 - Exchange string - Type string - Passive bool - Durable bool - AutoDelete bool - Internal bool - NoWait bool - Arguments Table + Exchange string `json:"exchange"` + Type string `json:"type"` + Passive bool `json:"passive"` + Durable bool `json:"durable"` + AutoDelete bool `json:"autoDelete"` + Internal bool `json:"internal"` + NoWait bool `json:"noWait"` + Arguments Table `json:"arguments"` } func (msg *ExchangeDeclare) id() (uint16, uint16) { @@ -1163,13 +1163,13 @@ func (msg *exchangeUnbindOk) read(r io.Reader) (err error) { type QueueDeclare struct { reserved1 uint16 - Queue string - Passive bool - Durable bool - Exclusive bool - AutoDelete bool - NoWait bool - Arguments Table + Queue string `json:"queue"` + Passive bool `json:"passive"` + Durable bool `json:"durable"` + Exclusive bool `json:"exclusive"` + AutoDelete bool `json:"autoDelete"` + NoWait bool `json:"noWait"` + Arguments Table `json:"arguments"` } func (msg *QueueDeclare) id() (uint16, uint16) { @@ -1297,11 +1297,11 @@ func (msg *QueueDeclareOk) read(r io.Reader) (err error) { type QueueBind struct { reserved1 uint16 - Queue string - Exchange string - RoutingKey string - NoWait bool - Arguments Table + Queue string `json:"queue"` + Exchange string `json:"exchange"` + RoutingKey string `json:"routingKey"` + NoWait bool `json:"noWait"` + Arguments Table `json:"arguments"` } func (msg *QueueBind) id() (uint16, uint16) { @@ -1737,13 +1737,13 @@ func (msg *basicQosOk) read(r io.Reader) (err error) { type BasicConsume struct { reserved1 uint16 - Queue string - ConsumerTag string - NoLocal bool - NoAck bool - Exclusive bool - NoWait bool - Arguments Table + Queue string `json:"queue"` + ConsumerTag string `json:"consumerTag"` + NoLocal bool `json:"noLocal"` + NoAck bool `json:"noAck"` + Exclusive bool `json:"exclusive"` + NoWait bool `json:"noWait"` + Arguments Table `json:"arguments"` } func (msg *BasicConsume) id() (uint16, uint16) { @@ -1932,12 +1932,12 @@ func (msg *basicCancelOk) read(r io.Reader) (err error) { type BasicPublish struct { reserved1 uint16 - Exchange string - RoutingKey string - Mandatory bool - Immediate bool - Properties Properties - Body []byte + Exchange string `json:"exchange"` + RoutingKey string `json:"routingKey"` + Mandatory bool `json:"mandatory"` + Immediate bool `json:"immediate"` + Properties Properties `json:"properties"` + Body []byte `json:"body"` } func (msg *BasicPublish) id() (uint16, uint16) { @@ -2072,13 +2072,13 @@ func (msg *basicReturn) read(r io.Reader) (err error) { } type BasicDeliver struct { - ConsumerTag string - DeliveryTag uint64 - Redelivered bool - Exchange string - RoutingKey string - Properties Properties - Body []byte + ConsumerTag string `json:"consumerTag"` + DeliveryTag uint64 `json:"deliveryTag"` + Redelivered bool `json:"redelivered"` + Exchange string `json:"exchange"` + RoutingKey string `json:"routingKey"` + Properties Properties `json:"properties"` + Body []byte `json:"body"` } func (msg *BasicDeliver) id() (uint16, uint16) { diff --git a/tap/extensions/amqp/types.go b/tap/extensions/amqp/types.go index ea57556fa..1adee51c6 100644 --- a/tap/extensions/amqp/types.go +++ b/tap/extensions/amqp/types.go @@ -93,19 +93,19 @@ func (e Error) Error() string { // Used by header frames to capture routing and header information type Properties struct { - ContentType string // MIME content type - ContentEncoding string // MIME content encoding - Headers Table // Application or header exchange table - DeliveryMode uint8 // queue implementation use - Transient (1) or Persistent (2) - Priority uint8 // queue implementation use - 0 to 9 - CorrelationId string // application use - correlation identifier - ReplyTo string // application use - address to to reply to (ex: RPC) - Expiration string // implementation use - message expiration spec - MessageId string // application use - message identifier - Timestamp time.Time // application use - message timestamp - Type string // application use - message type name - UserId string // application use - creating user id - AppId string // application use - creating application + ContentType string `json:"contentType"` // MIME content type + ContentEncoding string `json:"contentEncoding"` // MIME content encoding + Headers Table `json:"headers"` // Application or header exchange table + DeliveryMode uint8 `json:"deliveryMode"` // queue implementation use - Transient (1) or Persistent (2) + Priority uint8 `json:"priority"` // queue implementation use - 0 to 9 + CorrelationId string `json:"correlationId"` // application use - correlation identifier + ReplyTo string `json:"replyTo"` // application use - address to to reply to (ex: RPC) + Expiration string `json:"expiration"` // implementation use - message expiration spec + MessageId string `json:"messageId"` // application use - message identifier + Timestamp time.Time `json:"timestamp"` // application use - message timestamp + Type string `json:"type"` // application use - message type name + UserId string `json:"userId"` // application use - creating user id + AppId string `json:"appId"` // application use - creating application reserved1 string // was cluster-id - process for buffer consumption } diff --git a/tap/extensions/http/helpers.go b/tap/extensions/http/helpers.go new file mode 100644 index 000000000..9348ea3cd --- /dev/null +++ b/tap/extensions/http/helpers.go @@ -0,0 +1,35 @@ +package main + +import ( + "encoding/json" + "fmt" + + "github.com/up9inc/mizu/tap/api" +) + +func mapSliceRebuildAsMap(mapSlice []interface{}) (newMap map[string]interface{}) { + newMap = make(map[string]interface{}) + for _, header := range mapSlice { + h := header.(map[string]interface{}) + newMap[h["name"].(string)] = h["value"] + } + + return +} + +func representMapSliceAsTable(mapSlice []interface{}, selectorPrefix string) (representation string) { + var table []api.TableData + for _, header := range mapSlice { + h := header.(map[string]interface{}) + selector := fmt.Sprintf("%s[\"%s\"]", selectorPrefix, h["name"].(string)) + table = append(table, api.TableData{ + Name: h["name"].(string), + Value: h["value"], + Selector: selector, + }) + } + + obj, _ := json.Marshal(table) + representation = string(obj) + return +} diff --git a/tap/extensions/http/main.go b/tap/extensions/http/main.go index 3ca432a71..b58fc7a02 100644 --- a/tap/extensions/http/main.go +++ b/tap/extensions/http/main.go @@ -17,6 +17,7 @@ var protocol api.Protocol = api.Protocol{ Name: "http", LongName: "Hypertext Transfer Protocol -- HTTP/1.1", Abbreviation: "HTTP", + Macro: "http", Version: "1.1", BackgroundColor: "#205cf5", ForegroundColor: "#ffffff", @@ -30,6 +31,7 @@ var http2Protocol api.Protocol = api.Protocol{ Name: "http", LongName: "Hypertext Transfer Protocol Version 2 (HTTP/2) (gRPC)", Abbreviation: "HTTP/2", + Macro: "grpc", Version: "2.0", BackgroundColor: "#244c5a", ForegroundColor: "#ffffff", @@ -117,7 +119,7 @@ func SetHostname(address, newHostname string) string { return replacedUrl.String() } -func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry { +func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry { var host, scheme, authority, path, service string request := item.Pair.Request.Payload.(map[string]interface{}) @@ -145,10 +147,32 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve service = fmt.Sprintf("%s://%s", scheme, authority) } else { service = fmt.Sprintf("http://%s", host) - path = reqDetails["url"].(string) + u, err := url.Parse(reqDetails["url"].(string)) + if err != nil { + path = reqDetails["url"].(string) + } else { + path = u.Path + } } - request["url"] = path + request["url"] = reqDetails["url"].(string) + reqDetails["path"] = path + reqDetails["summary"] = path + + // Rearrange the maps for the querying + reqDetails["_headers"] = reqDetails["headers"] + reqDetails["headers"] = mapSliceRebuildAsMap(reqDetails["_headers"].([]interface{})) + resDetails["_headers"] = resDetails["headers"] + resDetails["headers"] = mapSliceRebuildAsMap(resDetails["_headers"].([]interface{})) + + reqDetails["_cookies"] = reqDetails["cookies"] + reqDetails["cookies"] = mapSliceRebuildAsMap(reqDetails["_cookies"].([]interface{})) + resDetails["_cookies"] = resDetails["cookies"] + resDetails["cookies"] = mapSliceRebuildAsMap(resDetails["_cookies"].([]interface{})) + + reqDetails["_queryString"] = reqDetails["queryString"] + reqDetails["queryString"] = mapSliceRebuildAsMap(reqDetails["_queryString"].([]interface{})) + if resolvedDestination != "" { service = SetHostname(service, resolvedDestination) } else if resolvedSource != "" { @@ -156,51 +180,59 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve } elapsedTime := item.Pair.Response.CaptureTime.Sub(item.Pair.Request.CaptureTime).Round(time.Millisecond).Milliseconds() - entryBytes, _ := json.Marshal(item.Pair) + httpPair, _ := json.Marshal(item.Pair) + _protocol := protocol + _protocol.Version = item.Protocol.Version return &api.MizuEntry{ - ProtocolName: protocol.Name, - ProtocolLongName: protocol.LongName, - ProtocolAbbreviation: protocol.Abbreviation, - ProtocolVersion: item.Protocol.Version, - ProtocolBackgroundColor: protocol.BackgroundColor, - ProtocolForegroundColor: protocol.ForegroundColor, - ProtocolFontSize: protocol.FontSize, - ProtocolReferenceLink: protocol.ReferenceLink, - EntryId: entryId, - Entry: string(entryBytes), - Url: fmt.Sprintf("%s%s", service, path), - Method: reqDetails["method"].(string), - Status: int(resDetails["status"].(float64)), - RequestSenderIp: item.ConnectionInfo.ClientIP, - Service: service, - Timestamp: item.Timestamp, - ElapsedTime: elapsedTime, - Path: path, - ResolvedSource: resolvedSource, - ResolvedDestination: resolvedDestination, - SourceIp: item.ConnectionInfo.ClientIP, - DestinationIp: item.ConnectionInfo.ServerIP, - SourcePort: item.ConnectionInfo.ClientPort, - DestinationPort: item.ConnectionInfo.ServerPort, - IsOutgoing: item.ConnectionInfo.IsOutgoing, + Protocol: _protocol, + Source: &api.TCP{ + Name: resolvedSource, + IP: item.ConnectionInfo.ClientIP, + Port: item.ConnectionInfo.ClientPort, + }, + Destination: &api.TCP{ + Name: resolvedDestination, + IP: item.ConnectionInfo.ServerIP, + Port: item.ConnectionInfo.ServerPort, + }, + Outgoing: item.ConnectionInfo.IsOutgoing, + Request: reqDetails, + Response: resDetails, + Url: fmt.Sprintf("%s%s", service, path), + Method: reqDetails["method"].(string), + Status: int(resDetails["status"].(float64)), + RequestSenderIp: item.ConnectionInfo.ClientIP, + Service: service, + Timestamp: item.Timestamp, + StartTime: item.Pair.Request.CaptureTime, + ElapsedTime: elapsedTime, + Summary: path, + ResolvedSource: resolvedSource, + ResolvedDestination: resolvedDestination, + SourceIp: item.ConnectionInfo.ClientIP, + DestinationIp: item.ConnectionInfo.ServerIP, + SourcePort: item.ConnectionInfo.ClientPort, + DestinationPort: item.ConnectionInfo.ServerPort, + IsOutgoing: item.ConnectionInfo.IsOutgoing, + HTTPPair: string(httpPair), } } func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { var p api.Protocol - if entry.ProtocolVersion == "2.0" { + if entry.Protocol.Version == "2.0" { p = http2Protocol } else { p = protocol } return &api.BaseEntryDetails{ - Id: entry.EntryId, + Id: entry.Id, Protocol: p, Url: entry.Url, RequestSenderIp: entry.RequestSenderIp, Service: entry.Service, Path: entry.Path, - Summary: entry.Path, + Summary: entry.Summary, StatusCode: entry.Status, Method: entry.Method, Timestamp: entry.Timestamp, @@ -218,45 +250,50 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { } func representRequest(request map[string]interface{}) (repRequest []interface{}) { - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Method", - "value": request["method"].(string), + Name: "Method", + Value: request["method"].(string), + Selector: `request.method`, }, { - "name": "URL", - "value": request["url"].(string), + Name: "URL", + Value: request["url"].(string), + Selector: `request.url`, }, { - "name": "Body Size", - "value": fmt.Sprintf("%g bytes", request["bodySize"].(float64)), + Name: "Path", + Value: request["path"].(string), + Selector: `request.path`, + }, + { + Name: "Body Size (bytes)", + Value: int64(request["bodySize"].(float64)), + Selector: `request.bodySize`, }, }) - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - headers, _ := json.Marshal(request["headers"].([]interface{})) - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "Headers", - "data": string(headers), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "Headers", + Data: representMapSliceAsTable(request["_headers"].([]interface{}), `request.headers`), }) - cookies, _ := json.Marshal(request["cookies"].([]interface{})) - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "Cookies", - "data": string(cookies), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "Cookies", + Data: representMapSliceAsTable(request["_cookies"].([]interface{}), `request.cookies`), }) - queryString, _ := json.Marshal(request["queryString"].([]interface{})) - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "Query String", - "data": string(queryString), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "Query String", + Data: representMapSliceAsTable(request["_queryString"].([]interface{}), `request.queryString`), }) postData, _ := request["postData"].(map[string]interface{}) @@ -266,12 +303,12 @@ func representRequest(request map[string]interface{}) (repRequest []interface{}) } text, _ := postData["text"] if text != nil { - repRequest = append(repRequest, map[string]string{ - "type": api.BODY, - "title": "POST Data (text/plain)", - "encoding": "", - "mime_type": mimeType.(string), - "data": text.(string), + repRequest = append(repRequest, api.SectionData{ + Type: api.BODY, + Title: "POST Data (text/plain)", + MimeType: mimeType.(string), + Data: text.(string), + Selector: `request.postData.text`, }) } @@ -285,16 +322,16 @@ func representRequest(request map[string]interface{}) (repRequest []interface{}) "value": string(params), }, }) - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "POST Data (multipart/form-data)", - "data": string(multipart), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "POST Data (multipart/form-data)", + Data: string(multipart), }) } else { - repRequest = append(repRequest, map[string]string{ - "type": api.TABLE, - "title": "POST Data (application/x-www-form-urlencoded)", - "data": string(params), + repRequest = append(repRequest, api.SectionData{ + Type: api.TABLE, + Title: "POST Data (application/x-www-form-urlencoded)", + Data: representMapSliceAsTable(postData["params"].([]interface{}), `request.postData.params`), }) } } @@ -308,38 +345,39 @@ func representResponse(response map[string]interface{}) (repResponse []interface bodySize = int64(response["bodySize"].(float64)) - details, _ := json.Marshal([]map[string]string{ + details, _ := json.Marshal([]api.TableData{ { - "name": "Status", - "value": fmt.Sprintf("%g", response["status"].(float64)), + Name: "Status", + Value: int64(response["status"].(float64)), + Selector: `response.status`, }, { - "name": "Status Text", - "value": response["statusText"].(string), + Name: "Status Text", + Value: response["statusText"].(string), + Selector: `response.statusText`, }, { - "name": "Body Size", - "value": fmt.Sprintf("%d bytes", bodySize), + Name: "Body Size (bytes)", + Value: bodySize, + Selector: `response.bodySize`, }, }) - repResponse = append(repResponse, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + repResponse = append(repResponse, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) - headers, _ := json.Marshal(response["headers"].([]interface{})) - repResponse = append(repResponse, map[string]string{ - "type": api.TABLE, - "title": "Headers", - "data": string(headers), + repResponse = append(repResponse, api.SectionData{ + Type: api.TABLE, + Title: "Headers", + Data: representMapSliceAsTable(response["_headers"].([]interface{}), `response.headers`), }) - cookies, _ := json.Marshal(response["cookies"].([]interface{})) - repResponse = append(repResponse, map[string]string{ - "type": api.TABLE, - "title": "Cookies", - "data": string(cookies), + repResponse = append(repResponse, api.SectionData{ + Type: api.TABLE, + Title: "Cookies", + Data: representMapSliceAsTable(response["_cookies"].([]interface{}), `response.cookies`), }) content, _ := response["content"].(map[string]interface{}) @@ -350,37 +388,40 @@ func representResponse(response map[string]interface{}) (repResponse []interface encoding, _ := content["encoding"] text, _ := content["text"] if text != nil { - repResponse = append(repResponse, map[string]string{ - "type": api.BODY, - "title": "Body", - "encoding": encoding.(string), - "mime_type": mimeType.(string), - "data": text.(string), + repResponse = append(repResponse, api.SectionData{ + Type: api.BODY, + Title: "Body", + Encoding: encoding.(string), + MimeType: mimeType.(string), + Data: text.(string), + Selector: `response.content.text`, }) } return } -func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) { - if entry.ProtocolVersion == "2.0" { - p = http2Protocol +func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) { + if protoIn.Version == "2.0" { + protoOut = http2Protocol } else { - p = protocol + protoOut = protocol } - var root map[string]interface{} - json.Unmarshal([]byte(entry.Entry), &root) representation := make(map[string]interface{}, 0) - request := root["request"].(map[string]interface{})["payload"].(map[string]interface{}) - response := root["response"].(map[string]interface{})["payload"].(map[string]interface{}) - reqDetails := request["details"].(map[string]interface{}) - resDetails := response["details"].(map[string]interface{}) - repRequest := representRequest(reqDetails) - repResponse, bodySize := representResponse(resDetails) + repRequest := representRequest(request) + repResponse, bodySize := representResponse(response) representation["request"] = repRequest representation["response"] = repResponse object, err = json.Marshal(representation) return } +func (d dissecting) Macros() map[string]string { + return map[string]string{ + `http`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation), + `grpc`: fmt.Sprintf(`proto.abbr == "%s" and proto.version == "%s"`, protocol.Abbreviation, http2Protocol.Version), + `http2`: fmt.Sprintf(`proto.abbr == "%s" and proto.version == "%s"`, protocol.Abbreviation, http2Protocol.Version), + } +} + var Dissector dissecting diff --git a/tap/extensions/kafka/helpers.go b/tap/extensions/kafka/helpers.go index 3a9a238eb..dffe30bd8 100644 --- a/tap/extensions/kafka/helpers.go +++ b/tap/extensions/kafka/helpers.go @@ -27,48 +27,54 @@ type KafkaWrapper struct { } func representRequestHeader(data map[string]interface{}, rep []interface{}) []interface{} { - requestHeader, _ := json.Marshal([]map[string]string{ + requestHeader, _ := json.Marshal([]api.TableData{ { - "name": "ApiKey", - "value": apiNames[int(data["ApiKey"].(float64))], + Name: "ApiKey", + Value: apiNames[int(data["apiKey"].(float64))], + Selector: `request.apiKey`, }, { - "name": "ApiVersion", - "value": fmt.Sprintf("%d", int(data["ApiVersion"].(float64))), + Name: "ApiVersion", + Value: fmt.Sprintf("%d", int(data["apiVersion"].(float64))), + Selector: `request.apiVersion`, }, { - "name": "Client ID", - "value": data["ClientID"].(string), + Name: "Client ID", + Value: data["clientID"].(string), + Selector: `request.clientID`, }, { - "name": "Correlation ID", - "value": fmt.Sprintf("%d", int(data["CorrelationID"].(float64))), + Name: "Correlation ID", + Value: fmt.Sprintf("%d", int(data["correlationID"].(float64))), + Selector: `request.correlationID`, }, { - "name": "Size", - "value": fmt.Sprintf("%d", int(data["Size"].(float64))), + Name: "Size", + Value: fmt.Sprintf("%d", int(data["size"].(float64))), + Selector: `request.size`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Request Header", - "data": string(requestHeader), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Request Header", + Data: string(requestHeader), }) return rep } func representResponseHeader(data map[string]interface{}, rep []interface{}) []interface{} { - requestHeader, _ := json.Marshal([]map[string]string{ + requestHeader, _ := json.Marshal([]api.TableData{ { - "name": "Correlation ID", - "value": fmt.Sprintf("%d", int(data["CorrelationID"].(float64))), + Name: "Correlation ID", + Value: fmt.Sprintf("%d", int(data["correlationID"].(float64))), + Selector: `response.correlationID`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Response Header", - "data": string(requestHeader), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Response Header", + Data: string(requestHeader), }) return rep @@ -79,46 +85,50 @@ func representMetadataRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topics := "" allowAutoTopicCreation := "" includeClusterAuthorizedOperations := "" includeTopicAuthorizedOperations := "" - if payload["Topics"] != nil { - x, _ := json.Marshal(payload["Topics"].([]interface{})) + if payload["topics"] != nil { + x, _ := json.Marshal(payload["topics"].([]interface{})) topics = string(x) } - if payload["AllowAutoTopicCreation"] != nil { - allowAutoTopicCreation = strconv.FormatBool(payload["AllowAutoTopicCreation"].(bool)) + if payload["allowAutoTopicCreation"] != nil { + allowAutoTopicCreation = strconv.FormatBool(payload["allowAutoTopicCreation"].(bool)) } - if payload["IncludeClusterAuthorizedOperations"] != nil { - includeClusterAuthorizedOperations = strconv.FormatBool(payload["IncludeClusterAuthorizedOperations"].(bool)) + if payload["includeClusterAuthorizedOperations"] != nil { + includeClusterAuthorizedOperations = strconv.FormatBool(payload["includeClusterAuthorizedOperations"].(bool)) } - if payload["IncludeTopicAuthorizedOperations"] != nil { - includeTopicAuthorizedOperations = strconv.FormatBool(payload["IncludeTopicAuthorizedOperations"].(bool)) + if payload["includeTopicAuthorizedOperations"] != nil { + includeTopicAuthorizedOperations = strconv.FormatBool(payload["includeTopicAuthorizedOperations"].(bool)) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Topics", - "value": topics, + Name: "Topics", + Value: topics, + Selector: `request.payload.topics`, }, { - "name": "Allow Auto Topic Creation", - "value": allowAutoTopicCreation, + Name: "Allow Auto Topic Creation", + Value: allowAutoTopicCreation, + Selector: `request.payload.allowAutoTopicCreation`, }, { - "name": "Include Cluster Authorized Operations", - "value": includeClusterAuthorizedOperations, + Name: "Include Cluster Authorized Operations", + Value: includeClusterAuthorizedOperations, + Selector: `request.payload.includeClusterAuthorizedOperations`, }, { - "name": "Include Topic Authorized Operations", - "value": includeTopicAuthorizedOperations, + Name: "Include Topic Authorized Operations", + Value: includeTopicAuthorizedOperations, + Selector: `request.payload.includeTopicAuthorizedOperations`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -129,63 +139,69 @@ func representMetadataResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topics := "" - if payload["Topics"] != nil { - _topics, _ := json.Marshal(payload["Topics"].([]interface{})) + if payload["topics"] != nil { + _topics, _ := json.Marshal(payload["topics"].([]interface{})) topics = string(_topics) } brokers := "" - if payload["Brokers"] != nil { - _brokers, _ := json.Marshal(payload["Brokers"].([]interface{})) + if payload["brokers"] != nil { + _brokers, _ := json.Marshal(payload["brokers"].([]interface{})) brokers = string(_brokers) } controllerID := "" clusterID := "" throttleTimeMs := "" clusterAuthorizedOperations := "" - if payload["ControllerID"] != nil { - controllerID = fmt.Sprintf("%d", int(payload["ControllerID"].(float64))) + if payload["controllerID"] != nil { + controllerID = fmt.Sprintf("%d", int(payload["controllerID"].(float64))) } - if payload["ClusterID"] != nil { - clusterID = payload["ClusterID"].(string) + if payload["clusterID"] != nil { + clusterID = payload["clusterID"].(string) } - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - if payload["ClusterAuthorizedOperations"] != nil { - clusterAuthorizedOperations = fmt.Sprintf("%d", int(payload["ClusterAuthorizedOperations"].(float64))) + if payload["clusterAuthorizedOperations"] != nil { + clusterAuthorizedOperations = fmt.Sprintf("%d", int(payload["clusterAuthorizedOperations"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, { - "name": "Brokers", - "value": brokers, + Name: "Brokers", + Value: brokers, + Selector: `response.payload.brokers`, }, { - "name": "Cluster ID", - "value": clusterID, + Name: "Cluster ID", + Value: clusterID, + Selector: `response.payload.clusterID`, }, { - "name": "Controller ID", - "value": controllerID, + Name: "Controller ID", + Value: controllerID, + Selector: `response.payload.controllerID`, }, { - "name": "Topics", - "value": topics, + Name: "Topics", + Value: topics, + Selector: `response.payload.topics`, }, { - "name": "Cluster Authorized Operations", - "value": clusterAuthorizedOperations, + Name: "Cluster Authorized Operations", + Value: clusterAuthorizedOperations, + Selector: `response.payload.clusterAuthorizedOperations`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -196,29 +212,31 @@ func representApiVersionsRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) clientSoftwareName := "" clientSoftwareVersion := "" - if payload["ClientSoftwareName"] != nil { - clientSoftwareName = payload["ClientSoftwareName"].(string) + if payload["clientSoftwareName"] != nil { + clientSoftwareName = payload["clientSoftwareName"].(string) } - if payload["ClientSoftwareVersion"] != nil { - clientSoftwareVersion = payload["ClientSoftwareVersion"].(string) + if payload["clientSoftwareVersion"] != nil { + clientSoftwareVersion = payload["clientSoftwareVersion"].(string) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Client Software Name", - "value": clientSoftwareName, + Name: "Client Software Name", + Value: clientSoftwareName, + Selector: `request.payload.clientSoftwareName`, }, { - "name": "Client Software Version", - "value": clientSoftwareVersion, + Name: "Client Software Version", + Value: clientSoftwareVersion, + Selector: `request.payload.clientSoftwareVersion`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -229,34 +247,37 @@ func representApiVersionsResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) apiKeys := "" - if payload["TopicNames"] != nil { - x, _ := json.Marshal(payload["ApiKeys"].([]interface{})) + if payload["apiKeys"] != nil { + x, _ := json.Marshal(payload["apiKeys"].([]interface{})) apiKeys = string(x) } throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Error Code", - "value": fmt.Sprintf("%d", int(payload["ErrorCode"].(float64))), + Name: "Error Code", + Value: fmt.Sprintf("%d", int(payload["errorCode"].(float64))), + Selector: `response.payload.errorCode`, }, { - "name": "ApiKeys", - "value": apiKeys, + Name: "ApiKeys", + Value: apiKeys, + Selector: `response.payload.apiKeys`, }, { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -267,39 +288,43 @@ func representProduceRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topicData := "" - _topicData := payload["TopicData"] + _topicData := payload["topicData"] if _topicData != nil { x, _ := json.Marshal(_topicData.([]interface{})) topicData = string(x) } transactionalID := "" - if payload["TransactionalID"] != nil { - transactionalID = payload["TransactionalID"].(string) + if payload["transactionalID"] != nil { + transactionalID = payload["transactionalID"].(string) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Transactional ID", - "value": transactionalID, + Name: "Transactional ID", + Value: transactionalID, + Selector: `request.payload.transactionalID`, }, { - "name": "Required Acknowledgements", - "value": fmt.Sprintf("%d", int(payload["RequiredAcks"].(float64))), + Name: "Required Acknowledgements", + Value: fmt.Sprintf("%d", int(payload["requiredAcks"].(float64))), + Selector: `request.payload.requiredAcks`, }, { - "name": "Timeout", - "value": fmt.Sprintf("%d", int(payload["Timeout"].(float64))), + Name: "Timeout", + Value: fmt.Sprintf("%d", int(payload["timeout"].(float64))), + Selector: `request.payload.timeout`, }, { - "name": "Topic Data", - "value": topicData, + Name: "Topic Data", + Value: topicData, + Selector: `request.payload.topicData`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -310,30 +335,32 @@ func representProduceResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) responses := "" - if payload["Responses"] != nil { - _responses, _ := json.Marshal(payload["Responses"].([]interface{})) + if payload["responses"] != nil { + _responses, _ := json.Marshal(payload["responses"].([]interface{})) responses = string(_responses) } throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Responses", - "value": string(responses), + Name: "Responses", + Value: string(responses), + Selector: `response.payload.responses`, }, { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -344,87 +371,97 @@ func representFetchRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topics := "" - if payload["Topics"] != nil { - _topics, _ := json.Marshal(payload["Topics"].([]interface{})) + if payload["topics"] != nil { + _topics, _ := json.Marshal(payload["topics"].([]interface{})) topics = string(_topics) } replicaId := "" - if payload["ReplicaId"] != nil { - replicaId = fmt.Sprintf("%d", int(payload["ReplicaId"].(float64))) + if payload["replicaId"] != nil { + replicaId = fmt.Sprintf("%d", int(payload["replicaId"].(float64))) } maxBytes := "" - if payload["MaxBytes"] != nil { - maxBytes = fmt.Sprintf("%d", int(payload["MaxBytes"].(float64))) + if payload["maxBytes"] != nil { + maxBytes = fmt.Sprintf("%d", int(payload["maxBytes"].(float64))) } isolationLevel := "" - if payload["IsolationLevel"] != nil { - isolationLevel = fmt.Sprintf("%d", int(payload["IsolationLevel"].(float64))) + if payload["isolationLevel"] != nil { + isolationLevel = fmt.Sprintf("%d", int(payload["isolationLevel"].(float64))) } sessionId := "" - if payload["SessionId"] != nil { - sessionId = fmt.Sprintf("%d", int(payload["SessionId"].(float64))) + if payload["sessionId"] != nil { + sessionId = fmt.Sprintf("%d", int(payload["sessionId"].(float64))) } sessionEpoch := "" - if payload["SessionEpoch"] != nil { - sessionEpoch = fmt.Sprintf("%d", int(payload["SessionEpoch"].(float64))) + if payload["sessionEpoch"] != nil { + sessionEpoch = fmt.Sprintf("%d", int(payload["sessionEpoch"].(float64))) } forgottenTopicsData := "" - if payload["ForgottenTopicsData"] != nil { - x, _ := json.Marshal(payload["ForgottenTopicsData"].(map[string]interface{})) + if payload["forgottenTopicsData"] != nil { + x, _ := json.Marshal(payload["forgottenTopicsData"].(map[string]interface{})) forgottenTopicsData = string(x) } rackId := "" - if payload["RackId"] != nil { - rackId = payload["RackId"].(string) + if payload["rackId"] != nil { + rackId = payload["rackId"].(string) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Replica ID", - "value": replicaId, + Name: "Replica ID", + Value: replicaId, + Selector: `request.payload.replicaId`, }, { - "name": "Maximum Wait (ms)", - "value": fmt.Sprintf("%d", int(payload["MaxWaitMs"].(float64))), + Name: "Maximum Wait (ms)", + Value: fmt.Sprintf("%d", int(payload["maxWaitMs"].(float64))), + Selector: `request.payload.maxWaitMs`, }, { - "name": "Minimum Bytes", - "value": fmt.Sprintf("%d", int(payload["MinBytes"].(float64))), + Name: "Minimum Bytes", + Value: fmt.Sprintf("%d", int(payload["minBytes"].(float64))), + Selector: `request.payload.minBytes`, }, { - "name": "Maximum Bytes", - "value": maxBytes, + Name: "Maximum Bytes", + Value: maxBytes, + Selector: `request.payload.maxBytes`, }, { - "name": "Isolation Level", - "value": isolationLevel, + Name: "Isolation Level", + Value: isolationLevel, + Selector: `request.payload.isolationLevel`, }, { - "name": "Session ID", - "value": sessionId, + Name: "Session ID", + Value: sessionId, + Selector: `request.payload.sessionId`, }, { - "name": "Session Epoch", - "value": sessionEpoch, + Name: "Session Epoch", + Value: sessionEpoch, + Selector: `request.payload.sessionEpoch`, }, { - "name": "Topics", - "value": topics, + Name: "Topics", + Value: topics, + Selector: `request.payload.topics`, }, { - "name": "Forgotten Topics Data", - "value": forgottenTopicsData, + Name: "Forgotten Topics Data", + Value: forgottenTopicsData, + Selector: `request.payload.forgottenTopicsData`, }, { - "name": "Rack ID", - "value": rackId, + Name: "Rack ID", + Value: rackId, + Selector: `request.payload.rackId`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -435,46 +472,50 @@ func representFetchResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) responses := "" - if payload["Responses"] != nil { - _responses, _ := json.Marshal(payload["Responses"].([]interface{})) + if payload["responses"] != nil { + _responses, _ := json.Marshal(payload["responses"].([]interface{})) responses = string(_responses) } throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } errorCode := "" - if payload["ErrorCode"] != nil { - errorCode = fmt.Sprintf("%d", int(payload["ErrorCode"].(float64))) + if payload["errorCode"] != nil { + errorCode = fmt.Sprintf("%d", int(payload["errorCode"].(float64))) } sessionId := "" - if payload["SessionId"] != nil { - sessionId = fmt.Sprintf("%d", int(payload["SessionId"].(float64))) + if payload["sessionId"] != nil { + sessionId = fmt.Sprintf("%d", int(payload["sessionId"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, { - "name": "Error Code", - "value": errorCode, + Name: "Error Code", + Value: errorCode, + Selector: `response.payload.errorCode`, }, { - "name": "Session ID", - "value": sessionId, + Name: "Session ID", + Value: sessionId, + Selector: `response.payload.sessionId`, }, { - "name": "Responses", - "value": responses, + Name: "Responses", + Value: responses, + Selector: `response.payload.responses`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -485,26 +526,28 @@ func representListOffsetsRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topics := "" - if payload["Topics"] != nil { - _topics, _ := json.Marshal(payload["Topics"].([]interface{})) + if payload["topics"] != nil { + _topics, _ := json.Marshal(payload["topics"].([]interface{})) topics = string(_topics) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Replica ID", - "value": fmt.Sprintf("%d", int(payload["ReplicaId"].(float64))), + Name: "Replica ID", + Value: fmt.Sprintf("%d", int(payload["replicaId"].(float64))), + Selector: `request.payload.replicaId`, }, { - "name": "Topics", - "value": topics, + Name: "Topics", + Value: topics, + Selector: `request.payload.topics`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -515,26 +558,28 @@ func representListOffsetsResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) - topics, _ := json.Marshal(payload["Topics"].([]interface{})) + payload := data["payload"].(map[string]interface{}) + topics, _ := json.Marshal(payload["topics"].([]interface{})) throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, { - "name": "Topics", - "value": string(topics), + Name: "Topics", + Value: string(topics), + Selector: `response.payload.topics`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -545,30 +590,33 @@ func representCreateTopicsRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) - topics, _ := json.Marshal(payload["Topics"].([]interface{})) + payload := data["payload"].(map[string]interface{}) + topics, _ := json.Marshal(payload["topics"].([]interface{})) validateOnly := "" - if payload["ValidateOnly"] != nil { - validateOnly = strconv.FormatBool(payload["ValidateOnly"].(bool)) + if payload["validateOnly"] != nil { + validateOnly = strconv.FormatBool(payload["validateOnly"].(bool)) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Topics", - "value": string(topics), + Name: "Topics", + Value: string(topics), + Selector: `request.payload.topics`, }, { - "name": "Timeout (ms)", - "value": fmt.Sprintf("%d", int(payload["TimeoutMs"].(float64))), + Name: "Timeout (ms)", + Value: fmt.Sprintf("%d", int(payload["timeoutMs"].(float64))), + Selector: `request.payload.timeoutMs`, }, { - "name": "Validate Only", - "value": validateOnly, + Name: "Validate Only", + Value: validateOnly, + Selector: `request.payload.validateOnly`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -579,26 +627,28 @@ func representCreateTopicsResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) - topics, _ := json.Marshal(payload["Topics"].([]interface{})) + payload := data["payload"].(map[string]interface{}) + topics, _ := json.Marshal(payload["topics"].([]interface{})) throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, { - "name": "Topics", - "value": string(topics), + Name: "Topics", + Value: string(topics), + Selector: `response.payload.topics`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -609,35 +659,38 @@ func representDeleteTopicsRequest(data map[string]interface{}) []interface{} { rep = representRequestHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) + payload := data["payload"].(map[string]interface{}) topics := "" - if payload["Topics"] != nil { - x, _ := json.Marshal(payload["Topics"].([]interface{})) + if payload["topics"] != nil { + x, _ := json.Marshal(payload["topics"].([]interface{})) topics = string(x) } topicNames := "" - if payload["TopicNames"] != nil { - x, _ := json.Marshal(payload["TopicNames"].([]interface{})) + if payload["topicNames"] != nil { + x, _ := json.Marshal(payload["topicNames"].([]interface{})) topicNames = string(x) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "TopicNames", - "value": string(topicNames), + Name: "TopicNames", + Value: string(topicNames), + Selector: `request.payload.topicNames`, }, { - "name": "Topics", - "value": string(topics), + Name: "Topics", + Value: string(topics), + Selector: `request.payload.topics`, }, { - "name": "Timeout (ms)", - "value": fmt.Sprintf("%d", int(payload["TimeoutMs"].(float64))), + Name: "Timeout (ms)", + Value: fmt.Sprintf("%d", int(payload["timeoutMs"].(float64))), + Selector: `request.payload.timeoutMs`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep @@ -648,26 +701,28 @@ func representDeleteTopicsResponse(data map[string]interface{}) []interface{} { rep = representResponseHeader(data, rep) - payload := data["Payload"].(map[string]interface{}) - responses, _ := json.Marshal(payload["Responses"].([]interface{})) + payload := data["payload"].(map[string]interface{}) + responses, _ := json.Marshal(payload["responses"].([]interface{})) throttleTimeMs := "" - if payload["ThrottleTimeMs"] != nil { - throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64))) + if payload["throttleTimeMs"] != nil { + throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64))) } - repPayload, _ := json.Marshal([]map[string]string{ + repPayload, _ := json.Marshal([]api.TableData{ { - "name": "Throttle Time (ms)", - "value": throttleTimeMs, + Name: "Throttle Time (ms)", + Value: throttleTimeMs, + Selector: `response.payload.throttleTimeMs`, }, { - "name": "Responses", - "value": string(responses), + Name: "Responses", + Value: string(responses), + Selector: `response.payload.responses`, }, }) - rep = append(rep, map[string]string{ - "type": api.TABLE, - "title": "Payload", - "data": string(repPayload), + rep = append(rep, api.SectionData{ + Type: api.TABLE, + Title: "Payload", + Data: string(repPayload), }) return rep diff --git a/tap/extensions/kafka/main.go b/tap/extensions/kafka/main.go index b677182d1..4c8ea0697 100644 --- a/tap/extensions/kafka/main.go +++ b/tap/extensions/kafka/main.go @@ -15,6 +15,7 @@ var _protocol api.Protocol = api.Protocol{ Name: "kafka", LongName: "Apache Kafka Protocol", Abbreviation: "KAFKA", + Macro: "kafka", Version: "12", BackgroundColor: "#000000", ForegroundColor: "#ffffff", @@ -61,7 +62,7 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co } } -func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry { +func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry { request := item.Pair.Request.Payload.(map[string]interface{}) reqDetails := request["details"].(map[string]interface{}) service := "kafka" @@ -70,76 +71,76 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve } else if resolvedSource != "" { service = resolvedSource } - apiKey := ApiKey(reqDetails["ApiKey"].(float64)) + apiKey := ApiKey(reqDetails["apiKey"].(float64)) summary := "" switch apiKey { case Metadata: - _topics := reqDetails["Payload"].(map[string]interface{})["Topics"] + _topics := reqDetails["payload"].(map[string]interface{})["topics"] if _topics == nil { break } topics := _topics.([]interface{}) for _, topic := range topics { - summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string)) + summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string)) } if len(summary) > 0 { summary = summary[:len(summary)-2] } break case ApiVersions: - summary = reqDetails["ClientID"].(string) + summary = reqDetails["clientID"].(string) break case Produce: - _topics := reqDetails["Payload"].(map[string]interface{})["TopicData"] + _topics := reqDetails["payload"].(map[string]interface{})["topicData"] if _topics == nil { break } topics := _topics.([]interface{}) for _, topic := range topics { - summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Topic"].(string)) + summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["topic"].(string)) } if len(summary) > 0 { summary = summary[:len(summary)-2] } break case Fetch: - _topics := reqDetails["Payload"].(map[string]interface{})["Topics"] + _topics := reqDetails["payload"].(map[string]interface{})["topics"] if _topics == nil { break } topics := _topics.([]interface{}) for _, topic := range topics { - summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Topic"].(string)) + summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["topic"].(string)) } if len(summary) > 0 { summary = summary[:len(summary)-2] } break case ListOffsets: - _topics := reqDetails["Payload"].(map[string]interface{})["Topics"] + _topics := reqDetails["payload"].(map[string]interface{})["topics"] if _topics == nil { break } topics := _topics.([]interface{}) for _, topic := range topics { - summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string)) + summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string)) } if len(summary) > 0 { summary = summary[:len(summary)-2] } break case CreateTopics: - topics := reqDetails["Payload"].(map[string]interface{})["Topics"].([]interface{}) + topics := reqDetails["payload"].(map[string]interface{})["topics"].([]interface{}) for _, topic := range topics { - summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string)) + summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string)) } if len(summary) > 0 { summary = summary[:len(summary)-2] } break case DeleteTopics: - topicNames := reqDetails["TopicNames"].([]string) + topicNames := reqDetails["topicNames"].([]string) for _, name := range topicNames { summary += fmt.Sprintf("%s, ", name) } @@ -148,44 +149,48 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve request["url"] = summary elapsedTime := item.Pair.Response.CaptureTime.Sub(item.Pair.Request.CaptureTime).Round(time.Millisecond).Milliseconds() - entryBytes, _ := json.Marshal(item.Pair) return &api.MizuEntry{ - ProtocolName: _protocol.Name, - ProtocolLongName: _protocol.LongName, - ProtocolAbbreviation: _protocol.Abbreviation, - ProtocolVersion: _protocol.Version, - ProtocolBackgroundColor: _protocol.BackgroundColor, - ProtocolForegroundColor: _protocol.ForegroundColor, - ProtocolFontSize: _protocol.FontSize, - ProtocolReferenceLink: _protocol.ReferenceLink, - EntryId: entryId, - Entry: string(entryBytes), - Url: fmt.Sprintf("%s%s", service, summary), - Method: apiNames[apiKey], - Status: 0, - RequestSenderIp: item.ConnectionInfo.ClientIP, - Service: service, - Timestamp: item.Timestamp, - ElapsedTime: elapsedTime, - Path: summary, - ResolvedSource: resolvedSource, - ResolvedDestination: resolvedDestination, - SourceIp: item.ConnectionInfo.ClientIP, - DestinationIp: item.ConnectionInfo.ServerIP, - SourcePort: item.ConnectionInfo.ClientPort, - DestinationPort: item.ConnectionInfo.ServerPort, - IsOutgoing: item.ConnectionInfo.IsOutgoing, + Protocol: _protocol, + Source: &api.TCP{ + Name: resolvedSource, + IP: item.ConnectionInfo.ClientIP, + Port: item.ConnectionInfo.ClientPort, + }, + Destination: &api.TCP{ + Name: resolvedDestination, + IP: item.ConnectionInfo.ServerIP, + Port: item.ConnectionInfo.ServerPort, + }, + Outgoing: item.ConnectionInfo.IsOutgoing, + Request: reqDetails, + Response: item.Pair.Response.Payload.(map[string]interface{})["details"].(map[string]interface{}), + Url: fmt.Sprintf("%s%s", service, summary), + Method: apiNames[apiKey], + Status: 0, + RequestSenderIp: item.ConnectionInfo.ClientIP, + Service: service, + Timestamp: item.Timestamp, + StartTime: item.Pair.Request.CaptureTime, + ElapsedTime: elapsedTime, + Summary: summary, + ResolvedSource: resolvedSource, + ResolvedDestination: resolvedDestination, + SourceIp: item.ConnectionInfo.ClientIP, + DestinationIp: item.ConnectionInfo.ServerIP, + SourcePort: item.ConnectionInfo.ClientPort, + DestinationPort: item.ConnectionInfo.ServerPort, + IsOutgoing: item.ConnectionInfo.IsOutgoing, } } func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { return &api.BaseEntryDetails{ - Id: entry.EntryId, + Id: entry.Id, Protocol: _protocol, Url: entry.Url, RequestSenderIp: entry.RequestSenderIp, Service: entry.Service, - Summary: entry.Path, + Summary: entry.Summary, StatusCode: entry.Status, Method: entry.Method, Timestamp: entry.Timestamp, @@ -202,49 +207,43 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { } } -func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) { - p = _protocol +func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) { + protoOut = _protocol bodySize = 0 - var root map[string]interface{} - json.Unmarshal([]byte(entry.Entry), &root) representation := make(map[string]interface{}, 0) - request := root["request"].(map[string]interface{})["payload"].(map[string]interface{}) - response := root["response"].(map[string]interface{})["payload"].(map[string]interface{}) - reqDetails := request["details"].(map[string]interface{}) - resDetails := response["details"].(map[string]interface{}) - apiKey := ApiKey(reqDetails["ApiKey"].(float64)) + apiKey := ApiKey(request["apiKey"].(float64)) var repRequest []interface{} var repResponse []interface{} switch apiKey { case Metadata: - repRequest = representMetadataRequest(reqDetails) - repResponse = representMetadataResponse(resDetails) + repRequest = representMetadataRequest(request) + repResponse = representMetadataResponse(response) break case ApiVersions: - repRequest = representApiVersionsRequest(reqDetails) - repResponse = representApiVersionsResponse(resDetails) + repRequest = representApiVersionsRequest(request) + repResponse = representApiVersionsResponse(response) break case Produce: - repRequest = representProduceRequest(reqDetails) - repResponse = representProduceResponse(resDetails) + repRequest = representProduceRequest(request) + repResponse = representProduceResponse(response) break case Fetch: - repRequest = representFetchRequest(reqDetails) - repResponse = representFetchResponse(resDetails) + repRequest = representFetchRequest(request) + repResponse = representFetchResponse(response) break case ListOffsets: - repRequest = representListOffsetsRequest(reqDetails) - repResponse = representListOffsetsResponse(resDetails) + repRequest = representListOffsetsRequest(request) + repResponse = representListOffsetsResponse(response) break case CreateTopics: - repRequest = representCreateTopicsRequest(reqDetails) - repResponse = representCreateTopicsResponse(resDetails) + repRequest = representCreateTopicsRequest(request) + repResponse = representCreateTopicsResponse(response) break case DeleteTopics: - repRequest = representDeleteTopicsRequest(reqDetails) - repResponse = representDeleteTopicsResponse(resDetails) + repRequest = representDeleteTopicsRequest(request) + repResponse = representDeleteTopicsResponse(response) break } @@ -254,4 +253,10 @@ func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []by return } +func (d dissecting) Macros() map[string]string { + return map[string]string{ + `kafka`: fmt.Sprintf(`proto.abbr == "%s"`, _protocol.Abbreviation), + } +} + var Dissector dissecting diff --git a/tap/extensions/kafka/request.go b/tap/extensions/kafka/request.go index b7ac67d7d..263755892 100644 --- a/tap/extensions/kafka/request.go +++ b/tap/extensions/kafka/request.go @@ -10,13 +10,13 @@ import ( ) type Request struct { - Size int32 - ApiKey ApiKey - ApiVersion int16 - CorrelationID int32 - ClientID string - Payload interface{} - CaptureTime time.Time + Size int32 `json:"size"` + ApiKey ApiKey `json:"apiKey"` + ApiVersion int16 `json:"apiVersion"` + CorrelationID int32 `json:"correlationID"` + ClientID string `json:"clientID"` + Payload interface{} `json:"payload"` + CaptureTime time.Time `json:"captureTime"` } func ReadRequest(r io.Reader, tcpID *api.TcpID, superTimer *api.SuperTimer) (apiKey ApiKey, apiVersion int16, err error) { diff --git a/tap/extensions/kafka/response.go b/tap/extensions/kafka/response.go index 574efa8a2..5441e784d 100644 --- a/tap/extensions/kafka/response.go +++ b/tap/extensions/kafka/response.go @@ -10,10 +10,10 @@ import ( ) type Response struct { - Size int32 - CorrelationID int32 - Payload interface{} - CaptureTime time.Time + Size int32 `json:"size"` + CorrelationID int32 `json:"correlationID"` + Payload interface{} `json:"payload"` + CaptureTime time.Time `json:"captureTime"` } func ReadResponse(r io.Reader, tcpID *api.TcpID, superTimer *api.SuperTimer, emitter api.Emitter) (err error) { diff --git a/tap/extensions/kafka/structs.go b/tap/extensions/kafka/structs.go index d9aa5c1cb..67064d85e 100644 --- a/tap/extensions/kafka/structs.go +++ b/tap/extensions/kafka/structs.go @@ -26,38 +26,38 @@ func (acks RequiredAcks) String() string { } type UUID struct { - TimeLow int32 - TimeMid int16 - TimeHiAndVersion int16 - ClockSeq int16 - NodePart1 int32 - NodePart22 int16 + TimeLow int32 `json:"timeLow"` + TimeMid int16 `json:"timeMid"` + TimeHiAndVersion int16 `json:"timeHiAndVersion"` + ClockSeq int16 `json:"clockSeq"` + NodePart1 int32 `json:"nodePart1"` + NodePart22 int16 `json:"nodePart22"` } // Metadata Request (Version: 0) type MetadataRequestTopicV0 struct { - Name string + Name string `json:"name"` } type MetadataRequestV0 struct { - Topics []MetadataRequestTopicV0 + Topics []MetadataRequestTopicV0 `json:"topics"` } // Metadata Request (Version: 4) type MetadataRequestV4 struct { - Topics []MetadataRequestTopicV0 - AllowAutoTopicCreation bool + Topics []MetadataRequestTopicV0 `json:"topics"` + AllowAutoTopicCreation bool `json:"allowAutoTopicCreation"` } // Metadata Request (Version: 8) type MetadataRequestV8 struct { - Topics []MetadataRequestTopicV0 - AllowAutoTopicCreation bool - IncludeClusterAuthorizedOperations bool - IncludeTopicAuthorizedOperations bool + Topics []MetadataRequestTopicV0 `json:"topics"` + AllowAutoTopicCreation bool `json:"allowAutoTopicCreation"` + IncludeClusterAuthorizedOperations bool `json:"includeClusterAuthorizedOperations"` + IncludeTopicAuthorizedOperations bool `json:"includeTopicAuthorizedOperations"` } // Metadata Request (Version: 10) @@ -68,188 +68,188 @@ type MetadataRequestTopicV10 struct { } type MetadataRequestV10 struct { - Topics []MetadataRequestTopicV10 - AllowAutoTopicCreation bool - IncludeClusterAuthorizedOperations bool - IncludeTopicAuthorizedOperations bool + Topics []MetadataRequestTopicV10 `json:"topics"` + AllowAutoTopicCreation bool `json:"allowAutoTopicCreation"` + IncludeClusterAuthorizedOperations bool `json:"includeClusterAuthorizedOperations"` + IncludeTopicAuthorizedOperations bool `json:"includeTopicAuthorizedOperations"` } // Metadata Request (Version: 11) type MetadataRequestV11 struct { - Topics []MetadataRequestTopicV10 - AllowAutoTopicCreation bool - IncludeTopicAuthorizedOperations bool + Topics []MetadataRequestTopicV10 `json:"topics"` + AllowAutoTopicCreation bool `json:"allowAutoTopicCreation"` + IncludeTopicAuthorizedOperations bool `json:"includeTopicAuthorizedOperations"` } // Metadata Response (Version: 0) type BrokerV0 struct { - NodeId int32 - Host string - Port int32 + NodeId int32 `json:"nodeId"` + Host string `json:"host"` + Port int32 `json:"port"` } type PartitionsV0 struct { - ErrorCode int16 - PartitionIndex int32 - LeaderId int32 - ReplicaNodes int32 - IsrNodes int32 + ErrorCode int16 `json:"errorCode"` + PartitionIndex int32 `json:"partitionIndex"` + LeaderId int32 `json:"leaderId"` + ReplicaNodes int32 `json:"replicaNodes"` + IsrNodes int32 `json:"isrNodes"` } type TopicV0 struct { - ErrorCode int16 - Name string - Partitions []PartitionsV0 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + Partitions []PartitionsV0 `json:"partitions"` } type MetadataResponseV0 struct { - Brokers []BrokerV0 - Topics []TopicV0 + Brokers []BrokerV0 `json:"brokers"` + Topics []TopicV0 `json:"topics"` } // Metadata Response (Version: 1) type BrokerV1 struct { - NodeId int32 - Host string - Port int32 - Rack string + NodeId int32 `json:"nodeId"` + Host string `json:"host"` + Port int32 `json:"port"` + Rack string `json:"rack"` } type TopicV1 struct { - ErrorCode int16 - Name string - IsInternal bool - Partitions []PartitionsV0 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + IsInternal bool `json:"isInternal"` + Partitions []PartitionsV0 `json:"partitions"` } type MetadataResponseV1 struct { - Brokers []BrokerV1 - ControllerID int32 - Topics []TopicV1 + Brokers []BrokerV1 `json:"brokers"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV1 `json:"topics"` } // Metadata Response (Version: 2) type MetadataResponseV2 struct { - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV1 + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV1 `json:"topics"` } // Metadata Response (Version: 3) type MetadataResponseV3 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV1 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV1 `json:"topics"` } // Metadata Response (Version: 5) type PartitionsV5 struct { - ErrorCode int16 - PartitionIndex int32 - LeaderId int32 - ReplicaNodes int32 - IsrNodes int32 - OfflineReplicas int32 + ErrorCode int16 `json:"errorCode"` + PartitionIndex int32 `json:"partitionIndex"` + LeaderId int32 `json:"leaderId"` + ReplicaNodes int32 `json:"replicaNodes"` + IsrNodes int32 `json:"isrNodes"` + OfflineReplicas int32 `json:"offlineReplicas"` } type TopicV5 struct { - ErrorCode int16 - Name string - IsInternal bool - Partitions []PartitionsV5 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + IsInternal bool `json:"isInternal"` + Partitions []PartitionsV5 `json:"partitions"` } type MetadataResponseV5 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV5 `json:"topics"` } // Metadata Response (Version: 7) type PartitionsV7 struct { - ErrorCode int16 - PartitionIndex int32 - LeaderId int32 - LeaderEpoch int32 - ReplicaNodes int32 - IsrNodes int32 - OfflineReplicas int32 + ErrorCode int16 `json:"errorCode"` + PartitionIndex int32 `json:"partitionIndex"` + LeaderId int32 `json:"leaderId"` + LeaderEpoch int32 `json:"leaderEpoch"` + ReplicaNodes int32 `json:"replicaNodes"` + IsrNodes int32 `json:"isrNodes"` + OfflineReplicas int32 `json:"offlineReplicas"` } type TopicV7 struct { - ErrorCode int16 - Name string - IsInternal bool - Partitions []PartitionsV7 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + IsInternal bool `json:"isInternal"` + Partitions []PartitionsV7 `json:"partitions"` } type MetadataResponseV7 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV7 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV7 `json:"topics"` } // Metadata Response (Version: 8) type TopicV8 struct { - ErrorCode int16 - Name string - IsInternal bool - Partitions []PartitionsV7 - TopicAuthorizedOperations int32 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + IsInternal bool `json:"isInternal"` + Partitions []PartitionsV7 `json:"partitions"` + TopicAuthorizedOperations int32 `json:"topicAuthorizedOperations"` } type MetadataResponseV8 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV8 - ClusterAuthorizedOperations int32 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV8 `json:"topics"` + ClusterAuthorizedOperations int32 `json:"clusterAuthorizedOperations"` } // Metadata Response (Version: 10) type TopicV10 struct { - ErrorCode int16 - Name string - TopicID UUID - IsInternal bool - Partitions []PartitionsV7 - TopicAuthorizedOperations int32 + ErrorCode int16 `json:"errorCode"` + Name string `json:"name"` + TopicID UUID `json:"topicID"` + IsInternal bool `json:"isInternal"` + Partitions []PartitionsV7 `json:"partitions"` + TopicAuthorizedOperations int32 `json:"topicAuthorizedOperations"` } type MetadataResponseV10 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV10 - ClusterAuthorizedOperations int32 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV10 `json:"topics"` + ClusterAuthorizedOperations int32 `json:"clusterAuthorizedOperations"` } // Metadata Response (Version: 11) type MetadataResponseV11 struct { - ThrottleTimeMs int32 - Brokers []BrokerV1 - ClusterID string - ControllerID int32 - Topics []TopicV10 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Brokers []BrokerV1 `json:"brokers"` + ClusterID string `json:"clusterID"` + ControllerID int32 `json:"controllerID"` + Topics []TopicV10 `json:"topics"` } // ApiVersions Request (Version: 0) @@ -259,742 +259,742 @@ type ApiVersionsRequestV0 struct{} // ApiVersions Request (Version: 3) type ApiVersionsRequestV3 struct { - ClientSoftwareName string - ClientSoftwareVersion string + ClientSoftwareName string `json:"clientSoftwareName"` + ClientSoftwareVersion string `json:"clientSoftwareVersion"` } // ApiVersions Response (Version: 0) type ApiVersionsResponseApiKey struct { - ApiKey int16 - MinVersion int16 - MaxVersion int16 + ApiKey int16 `json:"apiKey"` + MinVersion int16 `json:"minVersion"` + MaxVersion int16 `json:"maxVersion"` } type ApiVersionsResponseV0 struct { - ErrorCode int16 - ApiKeys []ApiVersionsResponseApiKey + ErrorCode int16 `json:"errorCode"` + ApiKeys []ApiVersionsResponseApiKey `json:"apiKeys"` } // ApiVersions Response (Version: 1) type ApiVersionsResponseV1 struct { - ErrorCode int16 - ApiKeys []ApiVersionsResponseApiKey // FIXME: `confluent-kafka-python` causes memory leak - ThrottleTimeMs int32 + ErrorCode int16 `json:"errorCode"` + ApiKeys []ApiVersionsResponseApiKey `json:"apiKeys"` // FIXME: `confluent-kafka-python` causes memory leak + ThrottleTimeMs int32 `json:"throttleTimeMs"` } // Produce Request (Version: 0) // Message is a kafka message type type MessageV0 struct { - Codec int8 // codec used to compress the message contents - CompressionLevel int // compression level - LogAppendTime bool // the used timestamp is LogAppendTime - Key []byte // the message key, may be nil - Value []byte // the message contents - Set *MessageSet // the message set a message might wrap - Version int8 // v1 requires Kafka 0.10 - Timestamp time.Time // the timestamp of the message (version 1+ only) + Codec int8 `json:"codec"` // codec used to compress the message contents + CompressionLevel int `json:"compressionLevel"` // compression level + LogAppendTime bool `json:"logAppendTime"` // the used timestamp is LogAppendTime + Key []byte `json:"key"` // the message key, may be nil + Value []byte `json:"value"` // the message contents + Set *MessageSet `json:"set"` // the message set a message might wrap + Version int8 `json:"version"` // v1 requires Kafka 0.10 + Timestamp time.Time `json:"timestamp"` // the timestamp of the message (version 1+ only) compressedSize int // used for computing the compression ratio metrics } // MessageBlock represents a part of request with message type MessageBlock struct { - Offset int64 - Msg *MessageV0 + Offset int64 `json:"offset"` + Msg *MessageV0 `json:"msg"` } // MessageSet is a replacement for RecordBatch in older versions type MessageSet struct { - PartialTrailingMessage bool // whether the set on the wire contained an incomplete trailing MessageBlock - OverflowMessage bool // whether the set on the wire contained an overflow message - Messages []*MessageBlock + PartialTrailingMessage bool `json:"partialTrailingMessage"` // whether the set on the wire contained an incomplete trailing MessageBlock + OverflowMessage bool `json:"overflowMessage"` // whether the set on the wire contained an overflow message + Messages []*MessageBlock `json:"messages"` } type RecordHeader struct { - HeaderKeyLength int8 - HeaderKey string - HeaderValueLength int8 - Value string + HeaderKeyLength int8 `json:"headerKeyLength"` + HeaderKey string `json:"headerKey"` + HeaderValueLength int8 `json:"headerValueLength"` + Value string `json:"value"` } // Record is kafka record type type RecordV0 struct { - Unknown int8 - Attributes int8 - TimestampDelta int8 - OffsetDelta int8 - KeyLength int8 - Key string - ValueLen int8 - Value string - Headers []RecordHeader + Unknown int8 `json:"unknown"` + Attributes int8 `json:"attributes"` + TimestampDelta int8 `json:"timestampDelta"` + OffsetDelta int8 `json:"offsetDelta"` + KeyLength int8 `json:"keyLength"` + Key string `json:"key"` + ValueLen int8 `json:"valueLen"` + Value string `json:"value"` + Headers []RecordHeader `json:"headers"` } // RecordBatch are records from one kafka request type RecordBatch struct { - BaseOffset int64 - BatchLength int32 - PartitionLeaderEpoch int32 - Magic int8 - Crc int32 - Attributes int16 - LastOffsetDelta int32 - FirstTimestamp int64 - MaxTimestamp int64 - ProducerId int64 - ProducerEpoch int16 - BaseSequence int32 - Record []RecordV0 + BaseOffset int64 `json:"baseOffset"` + BatchLength int32 `json:"batchLength"` + PartitionLeaderEpoch int32 `json:"partitionLeaderEpoch"` + Magic int8 `json:"magic"` + Crc int32 `json:"crc"` + Attributes int16 `json:"attributes"` + LastOffsetDelta int32 `json:"lastOffsetDelta"` + FirstTimestamp int64 `json:"firstTimestamp"` + MaxTimestamp int64 `json:"maxTimestamp"` + ProducerId int64 `json:"producerId"` + ProducerEpoch int16 `json:"producerEpoch"` + BaseSequence int32 `json:"baseSequence"` + Record []RecordV0 `json:"record"` } type Records struct { - RecordBatch RecordBatch + RecordBatch RecordBatch `json:"recordBatch"` // TODO: Implement `MessageSet` // MessageSet MessageSet } type PartitionData struct { - Index int32 - Unknown int32 - Records Records + Index int32 `json:"index"` + Unknown int32 `json:"unknown"` + Records Records `json:"records"` } type Partitions struct { - Length int32 - PartitionData PartitionData + Length int32 `json:"length"` + PartitionData PartitionData `json:"partitionData"` } type TopicData struct { - Topic string - Partitions Partitions + Topic string `json:"topic"` + Partitions Partitions `json:"partitions"` } type ProduceRequestV0 struct { - RequiredAcks RequiredAcks - Timeout int32 - TopicData []TopicData + RequiredAcks RequiredAcks `json:"requiredAcks"` + Timeout int32 `json:"timeout"` + TopicData []TopicData `json:"topicData"` } // Produce Request (Version: 3) type ProduceRequestV3 struct { - TransactionalID string - RequiredAcks RequiredAcks - Timeout int32 - TopicData []TopicData + TransactionalID string `json:"transactionalID"` + RequiredAcks RequiredAcks `json:"requiredAcks"` + Timeout int32 `json:"timeout"` + TopicData []TopicData `json:"topicData"` } // Produce Response (Version: 0) type PartitionResponseV0 struct { - Index int32 - ErrorCode int16 - BaseOffset int64 + Index int32 `json:"index"` + ErrorCode int16 `json:"errorCode"` + BaseOffset int64 `json:"baseOffset"` } type ResponseV0 struct { - Name string - PartitionResponses []PartitionResponseV0 + Name string `json:"name"` + PartitionResponses []PartitionResponseV0 `json:"partitionResponses"` } type ProduceResponseV0 struct { - Responses []ResponseV0 + Responses []ResponseV0 `json:"responses"` } // Produce Response (Version: 1) type ProduceResponseV1 struct { - Responses []ResponseV0 - ThrottleTimeMs int32 + Responses []ResponseV0 `json:"responses"` + ThrottleTimeMs int32 `json:"throttleTimeMs"` } // Produce Response (Version: 2) type PartitionResponseV2 struct { - Index int32 - ErrorCode int16 - BaseOffset int64 - LogAppendTimeMs int64 + Index int32 `json:"index"` + ErrorCode int16 `json:"errorCode"` + BaseOffset int64 `json:"baseOffset"` + LogAppendTimeMs int64 `json:"logAppendTimeMs"` } type ResponseV2 struct { - Name string - PartitionResponses []PartitionResponseV2 + Name string `json:"name"` + PartitionResponses []PartitionResponseV2 `json:"partitionResponses"` } type ProduceResponseV2 struct { - Responses []ResponseV2 - ThrottleTimeMs int32 + Responses []ResponseV2 `json:"responses"` + ThrottleTimeMs int32 `json:"throttleTimeMs"` } // Produce Response (Version: 5) type PartitionResponseV5 struct { - Index int32 - ErrorCode int16 - BaseOffset int64 - LogAppendTimeMs int64 - LogStartOffset int64 + Index int32 `json:"index"` + ErrorCode int16 `json:"errorCode"` + BaseOffset int64 `json:"baseOffset"` + LogAppendTimeMs int64 `json:"logAppendTimeMs"` + LogStartOffset int64 `json:"logStartOffset"` } type ResponseV5 struct { - Name string - PartitionResponses []PartitionResponseV5 + Name string `json:"name"` + PartitionResponses []PartitionResponseV5 `json:"partitionResponses"` } type ProduceResponseV5 struct { - Responses []ResponseV5 - ThrottleTimeMs int32 + Responses []ResponseV5 `json:"responses"` + ThrottleTimeMs int32 `json:"throttleTimeMs"` } // Produce Response (Version: 8) type RecordErrors struct { - BatchIndex int32 - BatchIndexErrorMessage string + BatchIndex int32 `json:"batchIndex"` + BatchIndexErrorMessage string `json:"batchIndexErrorMessage"` } type PartitionResponseV8 struct { - Index int32 - ErrorCode int16 - BaseOffset int64 - LogAppendTimeMs int64 - LogStartOffset int64 - RecordErrors RecordErrors - ErrorMessage string + Index int32 `json:"index"` + ErrorCode int16 `json:"errorCode"` + BaseOffset int64 `json:"baseOffset"` + LogAppendTimeMs int64 `json:"logAppendTimeMs"` + LogStartOffset int64 `json:"logStartOffset"` + RecordErrors RecordErrors `json:"recordErrors"` + ErrorMessage string `json:"errorMessage"` } type ResponseV8 struct { - Name string - PartitionResponses []PartitionResponseV8 + Name string `json:"name"` + PartitionResponses []PartitionResponseV8 `json:"partitionResponses"` } type ProduceResponseV8 struct { - Responses []ResponseV8 - ThrottleTimeMs int32 + Responses []ResponseV8 `json:"responses"` + ThrottleTimeMs int32 `json:"throttleTimeMs"` } // Fetch Request (Version: 0) type FetchPartitionV0 struct { - Partition int32 - FetchOffset int64 - PartitionMaxBytes int32 + Partition int32 `json:"partition"` + FetchOffset int64 `json:"fetchOffset"` + PartitionMaxBytes int32 `json:"partitionMaxBytes"` } type FetchTopicV0 struct { - Topic string - Partitions []FetchPartitionV0 + Topic string `json:"topic"` + Partitions []FetchPartitionV0 `json:"partitions"` } type FetchRequestV0 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - Topics []FetchTopicV0 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + Topics []FetchTopicV0 `json:"topics"` } // Fetch Request (Version: 3) type FetchRequestV3 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - Topics []FetchTopicV0 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + Topics []FetchTopicV0 `json:"topics"` } // Fetch Request (Version: 4) type FetchRequestV4 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - IsolationLevel int8 - Topics []FetchTopicV0 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + IsolationLevel int8 `json:"isolationLevel"` + Topics []FetchTopicV0 `json:"topics"` } // Fetch Request (Version: 5) type FetchPartitionV5 struct { - Partition int32 - FetchOffset int64 - LogStartOffset int64 - PartitionMaxBytes int32 + Partition int32 `json:"partition"` + FetchOffset int64 `json:"fetchOffset"` + LogStartOffset int64 `json:"logStartOffset"` + PartitionMaxBytes int32 `json:"partitionMaxBytes"` } type FetchTopicV5 struct { - Topic string - Partitions []FetchPartitionV5 + Topic string `json:"topic"` + Partitions []FetchPartitionV5 `json:"partitions"` } type FetchRequestV5 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - IsolationLevel int8 - Topics []FetchTopicV5 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + IsolationLevel int8 `json:"isolationLevel"` + Topics []FetchTopicV5 `json:"topics"` } // Fetch Request (Version: 7) type ForgottenTopicsDataV7 struct { - Topic string - Partitions []int32 + Topic string `json:"topic"` + Partitions []int32 `json:"partitions"` } type FetchRequestV7 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - IsolationLevel int8 - SessionId int32 - SessionEpoch int32 - Topics []FetchTopicV5 - ForgottenTopicsData ForgottenTopicsDataV7 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + IsolationLevel int8 `json:"isolationLevel"` + SessionId int32 `json:"sessionId"` + SessionEpoch int32 `json:"sessionEpoch"` + Topics []FetchTopicV5 `json:"topics"` + ForgottenTopicsData ForgottenTopicsDataV7 `json:"forgottenTopicsData"` } // Fetch Request (Version: 9) type FetchPartitionV9 struct { - Partition int32 - CurrentLeaderEpoch int32 - FetchOffset int64 - LogStartOffset int64 - PartitionMaxBytes int32 + Partition int32 `json:"partition"` + CurrentLeaderEpoch int32 `json:"currentLeaderEpoch"` + FetchOffset int64 `json:"fetchOffset"` + LogStartOffset int64 `json:"logStartOffset"` + PartitionMaxBytes int32 `json:"partitionMaxBytes"` } type FetchTopicV9 struct { - Topic string - Partitions []FetchPartitionV9 + Topic string `json:"topic"` + Partitions []FetchPartitionV9 `json:"partitions"` } type FetchRequestV9 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - IsolationLevel int8 - SessionId int32 - SessionEpoch int32 - Topics []FetchTopicV9 - ForgottenTopicsData ForgottenTopicsDataV7 + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + IsolationLevel int8 `json:"isolationLevel"` + SessionId int32 `json:"sessionId"` + SessionEpoch int32 `json:"sessionEpoch"` + Topics []FetchTopicV9 `json:"topics"` + ForgottenTopicsData ForgottenTopicsDataV7 `json:"forgottenTopicsData"` } // Fetch Request (Version: 11) type FetchRequestV11 struct { - ReplicaId int32 - MaxWaitMs int32 - MinBytes int32 - MaxBytes int32 - IsolationLevel int8 - SessionId int32 - SessionEpoch int32 - Topics []FetchTopicV9 - ForgottenTopicsData ForgottenTopicsDataV7 - RackId string + ReplicaId int32 `json:"replicaId"` + MaxWaitMs int32 `json:"maxWaitMs"` + MinBytes int32 `json:"minBytes"` + MaxBytes int32 `json:"maxBytes"` + IsolationLevel int8 `json:"isolationLevel"` + SessionId int32 `json:"sessionId"` + SessionEpoch int32 `json:"sessionEpoch"` + Topics []FetchTopicV9 `json:"topics"` + ForgottenTopicsData ForgottenTopicsDataV7 `json:"forgottenTopicsData"` + RackId string `json:"rackId"` } // Fetch Response (Version: 0) type PartitionResponseFetchV0 struct { - Partition int32 - ErrorCode int16 - HighWatermark int64 - RecordSet Records + Partition int32 `json:"partition"` + ErrorCode int16 `json:"errorCode"` + HighWatermark int64 `json:"highWatermark"` + RecordSet Records `json:"recordSet"` } type ResponseFetchV0 struct { - Topic string - PartitionResponses []PartitionResponseFetchV0 + Topic string `json:"topic"` + PartitionResponses []PartitionResponseFetchV0 `json:"partitionResponses"` } type FetchResponseV0 struct { - Responses []ResponseFetchV0 + Responses []ResponseFetchV0 `json:"responses"` } // Fetch Response (Version: 1) type FetchResponseV1 struct { - ThrottleTimeMs int32 - Responses []ResponseFetchV0 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []ResponseFetchV0 `json:"responses"` } // Fetch Response (Version: 4) type AbortedTransactionsV4 struct { - ProducerId int32 - FirstOffset int32 + ProducerId int32 `json:"producerId"` + FirstOffset int32 `json:"firstOffset"` } type PartitionResponseFetchV4 struct { - Partition int32 - ErrorCode int16 - HighWatermark int64 - LastStableOffset int64 - AbortedTransactions AbortedTransactionsV4 - RecordSet Records + Partition int32 `json:"partition"` + ErrorCode int16 `json:"errorCode"` + HighWatermark int64 `json:"highWatermark"` + LastStableOffset int64 `json:"lastStableOffset"` + AbortedTransactions AbortedTransactionsV4 `json:"abortedTransactions"` + RecordSet Records `json:"recordSet"` } type ResponseFetchV4 struct { - Topic string - PartitionResponses []PartitionResponseFetchV4 + Topic string `json:"topic"` + PartitionResponses []PartitionResponseFetchV4 `json:"partitionResponses"` } type FetchResponseV4 struct { - ThrottleTimeMs int32 - Responses []ResponseFetchV4 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []ResponseFetchV4 `json:"responses"` } // Fetch Response (Version: 5) type PartitionResponseFetchV5 struct { - Partition int32 - ErrorCode int16 - HighWatermark int64 - LastStableOffset int64 - LogStartOffset int64 - AbortedTransactions AbortedTransactionsV4 - RecordSet Records + Partition int32 `json:"partition"` + ErrorCode int16 `json:"errorCode"` + HighWatermark int64 `json:"highWatermark"` + LastStableOffset int64 `json:"lastStableOffset"` + LogStartOffset int64 `json:"logStartOffset"` + AbortedTransactions AbortedTransactionsV4 `json:"abortedTransactions"` + RecordSet Records `json:"recordSet"` } type ResponseFetchV5 struct { - Topic string - PartitionResponses []PartitionResponseFetchV5 + Topic string `json:"topic"` + PartitionResponses []PartitionResponseFetchV5 `json:"partitionResponses"` } type FetchResponseV5 struct { - ThrottleTimeMs int32 - Responses []ResponseFetchV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []ResponseFetchV5 `json:"responses"` } // Fetch Response (Version: 7) type FetchResponseV7 struct { - ThrottleTimeMs int32 - ErrorCode int16 - SessionId int32 - Responses []ResponseFetchV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + ErrorCode int16 `json:"errorCode"` + SessionId int32 `json:"sessionId"` + Responses []ResponseFetchV5 `json:"responses"` } // Fetch Response (Version: 11) type PartitionResponseFetchV11 struct { - Partition int32 - ErrorCode int16 - HighWatermark int64 - LastStableOffset int64 - LogStartOffset int64 - AbortedTransactions AbortedTransactionsV4 - PreferredReadReplica int32 - RecordSet Records + Partition int32 `json:"partition"` + ErrorCode int16 `json:"errorCode"` + HighWatermark int64 `json:"highWatermark"` + LastStableOffset int64 `json:"lastStableOffset"` + LogStartOffset int64 `json:"logStartOffset"` + AbortedTransactions AbortedTransactionsV4 `json:"abortedTransactions"` + PreferredReadReplica int32 `json:"preferredReadReplica"` + RecordSet Records `json:"recordSet"` } type ResponseFetchV11 struct { - Topic string - PartitionResponses []PartitionResponseFetchV11 + Topic string `json:"topic"` + PartitionResponses []PartitionResponseFetchV11 `json:"partitionResponses"` } type FetchResponseV11 struct { - ThrottleTimeMs int32 - ErrorCode int16 - SessionId int32 - Responses []ResponseFetchV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + ErrorCode int16 `json:"errorCode"` + SessionId int32 `json:"sessionId"` + Responses []ResponseFetchV5 `json:"responses"` } // ListOffsets Request (Version: 0) type ListOffsetsRequestPartitionV0 struct { - PartitionIndex int32 - Timestamp int64 - MaxNumOffsets int32 + PartitionIndex int32 `json:"partitionIndex"` + Timestamp int64 `json:"timestamp"` + MaxNumOffsets int32 `json:"maxNumOffsets"` } type ListOffsetsRequestTopicV0 struct { - Name string - Partitions []ListOffsetsRequestPartitionV0 + Name string `json:"name"` + Partitions []ListOffsetsRequestPartitionV0 `json:"partitions"` } type ListOffsetsRequestV0 struct { - ReplicaId int32 - Topics []ListOffsetsRequestTopicV0 + ReplicaId int32 `json:"replicaId"` + Topics []ListOffsetsRequestTopicV0 `json:"topics"` } // ListOffsets Request (Version: 1) type ListOffsetsRequestPartitionV1 struct { - PartitionIndex int32 - Timestamp int64 + PartitionIndex int32 `json:"partitionIndex"` + Timestamp int64 `json:"timestamp"` } type ListOffsetsRequestTopicV1 struct { - Name string - Partitions []ListOffsetsRequestPartitionV1 + Name string `json:"name"` + Partitions []ListOffsetsRequestPartitionV1 `json:"partitions"` } type ListOffsetsRequestV1 struct { - ReplicaId int32 - Topics []ListOffsetsRequestTopicV1 + ReplicaId int32 `json:"replicaId"` + Topics []ListOffsetsRequestTopicV1 `json:"topics"` } // ListOffsets Request (Version: 2) type ListOffsetsRequestV2 struct { - ReplicaId int32 - IsolationLevel int8 - Topics []ListOffsetsRequestTopicV1 + ReplicaId int32 `json:"replicaId"` + IsolationLevel int8 `json:"isolationLevel"` + Topics []ListOffsetsRequestTopicV1 `json:"topics"` } // ListOffsets Request (Version: 4) type ListOffsetsRequestPartitionV4 struct { - PartitionIndex int32 - CurrentLeaderEpoch int32 - Timestamp int64 + PartitionIndex int32 `json:"partitionIndex"` + CurrentLeaderEpoch int32 `json:"currentLeaderEpoch"` + Timestamp int64 `json:"timestamp"` } type ListOffsetsRequestTopicV4 struct { - Name string - Partitions []ListOffsetsRequestPartitionV4 + Name string `json:"name"` + Partitions []ListOffsetsRequestPartitionV4 `json:"partitions"` } type ListOffsetsRequestV4 struct { - ReplicaId int32 - Topics []ListOffsetsRequestTopicV4 + ReplicaId int32 `json:"replicaId"` + Topics []ListOffsetsRequestTopicV4 `json:"topics"` } // ListOffsets Response (Version: 0) type ListOffsetsResponsePartitionV0 struct { - PartitionIndex int32 - ErrorCode int16 - OldStyleOffsets int64 + PartitionIndex int32 `json:"partitionIndex"` + ErrorCode int16 `json:"errorCode"` + OldStyleOffsets int64 `json:"oldStyleOffsets"` } type ListOffsetsResponseTopicV0 struct { - Name string - Partitions []ListOffsetsResponsePartitionV0 + Name string `json:"name"` + Partitions []ListOffsetsResponsePartitionV0 `json:"partitions"` } type ListOffsetsResponseV0 struct { - Topics []ListOffsetsResponseTopicV0 + Topics []ListOffsetsResponseTopicV0 `json:"topics"` } // ListOffsets Response (Version: 1) type ListOffsetsResponsePartitionV1 struct { - PartitionIndex int32 - ErrorCode int16 - Timestamp int64 - Offset int64 + PartitionIndex int32 `json:"partitionIndex"` + ErrorCode int16 `json:"errorCode"` + Timestamp int64 `json:"timestamp"` + Offset int64 `json:"offset"` } type ListOffsetsResponseTopicV1 struct { - Name string - Partitions []ListOffsetsResponsePartitionV1 + Name string `json:"name"` + Partitions []ListOffsetsResponsePartitionV1 `json:"partitions"` } type ListOffsetsResponseV1 struct { - Topics []ListOffsetsResponseTopicV1 + Topics []ListOffsetsResponseTopicV1 `json:"topics"` } // ListOffsets Response (Version: 2) type ListOffsetsResponseV2 struct { - ThrottleTimeMs int32 - Topics []ListOffsetsResponseTopicV1 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Topics []ListOffsetsResponseTopicV1 `json:"topics"` } // ListOffsets Response (Version: 4) type ListOffsetsResponsePartitionV4 struct { - PartitionIndex int32 - ErrorCode int16 - Timestamp int64 - Offset int64 - LeaderEpoch int32 + PartitionIndex int32 `json:"partitionIndex"` + ErrorCode int16 `json:"errorCode"` + Timestamp int64 `json:"timestamp"` + Offset int64 `json:"offset"` + LeaderEpoch int32 `json:"leaderEpoch"` } type ListOffsetsResponseTopicV4 struct { - Name string - Partitions []ListOffsetsResponsePartitionV4 + Name string `json:"name"` + Partitions []ListOffsetsResponsePartitionV4 `json:"partitions"` } type ListOffsetsResponseV4 struct { - Topics []ListOffsetsResponseTopicV4 + Topics []ListOffsetsResponseTopicV4 `json:"topics"` } // CreateTopics Request (Version: 0) type AssignmentsV0 struct { - PartitionIndex int32 - BrokerIds []int32 + PartitionIndex int32 `json:"partitionIndex"` + BrokerIds []int32 `json:"brokerIds"` } type CreateTopicsRequestConfigsV0 struct { - Name string - Value string + Name string `json:"name"` + Value string `json:"value"` } type CreateTopicsRequestTopicV0 struct { - Name string - NumPartitions int32 - ReplicationFactor int16 - Assignments []AssignmentsV0 - Configs []CreateTopicsRequestConfigsV0 + Name string `json:"name"` + NumPartitions int32 `json:"numPartitions"` + ReplicationFactor int16 `json:"replicationFactor"` + Assignments []AssignmentsV0 `json:"assignments"` + Configs []CreateTopicsRequestConfigsV0 `json:"configs"` } type CreateTopicsRequestV0 struct { - Topics []CreateTopicsRequestTopicV0 - TimeoutMs int32 + Topics []CreateTopicsRequestTopicV0 `json:"topics"` + TimeoutMs int32 `json:"timeoutMs"` } // CreateTopics Request (Version: 1) type CreateTopicsRequestV1 struct { - Topics []CreateTopicsRequestTopicV0 - TimeoutMs int32 - ValidateOnly bool + Topics []CreateTopicsRequestTopicV0 `json:"topics"` + TimeoutMs int32 `json:"timeoutMs"` + ValidateOnly bool `json:"validateOnly"` } // CreateTopics Response (Version: 0) type CreateTopicsResponseTopicV0 struct { - Name string - ErrorCode int16 + Name string `json:"name"` + ErrorCode int16 `json:"errorCode"` } type CreateTopicsResponseV0 struct { - Topics []CreateTopicsResponseTopicV0 + Topics []CreateTopicsResponseTopicV0 `json:"topics"` } // CreateTopics Response (Version: 1) type CreateTopicsResponseTopicV1 struct { - Name string - ErrorCode int16 - ErrorMessage string + Name string `json:"name"` + ErrorCode int16 `json:"errorCode"` + ErrorMessage string `json:"errorMessage"` } type CreateTopicsResponseV1 struct { - Topics []CreateTopicsResponseTopicV1 + Topics []CreateTopicsResponseTopicV1 `json:"topics"` } // CreateTopics Response (Version: 2) type CreateTopicsResponseV2 struct { - ThrottleTimeMs int32 - Topics []CreateTopicsResponseTopicV1 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Topics []CreateTopicsResponseTopicV1 `json:"topics"` } // CreateTopics Response (Version: 5) type CreateTopicsResponseConfigsV5 struct { - Name string - Value string - ReadOnly bool - ConfigSource int8 - IsSensitive bool + Name string `json:"name"` + Value string `json:"value"` + ReadOnly bool `json:"readOnly"` + ConfigSource int8 `json:"configSource"` + IsSensitive bool `json:"isSensitive"` } type CreateTopicsResponseTopicV5 struct { - Name string - ErrorCode int16 - ErrorMessage string - NumPartitions int32 - ReplicationFactor int16 - Configs []CreateTopicsResponseConfigsV5 + Name string `json:"name"` + ErrorCode int16 `json:"errorCode"` + ErrorMessage string `json:"errorMessage"` + NumPartitions int32 `json:"numPartitions"` + ReplicationFactor int16 `json:"replicationFactor"` + Configs []CreateTopicsResponseConfigsV5 `json:"configs"` } type CreateTopicsResponseV5 struct { - ThrottleTimeMs int32 - Topics []CreateTopicsResponseTopicV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Topics []CreateTopicsResponseTopicV5 `json:"topics"` } // CreateTopics Response (Version: 7) type CreateTopicsResponseTopicV7 struct { - Name string - TopicID UUID - ErrorCode int16 - ErrorMessage string - NumPartitions int32 - ReplicationFactor int16 - Configs []CreateTopicsResponseConfigsV5 + Name string `json:"name"` + TopicID UUID `json:"topicID"` + ErrorCode int16 `json:"errorCode"` + ErrorMessage string `json:"errorMessage"` + NumPartitions int32 `json:"numPartitions"` + ReplicationFactor int16 `json:"replicationFactor"` + Configs []CreateTopicsResponseConfigsV5 `json:"configs"` } type CreateTopicsResponseV7 struct { - ThrottleTimeMs int32 - Topics []CreateTopicsResponseTopicV7 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Topics []CreateTopicsResponseTopicV7 `json:"topics"` } // DeleteTopics Request (Version: 0) type DeleteTopicsRequestV0 struct { - TopicNames []string - TimeoutMs int32 + TopicNames []string `json:"topicNames"` + TimeoutMs int32 `json:"timeoutMs"` } // DeleteTopics Request (Version: 6) type DeleteTopicsRequestTopicV6 struct { - Name string - UUID UUID + Name string `json:"name"` + UUID UUID `json:"uuid"` } type DeleteTopicsRequestV6 struct { - Topics []DeleteTopicsRequestTopicV6 - TimeoutMs int32 + Topics []DeleteTopicsRequestTopicV6 `json:"topics"` + TimeoutMs int32 `json:"timeoutMs"` } // DeleteTopics Response (Version: 0) type DeleteTopicsReponseResponseV0 struct { - Name string - ErrorCode int16 + Name string `json:"name"` + ErrorCode int16 `json:"errorCode"` } type DeleteTopicsReponseV0 struct { - Responses []DeleteTopicsReponseResponseV0 + Responses []DeleteTopicsReponseResponseV0 `json:"responses"` } // DeleteTopics Response (Version: 1) type DeleteTopicsReponseV1 struct { - ThrottleTimeMs int32 - Responses []DeleteTopicsReponseResponseV0 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []DeleteTopicsReponseResponseV0 `json:"responses"` } // DeleteTopics Response (Version: 5) type DeleteTopicsReponseResponseV5 struct { - Name string - ErrorCode int16 - ErrorMessage string + Name string `json:"name"` + ErrorCode int16 `json:"errorCode"` + ErrorMessage string `json:"errorMessage"` } type DeleteTopicsReponseV5 struct { - ThrottleTimeMs int32 - Responses []DeleteTopicsReponseResponseV5 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []DeleteTopicsReponseResponseV5 `json:"responses"` } // DeleteTopics Response (Version: 6) type DeleteTopicsReponseResponseV6 struct { - Name string - TopicID UUID - ErrorCode int16 - ErrorMessage string + Name string `json:"name"` + TopicID UUID `json:"topicID"` + ErrorCode int16 `json:"errorCode"` + ErrorMessage string `json:"errorMessage"` } type DeleteTopicsReponseV6 struct { - ThrottleTimeMs int32 - Responses []DeleteTopicsReponseResponseV6 + ThrottleTimeMs int32 `json:"throttleTimeMs"` + Responses []DeleteTopicsReponseResponseV6 `json:"responses"` } diff --git a/tap/extensions/redis/helpers.go b/tap/extensions/redis/helpers.go index a020b722d..3f8b61791 100644 --- a/tap/extensions/redis/helpers.go +++ b/tap/extensions/redis/helpers.go @@ -2,6 +2,7 @@ package main import ( "encoding/json" + "fmt" "github.com/up9inc/mizu/tap/api" ) @@ -24,33 +25,38 @@ type RedisWrapper struct { Details interface{} `json:"details"` } -func representGeneric(generic map[string]interface{}) (representation []interface{}) { - details, _ := json.Marshal([]map[string]string{ +func representGeneric(generic map[string]interface{}, selectorPrefix string) (representation []interface{}) { + details, _ := json.Marshal([]api.TableData{ { - "name": "Type", - "value": generic["type"].(string), + Name: "Type", + Value: generic["type"].(string), + Selector: fmt.Sprintf("%stype", selectorPrefix), }, { - "name": "Command", - "value": generic["command"].(string), + Name: "Command", + Value: generic["command"].(string), + Selector: fmt.Sprintf("%scommand", selectorPrefix), }, { - "name": "Key", - "value": generic["key"].(string), + Name: "Key", + Value: generic["key"].(string), + Selector: fmt.Sprintf("%skey", selectorPrefix), }, { - "name": "Value", - "value": generic["value"].(string), + Name: "Value", + Value: generic["value"].(string), + Selector: fmt.Sprintf("%svalue", selectorPrefix), }, { - "name": "Keyword", - "value": generic["keyword"].(string), + Name: "Keyword", + Value: generic["keyword"].(string), + Selector: fmt.Sprintf("%skeyword", selectorPrefix), }, }) - representation = append(representation, map[string]string{ - "type": api.TABLE, - "title": "Details", - "data": string(details), + representation = append(representation, api.SectionData{ + Type: api.TABLE, + Title: "Details", + Data: string(details), }) return diff --git a/tap/extensions/redis/main.go b/tap/extensions/redis/main.go index be0650bfc..387a82b2f 100644 --- a/tap/extensions/redis/main.go +++ b/tap/extensions/redis/main.go @@ -13,6 +13,7 @@ var protocol api.Protocol = api.Protocol{ Name: "redis", LongName: "Redis Serialization Protocol", Abbreviation: "REDIS", + Macro: "redis", Version: "3.x", BackgroundColor: "#a41e11", ForegroundColor: "#ffffff", @@ -57,9 +58,12 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co } } -func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry { +func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry { request := item.Pair.Request.Payload.(map[string]interface{}) + response := item.Pair.Response.Payload.(map[string]interface{}) reqDetails := request["details"].(map[string]interface{}) + resDetails := response["details"].(map[string]interface{}) + service := "redis" if resolvedDestination != "" { service = resolvedDestination @@ -78,45 +82,49 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve } request["url"] = summary - entryBytes, _ := json.Marshal(item.Pair) return &api.MizuEntry{ - ProtocolName: protocol.Name, - ProtocolLongName: protocol.LongName, - ProtocolAbbreviation: protocol.Abbreviation, - ProtocolVersion: protocol.Version, - ProtocolBackgroundColor: protocol.BackgroundColor, - ProtocolForegroundColor: protocol.ForegroundColor, - ProtocolFontSize: protocol.FontSize, - ProtocolReferenceLink: protocol.ReferenceLink, - EntryId: entryId, - Entry: string(entryBytes), - Url: fmt.Sprintf("%s%s", service, summary), - Method: method, - Status: 0, - RequestSenderIp: item.ConnectionInfo.ClientIP, - Service: service, - Timestamp: item.Timestamp, - ElapsedTime: 0, - Path: summary, - ResolvedSource: resolvedSource, - ResolvedDestination: resolvedDestination, - SourceIp: item.ConnectionInfo.ClientIP, - DestinationIp: item.ConnectionInfo.ServerIP, - SourcePort: item.ConnectionInfo.ClientPort, - DestinationPort: item.ConnectionInfo.ServerPort, - IsOutgoing: item.ConnectionInfo.IsOutgoing, + Protocol: protocol, + Source: &api.TCP{ + Name: resolvedSource, + IP: item.ConnectionInfo.ClientIP, + Port: item.ConnectionInfo.ClientPort, + }, + Destination: &api.TCP{ + Name: resolvedDestination, + IP: item.ConnectionInfo.ServerIP, + Port: item.ConnectionInfo.ServerPort, + }, + Outgoing: item.ConnectionInfo.IsOutgoing, + Request: reqDetails, + Response: resDetails, + Url: fmt.Sprintf("%s%s", service, summary), + Method: method, + Status: 0, + RequestSenderIp: item.ConnectionInfo.ClientIP, + Service: service, + Timestamp: item.Timestamp, + StartTime: item.Pair.Request.CaptureTime, + ElapsedTime: 0, + Summary: summary, + ResolvedSource: resolvedSource, + ResolvedDestination: resolvedDestination, + SourceIp: item.ConnectionInfo.ClientIP, + DestinationIp: item.ConnectionInfo.ServerIP, + SourcePort: item.ConnectionInfo.ClientPort, + DestinationPort: item.ConnectionInfo.ServerPort, + IsOutgoing: item.ConnectionInfo.IsOutgoing, } } func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { return &api.BaseEntryDetails{ - Id: entry.EntryId, + Id: entry.Id, Protocol: protocol, Url: entry.Url, RequestSenderIp: entry.RequestSenderIp, Service: entry.Service, - Summary: entry.Path, + Summary: entry.Summary, StatusCode: entry.Status, Method: entry.Method, Timestamp: entry.Timestamp, @@ -133,22 +141,22 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails { } } -func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) { - p = protocol +func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) { + protoOut = protocol bodySize = 0 - var root map[string]interface{} - json.Unmarshal([]byte(entry.Entry), &root) representation := make(map[string]interface{}, 0) - request := root["request"].(map[string]interface{})["payload"].(map[string]interface{}) - response := root["response"].(map[string]interface{})["payload"].(map[string]interface{}) - reqDetails := request["details"].(map[string]interface{}) - resDetails := response["details"].(map[string]interface{}) - repRequest := representGeneric(reqDetails) - repResponse := representGeneric(resDetails) + repRequest := representGeneric(request, `request.`) + repResponse := representGeneric(response, `response.`) representation["request"] = repRequest representation["response"] = repResponse object, err = json.Marshal(representation) return } +func (d dissecting) Macros() map[string]string { + return map[string]string{ + `redis`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation), + } +} + var Dissector dissecting diff --git a/ui/package-lock.json b/ui/package-lock.json index f6a4eff73..745364d11 100644 --- a/ui/package-lock.json +++ b/ui/package-lock.json @@ -1769,6 +1769,33 @@ } } }, + "@mapbox/rehype-prism": { + "version": "0.7.0", + "resolved": "https://registry.npmjs.org/@mapbox/rehype-prism/-/rehype-prism-0.7.0.tgz", + "integrity": "sha512-zSG46selA6v+3THhCatTyOt9DuTzxTIVTxTbcj15kFmxPDtjzZ5VoFVCLZfjWFouYa9PiXxcbMLLhJoVzCxh9w==", + "requires": { + "hast-util-to-string": "^1.0.4", + "refractor": "^3.4.0", + "unist-util-visit": "^2.0.3" + }, + "dependencies": { + "prismjs": { + "version": "1.24.1", + "resolved": "https://registry.npmjs.org/prismjs/-/prismjs-1.24.1.tgz", + "integrity": "sha512-mNPsedLuk90RVJioIky8ANZEwYm5w9LcvCXrxHlwf4fNVSn8jEipMybMkWUyyF0JhnC+C4VcOVSBuHRKs1L5Ow==" + }, + "refractor": { + "version": "3.4.0", + "resolved": "https://registry.npmjs.org/refractor/-/refractor-3.4.0.tgz", + "integrity": "sha512-dBeD02lC5eytm9Gld2Mx0cMcnR+zhSnsTfPpWqFaMgUMJfC9A6bcN3Br/NaXrnBJcuxnLFR90k1jrkaSyV8umg==", + "requires": { + "hastscript": "^6.0.0", + "parse-entities": "^2.0.0", + "prismjs": "~1.24.0" + } + } + } + }, "@material-ui/core": { "version": "4.12.2", "resolved": "https://registry.npmjs.org/@material-ui/core/-/core-4.12.2.tgz", @@ -1788,6 +1815,14 @@ "react-transition-group": "^4.4.0" } }, + "@material-ui/icons": { + "version": "4.11.2", + "resolved": "https://registry.npmjs.org/@material-ui/icons/-/icons-4.11.2.tgz", + "integrity": "sha512-fQNsKX2TxBmqIGJCSi3tGTO/gZ+eJgWmMJkgDiOfyNaunNaxcklJQFaFogYcFl0qFuaEz1qaXYXboa/bUXVSOQ==", + "requires": { + "@babel/runtime": "^7.4.4" + } + }, "@material-ui/lab": { "version": "4.0.0-alpha.60", "resolved": "https://registry.npmjs.org/@material-ui/lab/-/lab-4.0.0-alpha.60.tgz", @@ -2406,6 +2441,11 @@ "resolved": "https://registry.npmjs.org/@types/parse-json/-/parse-json-4.0.0.tgz", "integrity": "sha512-//oorEZjL6sbPcKUaCdIGlIUeH26mgzimjBB77G6XRgnDl/L5wOnpyBGRe/Mmf5CVW3PwEBE1NjiMZ/ssFh4wA==" }, + "@types/parse5": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/@types/parse5/-/parse5-6.0.1.tgz", + "integrity": "sha512-ARATsLdrGPUnaBvxLhUlnltcMgn7pQG312S8ccdYlnyijabrX9RN/KN/iGj9Am96CoW8e/K9628BA7Bv4XHdrA==" + }, "@types/prettier": { "version": "2.2.3", "resolved": "https://registry.npmjs.org/@types/prettier/-/prettier-2.2.3.tgz", @@ -2629,6 +2669,26 @@ "eslint-visitor-keys": "^2.0.0" } }, + "@uiw/react-textarea-code-editor": { + "version": "1.4.12", + "resolved": "https://registry.npmjs.org/@uiw/react-textarea-code-editor/-/react-textarea-code-editor-1.4.12.tgz", + "integrity": "sha512-op0aIRxX8hLi8OLwm/23dQ2X4o2xHUoK2pLr1DWBFgNbIh4L+RM5zByNFjV8mHfi/NvE6mQLB6LtMd1qaor5MQ==", + "requires": { + "@babel/runtime": "7.15.4", + "@mapbox/rehype-prism": "0.7.0", + "rehype": "12.0.0" + }, + "dependencies": { + "@babel/runtime": { + "version": "7.15.4", + "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.15.4.tgz", + "integrity": "sha512-99catp6bHCaxr4sJ/DbTGgHS4+Rs2RVd2g7iOap6SLGPDknRK9ztKNsE/Fg6QhSeh1FGE5f6gHGQmvvn3I3xhw==", + "requires": { + "regenerator-runtime": "^0.13.4" + } + } + } + }, "@webassemblyjs/ast": { "version": "1.9.0", "resolved": "https://registry.npmjs.org/@webassemblyjs/ast/-/ast-1.9.0.tgz", @@ -3689,6 +3749,11 @@ "resolved": "https://registry.npmjs.org/babylon/-/babylon-6.18.0.tgz", "integrity": "sha512-q/UEjfGJ2Cm3oKV71DJz9d25TPnq5rhBVL2Q4fA5wcC3jcrdn7+SssEybFIxwAvvP+YCsCYNKughoF33GxgycQ==" }, + "bail": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/bail/-/bail-2.0.1.tgz", + "integrity": "sha512-d5FoTAr2S5DSUPKl85WNm2yUwsINN8eidIdIwsOge2t33DaOfOdSmmsI11jMN3GmALCXaw+Y6HMVHDzePshFAA==" + }, "balanced-match": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz", @@ -4179,6 +4244,11 @@ "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", "integrity": "sha1-G2gcIf+EAzyCZUMJBolCDRhxUdw=" }, + "ccount": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ccount/-/ccount-2.0.0.tgz", + "integrity": "sha512-VOR0NWFYX65n9gELQdcpqsie5L5ihBXuZGAgaPEp/U7IOSjnPMEH6geE+2f6lcekaNEfWzAHS45mPvSo5bqsUA==" + }, "chalk": { "version": "2.4.2", "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", @@ -4199,6 +4269,11 @@ "resolved": "https://registry.npmjs.org/character-entities/-/character-entities-1.2.4.tgz", "integrity": "sha512-iBMyeEHxfVnIakwOuDXpVkc54HijNgCyQB2w0VfGQThle6NXn50zU6V/u+LDhxHcDUPojn6Kpga3PTAD8W1bQw==" }, + "character-entities-html4": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/character-entities-html4/-/character-entities-html4-2.0.0.tgz", + "integrity": "sha512-dwT2xh5ZhUAjyP96k57ilMKoTQyASaw9IAMR9U5c1lCu2RUni6O6jxfpUEdO2RcPT6TJFvr8pqsbami4Jk+2oA==" + }, "character-entities-legacy": { "version": "1.1.4", "resolved": "https://registry.npmjs.org/character-entities-legacy/-/character-entities-legacy-1.1.4.tgz", @@ -7534,11 +7609,121 @@ "minimalistic-assert": "^1.0.1" } }, + "hast-util-from-parse5": { + "version": "7.1.0", + "resolved": "https://registry.npmjs.org/hast-util-from-parse5/-/hast-util-from-parse5-7.1.0.tgz", + "integrity": "sha512-m8yhANIAccpU4K6+121KpPP55sSl9/samzQSQGpb0mTExcNh2WlvjtMwSWFhg6uqD4Rr6Nfa8N6TMypQM51rzQ==", + "requires": { + "@types/hast": "^2.0.0", + "@types/parse5": "^6.0.0", + "@types/unist": "^2.0.0", + "hastscript": "^7.0.0", + "property-information": "^6.0.0", + "vfile": "^5.0.0", + "vfile-location": "^4.0.0", + "web-namespaces": "^2.0.0" + }, + "dependencies": { + "comma-separated-tokens": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-2.0.2.tgz", + "integrity": "sha512-G5yTt3KQN4Yn7Yk4ed73hlZ1evrFKXeUW3086p3PRFNp7m2vIjI6Pg+Kgb+oyzhd9F2qdcoj67+y3SdxL5XWsg==" + }, + "hast-util-parse-selector": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/hast-util-parse-selector/-/hast-util-parse-selector-3.1.0.tgz", + "integrity": "sha512-AyjlI2pTAZEOeu7GeBPZhROx0RHBnydkQIXlhnFzDi0qfXTmGUWoCYZtomHbrdrheV4VFUlPcfJ6LMF5T6sQzg==", + "requires": { + "@types/hast": "^2.0.0" + } + }, + "hastscript": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/hastscript/-/hastscript-7.0.2.tgz", + "integrity": "sha512-uA8ooUY4ipaBvKcMuPehTAB/YfFLSSzCwFSwT6ltJbocFUKH/GDHLN+tflq7lSRf9H86uOuxOFkh1KgIy3Gg2g==", + "requires": { + "@types/hast": "^2.0.0", + "comma-separated-tokens": "^2.0.0", + "hast-util-parse-selector": "^3.0.0", + "property-information": "^6.0.0", + "space-separated-tokens": "^2.0.0" + } + }, + "property-information": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/property-information/-/property-information-6.0.1.tgz", + "integrity": "sha512-F4WUUAF7fMeF4/JUFHNBWDaKDXi2jbvqBW/y6o5wsf3j19wTZ7S60TmtB5HoBhtgw7NKQRMWuz5vk2PR0CygUg==" + }, + "space-separated-tokens": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/space-separated-tokens/-/space-separated-tokens-2.0.1.tgz", + "integrity": "sha512-ekwEbFp5aqSPKaqeY1PGrlGQxPNaq+Cnx4+bE2D8sciBQrHpbwoBbawqTN2+6jPs9IdWxxiUcN0K2pkczD3zmw==" + } + } + }, + "hast-util-is-element": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/hast-util-is-element/-/hast-util-is-element-2.1.1.tgz", + "integrity": "sha512-ag0fiZfRWsPiR1udvnSbaazJLGv8qd8E+/e3rW8rUZhbKG4HNJmFL4QkEceN+22BgE+uozXY30z/s+2dL6Z++g==", + "requires": { + "@types/hast": "^2.0.0", + "@types/unist": "^2.0.0" + } + }, "hast-util-parse-selector": { "version": "2.2.5", "resolved": "https://registry.npmjs.org/hast-util-parse-selector/-/hast-util-parse-selector-2.2.5.tgz", "integrity": "sha512-7j6mrk/qqkSehsM92wQjdIgWM2/BW61u/53G6xmC8i1OmEdKLHbk419QKQUjz6LglWsfqoiHmyMRkP1BGjecNQ==" }, + "hast-util-to-html": { + "version": "8.0.2", + "resolved": "https://registry.npmjs.org/hast-util-to-html/-/hast-util-to-html-8.0.2.tgz", + "integrity": "sha512-ipLhUTMyyJi9F/LXaNDG9BrRdshP6obCfmUZYbE/+T639IdzqAOkKN4DyrEyID0gbb+rsC3PKf0XlviZwzomhw==", + "requires": { + "@types/hast": "^2.0.0", + "ccount": "^2.0.0", + "comma-separated-tokens": "^2.0.0", + "hast-util-is-element": "^2.0.0", + "hast-util-whitespace": "^2.0.0", + "html-void-elements": "^2.0.0", + "property-information": "^6.0.0", + "space-separated-tokens": "^2.0.0", + "stringify-entities": "^4.0.0", + "unist-util-is": "^5.0.0" + }, + "dependencies": { + "comma-separated-tokens": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-2.0.2.tgz", + "integrity": "sha512-G5yTt3KQN4Yn7Yk4ed73hlZ1evrFKXeUW3086p3PRFNp7m2vIjI6Pg+Kgb+oyzhd9F2qdcoj67+y3SdxL5XWsg==" + }, + "property-information": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/property-information/-/property-information-6.0.1.tgz", + "integrity": "sha512-F4WUUAF7fMeF4/JUFHNBWDaKDXi2jbvqBW/y6o5wsf3j19wTZ7S60TmtB5HoBhtgw7NKQRMWuz5vk2PR0CygUg==" + }, + "space-separated-tokens": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/space-separated-tokens/-/space-separated-tokens-2.0.1.tgz", + "integrity": "sha512-ekwEbFp5aqSPKaqeY1PGrlGQxPNaq+Cnx4+bE2D8sciBQrHpbwoBbawqTN2+6jPs9IdWxxiUcN0K2pkczD3zmw==" + }, + "unist-util-is": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/unist-util-is/-/unist-util-is-5.1.1.tgz", + "integrity": "sha512-F5CZ68eYzuSvJjGhCLPL3cYx45IxkqXSetCcRgUXtbcm50X2L9oOWQlfUfDdAf+6Pd27YDblBfdtmsThXmwpbQ==" + } + } + }, + "hast-util-to-string": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/hast-util-to-string/-/hast-util-to-string-1.0.4.tgz", + "integrity": "sha512-eK0MxRX47AV2eZ+Lyr18DCpQgodvaS3fAQO2+b9Two9F5HEoRPhiUMNzoXArMJfZi2yieFzUBMRl3HNJ3Jus3w==" + }, + "hast-util-whitespace": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/hast-util-whitespace/-/hast-util-whitespace-2.0.0.tgz", + "integrity": "sha512-Pkw+xBHuV6xFeJprJe2BBEoDV+AvQySaz3pPDRUs5PNZEMQjpXJJueqrpcHIXxnWTcAGi/UOCgVShlkY6kLoqg==" + }, "hastscript": { "version": "6.0.0", "resolved": "https://registry.npmjs.org/hastscript/-/hastscript-6.0.0.tgz", @@ -7671,6 +7856,11 @@ "terser": "^4.6.3" } }, + "html-void-elements": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/html-void-elements/-/html-void-elements-2.0.0.tgz", + "integrity": "sha512-4OYzQQsBt0G9bJ/nM9/DDsjm4+fVdzAaPJJcWk5QwA3GIAPxQEeOR0rsI8HbDHQz5Gta8pVvGnnTNSbZVEVvkQ==" + }, "html-webpack-plugin": { "version": "4.5.0", "resolved": "https://registry.npmjs.org/html-webpack-plugin/-/html-webpack-plugin-4.5.0.tgz", @@ -13470,6 +13660,14 @@ "refractor": "^3.2.0" } }, + "react-toastify": { + "version": "8.0.3", + "resolved": "https://registry.npmjs.org/react-toastify/-/react-toastify-8.0.3.tgz", + "integrity": "sha512-rv3koC7f9lKKSkdpYgo/TGzgWlrB/aaiUInF1DyV7BpiM4kyTs+uhu6/r8XDMtBY2FOIHK+FlK3Iv7OzpA/tCA==", + "requires": { + "clsx": "^1.1.1" + } + }, "react-transition-group": { "version": "4.4.1", "resolved": "https://registry.npmjs.org/react-transition-group/-/react-transition-group-4.4.1.tgz", @@ -13693,6 +13891,38 @@ } } }, + "rehype": { + "version": "12.0.0", + "resolved": "https://registry.npmjs.org/rehype/-/rehype-12.0.0.tgz", + "integrity": "sha512-gZcttmf9R5IYHb8AlI1rlmWqXS1yX0rSB/S5ZGJs8atfYZy2DobvH3Ic/gSzB+HL/+oOHPtBguw1TprfhxXBgQ==", + "requires": { + "@types/hast": "^2.0.0", + "rehype-parse": "^8.0.0", + "rehype-stringify": "^9.0.0", + "unified": "^10.0.0" + } + }, + "rehype-parse": { + "version": "8.0.3", + "resolved": "https://registry.npmjs.org/rehype-parse/-/rehype-parse-8.0.3.tgz", + "integrity": "sha512-RGw0CVt+0S6KdvpE8bbP2Db9WXclQcIX7A0ufM3QFqAhTo/ddJMQrrI2j3cijlRPZlGK8R3pRgC8U5HyV76IDw==", + "requires": { + "@types/hast": "^2.0.0", + "hast-util-from-parse5": "^7.0.0", + "parse5": "^6.0.0", + "unified": "^10.0.0" + } + }, + "rehype-stringify": { + "version": "9.0.2", + "resolved": "https://registry.npmjs.org/rehype-stringify/-/rehype-stringify-9.0.2.tgz", + "integrity": "sha512-BuVA6lAEYtOpXO2xuHLohAzz8UNoQAxAqYRqh4QEEtU39Co+P1JBZhw6wXA9hMWp+JLcmrxWH8+UKcNSr443Fw==", + "requires": { + "@types/hast": "^2.0.0", + "hast-util-to-html": "^8.0.0", + "unified": "^10.0.0" + } + }, "relateurl": { "version": "0.2.7", "resolved": "https://registry.npmjs.org/relateurl/-/relateurl-0.2.7.tgz", @@ -15342,6 +15572,22 @@ } } }, + "stringify-entities": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/stringify-entities/-/stringify-entities-4.0.1.tgz", + "integrity": "sha512-gmMQxKXPWIO3NXNSPyWNhlYcBNGpPA/487D+9dLPnU4xBnIrnHdr8cv5rGJOS/1BRxEXRb7uKwg7BA36IWV7xg==", + "requires": { + "character-entities-html4": "^2.0.0", + "character-entities-legacy": "^2.0.0" + }, + "dependencies": { + "character-entities-legacy": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/character-entities-legacy/-/character-entities-legacy-2.0.0.tgz", + "integrity": "sha512-YwaEtEvWLpFa6Wh3uVLrvirA/ahr9fki/NUd/Bd4OR6EdJ8D22hovYQEOUCBfQfcqnC4IAMGMsHXY1eXgL4ZZA==" + } + } + }, "stringify-object": { "version": "3.3.0", "resolved": "https://registry.npmjs.org/stringify-object/-/stringify-object-3.3.0.tgz", @@ -15877,6 +16123,11 @@ "resolved": "https://registry.npmjs.org/trim-newlines/-/trim-newlines-1.0.0.tgz", "integrity": "sha1-WIeWa7WCpFA6QetST301ARgVphM=" }, + "trough": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/trough/-/trough-2.0.2.tgz", + "integrity": "sha512-FnHq5sTMxC0sk957wHDzRnemFnNBvt/gSY99HzK8F7UP5WAbvP70yX5bd7CjEQkN+TjdxwI7g7lJ6podqrG2/w==" + }, "true-case-path": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/true-case-path/-/true-case-path-1.0.3.tgz", @@ -16037,6 +16288,32 @@ "resolved": "https://registry.npmjs.org/unicode-property-aliases-ecmascript/-/unicode-property-aliases-ecmascript-1.1.0.tgz", "integrity": "sha512-PqSoPh/pWetQ2phoj5RLiaqIk4kCNwoV3CI+LfGmWLKI3rE3kl1h59XpX2BjgDrmbxD9ARtQobPGU1SguCYuQg==" }, + "unified": { + "version": "10.1.0", + "resolved": "https://registry.npmjs.org/unified/-/unified-10.1.0.tgz", + "integrity": "sha512-4U3ru/BRXYYhKbwXV6lU6bufLikoAavTwev89H5UxY8enDFaAT2VXmIXYNm6hb5oHPng/EXr77PVyDFcptbk5g==", + "requires": { + "@types/unist": "^2.0.0", + "bail": "^2.0.0", + "extend": "^3.0.0", + "is-buffer": "^2.0.0", + "is-plain-obj": "^4.0.0", + "trough": "^2.0.0", + "vfile": "^5.0.0" + }, + "dependencies": { + "is-buffer": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/is-buffer/-/is-buffer-2.0.5.tgz", + "integrity": "sha512-i2R6zNFDwgEHJyQUtJEk0XFi1i0dPFn/oqjK3/vPCcDeJvW5NQ83V8QbicfF1SupOaB0h8ntgBC2YiE7dfyctQ==" + }, + "is-plain-obj": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-4.0.0.tgz", + "integrity": "sha512-NXRbBtUdBioI73y/HmOhogw/U5msYPC9DAtGkJXeFcFWSFZw0mCUsPxk/snTuJHzNKA8kLBK4rH97RMB1BfCXw==" + } + } + }, "union-value": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/union-value/-/union-value-1.0.1.tgz", @@ -16082,6 +16359,38 @@ "crypto-random-string": "^1.0.0" } }, + "unist-util-is": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/unist-util-is/-/unist-util-is-4.1.0.tgz", + "integrity": "sha512-ZOQSsnce92GrxSqlnEEseX0gi7GH9zTJZ0p9dtu87WRb/37mMPO2Ilx1s/t9vBHrFhbgweUwb+t7cIn5dxPhZg==" + }, + "unist-util-stringify-position": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/unist-util-stringify-position/-/unist-util-stringify-position-3.0.0.tgz", + "integrity": "sha512-SdfAl8fsDclywZpfMDTVDxA2V7LjtRDTOFd44wUJamgl6OlVngsqWjxvermMYf60elWHbxhuRCZml7AnuXCaSA==", + "requires": { + "@types/unist": "^2.0.0" + } + }, + "unist-util-visit": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/unist-util-visit/-/unist-util-visit-2.0.3.tgz", + "integrity": "sha512-iJ4/RczbJMkD0712mGktuGpm/U4By4FfDonL7N/9tATGIF4imikjOuagyMY53tnZq3NP6BcmlrHhEKAfGWjh7Q==", + "requires": { + "@types/unist": "^2.0.0", + "unist-util-is": "^4.0.0", + "unist-util-visit-parents": "^3.0.0" + } + }, + "unist-util-visit-parents": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/unist-util-visit-parents/-/unist-util-visit-parents-3.1.1.tgz", + "integrity": "sha512-1KROIZWo6bcMrZEwiH2UrXDyalAa0uqzWCxCJj6lPOvTve2WkfgCytoDTPaMnodXh1WrXOq0haVYHj99ynJlsg==", + "requires": { + "@types/unist": "^2.0.0", + "unist-util-is": "^4.0.0" + } + }, "universalify": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.0.tgz", @@ -16306,6 +16615,42 @@ "extsprintf": "^1.2.0" } }, + "vfile": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/vfile/-/vfile-5.1.0.tgz", + "integrity": "sha512-4o7/DJjEaFPYSh0ckv5kcYkJTHQgCKdL8ozMM1jLAxO9ox95IzveDPXCZp08HamdWq8JXTkClDvfAKaeLQeKtg==", + "requires": { + "@types/unist": "^2.0.0", + "is-buffer": "^2.0.0", + "unist-util-stringify-position": "^3.0.0", + "vfile-message": "^3.0.0" + }, + "dependencies": { + "is-buffer": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/is-buffer/-/is-buffer-2.0.5.tgz", + "integrity": "sha512-i2R6zNFDwgEHJyQUtJEk0XFi1i0dPFn/oqjK3/vPCcDeJvW5NQ83V8QbicfF1SupOaB0h8ntgBC2YiE7dfyctQ==" + } + } + }, + "vfile-location": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/vfile-location/-/vfile-location-4.0.1.tgz", + "integrity": "sha512-JDxPlTbZrZCQXogGheBHjbRWjESSPEak770XwWPfw5mTc1v1nWGLB/apzZxsx8a0SJVfF8HK8ql8RD308vXRUw==", + "requires": { + "@types/unist": "^2.0.0", + "vfile": "^5.0.0" + } + }, + "vfile-message": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/vfile-message/-/vfile-message-3.0.2.tgz", + "integrity": "sha512-UUjZYIOg9lDRwwiBAuezLIsu9KlXntdxwG+nXnjuQAHvBpcX3x0eN8h+I7TkY5nkCXj+cWVp4ZqebtGBvok8ww==", + "requires": { + "@types/unist": "^2.0.0", + "unist-util-stringify-position": "^3.0.0" + } + }, "vm-browserify": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/vm-browserify/-/vm-browserify-1.1.2.tgz", @@ -16590,6 +16935,11 @@ "minimalistic-assert": "^1.0.0" } }, + "web-namespaces": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/web-namespaces/-/web-namespaces-2.0.0.tgz", + "integrity": "sha512-dE7ELZRVWh0ceQsRgkjLgsAvwTuv3kcjSY/hLjqL0llleUlQBDjE9JkB9FCBY5F2mnFEwiyJoowl8+NVGHe8dw==" + }, "web-vitals": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/web-vitals/-/web-vitals-1.1.1.tgz", diff --git a/ui/package.json b/ui/package.json index 463ef8c4b..088241fe0 100644 --- a/ui/package.json +++ b/ui/package.json @@ -4,6 +4,7 @@ "private": true, "dependencies": { "@material-ui/core": "^4.11.3", + "@material-ui/icons": "^4.11.2", "@material-ui/lab": "^4.0.0-alpha.60", "@testing-library/jest-dom": "^5.11.10", "@testing-library/react": "^11.2.6", @@ -12,6 +13,7 @@ "@types/node": "^12.20.10", "@types/react": "^17.0.3", "@types/react-dom": "^17.0.3", + "@uiw/react-textarea-code-editor": "^1.4.12", "axios": "^0.21.1", "jsonpath": "^1.1.1", "node-sass": "^5.0.0", @@ -23,6 +25,7 @@ "react-scripts": "4.0.3", "react-scrollable-feed-virtualized": "^1.4.3", "react-syntax-highlighter": "^15.4.3", + "react-toastify": "^8.0.3", "typescript": "^4.2.4", "web-vitals": "^1.1.1" }, diff --git a/ui/src/components/EntriesList.tsx b/ui/src/components/EntriesList.tsx index 53e653a05..4fc05895e 100644 --- a/ui/src/components/EntriesList.tsx +++ b/ui/src/components/EntriesList.tsx @@ -1,10 +1,7 @@ import {EntryItem} from "./EntryListItem/EntryListItem"; -import React, {useCallback, useEffect, useMemo, useRef, useState} from "react"; +import React, {useRef} from "react"; import styles from './style/EntriesList.module.sass'; -import spinner from './assets/spinner.svg'; import ScrollableFeedVirtualized from "react-scrollable-feed-virtualized"; -import {StatusType} from "./Filters"; -import Api from "../helpers/api"; import down from "./assets/downImg.svg"; interface EntriesListProps { @@ -13,114 +10,36 @@ interface EntriesListProps { focusedEntryId: string; setFocusedEntryId: (id: string) => void; connectionOpen: boolean; - noMoreDataTop: boolean; - setNoMoreDataTop: (flag: boolean) => void; - noMoreDataBottom: boolean; - setNoMoreDataBottom: (flag: boolean) => void; - methodsFilter: Array; - statusFilter: Array; - pathFilter: string; - serviceFilter: string; listEntryREF: any; onScrollEvent: (isAtBottom:boolean) => void; scrollableList: boolean; + ws: any + openWebSocket: any; + query: string; + updateQuery: any; + queriedCurrent: number; + queriedTotal: number; + startTime: number; } -enum FetchOperator { - LT = "lt", - GT = "gt" -} - -const api = new Api(); - -export const EntriesList: React.FC = ({entries, setEntries, focusedEntryId, setFocusedEntryId, connectionOpen, noMoreDataTop, setNoMoreDataTop, noMoreDataBottom, setNoMoreDataBottom, methodsFilter, statusFilter, pathFilter, serviceFilter, listEntryREF, onScrollEvent, scrollableList}) => { - - const [loadMoreTop, setLoadMoreTop] = useState(false); - const [isLoadingTop, setIsLoadingTop] = useState(false); +export const EntriesList: React.FC = ({entries, setEntries, focusedEntryId, setFocusedEntryId, connectionOpen, listEntryREF, onScrollEvent, scrollableList, ws, openWebSocket, query, updateQuery, queriedCurrent, queriedTotal, startTime}) => { const scrollableRef = useRef(null); - useEffect(() => { - const list = document.getElementById('list').firstElementChild; - list.addEventListener('scroll', (e) => { - const el: any = e.target; - if(el.scrollTop === 0) { - setLoadMoreTop(true); - } else { - setLoadMoreTop(false); - } - }); - }, []); - - const filterEntries = useCallback((entry) => { - if(methodsFilter.length > 0 && !methodsFilter.includes(entry.method.toLowerCase())) return; - if(pathFilter && entry.path?.toLowerCase()?.indexOf(pathFilter) === -1) return; - if(serviceFilter && entry.service?.toLowerCase()?.indexOf(serviceFilter) === -1) return; - if(statusFilter.includes(StatusType.SUCCESS) && entry.statusCode >= 400) return; - if(statusFilter.includes(StatusType.ERROR) && entry.statusCode < 400) return; - return entry; - },[methodsFilter, pathFilter, statusFilter, serviceFilter]) - - const filteredEntries = useMemo(() => { - return entries.filter(filterEntries); - },[entries, filterEntries]) - - const getOldEntries = useCallback(async () => { - setIsLoadingTop(true); - const data = await api.fetchEntries(FetchOperator.LT, entries[0].timestamp); - setLoadMoreTop(false); - - let scrollTo; - if(data.length === 0) { - setNoMoreDataTop(true); - scrollTo = document.getElementById("noMoreDataTop"); - } else { - scrollTo = document.getElementById(filteredEntries?.[0]?.id); - } - setIsLoadingTop(false); - const newEntries = [...data, ...entries]; - setEntries(newEntries); - - if(scrollTo) { - scrollTo.scrollIntoView(); - } - },[setLoadMoreTop, setIsLoadingTop, entries, setEntries, filteredEntries, setNoMoreDataTop]) - - useEffect(() => { - if(!loadMoreTop || connectionOpen || noMoreDataTop) return; - getOldEntries(); - }, [loadMoreTop, connectionOpen, noMoreDataTop, getOldEntries]); - - const getNewEntries = async () => { - const data = await api.fetchEntries(FetchOperator.GT, entries[entries.length - 1].timestamp); - let scrollTo; - if(data.length === 0) { - setNoMoreDataBottom(true); - } - scrollTo = document.getElementById(filteredEntries?.[filteredEntries.length -1]?.id); - let newEntries = [...entries, ...data]; - setEntries(newEntries); - if(scrollTo) { - scrollTo.scrollIntoView({behavior: "smooth"}); - } - } - return <>
- {isLoadingTop &&
- spinner -
} onScrollEvent(isAtBottom)}> - {noMoreDataTop && !connectionOpen &&
No more data available
} - {filteredEntries.map(entry => )} + isSelected={focusedEntryId === entry.id.toString()} + style={{}} + updateQuery={updateQuery}/>)} - {!connectionOpen && !noMoreDataBottom &&
-
getNewEntries()}>Fetch more entries
+ {!connectionOpen &&
+
{ws.close(); openWebSocket(query);}}>Reconnect
}
- {entries?.length > 0 &&
-
{filteredEntries?.length !== entries.length && `${filteredEntries?.length} / `} {entries?.length} requests
-
Started listening at {new Date(+entries[0].timestamp)?.toLocaleString()}
-
} +
+
Displaying {entries?.length} results (queried {queriedCurrent}/{queriedTotal})
+ {startTime !== 0 &&
Started listening at {new Date(startTime).toLocaleString()}
} +
; }; diff --git a/ui/src/components/EntryDetailed.tsx b/ui/src/components/EntryDetailed.tsx index ffc5151a9..2c5481a13 100644 --- a/ui/src/components/EntryDetailed.tsx +++ b/ui/src/components/EntryDetailed.tsx @@ -3,7 +3,7 @@ import EntryViewer from "./EntryDetailed/EntryViewer"; import {makeStyles} from "@material-ui/core"; import Protocol from "./UI/Protocol" import StatusCode from "./UI/StatusCode"; -import {EndpointPath} from "./UI/EndpointPath"; +import {Summary} from "./UI/Summary"; const useStyles = makeStyles(() => ({ entryTitle: { @@ -28,50 +28,79 @@ const useStyles = makeStyles(() => ({ interface EntryDetailedProps { entryData: any + updateQuery: any } export const formatSize = (n: number) => n > 1000 ? `${Math.round(n / 1000)}KB` : `${n} B`; -const EntryTitle: React.FC = ({protocol, data, bodySize, elapsedTime}) => { +const EntryTitle: React.FC = ({protocol, data, bodySize, elapsedTime, updateQuery}) => { const classes = useStyles(); - const {response} = JSON.parse(data.entry); + const response = data.response; return
- +
- {response.payload &&
{formatSize(bodySize)}
} - {response.payload &&
{Math.round(elapsedTime)}ms
} + {response &&
{ + updateQuery(`response.bodySize == ${bodySize}`) + }} + > + {formatSize(bodySize)} +
} + {response &&
{ + updateQuery(`elapsedTime >= ${elapsedTime}`) + }} + > + {Math.round(elapsedTime)}ms +
}
; }; -const EntrySummary: React.FC = ({data}) => { +const EntrySummary: React.FC = ({data, updateQuery}) => { const classes = useStyles(); - const {response, request} = JSON.parse(data.entry); + const response = data.response; return
- {response?.payload && response.payload?.details && "status" in response.payload.details &&
- + {response && "status" in response &&
+
}
- +
; }; -export const EntryDetailed: React.FC = ({entryData}) => { +export const EntryDetailed: React.FC = ({entryData, updateQuery}) => { return <> - {entryData.data && } + {entryData.data && } <> - {entryData.data && } + {entryData.data && } }; diff --git a/ui/src/components/EntryDetailed/EntrySections.tsx b/ui/src/components/EntryDetailed/EntrySections.tsx index f364b4a51..b702b4db1 100644 --- a/ui/src/components/EntryDetailed/EntrySections.tsx +++ b/ui/src/components/EntryDetailed/EntrySections.tsx @@ -9,11 +9,29 @@ import ProtobufDecoder from "protobuf-decoder"; interface EntryViewLineProps { label: string; value: number | string; + updateQuery: any; + selector: string; + overrideQueryValue?: string; } -const EntryViewLine: React.FC = ({label, value}) => { +const EntryViewLine: React.FC = ({label, value, updateQuery, selector, overrideQueryValue}) => { return (label && value && - {label} + { + if (!selector) { + return + } else if (overrideQueryValue) { + updateQuery(`${selector} == ${overrideQueryValue}`) + } else if (typeof(value) === "string") { + updateQuery(`${selector} == "${JSON.stringify(value).slice(1, -1)}"`) + } else { + updateQuery(`${selector} == ${value}`) + } + }} + > + {label} + = ({tit interface EntryBodySectionProps { content: any, color: string, + updateQuery: any, encoding?: string, contentType?: string, + selector?: string, } export const EntryBodySection: React.FC = ({ color, + updateQuery, content, encoding, contentType, + selector, }) => { const MAXIMUM_BYTES_TO_HIGHLIGHT = 10000; // The maximum of chars to highlight in body, in case the response can be megabytes const supportedLanguages = [['html', 'html'], ['json', 'json'], ['application/grpc', 'json']]; // [[indicator, languageToUse],...] @@ -107,8 +129,8 @@ export const EntryBodySection: React.FC = ({ {content && content?.length > 0 && - - + +
@@ -132,17 +154,23 @@ interface EntrySectionProps { title: string, color: string, arrayToIterate: any[], + updateQuery: any, } -export const EntryTableSection: React.FC = ({title, color, arrayToIterate}) => { +export const EntryTableSection: React.FC = ({title, color, arrayToIterate, updateQuery}) => { return { arrayToIterate && arrayToIterate.length > 0 ? - {arrayToIterate.map(({name, value}, index) => )} + {arrayToIterate.map(({name, value, selector}, index) => )}
: diff --git a/ui/src/components/EntryDetailed/EntryViewer.tsx b/ui/src/components/EntryDetailed/EntryViewer.tsx index dc8b8f4c7..dabf3d5b7 100644 --- a/ui/src/components/EntryDetailed/EntryViewer.tsx +++ b/ui/src/components/EntryDetailed/EntryViewer.tsx @@ -8,7 +8,7 @@ enum SectionTypes { SectionBody = "body", } -const SectionsRepresentation: React.FC = ({data, color}) => { +const SectionsRepresentation: React.FC = ({data, color, updateQuery}) => { const sections = [] if (data) { @@ -16,12 +16,12 @@ const SectionsRepresentation: React.FC = ({data, color}) => { switch (row.type) { case SectionTypes.SectionTable: sections.push( - + ) break; case SectionTypes.SectionBody: sections.push( - + ) break; default: @@ -33,7 +33,7 @@ const SectionsRepresentation: React.FC = ({data, color}) => { return <>{sections}; } -const AutoRepresentation: React.FC = ({representation, isRulesEnabled, rulesMatched, contractStatus, requestReason, responseReason, contractContent, elapsedTime, color}) => { +const AutoRepresentation: React.FC = ({representation, isRulesEnabled, rulesMatched, contractStatus, requestReason, responseReason, contractContent, elapsedTime, color, updateQuery}) => { var TABS = [ { tab: 'Request' @@ -85,10 +85,10 @@ const AutoRepresentation: React.FC = ({representation, isRulesEnabled, rule
{currentTab === TABS[0].tab && - + } {response && currentTab === TABS[responseTabIndex].tab && - + } {isRulesEnabled && currentTab === TABS[rulesTabIndex].tab && @@ -110,9 +110,10 @@ interface Props { contractContent: string; color: string; elapsedTime: number; + updateQuery: any; } -const EntryViewer: React.FC = ({representation, isRulesEnabled, rulesMatched, contractStatus, requestReason, responseReason, contractContent, elapsedTime, color}) => { +const EntryViewer: React.FC = ({representation, isRulesEnabled, rulesMatched, contractStatus, requestReason, responseReason, contractContent, elapsedTime, color, updateQuery}) => { return = ({representation, isRulesEnabled, rulesMatc contractContent={contractContent} elapsedTime={elapsedTime} color={color} + updateQuery={updateQuery} /> }; diff --git a/ui/src/components/EntryListItem/EntryListItem.tsx b/ui/src/components/EntryListItem/EntryListItem.tsx index fc418631f..ae771801d 100644 --- a/ui/src/components/EntryListItem/EntryListItem.tsx +++ b/ui/src/components/EntryListItem/EntryListItem.tsx @@ -2,7 +2,7 @@ import React from "react"; import styles from './EntryListItem.module.sass'; import StatusCode, {getClassification, StatusCodeClassification} from "../UI/StatusCode"; import Protocol, {ProtocolInterface} from "../UI/Protocol" -import {EndpointPath} from "../UI/EndpointPath"; +import {Summary} from "../UI/Summary"; import ingoingIconSuccess from "../assets/ingoing-traffic-success.svg" import ingoingIconFailure from "../assets/ingoing-traffic-failure.svg" import ingoingIconNeutral from "../assets/ingoing-traffic-neutral.svg" @@ -15,7 +15,7 @@ interface Entry { method?: string, summary: string, service: string, - id: string, + id: number, statusCode?: number; url?: string; timestamp: Date; @@ -40,9 +40,10 @@ interface EntryProps { setFocusedEntryId: (id: string) => void; isSelected?: boolean; style: object; + updateQuery: any; } -export const EntryItem: React.FC = ({entry, setFocusedEntryId, isSelected, style}) => { +export const EntryItem: React.FC = ({entry, setFocusedEntryId, isSelected, style, updateQuery}) => { const classification = getClassification(entry.statusCode) const numberOfRules = entry.rules.numberOfRules let ingoingIcon; @@ -115,10 +116,10 @@ export const EntryItem: React.FC = ({entry, setFocusedEntryId, isSel return <>
setFocusedEntryId(entry.id)} + onClick={() => setFocusedEntryId(entry.id.toString())} style={{ border: isSelected ? `1px ${entry.protocol.backgroundColor} solid` : "1px transparent solid", position: "absolute", @@ -127,14 +128,26 @@ export const EntryItem: React.FC = ({entry, setFocusedEntryId, isSel width: "calc(100% - 25px)", }} > - + {((entry.protocol.name === "http" && "statusCode" in entry) || entry.statusCode !== 0) &&
- +
}
- +
- {entry.service} + { + updateQuery(`service == "${entry.service}"`) + }} + > + {entry.service} +
{ @@ -152,17 +165,53 @@ export const EntryItem: React.FC = ({entry, setFocusedEntryId, isSel : "" }
- {entry.sourcePort} + { + updateQuery(`src.port == "${entry.sourcePort}"`) + }} + > + {entry.sourcePort} + {entry.isOutgoing ? - Ingoing traffic + Ingoing traffic { + updateQuery(`outgoing == true`) + }} + /> : - Outgoing traffic + Outgoing traffic { + updateQuery(`outgoing == false`) + }} + /> } - {entry.destinationPort} + { + updateQuery(`dst.port == "${entry.destinationPort}"`) + }} + > + {entry.destinationPort} +
- - {new Date(+entry.timestamp)?.toLocaleString()} + { + updateQuery(`timestamp >= datetime("${new Date(+entry.timestamp)?.toLocaleString("en-US", {timeZone: 'UTC' })}")`) + }} + > + {new Date(+entry.timestamp)?.toLocaleString("en-US")}
diff --git a/ui/src/components/Filters.tsx b/ui/src/components/Filters.tsx index c1795a67a..3c7e38f74 100644 --- a/ui/src/components/Filters.tsx +++ b/ui/src/components/Filters.tsx @@ -1,137 +1,303 @@ -import React from "react"; +import React, {useRef, useState} from "react"; import styles from './style/Filters.module.sass'; -import {FilterSelect} from "./UI/FilterSelect"; -import {TextField} from "@material-ui/core"; -import {ALL_KEY} from "./UI/Select"; +import {Button, Grid, Modal, Box, Typography, Backdrop, Fade, Divider} from "@material-ui/core"; +import CodeEditor from '@uiw/react-textarea-code-editor'; +import MenuBookIcon from '@material-ui/icons/MenuBook'; +import {SyntaxHighlighter} from "./UI/SyntaxHighlighter/index"; +import filterUIExample1 from "./assets/filter-ui-example-1.png" +import filterUIExample2 from "./assets/filter-ui-example-2.png" interface FiltersProps { - methodsFilter: Array; - setMethodsFilter: (methods: Array) => void; - statusFilter: Array; - setStatusFilter: (methods: Array) => void; - pathFilter: string - setPathFilter: (val: string) => void; - serviceFilter: string - setServiceFilter: (val: string) => void; + query: string + setQuery: any + backgroundColor: string + ws: any + openWebSocket: (query: string) => void; } -export const Filters: React.FC = ({methodsFilter, setMethodsFilter, statusFilter, setStatusFilter, pathFilter, setPathFilter, serviceFilter, setServiceFilter}) => { - +export const Filters: React.FC = ({query, setQuery, backgroundColor, ws, openWebSocket}) => { return
- - - - +
; }; -const _toUpperCase = v => v.toUpperCase(); +interface QueryFormProps { + query: string + setQuery: any + backgroundColor: string + ws: any + openWebSocket: (query: string) => void; +} -const FilterContainer: React.FC = ({children}) => { - return
- {children} -
; +const style = { + position: 'absolute', + top: '50%', + left: '50%', + transform: 'translate(-50%, -50%)', + width: '80vw', + bgcolor: 'background.paper', + borderRadius: '5px', + boxShadow: 24, + p: 4, + color: '#000', }; -enum HTTPMethod { - GET = "get", - PUT = "put", - POST = "post", - DELETE = "delete", - OPTIONS="options", - PATCH = "patch" -} +export const QueryForm: React.FC = ({query, setQuery, backgroundColor, ws, openWebSocket}) => { -interface MethodFilterProps { - methodsFilter: Array; - setMethodsFilter: (methods: Array) => void; -} + const formRef = useRef(null); -const MethodFilter: React.FC = ({methodsFilter, setMethodsFilter}) => { + const [openModal, setOpenModal] = useState(false); - const methodClicked = (val) => { - if(val === ALL_KEY) { - setMethodsFilter([]); - return; - } - if(methodsFilter.includes(val)) { - setMethodsFilter(methodsFilter.filter(method => method !== val)) - } else { - setMethodsFilter([...methodsFilter, val]); - } + const handleOpenModal = () => setOpenModal(true); + const handleCloseModal = () => setOpenModal(false); + + const handleChange = async (e) => { + setQuery(e.target.value); } - return - methodClicked(val)} - transformDisplay={_toUpperCase} - label={"Methods"} - /> - ; -}; - -export enum StatusType { - SUCCESS = "success", - ERROR = "error" -} - -interface StatusTypesFilterProps { - statusFilter: Array; - setStatusFilter: (methods: Array) => void; -} - -const StatusTypesFilter: React.FC = ({statusFilter, setStatusFilter}) => { - - const statusClicked = (val) => { - if(val === ALL_KEY) { - setStatusFilter([]); - return; - } - setStatusFilter([val]); + const handleSubmit = (e) => { + ws.close() + openWebSocket(query) + e.preventDefault(); } - return - statusClicked(val)} - transformDisplay={_toUpperCase} - label="Status" - /> - ; -}; + return <> +
+ + + + + + + + + +
-interface PathFilterProps { - pathFilter: string; - setPathFilter: (val: string) => void; + + + + + Filtering Guide (Cheatsheet) + + +

Mizu has a rich filtering syntax that let's you query the results both flexibly and efficiently.

+

Here are some examples that you can try;

+ + + + + This is a simple query that matches to HTTP packets with request path "/catalogue": + + + + The same query can be negated for HTTP path and written like this: + + + + The syntax supports regular expressions. Here is a query that matches the HTTP requests that send JSON to a server: + + + + Here is another query that matches HTTP responses with status code 4xx: + + + + The same exact query can be as integer comparison: + + = 400`} + language="python" + /> + + The results can be queried based on their timestamps: + + + + + + + Since Mizu supports various protocols like gRPC, AMQP, Kafka and Redis. It's possible to write complex queries that match multiple protocols like this: + + + + By clicking the UI elements in both left-pane and right-pane, you can automatically select a field and update the query: + + Clicking to UI elements (left-pane) + + Such that; clicking this in left-pane, would append the query below: + + + + Another queriable UI element example, this time from the right-pane: + + Clicking to UI elements (right-pane) + + A query that compares one selector to another is also a valid query: + + + + + + + There are a few helper methods included the in the filter language* to help building queries more easily. + +

+ + true if the given selector's value starts with the string: + + + + true if the given selector's value ends with the string: + + + + true if the given selector's value contains the string: + + + + returns the UNIX timestamp which is the equivalent of the time that's provided by the string. Invalid input evaluates to false: + + = datetime("10/19/2021, 6:29:02 PM")`} + language="python" + /> + + limits the number of records that are streamed back as a result of a query. Always evaluates to true: + + + + +

+ + *The filtering functionality is provided through Basenine database server. Please refer to BFL Syntax Reference for more information. + + + + + } - -const PathFilter: React.FC = ({pathFilter, setPathFilter}) => { - - return -
Path
-
- setPathFilter(e.target.value)}/> -
- ; -}; - -interface ServiceFilterProps { - serviceFilter: string; - setServiceFilter: (val: string) => void; -} - -const ServiceFilter: React.FC = ({serviceFilter, setServiceFilter}) => { - - return -
Service
-
- setServiceFilter(e.target.value)}/> -
- ; -}; - diff --git a/ui/src/components/TrafficPage.tsx b/ui/src/components/TrafficPage.tsx index f1e3be2b1..b8ea48ff7 100644 --- a/ui/src/components/TrafficPage.tsx +++ b/ui/src/components/TrafficPage.tsx @@ -10,6 +10,8 @@ import pauseIcon from './assets/pause.svg'; import variables from '../variables.module.scss'; import {StatusBar} from "./UI/StatusBar"; import Api, {MizuWebsocketURL} from "../helpers/api"; +import { ToastContainer, toast } from 'react-toastify'; +import 'react-toastify/dist/ReactToastify.css'; const useLayoutStyles = makeStyles(() => ({ details: { @@ -34,7 +36,6 @@ const useLayoutStyles = makeStyles(() => ({ enum ConnectionStatus { Closed, Connected, - Paused } interface TrafficPageProps { @@ -52,25 +53,52 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS const [focusedEntryId, setFocusedEntryId] = useState(null); const [selectedEntryData, setSelectedEntryData] = useState(null); const [connection, setConnection] = useState(ConnectionStatus.Closed); - const [noMoreDataTop, setNoMoreDataTop] = useState(false); - const [noMoreDataBottom, setNoMoreDataBottom] = useState(false); - - const [methodsFilter, setMethodsFilter] = useState([]); - const [statusFilter, setStatusFilter] = useState([]); - const [pathFilter, setPathFilter] = useState(""); - const [serviceFilter, setServiceFilter] = useState(""); const [tappingStatus, setTappingStatus] = useState(null); const [disableScrollList, setDisableScrollList] = useState(false); + const [query, setQueryDefault] = useState(""); + const [queryBackgroundColor, setQueryBackgroundColor] = useState("#f5f5f5"); + + const [queriedCurrent, setQueriedCurrent] = useState(0); + const [queriedTotal, setQueriedTotal] = useState(0); + + const [startTime, setStartTime] = useState(0); + + const setQuery = async (query) => { + if (!query) { + setQueryBackgroundColor("#f5f5f5") + } else { + const data = await api.validateQuery(query); + if (data.valid) { + setQueryBackgroundColor("#d2fad2") + } else { + setQueryBackgroundColor("#fad6dc") + } + } + setQueryDefault(query) + } + + const updateQuery = (addition) => { + if (query) { + setQuery(`${query} and ${addition}`) + } else { + setQuery(addition) + } + } + const ws = useRef(null); const listEntry = useRef(null); - const openWebSocket = () => { + const openWebSocket = (query) => { + setEntries([]) ws.current = new WebSocket(MizuWebsocketURL); - ws.current.onopen = () => setConnection(ConnectionStatus.Connected); + ws.current.onopen = () => { + ws.current.send(query) + setConnection(ConnectionStatus.Connected); + } ws.current.onclose = () => setConnection(ConnectionStatus.Closed); } @@ -81,11 +109,7 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS switch (message.messageType) { case "entry": const entry = message.data - if (connection === ConnectionStatus.Paused) { - setNoMoreDataBottom(false) - return; - } - if (!focusedEntryId) setFocusedEntryId(entry.id) + if (!focusedEntryId) setFocusedEntryId(entry.id.toString()) let newEntries = [...entries]; setEntries([...newEntries, entry]) if(listEntry.current) { @@ -103,6 +127,25 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS case "outboundLink": onTLSDetected(message.Data.DstIP); break; + case "toast": + toast[message.data.type](message.data.text, { + position: "bottom-right", + theme: "colored", + autoClose: message.data.autoClose, + hideProgressBar: false, + closeOnClick: true, + pauseOnHover: true, + draggable: true, + progress: undefined, + }); + break; + case "queryMetadata": + setQueriedCurrent(message.data.current) + setQueriedTotal(message.data.total) + break; + case "startTime": + setStartTime(message.data); + break; default: console.error(`unsupported websocket message type, Got: ${message.messageType}`) } @@ -111,7 +154,7 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS useEffect(() => { (async () => { - openWebSocket(); + openWebSocket("rlimit(100)"); try{ const tapStatusResponse = await api.tapStatus(); setTappingStatus(tapStatusResponse); @@ -139,14 +182,17 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS }, [focusedEntryId]) const toggleConnection = () => { - setConnection(connection === ConnectionStatus.Connected ? ConnectionStatus.Paused : ConnectionStatus.Connected); + if (connection === ConnectionStatus.Connected) { + ws.current.close(); + } else { + openWebSocket(query); + setConnection(ConnectionStatus.Connected); + } } const getConnectionStatusClass = (isContainer) => { const container = isContainer ? "Container" : ""; switch (connection) { - case ConnectionStatus.Paused: - return "orangeIndicator" + container; case ConnectionStatus.Connected: return "greenIndicator" + container; default: @@ -156,8 +202,6 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS const getConnectionTitle = () => { switch (connection) { - case ConnectionStatus.Paused: - return "traffic paused"; case ConnectionStatus.Connected: return "connected, waiting for traffic" default: @@ -176,8 +220,10 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS return (
- {connection !== ConnectionStatus.Closed && pause} + pause + play
{getConnectionTitle()}
@@ -185,42 +231,51 @@ export const TrafficPage: React.FC = ({setAnalyzeStatus, onTLS
- {entries.length > 0 &&
+ {
-
-
- {selectedEntryData && } + {selectedEntryData && }
} {tappingStatus?.pods != null && } +
) }; diff --git a/ui/src/components/UI/EndpointPath.tsx b/ui/src/components/UI/EndpointPath.tsx deleted file mode 100644 index 2561aab44..000000000 --- a/ui/src/components/UI/EndpointPath.tsx +++ /dev/null @@ -1,15 +0,0 @@ -import miscStyles from "./style/misc.module.sass"; -import React from "react"; -import styles from './style/EndpointPath.module.sass'; - -interface EndpointPathProps { - method: string, - path: string -} - -export const EndpointPath: React.FC = ({method, path}) => { - return
- {method && {method}} - {path &&
{path}
} -
-}; diff --git a/ui/src/components/UI/FilterSelect.tsx b/ui/src/components/UI/FilterSelect.tsx deleted file mode 100644 index bf6764ad0..000000000 --- a/ui/src/components/UI/FilterSelect.tsx +++ /dev/null @@ -1,28 +0,0 @@ -import React from "react"; -import { MenuItem } from '@material-ui/core'; -import style from './style/FilterSelect.module.sass'; -import { Select, SelectProps } from "./Select"; - -interface FilterSelectProps extends SelectProps { - items: string[]; - value: string | string[]; - onChange: (string) => void; - label?: string; - allowMultiple?: boolean; - transformDisplay?: (string) => string; -} - -export const FilterSelect: React.FC = ({items, value, onChange, label, allowMultiple= false, transformDisplay}) => { - return -}; diff --git a/ui/src/components/UI/Protocol.tsx b/ui/src/components/UI/Protocol.tsx index 0b031bf18..0befffce0 100644 --- a/ui/src/components/UI/Protocol.tsx +++ b/ui/src/components/UI/Protocol.tsx @@ -4,7 +4,8 @@ import styles from './style/Protocol.module.sass'; export interface ProtocolInterface { name: string longName: string - abbreviation: string + abbr: string + macro: string backgroundColor: string foregroundColor: string fontSize: number @@ -16,9 +17,10 @@ export interface ProtocolInterface { interface ProtocolProps { protocol: ProtocolInterface horizontal: boolean + updateQuery: any } -const Protocol: React.FC = ({protocol, horizontal}) => { +const Protocol: React.FC = ({protocol, horizontal, updateQuery}) => { if (horizontal) { return = ({protocol, horizontal}) => { color: protocol.foregroundColor, fontSize: 13, }} - title={protocol.abbreviation} + title={protocol.abbr} > {protocol.longName} } else { - return - - {protocol.abbreviation} - - + return { + updateQuery(protocol.macro) + }} + > + {protocol.abbr} + } }; diff --git a/ui/src/components/UI/StatusCode.tsx b/ui/src/components/UI/StatusCode.tsx index b35788ac1..b743aec02 100644 --- a/ui/src/components/UI/StatusCode.tsx +++ b/ui/src/components/UI/StatusCode.tsx @@ -9,16 +9,21 @@ export enum StatusCodeClassification { interface EntryProps { statusCode: number + updateQuery: any } -const StatusCode: React.FC = ({statusCode}) => { +const StatusCode: React.FC = ({statusCode, updateQuery}) => { const classification = getClassification(statusCode) return - {statusCode} + className={`queryable ${styles[classification]} ${styles.base}`} + onClick={() => { + updateQuery(`response.status == ${statusCode}`) + }} + > + {statusCode} }; diff --git a/ui/src/components/UI/Summary.tsx b/ui/src/components/UI/Summary.tsx new file mode 100644 index 000000000..3c05a89a8 --- /dev/null +++ b/ui/src/components/UI/Summary.tsx @@ -0,0 +1,32 @@ +import miscStyles from "./style/misc.module.sass"; +import React from "react"; +import styles from './style/Summary.module.sass'; + +interface SummaryProps { + method: string + summary: string + updateQuery: any +} + +export const Summary: React.FC = ({method, summary, updateQuery}) => { + return
+ {method && { + updateQuery(`method == "${method}"`) + }} + > + {method} + } + {summary &&
{ + updateQuery(`summary == "${summary}"`) + }} + > + {summary} +
} +
+}; diff --git a/ui/src/components/UI/SyntaxHighlighter/highlighterStyle.ts b/ui/src/components/UI/SyntaxHighlighter/highlighterStyle.ts index a5be67b25..1766e309a 100644 --- a/ui/src/components/UI/SyntaxHighlighter/highlighterStyle.ts +++ b/ui/src/components/UI/SyntaxHighlighter/highlighterStyle.ts @@ -112,7 +112,7 @@ export const highlighterStyle = { "color": "#C6C5FE" }, "operator": { - "color": "#EDEDED" + "color": "#A1A1A1" }, "entity": { "color": "#fdab2b", diff --git a/ui/src/components/UI/style/Protocol.module.sass b/ui/src/components/UI/style/Protocol.module.sass index e702f5ae2..9ffeaab14 100644 --- a/ui/src/components/UI/style/Protocol.module.sass +++ b/ui/src/components/UI/style/Protocol.module.sass @@ -6,7 +6,6 @@ background-color: #000 color: #fff margin-left: -8px - margin-bottom: -4px .vertical line-height: 22px diff --git a/ui/src/components/UI/style/EndpointPath.module.sass b/ui/src/components/UI/style/Summary.module.sass similarity index 75% rename from ui/src/components/UI/style/EndpointPath.module.sass rename to ui/src/components/UI/style/Summary.module.sass index 2fc54c0f3..59bc34893 100644 --- a/ui/src/components/UI/style/EndpointPath.module.sass +++ b/ui/src/components/UI/style/Summary.module.sass @@ -2,7 +2,7 @@ display: flex align-items: center -.path +.summary text-overflow: ellipsis overflow: hidden - white-space: nowrap \ No newline at end of file + white-space: nowrap diff --git a/ui/src/components/assets/filter-ui-example-1.png b/ui/src/components/assets/filter-ui-example-1.png new file mode 100644 index 0000000000000000000000000000000000000000..244c1b5113a305b006d82882448c7c8962ba4de6 GIT binary patch literal 41498 zcmbTdWmuK%^98yU1tmpF8l}5Ix|Htjkdp3BrKF|1OS-$eySuxj8_v!9tN;0OJ{-8< z+VSlD+%+?6)~xZDmJ~vO#e#)EAP6GDUt}SWSGnM^{~KuVe>>GA0Rnk5>@2TrE34x` zU}kx>V7nFih&Gc7G+ab2^8|#s6lOCuKCH`DmG^ z$Zu*Hi9F_g_xbI3*FKro{^tIVP&U2C2E4gD471}O8#XK35k~`8t{zC?`?qy8H(b&@ z#SwD}Zb)`bFsjGcDbh>J_n1@rmQRR0G#|(g+rEfg5Q&`3Rx7>QbtevXNovdmP+I% z982NhHM-MhJ1)L6KiaOW3#0rsRh^Dk!t<_#p^a`1HQqOXH+_NAN?zVJcWP~htLspbp zQI)JQD4=~()}+!|!d$t`v@oO@Tf?OdO00O%dA~6m3Rxr9lrs;3{!PUYE1a#!4Udcpav2fA-+?PX~U>9SR{7igf~sL~piiVQi& z)hBiq5q*{UNfnb&vdIALbMp&5M(U_3_D-$AXY3K(;&wR}E~f4Y#^%MI`{p5Q!J*pr zRQn7i9Bnpzm|Jqa7;J)%K5H#qCS%pa5$ulH+|hq?JoD)_?7VYc^{0|OO}h|hW&2}CJ;kjT*uw@adYvFgft$h3dEt7zkxl} zMpO~H({eZ}DUpLv=S{z$MY&EL+&5(@oVWT8Y!cg+Bx02rp`1Arpw)>9|7|TEl@l6b^9!c-D@mYB6~lbeo8A$uNyZriu8J3K8ie)z8WE^(=8^fz zC#6br7T^Uw=ZC1uY>tJ+%ra1K5Y?5Kd8Rg+=F4E5)+yLzq7bYUB+$rZmoF$cR3or& zP;y|=q;zuz%i6OW#^g?387HHC!d-1O3S{(zRpjfP-ufQGX!+4=>Z?|n6rud=JehA$ z&^0pqY+9hqIurgENz)%BM)Y#^lv~+ddAIIsMJZ6_saC_v0ttq-!M$UZ15N10$f*0P zo{pM7k>MkpT(6w^%0AD(2;ItY{rD~Amn&oqh3vD9kdPy9o{(dS>PavR6LFN#p3?bw z1KYCg@ys_4?7<5+oL8vRR@V=FNN6w(U%}h7_u1jXfee{2HjZdyX2rv_CAL zG^AA^#?wxB^igqpo(m`{oz7N*CbI1a`&=*O3NODS(jNGU7a&v<$ft3Qb>un7xyvQK zo_F?R50qS9C3R&#^A$gMn=jUjfhe4Re0?#8;&PgkB){R&`{*l6knAq~HL%Ixp!Xt` z?~F6=D#@G$XM*sH3{NX0!#jA7`$Z1PQ)J9L5xttJii0xSy`8rf3=Z?-7Aoec zrN2o{*E33^$#IW6m`}xjoOU>VwN5ah#&|4k*jw782_TJ-42X|{FHruC{e4$MKsI55 zraPi;MBsgyCd*l7kGzm#>|gF&mrzGcU44rFrr%}n3^fK}-8BnEh=*^;MYPhwF}nMW zgt=FP1m3q3>^PH%I1O4JdQCX~`n-CmU8hu=KwINbR$q-`Z1f?HXN663+-V-ddP~2} z<;T^_CNecjCdVkJUNZ3OOG6Jzc#A-<-<+1!42Q4;qkbArl8Ob03L) z+UOrr!<-*FTJWhpzzm~DQ+X%m=38lR(99t$f8pt-+3#6BZx~AweSwcD8wJg^`oRxt zsD9+qa?)Q{?&`Djck;fqK#{#Aqr!MeA_XfQ%NZXc?o@`;N?j&iL_M6Qz}N9V?b2KK zx=3t?1o*~8DB}~BU3YL@2glr+p-Qa27Hv9Jl37V(?ldd&*K`)hVT!!Kz=Ekmc8*KT zqc$VQ9u!u=WD9m?VjY`2Hl)n9aiDE!Qz2bD#tXdNki;N1aQJLx9hE)Zo8=URM zvzjSNC7|!}^}L4-O^SodCiDovBK(*u+QO2Xm_2c!$?m8yoa|$gWi6{esk5O@Ump-r6VUN>5hL4G9PN$ zeHvcAGbIbcz{sG^c6dve+z}rnZ^&Rfed2oi^O4%yV%rC=c4THsTyoV=o{|Ff8*KFx z_89vjYDCMc8(JH=j_*To1oFb$NI*bZL_pv_&;|gS zM7J<5;dWk}PA#>sG=ri$plYJ12t~;hZi1_748E;7t~TuL8u$pF$||1aBUL`eC=Jhm33c z;;%Mn;WsOy^oz!W2!4E};`4@JDnv^A`g|zFA}5ipz`y#7#O@xZ&QL3=y`jwenu4es z=@F*6p7|^S#axyQ(Y9&kvir?QB^;A+)q|Vqd@u^ZUwCyRYL6_$0oQO%6QfbWR0r&u z8grvlJj+@rc+TDMk8HA@**qyQOT#+}WxUJO@6>HvOpeyiUkJ-7^8VMjy=x<^|Um#Dj z*|yoP`pyP1*~3OdCmUYuU**T)$7=qmz$Qt+pwQ^14Tp8L;8hdwwWp(z=hi`@W!PMy zTC!u``TO-j>ZFqLO7OJaUVs+=$9|Aii(PG z*sQ(rxSUtEwz@h4G1wgc5?EPTeZa<+t1?xIXEOTv(F5n%CBrNv+zPn1SKFjg1X>wQsJ{WOt^NaI-&V*k%xX$>aT*)Td9MdU|?V z+S-0PU+k@}uV*;%kdX}cPeMYHnUz)gee@F!4$k-aDr$Ci_M6Ly#d?RczpDZh#>|L_h)gC^@Ipe} z7?){@r#PIBziqaM!XhKToUC=`YBsV@<|_#a3Hdd;Ts4&-z5VF@j{nUG%h%3K9TPJm zgpVDR?(UjVnlh+06%lKNn#Fl;PFPXq9m)GHOS{;MWBHrdaME1+#m$fDU_($T6@!vE z9NvHU(6cj|u{&E%%JX#3sE{Y0X1-VlAtECB9UChgipK?!CPU?Pz5YFxCC2FZS36Ra zIA5{wTe~m9`TjH#golU6)YP=%C!TR778o{XaZ+L;9W(RhNSZ(p4%>Urc0VK(Z0tZU zFDOF6pq8GVUkR)hW$q8obJZ51F)(mA!$5`g$u@6vzG!P{@myKa zehCG2b5K5fTBycnW^=T_R%d@0P*_OqaJ-~HnWyjp3rn{2+gDWB*I{8{9o^l}g9C?2 zESl;g6I0|+5=Vw&zDl_Pg1x=H?eS7lK>?Nh!ORa;4NKF{L`2?^k*L0ixB;c5^h(8= z`d~ISH8s-dU*1j?s=@F+JwDL0u#iwu1%VU5Xt~0BaCpe(dQFExqn-+mt+KMR<=+*) zXj-jg(swaYQJs5}`K1*VnP5?Gp#MNvOlLg7)VQ3_c?%V7z!7ILU+5{(Y1f?bgIpdj z59i9|T(5W`l~h-6Umnf_3xTdwl^Bd?hMV+z-p}Vvy}v>e`E7h&+^brggTT11m>Hlo zo%5(Wecy8hwM40qBBAuN=7Uq5cemAX2Lgq;sfbg(b;XrwfEs~N{ML_-LW+van~H75M@Lsr z*1NCo?sUI@f71dUk}Kq)E-fuVlvPw@awKuVj);i4U{zk6owc{LKp;|5Qi%B6;>O0t zY5Vr)J2LkV52e-BFfE;(ueG$a+S=OwD3^YMh-QWRB49t?o+E5EBb7u-+ZZdRZKom>5)4L?b9D2;7ZqkvdyfD1K;22%?OP%tC`x#N)%A zT!AvZ1a2rk&(LhSA-LW4OsVcfuAE;uvDnKM;2wj65#r+F;!XZc8Uq9O2KUtM4tZVW zCh2s3aWOhQJ-xAgd|Zq+;jD?zZngFbtkzUy_u^vG>1Myp)&QpQRDsGLZlZs0#aeT* z@K5>H?y?FbcrR}~g{5_iST%YfLB(Tg2dt6%VD1DQc#)~2uzf#iE z(=S8XFZQQ%<#T0!6rbrD8|S^1e$mm{xw^F#odawoUMLjLC?twZX3faZL$>RFFFHQH zFO=u;kCR4*AOdi5{Ias|flt!kpDI+{O&OeCSWsuQrrc&{n-&%ph5#@0Y58(UYP~C1 zrO}yrJV&a#z1<5~s9YNVYj}A0UxEFIMXG9QCP}(RMk21RoS2xHM5LtPNv_i{FfMR1 zAn&lT;aY&1)Vkg<0NWg?5OX&hN&gD>?7OSX=DWb+p8WlrYC2mMNv&2T?M6mT9Xv5H z!MRykRyGDZ(P)bQ{$Gdn&B<~GsRRLR{FZ_;MHI#I^e{B8b!JYE&G8Z!F!Pk-{g)hh zOM&&q?s7Y@YOI8Wgud{Y=}u;*_qL7YG#Yg)o0}J+hdJ^Ta0)|)v$L}&C*Alqcb8_s z;eH0bAHU*dwDJUrT z6%sPVj*f!jRaA5&w*5LaKR>XruyCt;VF3pU3Tj|*(CPYbn&MnX$Ct7)Sk^t-9E7#? zb>{B$^z>vGNn>LQ;0IY*S+SUnJAQu@=ouK;0MSM@uyk8Gf3l;i>zls54+vid1_r9r zJ$-%8aY|KxXDkaYiQR7P_F_NrvsHQvi|4_vJ2W)3qm$EUnZ6hZH=V#}nKSB=yL)=x z$jObn9#58M6l-SxI|6+4zP6wUCx~(p!7apx2t3?}2Bv2f7fY$D;{YoWEL6+@PriNo z1fEQZ8_AKOdV72S4h!QG6@~S9xWLfwiyRdn=5RhITU}k{y}Xsn5{(+$NV&7qnJiF= zz-BeKTy1*|JZoubDGs;m2e9OYMn7ofs_!=nu58!45Wri&15;@>#AXPEqVHizN=nAY z#g%$Ixy#DQIi2qy-(4P#t3vI8vu3*7^z4X1q&wRlCM6^L)g6kTA)8wRz{l?HE^snB zy1HI|elRYV2Ortk;=rXLIBd_R5Jjm3U6!+p%j2F1tRf5;OOxB(XXo=jQ#*7T8X92V zzK^E+7;HD)KSu(aE%E0ttf+{tqN1`;XD5czLV3{mZ0tz*++VU}k@4_`z^3Oa7RHkd z2HyVux7Ypk)buG%tlUX-$!wELH+u>fs#XGoK|n|M8$_ppfq}ml7g@kT`qcRX zALep(bTCuumEwN;_8Fuwn{)x#z%2JdS67$K>68e>p05CqA#q=O0zhOioE$8hoB756 zeZ$p~OFVN8u)|@n{x=uXTHxHbPEAE~yWM`bmoAXYK{PTlD$r_9K0mhws02U(KZ8~K z!eQmjzJ(eqF9d8BIuHN{Kb^4wB((!T5eyQZUUvxY&d!dup`lQ<`Qm9NweVt{U2jj1 z_GVucFnnDd9lri(YG%I5w)2nwU=;B0$?YF-YVj%K;%;&PNt{-GjnDz_xg@WOKFl?^)3^DGrIe%Kmrtqn5t@gV;}bJVfI@7qX3ze?s@a z;y1piTjGW4c5^9#f?{;M#Vw|`!6(+Q|Gi-CNkr5b9WyR(x^F#rXp~YX-C!S8mwWfy zui_l>CzSu)yI7WDl2cOdJw}S8sPUA^b`A>WgEI9X#s3Bo5DxF~pS1w<`YQQT{=Xl8 zm);}tTlBwo!*D4YlE;DEFRs@O3U;PCvr`hIH+rC&wRi-7(!SpcF6jf7YM*|GoY> zAZ&j7_U#Pde zdn#9%VrB>iw@pr7Xd$QmJP4BqiB}LNqc5>o^g9d;QdY5l4f|y3OJ}AC;TxG(0l@6a zlMvy$6Ld{Zm;|egwAdrixuZEw(S5wV0o?cVu*2lA-Ql&gq|>Og4Z|k+&!XS^q4f&q zrQrsbR*r1C2L)tcl=bv@Um_r!oJG(2S&RceR$5i{s1!5W)#Y37a17R4M5I?a(o$z| zFZj5_tyg2IrpA%`#t~j$^zNI{swi>F-2Lep=Y(=FJNlw6*|V+qRRC1M$+7nSS*&3bMRVQT~ZR7 zYvTwbNF<%ISo+At_NO!~Z3;cJd?+_L^GCC6EPnpHY>-W6+gz+=yp9^WI4Sap%EFe* z8G5;Lc~}pB^POn3F;CZQQ_|;@`2&kBDsYLT?5xZe^gOOSNfb>NC8d`w_38M8+?Zmv z@&j=Fmt|M42nY$ij+-9BU^h*)!HmEsuaD;OfeXFvzO~iady^_p5yxhO*c7!7CLG}B zheXKp@Ux>MlV5{0RRo)p(^WHesJkA$@;)B*`mxHao{rUK!WDXaYz#cmTd47W+I>O^ z!QBU`5i_? znxk#L)#F{<6!BEz^K9{tQm7*GJ=1cbFr^f8K_WzqD8@HCCwJc&_Y}w0ca~MGm6pOc zHa0v4QJa6pCL}~eMxLuW9V~mqf~~<~E_#~kTP!kT#k!qdkC5VaEXNuV zutDCw^@rH*Ekpyjv_|pH1S?xVaO0o&E$I~$|r(}Ic97&=kr$zdACircOYtwmv{5pAHioQDyiIF9{Pi{ z-r;yrcip$p_WQC|sPFl9H$G`&<=w-i(sEdMc;EdEj~t*y*4EZeEN0&8>Jm1^)1kAn zDt-E7{&%sO=X85WTQYGbG%D&fIDh6EcV!mKAHnzX=`V|_H#o-eY*BBW@7f&9*5mqo zdfFW+ww8hbSloMS_X8f6#WPQV0^}mg;wg`CAl)@QYEZFgFw@Kq!AHU$EYmmHS;)fG z?Fw1hi!vEV=86%EDf+uT6bwRg55^VG#@%r?y?&q6%{Q2y3$ck^T1$2^^MgY6~Dk03L2Wk z+rTyXlS07A2>elMW`WStTLJTxeH%vtRKx&D338u zd5hw%4R)6MQIBrf{f@YCFO{2Vev`|&r>m5AQE9Ls z$;qJ%x?~7>420nFogdJ9BDo-ZvK?kQKi7Ov@{{7m^J@0svOk3Fy?SVLL4=HqG~3 z4eXpizGBkN*)T!jxv#Hp#qkLHaA;sY<6u1NXakK*`ez6r(nfH~OK-NbaeciHGOX62 zlGm;)23tW+e?ji@OVrwhszX7tE{_$e_ijAAR;GMi7>c2 zk_I7Rw)yy7p{iZiTwl^=|Fld)EQ-Ish%4Kp8h8&JEO<|a{3*F&b4DE8zHb&v@y>P$ zrOp?=+tF7YClQbelN?;r-RWiwkT<`2jbUz_ke2ol0c(O7xNYEzUogr20hwTm%Mm#8 zDpjTk!B~^qfwY2Kn1kv0`Tf(&u6&Di?gI~QA7x}lUvEXl#(qfwj<~LZL(bz~alet# z@R0<#@q)qy5Vi~sW{TNtcgVI3LgI5xzy}7Nbl*YwF}%`o@BN5mYS?W_q`wm7Ztkdi?>V={966i$d3Aov*QO z@U!h-S_mTE=bymagBzGG*2yt4lN)Y?`zz+HFr(mF<#2m{{3x&R~2q7gR`UYZBg%J

~Q*A6uQ&B;i6oP z%WZd!2c2k*LxD5R2Aij>Qnz}r(FvpBY_q$DHh2eU7K`SjfXYhZK=h|05Q_1Ei;?TisWM*q7L z-O|prmS&JLjT;vhE}Wbcb!Pg@^$gsYB>aP<9SH+Wk;!aOENOGk7IayUa+i0QL%FUK z6t=71CA=4KHrV5^@`AsRC}F6~R_kP$u>I?vKaIz+MTy5H`s@V;5?IB~Vr~W#s_1pv zt`d1X{L!hCLrlpO7ZS6?wzm?d%QN`)Ce6Cy7(Ezw=TEM$L6|+i&n@e7()*NHuWM^kLIyu?d!mE|9DJdBOQE#xU9Wk?7 zZ6gNMF0mN99!${tGB5eg}DeAoN<;PLI-DU&A({?jj(LLOc%trRvJ18p^56kH8>sr{?-u?8m*F08cO5}N=hz-0L%)(XgkCD;IkL6kS87k@c#H-w)6(wV!mCGF zJZQ%J;ImKBs)?l9mCe>MRf~hRs4PnjU{(GP-YOe0+%GKIHhO zyB^D_Pa+Xtu~nRrR#WrY>Sj#Ae4(Z@wbhMWv0%m1uKBSsmeYyk$B%iV`Kqj~>n1mt zSzOe<9+`G}7KXKHt!;YK9Wor8gYR<{VI&&1^6*+ADIGiXDbm^Exi5=RumFRa`KLw3t zam8|*v7j`^i3jf~{8>&#M8f0D)R;HFzP+Wp+crXFWL@v*9T!hJMd>BawXicT{l_X_ z{&=C#+Dy%<_dJs*%i;;9=v7ibf^P8gN|7Gl(9{(OUSEN*ET0@!~pzuH8_=nuie?zhoPUK zP$;T&osh!2KV;&o85w!l+23b0xs)GlcO1_SCevC9TpQ;6utj45l0@lTc^nSs@VX0Q zqp>8mGgbh>N>wig2YLudNRHyt|2TcX>`nhl`MKnY&WmR+r#9xT|LpifmFeuzFJxW+ zUp4-qSb%^%rGFaf9|`C;PfDd?Lk~Pk9-ep7xz_d?grCee?jC`|oUPG;0}<`Fq3CQ~ zWrBZTpyBvd6c|3#Ea9PQ)xoROx566riZx}Nl-ugGv{GF`H?JT{N`~%_`s{=8Rr-hV z60Y~XD=&v_>}hgUy5T@pzz6?siuLrBa_O*ui0v3-ig zr9xk_WL16!vq+O8xb|jVOUdHsI53vg0a+~izDQw;m7EkhAu(~_{&n+r?rM*LC!gQH z-Kb)lBMkGFF4$8ZKq+=<%?ya`l9v#b$C1wd*YUM7oLxD5q7ln5fq|`rKXNkNmM}e+iWg zz4ac%-j-%>j{`Hc?{K)EALc0(Zy3X7gRDt%C}?y!K&LOJ|fC`5{FC)ATIy_#*G%=zKQF}JBC%N2={P*NfCYL62Jg|(g1x*Nw;a#5UeHqcmD`9gFf!iQ zkIDNY;0Hx1TtY35=s+c_Efp=Y)7$My!DBLnYk4F|-oR=men#%Pwu+!*Vj>dDZ5(X@ ziSSGnwS`u730W)10&a}klm|-_nR7vns;@)`%F4=aYoQkIp6N^}p4qHlOGv=d)|TL+ zw@8y#RgcFIRnS#^Y2`GEavJkt)={<6INHz_v8A^}4l?0F@(yMs+YS!#fTzGIDg5?Y zL`7Vyj&>tRvR@`c5Df&XoZ&>bBd1_2<`L(^mET;jbuuJiA(xYo9>Hn2F8DM~E6-Fsa2s+;a&rD}8PLZ0`a z$iBM2-@ZG(oji{RbBWKFD}SmPn61iTH#eiax-Rzv>JCZ&Dr=(cA}jF`6~Fl5rbw%~ ziiGzyD5HnvxB@Qa*%{P(y<{frfW2*EVPypz6NQg?O52p&S#7b31Z9u0MG99ynPMT9 z6ZbmXau^$-lHt&`54jwvY+fXX=N1y~KaAHJE|P3xF0!mo<;iK$Q^%lehQV1>YW4ff zzBJifrl+C4zP@tFLBK{!la#` zbF#Z1LvwSo`;I_6fbahK4Iuxv?pYX@Z3qc7$P@=Shu;w18UFm?xux^@%F9}3X0uMp z+1VMDRx@a3re-_YFv+B`A~DR8LZNj$OsPOwNO}a%{y@1H(@3tm2p`b$Sx)8lR4NVO zfXEnU-=3?qWUn~p14pOc-V8_=DqHTgE|NcBV!p?Yd6M5gKWCD-i>BBNNzt;KDwyqf zMs7+6{^Mobi;5IXz@oq7cooB<=sdHGM9_}jZn5T>@({v{Ckf&UsjUO z^k;w?(uBpPvjfV}y`lNcd|qHg1R<=?PnRpacbJ$yz%zpyG!d~O&$|k<@hA{#=a8U= zRU-1pa31zwk(1{#4Ke>Dp{9PQ-9DUm`F*r_I||(Bv;PkdK_ojrk7c!xX$lnC$KGka z_R7xb9&Q%bJ`xM%zYzU%(89eVRUXAA;%%=)%cW6LRvv3yFuk|d~sqoLNpa`s1k zBq50gB^mX_3j9+uQ?tJo=eA0%zK?dZRTolCPwrT_9F>!>-vQya$?-l5B+8)752QF> z30xeQdo*X*4PWuO4{>6$$qe<&?O$VPiEljfC$%=Z$uJ!_doP<{85kKVG-JpQU$mU) z>d5DBscEZB7~3@k`fq2;=8mim9?w{=w0s1$>F!7fFDHxs@=QuZQgT!@29!Xptzp5*&CD^hy#7m{%{l8| zRsyRwJUv;gCB<%MEH-k)Pzgkr0BCX`rTJB5-Ac>pT4z5!9bKw^e@@0ODEoKb^F;r>*-nwoksN$E zTv=5`L{IPE9XehSvRvyTwQm;z@qkA_P$|(S$E12CcEb+~jJDKX|}p(nU5m zoE5M<3ym&Xlp0?;B%bqIYkmB|Qbb%;T~pJJYzej5Qe9TyumO=kGu+-RTG+?ha#j#i zs0$E;e9=L0Tpan;G0zHcYsM1=mt9Ape)W`BM_dFnG*9FShLVO-F9sW19PSVcXDghD z+j;Oh@XyduP%E~xiVO4eBP=XQDJV#m&uj0f^b<_Ly zw64B>?NH-~L_!s^1}`QAomxG4k=9zm%Jq6hC%IyDls^9Sa1N6noHvm9uW4%uo;=*g~NHE(s zy|`44E_`_pOh)MsE#oCRmt+~i*j3+oT&wma^P`hq2#bog_4G)aP+kHCYqr7ZE`lIr z&*w+l!Y+wBBik?;6)CAuU*yS%_# zPcngpTDeRw%AnP*XX&uWG>$^O$yI{#q0TO$x34eVd~q%wGDeyLzco`D5roZ3KHLtN ze1owLa5-RYT|cm0uR zud%02Z1ocQ{#wfBrF^3^;=X)(qga=&T=V4xvt&yhj`<4L zVGwcnvSejutHZNH-0obxf%puYg^KoERShHtOCHT^Kz2u^S|d*R@bk48D2Jy>B%rIB zJ-F{*&D5EZ<||pGx_hxlQr*}NvIV~HJVOvYe7hJ!N~aXd4r&*xYPB?=l#_4g+7+Y) z49WIr;V5`9g$CdDOQGV~h4=AU#p21jPJwc1NIY}?=3Cj#gP@y1RxN2znDByjJ6&jz zbO+qw$mHZ8O5|LGiQZnh8wv5|F6tmCaskzXRHboLic6C7g-M1qnR2Cd(v($}Y;O)B zH!0D^u3P+PBOFj1J3nCKEk_(L2lWbrtpT=c*1Q3uG$|H%~57pVOfb{A* zN^Rlffp9t7>NrRfpiw#_zb`{VLaN@Dv>Ci{cVi(tXSi@xFj2J7oA`60+Wk+OsuB#$ zUD&NmG5KFCm7DX}tJgNAwmB}HRgtB(e1(cACp&ogLl&!o2X+$i?@6Z4ixx2P=NWc( z_pDe>DFTNINVti&vc~SQqsZj~pkN&6^-sk?04Vo((h0y|&CzMEJKq`gxx`g9kzf;% zk}@3cDs_na2O~hU zhg)U7*sG-F-hEfNybcWC;DN~x@bo9+Qq840?dXNb#7-|gIg9B6LCR66^ zVJ*G&Y&4(bzSBbotrI|~2Vt>={2nO;^%go%@C5Qc(SCH?XD1yJatUOoA)SGW(f}wm z-q}tk@;vsBF4VcjXnDj53ID|d>MAZ%Gc$is$Gr<#1YB2KG8d2b#2=k_8)$@E@9S%! zi9a$?ug_vO)o7pupa5?s8G|%;){Zm|5{Wlm9&FHJf$xE4kbc=lF!aXz?=S4<;sBvv z=y;t39NUOf2G8w}(`o>7QhU%Vlait}%5#C^CkEKVtRy-@|Ha=3-hO}X0KCfT(w znp$Ipog37v`Xb4{HoJ2@mrn4AiDWB{^(AqTbXVIp;UE8KCFgo^)p@i4LSVM~O1lV` zsK`Qk3(uSVvpCW$kOTOtGd7G!kBGmjos3A}bI9=4#YKt5qGO+HBiL(#Qq0`-PO* z&8OUCS67y=J}4$(KAvcUtK!(!F~-KnF*$3NZHjeGOxnBgk=Alj+%tfkfdYK=zyK9G zje1`zl*eeUjf=#bC&Yt9>e2UPom8e)){NcORE!+SK~`3d&2a@}{5Yw`lYIjzPh!bN zzChpjj49LFp}&C<7Nl(bj*iYIUp})(umv2*?sD2k7dYmCzNMv*k`l%OwR%hpCDQx* zJdC>J;yh(5o5$Gd*w|QQpy>(D&3&sxqqTmuj4=kpI-gWpw1D_U)Y^aoN$wlD2ER*i zn^3Ru;7i!-F7_Sr%tG&X$I8F+WSF7&J=`8Qv5_e23X={4*cfJ?7~yR%oFb#WLlEVw zE;7@f$ONIETD-hD$ZRj-CjQFdh@1Tq6C>iHMBx##SP0@?+v-d_ODmXQrZ9X42ExmWeB@Z z`isY}kLrs{%F4bC#v|Nc&c6itHy)n(J(CG6Q2gitLg^-23FIph35s;)3q0Cp=aJQuPLF`+Qiwbs29)jXFj0XGFkPQ+B)pZn$nQE!t3G7Q& zi|NQl0OaAZS;&@Q)=$=jY&Lr6p%Lz5J391S-P~fi*If!nJuL!&^}1Y|>yCBg0ul)h zkb250YGZ$7+?pP5;RgA)a@xu;Vrafs=oD+>qkhPAj&gZPXLjo_!!c8Cx=RsFr76-I zjtEF@96~|YNK!L_r@GHF*H% z#zf3OaZ7C4s&B#{BpYS!ZUlkoG~I)Pal7NqhAXK{K2Mu$Y(B%>*eS;@2oL)*YrStu zoID1y`P%EKG|IwjZ7{-;llh)+4(iKu(>k3N|7R?Xg^tw&omKHi`D``d- zoXRnot#P=V;dyiA$4YcG&}r_YM7b+_2PTpg7b=Vh7Zw+QGjZ`xhiz|WM-Yvbw(I@+3*wt1q;ekO3a*aAv zp22wj;be^pdrWbu;pDQpG+k^eD$xen8>SZ}Pri>2kkc;sW$?fyva1rw^h(L?)z z=?{{-`=nN0axswgXBQ~fdII0x4QO(p0L%fn9_SK!0l&S;D)O=8DGIY2!{+f(1?cbx zgqk!UpRL#eBjpb_VSo%tdQJq15Xz{W^&=hKJ5W*rvftCL(?n(ybf5%G2UvHkWlifV zDZ}jvwn}5eZjOF*mDjsY~HQb74l`s6Ebd6mhdm;Yy*u}0~B@8v^o(=mcgB;=z z^oq&TXk_)N_E{xY0eiV`d{O0ffgrv)DYv%xUx!phdbSm@g}@lSEq~&ByEEamv4(<+ zq(?_W)+cb7SHa)Sw8NM~R^r+q9O9DLo&BOpwYyQ%i=WO7=lwekCRV+5#x{yjNK6Nw;+ zAH86k{uKZyPU)4M9pMNP2~e8wAhg4-cfdlXpB4N18P&l+Jt>rKymHOeJG{d7uv?Vt zHBdeI#hnDQ=Cauzd;kD~;N!;^K!prEkpM3*q^qmT$2@_R`X7;D#`Ms3S1>lHkfQnU zw*t|A#S*1yqrpHdAJ82C{rh(XXsmHQTYAREes~_$`8g2*0q&N=|34NtY{34L?Ajf9ewX( zMir2|03B5#yIla#DFZ#ZV5RZoAC3B|t$2`30v_Z&4o*;LC=xgv&bw6uKq(6P7*I$h z!DnD76epJKZogjiNCB}oP{?kANq`cUVQd^1?#Cp6;DKDW#&Y##Z*MP9*&@Gt_jjRX zDR6F2;dFq_qALiKEJ{=`1Sj{#bVIq~J94$f@>V+%PsbwHahGU+3@s@qXQF1KGmre z1hMGrc(w!(24jJ2^VRFurLH&DU}60EPYkwKxP)uk4O^G#Yi|9nqs}(ZnGcZ_R75}H zr)8?LHEv5jfo;V}OG^XYCb$?F81GR~RI1H!)v8Qay^y#ejym)AYCgWc)}UYN;Nakw ze{cPSn9d#=7^tc$77Y!}moHyRKw}JOKMDZdBPS;(Kq8zDYOv$k2%v!i2%w+!!o`}6 zpaQB3Z0UJUKy;U$6Z3G5kB1imq{X0*igp0_kq96?UI$trATF1ohF`BhVHDl+l4;t+b7eMS1K`Nl6JOUmh?TtE(doU>3{%$0MKAK6BCVb>{Z>Z9k%B z*<4B5(5*p`P|6%i8Alk?qXjO9l&asCsn)wGEAX>oT1exJ;gHo<6v8=2tXxVTyuO0rPp)>1a zMtkosF}d`SgNre0R>T+FQwJsi}ba>fz2Xwj2%+0Os?FE3&7ob${ z9v_dmyDCqqg&k*kQUq;e@OYdbp4+XC(W*3oS=Q9mrRU@b@$-P<30ectvIDppm6bu#n^P(>_Y=DiUU7a0_?bnONmhGHUC=O zv(AYE#5K?}H&$z7SftsQcahxK*fW$OE94JJ*u3XD1YDd#Awbq-T?e@W&|Y=` z+Xuav4J5xV(yMN&`z$KNLfWP%IXTrCxhP2q;y6BolYVK<&PI5hjLV#plj8@R@72*_ zdw)M1;2A(8p3z7uU)pJd!!h3F<>iMDoEq8#b8}dr&EROU-VZe2fKIi3lob(i@rj8+ z&;!Q9(>!h5kAa}>D8mJLZYdL_+hQFWw|0vF&4H@72`QK)RW~>jFX^Xloh=Mn|LBa4qQ?gAO|4 z6r`~*EGU-!p&3RPzu6|~$X8MK>*}*dJjM5~2OS(%+4XV^&^efv(Se|nl|>BNBn{>& zjBIyCUVuA$bWYTP@B&Oh1*EMYQy9PfhWw-jaz`y=nMV(b&?i>IVcal5$G7-y6Dc9M z^G$RbIu7fv=olEn-6=up6pPDYFAZ7_7Zw&yU*%c@Rfs1LT?47^$?0i2aAv%hWXJ*n z?VL!(zz{$miV0qgRN`kofKosdXL)W}1@Y^-t1U|uO(woMKHuwlU*~<^=bXz~KYt#}ur)m?sNl_>Jr3gH;t+Sn zn>Iv9{wmI&ctx(v%1%p5Df@(hpPhM{C~|HX&eu<^%nm^F z6LG8J`Hqf9-GC;8`&D+p1M?IEGKlQ5Z?W~P`VJN|y_cdX-jOMLxL@gx>G9v&XpP6J>L{o7M? z4W;!}XbqfMsQG)zsL#R(@~tEqZ3T(LF7T+cy-Ulz9t3MG{JhI$y+y=i)K^g>y${l2 z7;dsKraE)*qeYH3`AwA+HJGi;6PdKMwe4^u5uA%E2?uZpr|agwhNe=b$oT=BulW2x zIVWHQJ(z!}H1jH1{x;nKztF3F%8EyD{ zF{l34$T7`%3HBMn5$-xuN{kY*BNq0oP!7HJEE5u-JSo_CXT%1Ii#OX5F zX}>zO$~hGSvnk5*XKYTaZH}zPGW|rlrWVv(z?g6}4hNp@L`f+uEd2an;ul+QOK1a^ znXw`CZroL=YAI##qRGn0#5&IQYmcFH?LTm!AATjA9(LnDSy(mRZ>V^%pSO@a8c*Ha z6BgxR@?qKdS4pnn{qKfqQvoxd?haj%<0fBf7`$$gt{&kc`%3AAipLk##~J2Jp0!cm zh@UE|{)B!3Z^(DUo-k~~d3#MxR@M+u3_PipWrR}kAp>V;oNc!(EqzfhRg)Bl;Jbuh z485VEx_XGI?ZlhHd%tF2p{(JOE3d764*a>RW11|R{6~V%;n&`*WgiS>upoFai z&G+2Ya#OB{ogF8(R2cvb4s0W1?bAt>36$!I=c8D zO!VQyMXV*L@O~XQkgCQS{juoLe_8-6yxTvMU#UAotjnsNE~BYQi6xI`eG05stQtdx z0VI(^El+9N`;%j}q{SVgf=Uk${81a+v$M;hZ)CJQkwk}x)*`fit4^xDoaCwEng6*k z9C_i%z#NA@+kQ$Lnh4_W^d1|p@LS&-Yi7g@|NZ5~(T-PS{3MpW`S+x=xk6P^24P9T zJKObVaPS*m&!_jaG&E_K=T9xmFy&}(mK~9%SHGjcsH4;6^dqygO zX7<%Iv2Jm~pkew_7ROS^1-jB#1zr4>Y-6LN$5O3&B%h{eI6CqXe76ENaIl_bWW03q z>qDVOr@+Vp?AY4bHT!+YhbIv3^D^uw9Qt%Zjy<8AD?9f$bTyS_PbxD3TrT4p(NYkjV!B0PAAr2Il=}< z9gR*2#NOzKdl=2csiq*PywkGokEy`#4M^!eM0?yNpqtz>3rcc<6xuU((z-!}cilAZFoQ03&CF0>vVSYeuAyO63WIk7g7Z`R#B^@^Z z*9f2$r7*4;bNEfBEuos(OYTbTx3j63m{klACP}a&2|>qjAkq2wVuAA&!{7`fx5te-2dlbW9-b;H03vAD-!* zN=kcy!v)N6opR%wGO{w5gRO)OG~0ZCXncCC=H*;T`u+!vs7vqmM~7ZR1wD_GRlTz9 zk86*qolNGl?d$n@T_^AJGF^stUWR>gMuv-;6|JaMw|2$@8?o!hmV5e796x;Jij>UU z`mTG7>?ztAi|fwOLW4bBT`qt5Wyd!c-rVDNW?^aAP*o3M?K;VIC{uf`TxjmX?;no&P;vmi&|UdF-%6<+tL44UwT3D)fzx-sZeI zX|fJ?(6`3M7qDlm=2`0zzK+MLtfhsVIce~SIC?nPbiJ_GBjxFlAn-9KM>4JTC`G6c znIe3s=ndrbP`=)&r|sXj4?g#&V4MH6TvfG2H=m#&+XHI2i1jm> zd3c-}$C8v{-nC6!;^nRS{W}5~0=58pzjLj8qoQct-Q90MnvaNG} zahyGkwzQU(L^V}6pYzI?{wVJq|SR>az@?SzNrB`(;cC_|pT|@j;nHhPF18 zTnBvuT*#|srfc1i#fVMcxKN1W_ld8suWV-&GCz<)GFx|{B%Z%#rmq|yUjt3QMo%HvHbac|&>eCK05W#sG=6 zYd9AO%_|Y|BIK4PJNR9e*wKwpd8t8LHT=zoymk&}XvdcaY)J1QrRF|&ZG-@6eU35) z31}8Neyo`(T%8};RdY||GQb;<2=XxMaOMRs!Tga{qpQQRobuzgzP5%|N79cUank6h zn9bTC@I`kOsBWn4q@gmc&}0}ahis4QAr z*NhZvMLx?|{4KJbvVHiyRgu`%j|zitizHrCYBk_=F=<;Dg;6VRW|tf;pBt^BD&y4 z{AdN$sSa)MNJ>gxa^G+~dh}?s-quTYy)Qvlow-FHH!}dYU}df$*8RtRSa>~mm`6@5 zSVd7Mc<$vxxx(jQXkK9hr0ip02(Bi8=Fe{HxmhP^X*2xj-Dw#Z7_i$lsHCHvS7SsD z{cnNe6>Jm~2EMeYPNxi+a&!?k&C>e4X+}M16;p>(RlY3Bm*PV4$Cs=_@or-Z{mV?F z_rKh48t5%^)cN!C|G!7S#Ju8g`~kj@1@(YXUT?wBa*hpVYId80%e=)+Gc$LQ8c_~+ zc+IOb#9&+EWnP_T*U9ypXv@5}IDX>=tsu-BL+yDs^hBatD`x=SphUa}A3h;-L@`E0 z0dxxtnB3;gSMy0qQNvlTzuQel%}w;$rpg%}d(`ldzO0xSdejjn28KuY#~ATTpc5hb z3|l(P;fz8;CxJ~496R?FGpf`sOiVu8;`=>;9UOeq_uJe|BbPwVh)YOx9Adpi*!005_slCkkwRANZfR5JC2 zhZHXml2}F>qq<5w=q)WR0WOo|4u8*SQrSx){?Xvzf3}WHpN-dER{iC?D`#zdjD}y|_Z?Z-3jMwmwd9%ezh?ZW zmqCvbIG)Ll>}FeO855^Rc0b?YBIc>SUfW! z;3i733eQC#@{o+Q`*k@PT8$w`tYk{ky^M$DicTIo_6^(Hyth~w?eb)ANsLKd2;n{B zTtRb}2>(9jl>Q(Zf4$eM19ra+esRwGcfH>k=@*2!4OAOUvgGd3A?YIy!2JIB&;5Ar zN4tnuVE3XzAz}@#jdw@B#0rvo2+SRdkC4lfA6D51vnD!1ToKBfR)PI_aC!0g4p*h6 zvmrhn5+;P}=;j|qU<0a-G(wI*=!wPsP)0Aa;4G}25dy{&*sl>lZ|wWqU08o(xowF3 z;73pbsQ0QLKVAWnu0>4aqP zc~?gZ3$|q%n4u3qd$V~)xaVK=BmT_csLJuib$hdxdHeIF(oaRpwj;J(x;6cu^S;x4 zf6K zLV-wUD<}-Yg1SpfucSEO&)i^*1kg)Jt))q@Yk~5G3m2eJKlsqwn;5mDACrfp|L@*k zktP(hZ*M3Ahgf#q$zN1%dXv+%_0eeQLfZ?a1Fg3pd|Km@ z_6aMoN0a&6uZ;Kv1X!E=-}D>BJ+mvw&|N87NS1dLDN7N}1I&wd@Z+Kj zlg5j>qrKHFRMG|9M2WFo4?qQVLh+=hVNZ$#XvAud543*iWJ z=)`+b&9S%<0vAwP8XH)HpmW#7$5xM1QOQ2qxu^MOO6kp0tgO$$PJ;DHd2*Ks`$#&e zIe2QVEaUxqJsg0i`IknJcu=-UH(wDHoIFa=+H_{Z~?d>fa z8}voNf6C%cOK5OgyC>VCh}~fItm5C@EdMUQ{_+_`@zic_)dtziMDL6i%-c!~_hgNj z-mb829#4sHa~YlfKCw8)kwp654<{3$%f^RHz#hY5$8RDh zCiZZQxRZT8AQzOqr0)qp)+Z?W*@oqNF{4CLaQm`Ih_g{ef(QshIV@OAd&;V+q^U&0 zw8!U?97LRpqEUHZx%bY?<0S&-BQ$$h_wRn(PQD(u7wxNDxL;Aym-}Pl*mP*52e1$( zy9+|`%cy?1d1XkGB_P~cW$7pRCz?+7A?GMXvn+qbBy`SlO!gkWz32Zu1Xo_-4E{FZ zytcps5L-J3nez1%H6~#fKa>DP#H?VFrsU93R<;uqoXKr!F`!+*HX-+oFaV%|{(kYA z*Nnl@vfv8WJ59i0mLdra%b_Lt`WgIkWk7^MvSq5x6N`kH8zh4O5}J2^xeP9906WSKERSYCBi9Cu|j$MvmrLj#TQazM?Lh8(=ON1YdvC`F7rzJmp z`n3BLA72>SvQzwTWcib>kDELMH)A_keFV*A1-1^Mts*oXIy!-8qz{5ygtA8}sl^ug zV3>`7Y{9UR0f>-_s%BxVh2L$BTUJ&U9WQ~aL!ios>}%0KdqLQfP`qaDgx{L9WNVEL zecYAIHgXCI>D#xHIZwW~`(G9dGIQe45Ft?q{Q&qXcoSt)R1%OZH2FKn7u^xzZ2x3V zj(yC)!ct>Q8LOQkmD!eUQg;F+trEX2$`vz)pQw$BNk-VR>DoDj{YScY7qWqv&{Q|f z6fW#M>REbIMIQ@EMSq}slL&b%O#m$4$L?c?6N0O*r>e08Zn`H2wCzv-ud>HaN0BbG zg)vVZro1-)X~L!}kmQR)2B7<7Pf;X1H-r!w&{BLJTS7RTRQ@K0a+xXOLxug51MMxL z4NO#sWL@8P@L(b)D#uvWp5d)ibKkysR3$-oZ!)s)2y7g(;=rx}_Kf~aA;;9o(ZvOg z39b9;5Cz`mIgR%x5p6b**XwQsq)r%sY6%4303(DEf#yk-7wN&U6kqvpP;hkYzJOfN zxhu#wM5AE}s4wKUb_`7rAe0z(DN%Lk1$rDdXD~IuS$h-4JL~|$fas;r)l3QRgM>Z^ zfCiCHk~rxIGeewxaxHGnI)Q<}vE{j!>>Wl8U`-J`MVL9`+n?s(IEbE= z@EgE2!by5XW$#%`K|t{Sj4x!A3sSDNR0G!xWCW!o2k1(;p(|nAuL<3$Pt9Y(Q{S|r zPj%qjzwbZgeFB|VRne=n$)zt>*lk=av>Ms1G;;5$d1d~uGx*i~cv|!3%~ODDbr(+_ zP4wIG6s;*%JIYl{v6~B8wq0U)3@y-VKnCyi3bO2mnz~o|Cy$72qz`_ zLDt}L|K;ziTkF$=a)^(s`L@<7L&Mp-&RUd=hX3Tmd@E7Xyeh<@sIGEEd_%!RQ2L`~ zWtxXWef$AYd0v~IK@hVZ8E$9?cv?(esn`0{JrJOU6w};mJs7UrhI!yz0uh6j9(jQM zz;LyLGd7%qD@&a=3bhs&DJezW3u_SX_UWzW$6L)|7Yy%&2JN`zY-Mth@@(>dF)cmU zWrDR1vCP^ZJE#1|-&^jeib*}?%IJLol$P%nIcJ%e2xKodd;A@T@t-N7B1&>{?~pY{ zI3J5rvYG=~**fEk>tggXM}WN=?-m>?P1`NQ;*yu(>LK+gX!nhOVCXhSv#;DU2MK2t zpEi{=_46oAjK^n>R({3nJ%W71;l?Om{mdj_X(r*6EOEqfwTy8_)vrJB-u9;xUh|(E zDQUJ(DZ1lmQ2UVefhOaX|AKbnmBd*YdUgUa=x3UNGbTiA#dvH{0UqLoC@?8D9q^Zu zP*HhZv!O0Ewukj7cTql%yk61;l4zA;-t0fDL+*_Ky$jdmv++m!&jXr+`UkH;U?LD1 z2a(3b+uvhLa_9$kk5~3+r7$ao#*Q~!v$0!Sc$^gE!PTi`+86TggNi>p6#PU@yl~H6 z&mI4ZEX;S`6ad6Bgn|Gp^$Y;(c7_|3i~iW+8aG30T^Z}_n#x|8$`O`r(_?BTm#hE% zV4fXnr6Ea4KhMB%1^avfBq^#dy5~=~f9yo-kK-bo$BG(=L92(5TL$u$Xsl5-SYOlR z?Az9~JUyui&I3uD<1E4Y%8W(-r4()N>AUVcnO)K&(xWMJqgkTiqp!31afW6XhEO4) zdg#Nr5uo)FB`GPr$Qx{Al@B*d*{4Q;?+5F{$^E4WWDN_8Y<@v+-DB4Mh})cM)={KY zPH>H$d)2d*byV%&SeP_=G%P3pE?-iYq4J!=|4dAOc za#8{F@Uas*UC22?`do}|-TG`lMU9e`fK>vlM^sZ9sFdLpyCa<|<{q|)o>QTKwlA&^FzdWtB?6ZFZ zuvYR&0-DL~G^*N2ZiP4Zp41>-oY3e`)lOI~K#`e3Q#{_paqGL0 z%yS6}`WHNZIlg>5_ucXmr2#Kxo@(4L&3C!zOvtSVSXzWRbnto^3G%b5;9Y88g3qarDOo z8O}_c*h=EALIOp9odS{v+sM`itL5UZ8(CXlxzr?tF_UEKtnl6bZ~Nct9g@gU$LD@M zO)WWERH<|Qz@G|(uY>vjq5#)@C__&iUF4-Z8NH91nw~Ju{_*m>4v$H7w7!kZ!4%TH z3u}WPP^n;s#!Zo-6Jcp&V+840R)C7{Xt zGIU!WcX}nCA0rf~h3VcH=sK2@eH3;{85N`k*{chC9IP=9j#vM81s)0|R5vyjZ;2GP_M_ZXJU?#U9Pg7A06v)ufq_u?l>tg=7Q6BzOZE?_7qwh`!G3TH-x?bH0mTxP zVj3}MH_-O_39G1hvFj@3Ws76vO!Cz;h19&(@`$FzeB?CpzCe>*GHX1^*iJ2Q{~VNJ zr#0T|jxIVA&>F;F=0B5J zDa=@rm0@d-ikVua;8#Y`rnT-~fJqGb2^0`YNm79VT94I@`6j5c`F9Z?9K`GhG!-#g zVImxz5}<~_ewnSz^nnAR$Ek;A*0Z3XAS(vt_1@~72_7o+uPq^mkhj;>6HJyT@gY-? zW_X9^ubIf4M}8-RgcpIiN9c5P->L!lZb+4y5ovaUT~iD*TSCI)oHLe5J{q%Kdr5CX zw~LdS6V(+Wks1iwYc^KrR~K58Aa%JBGQrkjuW+fVecnt?FfsJnKjuHoBfaGbhw4X) zzCeZwyV{4fd`hj*4j?NEG|*GDU}$#CKsW$^$~bj5H@}8(ZVuEjL7GMQDfs1gi6QMU zc*?q`5M-Y;A)=JfmEq9(_U)T8&VQ3g&@BVg(=pckgh60$i75J1Q{d5)0{5SROSp@f z7kel}4k1L+Y0elwR}xXyE!_|TJlR}dzPGkuuAZsiQ0hrMyR^j9G2Bz+auT_hVB|mE zEZqr2M8k9E6fh>p5E!{_K@BeDjk9G}-VkRg_tj+GdJwCn`Mb^8ZHOuw%>1w4Ja2V;T8yD{izAe2KOb~(;Q8JL=Ch?fmSj-*jU8{3W4s}QsW9ljR#<1XG0>9X169#b0|B{aH3 z%hSD3JYW&SLk!|Uum?Pi${ zacO`N;pVvHfV(eax0?8~biuoSh+cNFjRNNZuc;<0GhoKBoVNrIjXRp#tI(9JV4-6( zI(c}I#A8?{6;lO0?rdgYKP6PmuWXLrjK?~28ThOTb_T1cWNPoKU5 z;j8h!pzVCxe`VlYR!7)dh33E&YT;C%s0W*mo?qi#Y-SZNyyD@ItO2nWKDjgKcadsp zWM)QyIQaN6WF!US1r@hcjEI|zygcZcGlY>ouZ_MsG=*cu0QZEXSwQ)nHvz$@tkYU8 zR(rR%q)Xg4;Y;g>aD*T!0rP{*X@h)HDYM2mhA5uUKuc;lfIz=5K#Np2Lwpus2_XFq z8!~)N6;(A6vUsCY<$pZY~Nf;46;#ZxUE7-Nt2D>w%$pJ8Fy4>vVf%O^!e z!eF!r1Pi(`B+Tp~f`OwNi(w4iB*a-GknBTc#((W?9W)jFpzqJ^Bq z(LarWJI1$R*UEZ=e_EnW<>$V+r~yq}&=3Y57ftIaotOVA;8z9G1kYf~XQQrO*dTZ#X<+hR))g=v;g<6i7W^Ev{bC0VIxnX z*-nh+?p_f>Zx7~6pLPBpq=TYr#%bt0Uej`eHCXWmANJ%al@{;LQKj?4_fE*!^w zE8F{SZTG)(+3eX}dEt5B2`67fN^5?(&w?Ggt`S^^jC62aQht=BV4uc*#8nZu#nrCp zI;#|IPpxJ}|1U?izn7W)G#v#q8%9n&S>UhTdTOaDTrL9;TZ{fw?P>7~dBEQB;Oe!R zj$d4$Uo5HPcqRtGuHHNpS{14aWUdZn1kmrX^xOSS|&?WeY*Ml4C$(Okp4ag0Ux zMK~98e`hMoj~_}aQ29^x9RIpVLtx>bxH|Td$4R+gQ&zrs{lS2oYb%3+YqeX8&cc@4 z?O$tyJ?_FX_%1G{e{k?wsK$PQ2iDTQC;VH2SilC<1e#`9vxpW zA3m%|kN0(C^O-h=VQ0{w@JHLdFsk%ZZagBE)N;x4mk?ZG6&X4X)#fpt!wuAz?7MDf zndP~R*^akaN>wk?=$5!MwM@ym++c(r`s9J0z`qlbywkA~>vZh8e3AB3Zax74RUI{+ zk384r1_F9F%|tUxOQpkkZ7!91zc_m0gyPat( zKW5G|w_p3b+Ig$W%rkdVu2;&7r{;}fI}rvJX?3HMll7Q@7+BJ;FYUqg3*V$4SQh2j z<=yUiBp`BmwcmGZtT_5hm0Ilxyr$ww*WSHQA_HSQvpDt5ezvgrlEWXFslO+~!Lu95tl0Ndvb$v?a z`H97EjGfU?vHhOCn|^< z{^-xw)GAsjc^Koy6O`j|=g<6(qFD}u5ah7pyuS0dhs>mBBiv@AfbbK)7Zbw=87m#2 zSj*25wyhI!?2^aS)9d1c+9s;vtrgp2RRJ-#{(LRg)C_Cc z&em7UP}g$8IhB8s^nhu1OkMnsf9Sch@y=YjcBvirxQ}G;IIUK%4&JOSRUGbeibU6< zFF@Sd5LrX<%iVn)kIbCpcCa>-t^7}E%=K*b%S5PHESj-uUhA5l@A3pPHw})rz9h}- z$hC^fDha||*AWg;%c!K@XrjVGIq1@w8m$sq#z2)5CKM<66=`E4sP?LcXOPSsiic1L zB?OL6dfnpt{#0BZgzbbSsGWP~@zeZ0^CR1eBaKrZ9jblwGl{&FK`g=eiFngRlVt$ z*IFq__}rjXUHz)2PdQr1`jcg7?#SF@;bEe{A1%#(8^}Zj$UmT`%e2IV`_-Tm( zvq-6b2D~~`YzS{24!e-OWFob5?w6@ACu7J+jnQ!PC)M#17Wx1>NqF+TDiuUv_lD_L z{q$+NkGE$paNp6S06&FM+{cf(XsE7Cc94q|*cVTByAXM7i&c7vO6}*y3{X+=g&#fd zt8(uE%x4nr*9qEvYa^_|A4XpDL!lD#<9!o0QgfRtvhq;^k<-QNKC*$8JMM0EN&ZUL zhh|lJ246%pW7SGpFlp~9>V9+=@lt|uO25s5OW!p8vzT5H_plj%+1nFHO1FE#z`%>6 z=Q6FpC)2IRTDA7klD^-QmQP_Ktbmznv#^x7_~WB5o|hQV6RPw#AFh>^fiLV@0=@B7 zKc5}(Z2$6rB1&*E(K57nWz|M;ese{zn3DS_eM|&wEbbR_iMGXv##erR!%(Isuqv{U z&UN$|_Xon6{mr9YQ1+=Bt4wo3;b)JJ`Wr5o35z#S(`Vo5u=pgajl_iKmhW{Ze-Gh0 z39I&8tI(PyFm9g~tRjYAT__Qx=S@#yAE^)D^$@lJwuv`U+xc{Q7?mVgcT*V37!Mll zB|G`ZTiX0hlu)vIj)MRZ4Gv z+~%`iI9u^t{m=EQXAA82C|Adk1~NB%`z??^qQXN%6Ij!d-C?@xD`w!Z?Y-A+^rPjC zctQI+%yoy1%?-YwQWS)N4Oo9!LLrhSilTTVJuJQw|t-O%v2dk1o2PE6jpNB*az z*Eq5v(jhFpDY!0tw$~$S_#~Ug91nk$k+jmz@6CBBHTIDGeEY>iqr+oE{M!5AZ+y33 z=>}An#|3rY2qQvgR%hGo=fvVC>(-}T*3rD`G;wnl`1SKSnm)k6-t$F>;7!^@`SUul>deoqJ>>`E*ouagx79|Z(DAf7H`_` z9BI@WY*8{;xKkZn{(|w^a6&<5P|+2OUn^mR<8o{L5@h=0^updRU((=Gok(z3MDROZ zoj0A@zdyeekgq02lpYGpw;Gu{Mi&GG?%&2pInF#V_o~Q?{)tTK*JwfDc&ShlRXoGa z{$_jg!fIJ{nUK>~bd9QMUo`I_j4I#R1wT(;zeknfDmcITPy{B7=VebxKDw=2uPv=U z{@X1$*N_*7g9?uQr^Rl?SfXA`3e-FAZmuZ+MJRNC$%!FWp~kc+B;vE^K7Q{kV_cn)d}{t{rQ?;Jf;KVKh471gI+L`M z9k~yfMYs5l6S1+S4gNW|LWy4YllerXq0K>JVi&BFcy=8vMi1+%fXF8{X{HO=bxl!Ait$HyA7SXhS}No7Rj%T1e=C9pWw%6<5Q{o(kXo{+0M4 z)S!QcZg46nWlZo|?^OP9^U?h#+crf}XnTa6?Q0h1d$Y|n4xZd+IzLRiFxe44Kf(bm z_ro+xj=(8Qy+I@X>FSvGJGr_CrPmbOceFbxvc}#cqt`Fnmz>^B%@=7mW@Cb*LpP3I z$b4;H64|EE^kXGIJ2y*?94u?TU0+$k;B?eeCp+57IqAr->Uyofb2^n18QUT+POhE@ zpaENC@?~WCi{HHGd04j^=@PcVbC4< zbpeDx1uK(^H~jNdGpiT7y6nOij7bmq{+YNmKy_)3r|7n^X4@sm_o3JNe zuRPuI0He^NA4YV=9wEUaJagxG9`ByYhDL|I!Rl`P@KtY>hUCemC5~^mjAD)){y^_9 zRsNGjG>nwi{72UQ^V&R47si`xn)K5%GLlttP7_7)SVM!wm9Clp3>bMwtX8&C{@p&# zlI`3#$WFo~N_t)0^PF~rhkxf7uI*jV`}tCaeW&nl)cEKtto6uZzHWNq8>tx)QH$wjXqknbN; zv6Iz)ddaBkGkyMp2@Yy5ves1fTR2&dap)d?NW*KhgB5Vrlv>3pm}5us=X)d8UphBu ztM=wvbyu{1G*BtLdMU6i*SbSe?I$@xk0mm+TX(=?B@iXJyhDHCU;$jiu`tjd4G&>V z7@SgbbUtDFb$<2p+=zo&*^}wpz3+*2c&6jcGC7(iF6Jji%sR%NR-)te3ZRVtgs>1j z7(lp38ujC)8dza8JPKK*opD~ATFPrVn>>X`fZ%1Z@n2TY<@Q^$LCjzSe%#Els`A;4 zwETwoUJUw@mq?;i|kF?Cpa$cPm%K5*sGyI;dO9RsN9GUW%K>}x~p2AqSvFO zr%(A|Fs`++RBvio(Q{cFWZ^PX5Qp|KUWNfE=$I(jYymK6gbdFDN2JRwY9TX+xha_ zG12DnB=&#`NL1xh7&9720!)p02|H4YeA6}lQR**Cb3g>>D}{v9#;|fFZ+`z>-KCHDf0o36Omo^ z7a~;~+qs5qjqU!#ynOlWX};5!PVF$A!}Kv-zYuANDC`)moVi1lBHJ%)y1$TTs9(d; ztuGimw>T9ve6kLaLyP0V;r#Oyb3+bE?r!vizW-;n4RrntEJKvGu~Po2(`RFUBP2gg z%B?P5c3{@3b&_6G-=F%1!WQ$RoSfqb!yHD<4F4tlr`Qu(5RuhSkDA{T!4U#b^!lqC z@oUj*O<7A?obJ(dqCA^1(qKPScd|nCxz0V*StR&Sw9U$R1d-)icz_|SB&9_^a-rIo~&&5&}M`F-;h(`Ln%G? z52b|iliLp!ktVBj+M5gNH%2=agg9DOaVu6^k*26Lsd8ETyvZFg81M>?R)1=bJ9A%J z1>e+b!j(cv`3+L1lynj@H?gHjdpWU}uW)2@+4P*|(Et3cCFvuoGNdleKV|1672k)3 z49z#WklRfO;HWhOOX27enowHx05$~Se7pQ}L)|6YrwlxVRr2pT*xB6#;o|zs<6LoZ zR5X24@EDhSyiS{YyJ@zAwKCQcR&qpz(OV&3S5Kh_FfZERzH}t% zJ#M->N^SlEzVO27ii_nGZ-Xuwb8=+r<{!jWhPc}7F>;A8HiA0h0A>mawsF6ZM?;7o z3JaPaZ#}HNYA==6nq^dFaEv`nwEP}tuB@8cF+_P$SyLQeylPG^L$|haj&oB#e*gYi zxy=)-YA+#C%5q+GF5dim4FDC0N&aJhbP&t@xWxTTeX`O#lO)n6+tRWQTxsCz?nb<; zhG99>fb7iO-W^DS3>yY2H89UVS&!h%J?y&rYQHj4T9+UG~F z8z(N>Uf@3VkSC+Sl#(ypJ>;E%c1ZH*?YWVAyjr1c!ZyD-N9+g@drcA_`1X2SdQK-S zVB$#rkb>a`QI4g?whno=*`ko@uc#W)$I3NDuqX` za4khQf2X%+z!92tzkq0^IqO2PrP-94@NDAz`(p%W#ND zF-piY&!NMwgl(^yLGo|A5(!g#?VaJh)`L|9fKYv9ZAKcy03 zkvht8rZ)m*XW<`xhJ6jPd$g^@TcvWKN~y0iiyTvKlc_Hz7szf3`7-6z;1UTgt3cHz zm+WI~`fB?Lyyfe^xrgYzRNoEy3a&qx{2B6Smshsv%qf$);7U*sNG+NRE(xf55fyLZ z`vU69&<7qI*Lb@cfTbU;JKUZ6{Q%q}i*EZaoz6S#I$xge+%uQ3)}vKur*$W*LP3Zu!BsBL-AqKtcxQ77VXz=TD@+tq~AuOFm&SFzi& zfq@2G950!btW6YY9|}2jVOV+gblE2gaa?Ng_U-Zx&7w6W$V2geym;idS8Hw^JFl&| zUtZt4z=dvYVXATba*6Hu{lg&0M1C-w2s4qRNO z7jmQ{Vx(V=`;vpeP~)-YR)U9V`8a)ySdnvs<`?4|&p*gYwwGTFoC1Quoat`Oka1R2 z?b%N%EQ{;?Z&k|J1_XjxoenY#*onKI7SMZb_tDz*e-jIQL~=9+Mb1&bG6Ym>({^_{ zq*t}JGL{>)q`u5mx2pXM*OY+Kat`8=pHn4-9Kxbt00F z-^^VbCQ93Vt+|RBxJBsE+P_p`Htco>9gUx%=g zc3?th=B#V<^`%{bt&xXZ3kjP|PuDS){NX6|wF6`p!g)|?_wuqkDi(HC|2VG|+Md7Kg9Q`rSvYQS79#q`X zWB0?Z8yvjuTz}cT`7r3`e#vVJx!b+lA@H@{{xzv;g6M$r3K8yQi4C1vITo~NW5aG3f;8)cDUGKJJxRKfVp@ooY3AeFKE|~GhRDR zcqg1>_RUac_-bb;lFO|ojfYzt!(^qsokD{9%o%!Ui}-=`Bgbqlex-V$9t}MgmH)-u z6uQLdvQ&6G$#y_`9z_LhwED229OAaRqE$z};Jq!vn6~x7C0;csAu!HQS>{ z$*K6~bWn0s6D%0E!1hPrM}w19X_bVs)QUk?zIMRp4mTC8b$9o;Tf9 zQBmP@yiVg@Uzy~ujgPxQ9gyyoc+PM>mAhPr@DciN;!31Vs?Ao-Tx%U~5Ge-R(eS&F z^@h-n{m3SqWO{V@8$B|*Pmy{)zY*Xou6)^t)jvj!_m8Y{S80N zf)aLL15Y2@MJaHHh}Wk3R`+lJlLti<<0{E)A2d{8dV}BTY1OOiIOIYw@s1|8s9E1v zwEgnYF6r-CCgf%1^p~u=l=#{^5j%U2F-XpdJ-C)~BSqnHVo*7e>aPd2bgG!2_vclx zjkZmbNRiUZ|F|Xjna<8rHqfkpJi-_KEYSxHBH<``A>vkm*{T=!BJO43D|d3ObQONA zlQ+$2w&c1rxs)UusC3s>);jRKj)Ps_2BUz3b&_kMU+)-9hFlo@zIzw*XG$+5PWnam z?H`w0>?a5jw3UE15wc%7^a#%6v zETos*mP;&{=i%x<4RHES01OO~8Rwx88fkhE`N5&OCUf4@*pU=RxVz2<31ZUY4N6ME z5<7)>gaRwTF>(|fdjg_tk8!9xi``Xv=Xc@x?kPPx{+4)K8jMiAVOnfSY9JVXAwM*A zK1o4ghFVzXM}>Y4H84u4426JYoDs1RIXrDrbRZETGQN{8S$E zscS{ax_bAJx52WdIZv@VH-ebB=UO678o|Qev?RmZQfSjKJeJG7!-I)(uz`o(RnYdm zHM_U|(c&CcDKezL@!_kD;;?t7;GgcE-~2HWR|G;V6pwgAu)O!Zbhpt^zJHhFeT5FD zB!BYm+T6FFt{tiWY5K<9@o7bp`Yaz$zm3(rO|WQ*OCp73&09ZDPK_JtlcFEyGlKLG z@m-ZJz8qALldXU_dh2+NdC0NETzJ zl;CKqw6VDh$#zV+9e2)cxkktQ#QhYj77G_m8#bOAN=S3LG#+zb>7#+KT&u{Lwq#>Y z6b=O}g0M^XP9shGR>SzBoRj~KV7cHf&ygn4P^2eg%EJ}V-oAq4dHA|QwAQX|h8;U) zi5`8Z9e>Ps;C)Q;&s5b;wqVp;FfJ(nvDb4S-2YtjIKA8@XIeSzlmNj;I`pp}OIgxy)X}7vGjH`^}3i(xO+Wu^b?Zi)?jxjScKYqOR6%Lr{ zc-gfa3l6>doqrH%X#L=YF(=ilS4uDk*7t>N7Uph|>FbnGh0E9ag)3#9nlq1fGxbr- zOe89OuP5l#?~`(MbN-aU*^@0%>npQHS|4xu=-b+4>vUfDZ>FcYkRLGp$m}k?*q*UaIeo+_8`SV7y5bT91OGLYk`f;GzAxK~wY+tw0IW0g}vD z!NCn){+ZOY>&(t<^JhuUTarICO~j z#D(alh@7;ig0eeyJo0`{A{n~+cE!weVf~B>$<+H@CJJX5gZ+P|QXkO?XUN^nTtN4% zevb=cR_@=V^@#gC{9f%sz-I0%k8dgR>vq0byym((dPLjDtSf566I*j4XbV1duYGR+ zGOs`C%~|9%nzA~wyTE|cuhpBAJ8^YB_eIX~y}tx$-b8vO z!MTNulnLYDpdj3MdWn$2IOW%sbO;L$M*rdW<2u*%YT0}u!+9MKa;y}l9f*h{Qb6srJFg7u{uA$*w zP#_4}yRxAnqi($t{$>DB*o867QXiw`x{(R$GEu4R?;4`=1|c8~3JN-P!gqFgnVFkg zEp%4*`t`T-XFw3Z8IHu83Yh$vGykuoa}S3yedD+`ov@o)9n_{AwuGpOT85CJF-An1 za@>;Alu^Ra#9}V}$c!9vULnyKIYdL_P`1sXjHp2lIiz#Sv`%HC{e9ov_dl*{F7G^^ z_qp%)dG62W#%db9JF>dE+LpwhN!fg-x2K27Wa{9X11&+BV|U)@2N-2=Z(G5&V_XF6 zXX$Dev<(b^?M5Psm9qv+>P2V5!pv|M?OqqMtiUNYKVK6*3crQZyk4OuDQocaSgi~t zFaT5dV|~6yWP|nY_2*I`N*EsPulUNw-lnm!F)-}mH*cO6lx6HybV_kyQrq9xtUL20 zytc*Q72|<^o3&J6*6yWKN@{q#RLl?NbZj{OF}Q4dw5q`0hmu`Ct%)fkFP8rlHmI81 z6u1&j3Vi>-cN<*JoY{sSmd^6+-6Dmry*>;W^-OZO<`=5E2OLz_Epf_N6Y>7yMCb?w zZE4JJ2DNg~#njjrXw*$m^FVblvwp;9C>rV@C;tBX z91`WzY&M~&APVKw%pL;eh>3~0z3JTgn$ziLqu+%3NzV4P-aq>4U70LEp>y(bhPsF~ z7ybqu(=aBr>4aoE5bS*}F4j0@L+T{SCa;P>75>GG7g4vOh9H3IdT59nf&RGZb=NK@ zASjqw?h($Yq|^dW`r~4OAf~U+6w;UxQK_ukS`8FcGk`0Mec+O8QyCxI}a&vR#6cyuGe}(l~eX%{!xJ03SV(6Lk z?14ja5d~C<_D(;}^eoM9uE3Mzw{|Kiv!dTJ<^A${??T=?bP2bLkfNz1&2Ml`1Q_y%nWWTaMZ>sqNS&&kPaPx5q*~Jk}AYy0*3;x~{3o2HVapAa?+LH+FZYf(%2mKszQ=XqnM;%121I`kl;< zZ)mW9%+1!`-Y9(-${z>Jxvs9Rb7fv|{sL5CXN?dagN0Xpz)(u)oTWV6w4SO}|Ay7nU%k}`)3rvNL0pdElQzg#Y zDk`g15k4)5D2O`d!&hgjc6ocd=99Z~NIy~6Irz9(UT^g+o0D~MWUExof46K11d(#@ z#-Zmn&7$MAMF;g*3@e3y!_^}nFzz*XyO>73y}hVc&xU`1`vNhRFA^Dn7?3X5p6Snr zDO79ICe4*`L2ec3d)qk@UM3XMsz-Cn%aze{0`|)wZ$jx)RaJ=JKo-P(Q!!I}NoiqD zA6IxMv-(b0n%L4|TTp}dHFx2jFx?$4PI>cW-Ngd%PB{p3cpg7ZOijL zrJjX9n~(?qp1F!g-R^62-l?>W^e$cWYpOOo*^;2Xkh|S}^k=@!$-uJ{{nIOBDG1;S zii*gbEJ_X}r=>OLU)se?qC3w%O!s$nb%jQLw)WJCQ*8b=Gr1@(cd8=gyZItpZCb&T zTsq$-|G1qThr_9#IiW92FL@atwk655%*}H~Mvg>9MJdNpfXydmWfdzL?(iCd!5}#? zF>j(Cvx6WH5eIP`lcao&$ye~v+vg;Lfo(nF^+wDUZg+;P?FZ~XMsZW) z+Q=f35B<1CTI->@kZlNpSlE$pn9Ti-_X-3N2(&)rDLRUF9_-4VufBBjyaIMKv$J`@ z^HpG(Qx5qBm3Ujhe?n2EtfsaXwiP!Xgy)2%A|_@r_wyp47x{&Ss8L{m^XYkSo=E6; z5j&w?lE0O)@dDGqRx9bis^<-|9~_O|RPt6Io8~AL-RIGwr`4+dpt7RiR3t36kOm35 z`u-r4!7G-Om6pOvCv3dusgdaD={e{Z)~HgyX3d(G_I5=6&Rky3Z{IR_JS9+zauKky zI~835C zja7lyU>Hzlh5YG#*KH>8-=L?2Zo>Wok@*oazP<()jYf;xu!8Ubpr>8;f}()4XCM0- z4hSv6uge!*$vWJ2Q#Sf#qbSdAZ8Pm?+vtn#;UQ_(c%ROo>D)3#E9<(V&ZsDQs6^sV zzTD%!*2txkk!4y~dFJ%#Utxr}=V85bG0Rk2)oJOAXP8dJW%`8|c(qsmd6s)!P-0cqpCeABj7TJO91Q{fP$M+i2M#19Bs3sA2O1TbTxGBe#GW%V zGX&2+@}z<;fv?JzMmqu$$;7LoyqvggEZKzfgrf1UCnY50ZPa1B!7IYExX|x7^%&>F zkmpbx_4ScX2mi`q#Ur7>N`Op(9kRDyw{mkO0VUQvh4=3vbXpz=o$t7fExCy*rC11Jn2-8D$XnqkVa`C)B$b)GpiSeSEk?9s^4QBR5X~r zyOHk}-RaM!olf=r%_6*{hBdx=-aPfU>Kp#MI_6FYjbA-~9_dtsGZ9+@Y8Ns?8=Hjy ze|=ppAj6A~kH=rD~m&{d^>dUBHJ!O+>=t+6~7i^7MV zn^tLEY01g?)n;AkbmuwYmZ3VY)s^;~#0N(uIp#UYxv+%-WecRI??{T-y6pf7E+M!R2oNM_a7}Pe(BKl>-7Sy=cb7qeySux)4DRj@gTBr0)V+0X zy?Xz=bD`)OX4t)ZcYnRUwbs`Jew7nPLncH9fk0@I5+Vv95Ue2ZdJPdCXlW3pK7&BW zbZ$!D9TfDO$!+YcjZ7^J$sJs649N{$OpQPwm-*rZb30;g^k2^x*zT`&mUU^ zt)uSuj#@S$BAqKOQ{xLe#U&7tYrl z(%q&TXtDz{qri%?#;U@nW@<|E6_S%j!i^n}C*J0b^;+*PlF>S*-H@W`p_?`X~mW6BDpP#`I2N&vFleZr| ztVfMc_ZWT%wPbd8_Zmo0+|fwHeN+s)wP$CkNr=wvVW)!2b1yZQrX;`*+`f;DPC{Rr zZy$|9NFPiRxTY9w*iakIsyxU2L#@ZU7OUgcaYFhuOq241Q^d{>>GB-jreR2*2(SJ= z{u_v5=eLF{?8B35JDDKnfdV_*;h%c@HUh{3c3~f<)t+A^Yl)5L2KZNv4%Qor8Ab~+ z>{t3Y9~G!-Vv-PT4?&;iTqja+5hZ9%EBj-V&5=J|IY73-)T6UX3bHpG^RM2B7IKsk z@W9aTzIjgP(ers~m z;<3GvRugK{dPVNgkkHM@og2K^%@8J{5Yt6H*iCIw9{{80B)Qdx;ItBPPr3MI&-<6g z9nEIP@8jp6nxKX>+k4n*&#e*CuH}l;p3AsF7m%3zY&G{3;*;vu~n!vh?-Y&RCmH483zl{fzY%Tin`~99~j>I z(v5!c`?4vE+#9dbxem^(;4qgY^J0Z^`wK-Sq7a+aIHYvT7&?YyH!QtB5x5|%mLF{h z8YXt58(wAQL@!xKuNvr*twA?;-9H}G8rr8-fVU|3OjG58ky#8U!g{-kdO&P4!97_L zoSr?>MPlSyKwvc^okFqNw}(NiRAp(3Iw~bn8J?d({BhFyRwb-3W5%$1pq;+K>I{2< z0-UbM%|Rn~^N9@uv=@W2UJ)LN-}yQ|;c?|>-W8b>kKntDw#m##94;(0&|Du7@eSh~ zj@14xwx3P%`2H(bbat3vCYpUhe5;CdO;SUnKe`%M4Zj(jML$yY=Z|6G$rtKcBodCn z51D;|N^@^be-*;?{V;d=X^?IBjvZlo!o{?x97Y#1Hl9jkmXQUQFtSOZoyafmTw(rJ zyNi@+_vYZrNk^-56eACte2{b&LfzyhFkvDb5Cdao*vgHLYO!DUqsqgEajp+#T`XUc zJPgCb`xqac)kU}$eE0D{R<}vUN0dXsmm>S?g0KZI+9e1deTTcUuR@rv7_N{ue>1E` zM;682U_(F}O~st-%VE`wFA7VROVo!uXsmqUR04chW(VI%#YE_dfJz6}ZhniZyD$1r zabB+n0@(EHEp03sg)Cj`EIKpKWD{^pn7Nfs(PhmWI@6nlgRT`hbsKH8iBh5F&t6#BewR7m9~`WtPlgdhi(HcbD44D*4Htev{}rnIJ1b>jiA{SMv#6zh$^xBJ<%!Tnl1Zxa=lt~P%wBu6O3o=c5r zkOvZ>A?lZrh4c6~>g0c-+r5;OBzcV%TadiSG$n~#K-q0*USUUn|F;|Y6afh)gTDCy zzgncQ(st-1 zF=AvW{A8f4DF-+sr^0;x5dIVMoO?4H`Y_@Vf$ig}*6%6wDcjWdHG+Fi4nu`(bwB=& z8QsjQ7&RHc9pa>dWACg$cF4hIMm1)JhR<^a)*@dOI;|LQ(7Wx{?rW~F`0FY1m3aO| zYdX_9SK=LKQWVRPN)%4lD^w^0XYw=pfK?!?jX4K{ySintQ_Jvh2RTA*C}-uYJN)s8 zQ^w}szn}8Gr=!@6c>C9d?L@DV>}QdiGzNt{*^!^G^unLhD75Q|RiZ1Rr`s#`hy8Zz z5-P8mvv?BZ=UxBhoxTCAAvoT&Y7yd81(4~Rd{tsWi!b5B@6HXcspP2Muk4NUOrypT zqLG^(sOdT=M&+w7Uc_~<^hc>N+8C|vIvS2xIU0$vqvZ&g46cu{!NzMX!p|#JR7uooag1yLaK_2-30L1GvSXGtm9x*PtACSf--iL2f=qvTTEjmB$2C6JbGQFjnoLV)bL0ij^}H5e=F9$ec8valyg$Fcuw_!6D6kcza$UvBgG8L zlyf6b`p!A1arEU&)3e}Nv6~E0Z5IQ^>DbnjzA5ODH@JMpe;qk9ZK_17a3$zW+1l{5 zoc)W;Qi`6NP*#N7@STw9baaK|d~`viL$Cfstq@_y5blCRF_9kPEWy+qqx`DN4GFf9LTQ%yhE-UXc+R4ke+DFtK zvfScpxHNy~Ozi1xlN>4xBC-f(WW8FKF#3D{Abfi;Bz@Q>e0ont-YIVCMaM?^ivzlc zMBZVKcB1nHIk2Xl@Vg*cYu#5H2dkneF@iuazf6UNze)-V{}*cnxMGTDB)>$b5OKGT zhUiBe1o{AnzA+xPH#oLabl4eR@^I80nd>i6l0LnsWeoE*Z*LFpF7OMkEI}{vCBKDP z-`>vJrJ#CZqOA%zTdgBIIp#lJhItbEoo@u;N;iv(l%xJ-VWqt5D*6T!_1zzVz{I_ycEyrsc2U;n{u4(u7m-MdDFakea;`aWCHht!6Hr-3P3G6I-PusQA^#fij;x_W2 zV$eDz82#SWXtpnFC$O{YOS#_`x)GHyRVqaoU4a<%sb~;+=i`W3c8~haCKKvbb*y2@ zWC(enD8uJpAOiN7Z$E!?6%*1@PgJFyo`!RIMQXCv59n@w7s8=?-;42t(Avn35{+f0 z@Bzl5b@o~ub$9M{lH3SxzWHoNH!8`_8<@n*2@nJC@BRWcoV@vQdSgU3z-;) z>Jw5x>@o;M4w4l4tmHC(wB+oesJzj8CLfh!+3>+Iyr$0##dqur0jmMV)i)TJ&mwOi z8zL~hnNOgd4edNsgo^g|^KS*)G2Kg*jDC1R6fp~~@u9f$Y}IbDusf;e zzS)fNa387X{1Y<;OM~;~jqbM8>Gh{~6ciM8`5|Von3$N~Zh_{1q5Vzv($9ajk%cl- zXp>Wj6cm3?DS^8o3j)pmXjKU?*gr#Y7BZ%n5)G|TP<_UdsE>EoL`ykR@ljiKwmmk) z)#|D% zl!G1X1x4D(m+C3WA-!_lUhP8$MNCnCqV=eUv7|TW(X6a|ngDIto(iqF$9ixvf z(@yj#Dr%Bk3Nk8;2{Kh0kzSXa}6;JmUcn@pYkEEy0;Z?qir zMIU3nf5>Tl3#pww#37KCEi0GvN#GAk>ir?!qlrT4CX2sp-@9uc@EWHZi(A=NMe3k+ zb!-XgT;vDc`{st}Km_cGV9m3c^QSJwbW>+vcGvdL!4;BIKjd%r8*Es|Tn1 z?ahuH{%(u;=s|jnyRq8l+(w<&TT#-LqKa5Pe^?2wb@DX{Or7GI^YQP3gwyP?u7|Zd z*vv-+&w@vqA|lvcPvnk;L+gbFJwSoS8qu9_Y zR2m&G9VPusxYg_9R6e)H?4gZBnG!Lks*cUyC1r02tky%tAM)Pnrq80=$KjpN6n&A) z=7yRnzq?s6tJei17t1*vF&V8JL$R?t7VJ@WCr8-rpmP(j&ZnltW}7RIO{T=_pXFNo zvJWhW9x?)^9B9IuVh16mDjkqU2hZB0>Po%v(1T#@H-mDg4fbE<3(Wr7R>F$$Vo~(4@tViC{U|f^ZGA@PlePD@Iiepnl zFH(Vwf;EzsJli^7b&kmKt6X!wB{jPP^AO}sk3{n$1SMW%#sYH%49DZ6n1IGQ^80CD zRF5+ckihYf*?Wrs1YGNcb4$$JiIgE7wYC5v>1&7)nrs(o*~Rf9>^vu1JmOjWf#$%Z z3I#>Bfb{gMcvbx|C{<=a_p zBe;c);!T`7((te8e(!`D4i*N11NG*dKt9r6-W#6eG$m5(f)rgkL{#J3 zor4YIZmRK_2{kE6JMY}i=GH0Xa=7bnwlrZM(hYxLP;~O`67NYiC$*Bb1zVJTTpn0= zEU{52t@$bC1&>a>uyn7vkbc;*dw`ZvEpy0ku+t~$o`0ka8@jl>6g?;S9eP>_>sE8t zRT(0z_1`hY=*P|*w0C||q%eAa-*HOR?2;A2j%e$Z1FgDx4)tfHC0zMD>Dme4%(NFpb}rBCx=?CUmF|5o{p1PhRx(obQ{RW(HGV5jmO z8hN_ac1ns04y{)k{*Ydt?7Ow%7MGW^mEZmiZ!~C*^=?fZ^f9iKZYEn!ExBR^)k90% zQNSmQI0`Q*Tdq{>kL%aH!z0(0hnBpdua8jY-^Y6;0!Krwnmpr!6wW)oUGG#EcvsDeZJyl-IS-UOx%MjikF-~?sv!JY-K(Ou!=X?$&& zwY&V=XkrF69Uia;wDn7rLdxRA_{HCYnv*;*#x|Z-nf8h`9S^Trx+5;$`R;uZj)mD>a=zV0BwZC!0AM+TLP9PJntpQO$4=OWQIZ^md^5r;<}@Y1I4n&t?b^ z%Z+ShmIlq1^cz8*v5p!SqEYz~<8Bpt$PA}m zW#{!T{hHndQ}vZDxZ2p%FgoicD~prvieU}`4b85wU-~?nJDH7aB(!}(mmQYY6|F`C)`q*C=#yB1p6nE9 z*ZBd}rEf2UgsXiz6oC5vBC;X|bq%HU`?6~?$J3q=6oimG#Ax1!+k5mtDm73>NJ zzgS5W}%YTE}=wp3W{Q(ThXRpmt9TVJe?EaQ64K7{>TYPxk(#tf81 z2MAk)wk1E(Y4I$1(Kra}iyQEA-bTk4fDL{;c3K22>+*E%poykT!4kOv` zMEjOH1TBx?N6=h3+B(=!PoAl5PF&z|-_ROq@3Jc9Go2~Ro9Y`)b7Z8T*uhs&K%x7@ zLHtIyg#zd5`e8>dn+rU4wI&1WfyUt09OZT-8q}aixZq8g{pSrP)PFKfK~?w*yHL1^ zJTSEtR4w1p0674l)eYx5wKc;s^Kqx95hJ}pzngg=(Rg6M3+r8xDy9?55jYSB7f>&! z>Gb*Q;CwZhoMr$n#L0d=sv2otdZs(`%A5UHKtLbWSYJ4B)rSY533$-jQ*Z{v2vmIh z-`@S#XaC+l817N~Q^Y5>>R7UO=t#&}RS};QBEZ+0 z`2R|a{ZD8A-j;}wdhbNvoBGS1lmX+9(Up+=Xrq!PB;*4PQkq(r%yO0n?#!5&)BL|O zfPX#6zYpL)dHugMD^F{DQ?GDpsag``TWEEkS$ONA*|qmjp^Z0K7?+iANWPU*FE_a| zZRp(o-Avxpqabhd?+HDasMdH7aW)AXH>t~O$jve+d+05$GGB>-N84A*J4{_da5Y;r z04;kQd`{AKmrWc6LhB*cbTikN+Ec5|vvjk9GUIR+E+QF&_wC{0jTB>a3uE0Tgn_Xp zIyF^TMuxAni7)t>kL`C;Q%5k*%mB_pBD?)-fArKM8@0+gkyttyrTn)(b480lRI1W= z`YN}@-RkEsG8BAG%tr|}S?bbU!zA}HP9^zbpoGtn4F1t8SFXEkR%0!104ecng7#=W zSwAe{2pA3J)CCJX2T)%naNlz~zIqYl(A?Ww3UOP3rv{(hodjiuh*O@ejWeBCwr1snO@VBIx$U@&DL%AzeE1ZTI6%;E^FDl1Mw6Kmz=i+@pWHpwu ztI!<*d$RkZ+U<*LjSHyDxyk-?VfWr-;!rA-lGmA(oRTu0P8rr@fx|bW)2-p)Yb0$6 zrMMic@ipE04Jj2Ho4*f8CyBFkG$LaupGZ>+EGZ&Vf-mc?ETkXxDF5-s9NzC|}>u1r1mt;IOnUTMXo|R8AJ=xrp5883jW)- z(9?34=w!vVIu^1@i#d)IIhx4G(7u5I!?g~yrn~bYWC$A-m0vH}V??#N)5g};=U`m4 zvvbx;w=2d}k9Eq4iHhk&5s}TH@jm(2nkJ#6WXdHB?=slWE-E9EHG~uCK;xI6gZ;!&`XtjoH#{D`g+h~n!bz?a-B}FCUbmPbno}7v*8d&7921l+dqN#1c z(WPuo`4`=nrGMyzu(!tzdIf(66XC`)dG$Ocb9YIoS}+v2VcP0C^p}EYQt+8Ah>DgrHjyok&A!L3shQt&yyD&Conw~T1-u)jD1duAW>?o6 z0Iy|lzUcIZaBlCbGa+!F5_Y{ z)RdHB*YDnE@#N>Ld3-Wm6P+F(9qn)@3t zM-vR-EVfsP7=-;YU5ZNeRsrXyfusqEO&h(zYea&wYfAwmX5bjdU3se zrxpGzeSPlbSnXcxUpc6JtdEVO-aq`YZO9VMi5hKw|ubrpA5f`!CvZuR1Vq(3+p#urOTTO?--PU@`)RfH4=ym0!I)?_!YD4~d z${hXD0t!kIPQgu4;^17}n(@4j_ebHA!=LTC=HPR#73PkuJmwPe~kDiDqC`KzS?^|vk5bW%c z5z%Q^9$f)2=pP~&T3m&z}s35CyY4SzF8#9T1?&(l9Fjns0XHJU5P4 zM>n19qxz@SJ>x&84=9~yJno`6KK%xpTdF6S&pvWWg}4&>>qyARMGpImDbCBD0X&Wc z4L6>LXE4A=h(f>=rlyiEx<3y)>`lN5dZXN5lh82mB_J_{8SH|WkWeo-h7v6*DlP`J zAP2mkemnqyF<0>ph$4%Xmb8!%&H1Xp)hkZ2blC;!eHp;a?0#V+DLAx%!}$**`SeId zXfXkKjdK?i>~4Lvo-z35UnzViXzs1fP_Is7%QSy!h;`FxbzM#a>7jLf-pM5}|3FN% z{u3LKh4C~0kco~LUIFai|FHdc-tqtUI%QN?mfRapF1K=M2Mykj@SjbxIWRE%aW%OV zQsK2k3cw(MKCfOwCv#+9b^f{O=O3`cw%UfpouvE?G3wdh7A zc%T4wi{993i7Fq+&~WPv3T#_LX;>&!^w%#X<5?|!QxcN3W#jgyX$Gx^z{^WwvuR}g zzOm!QvFvR3`?IV`kB4!J_4P1a)|yIH7P+gahCfwt$?)g^pP-sl?~?6s=hu;E%zuy@2y~2gE(ztXp^&98;-j4iPsYc1pFy}TVkvpOffN-M% zY-rZOG$-D-A4*$Y1#=ZAT#sTC_a!JO@IF%tp6AjtR8<&fDntW=9?v%txCtDyoZM`u=MwAw_zopOui!cMBXg+t&0-P5jiK! zR~MEUoWa8{E+@WV-)c_s0v3^bKzH!WWRaRDM{ido7~DH+>%4k&gw%%h1KxHekCWT& z5<4SVZ71FyO_McJA-)OO z_jJ14xjCqI8HYC-x{Vx**ynrbv%K6V8Q!061ZE&OSU{DG%=+nKk=gkarmwFrqi$9% zlrXj*0Dd6vP|NPeet=Z%u^&oH!0PIJd5cCk<%B@yv%0!}3)NV>4+xNenr@1h1dPis zjwD+T{W$v@%Aj2p32>DFlebb14zD;JOn=TDeqL~r1rkJfR*n%S35lGG$Oj;fFAr9b zGRGxGwTofCw_0JiUlWsaI0??psfmb+!su4xjup9n=lG*!cTJ9>ek?zNpj3e6Xm$jZ zXWT4v6;&$&0j1QZ+`X_64WR4B9r`dxLQbGl@rNKa%h{`iTB@-ctw{w#4(sJX@n~Fy zYBLZPbr^YBg`jmaOZ@FEI3tWf2`6Uk;s!9;1)ZLrwy7(<;(~$r z?)LTd$fTaX2S}4F@bv2&;ctoD6JqIWFV@5}&n}y8b>eTE#S@slcs#8w2Fn`V8ib{R zqyhqxDb@ApP#+cZSoKJr-B%FJMpz7wk(gLXW7CVk>(iO-K@((yMt3Wf9F87X3+$rf zu$!w;lefiJRaucOvgr}fiF@}PR&DA!#|zFOc-prq{$wqoBb-$q%7^vOvb39%4j-Y20BcauAq{DM2=WnWIug;;8T(&fYGLQ9kRsQ|$_(rZ(!id;@%AQD z7xjl*J+X$-eQHYEon-;5R~LZF>~~1(m@cg3BY7*}6s8wj2z4W4s*SuXb#?j4S`}z_2(|N=YWGE+Jup1`y|$U8KQ5b2`KO z3{T!5i{)GJ>a^#evff>UEGWpL>HZpaz6vSV%Er<017;*LYgQJn(k}c57?^E(g>TmK z(aXiHk!;2lP_rkq8mlF7b#+_-b%C8+cieEd$Wnz+ZQl50`{RW*4oASL|<6?H19jX8p{Q7BO}t5mgQgm2y7NErEgWul|>Fy{X&R%>xV2o?`zf~ zd+*>*j{xKD_Q-7{r9a1N$qco%k@IN&nf1t-6^cqGdlv^N2p5+G6G#gZ2A>N((Ax|7 z09Ml3`8emdQzLkNUEhL(NTo1#qc_}sl>es8^Zwe7jGm4zh)O!{Yqk{5kdb{LyD$aC z`AwE76I5|)3x_p*1tp%?3l-jm>+pMVZa+O2Di;eup)yw!5DtHqpl^*TF}Tq1n%wC4 z_!69#-9@a^U+9QL_wAkA#bX78iTh#Jf+LPWt;0CYvxi1epmWdRybDV{B2~K_^GkGp zSCxFPLZg!7tX*9QiME|X9C9LY!+7(1O9f>~zM1vvMw$3Dfs z^xvlgp4s|fZy$tKRU@@ugANl=cjudp16L42@`-6z5HczyKb8SD<(F! zNUPUlKuCzePy!3FfJegJ)j|9{mr5ZH6BAR2SSD=5zy6%}0eG^e=4K>x^c?T!r%%?) zizGe+=Fd-$N+p^Em2-}-Gbi_yHiBN7rNLUcHPgUauRolx4!&7-50?Y;tQ{Q{kCS0WdQTV!9A8kXR@&d(_m7U!(9#A1vtMR9Nev(r zBUjXqPlRV@XBo1DFe8VQn5OYyu5rUKo37W0Dv@&=QFbOoSUBR^Y<^sO?*b&?kPMRDJiLo zyZgr8UZB4}0=(bnc5#tSG7(Phm{##1K_cyu*GWD;J}YR_JI804s~_4H>_8wKp_j7& zlrI%?J7t9Qi`F?UK+{4Ts(}Rwc=4ll$f!AMeXLuw2k#u+C8P&C)@(1>+2hLZFJGDR51K>>4URD-RUXez_`)tq+Q3lPrcC8)@j&hg1LcYsYUSb}H zD9#nPh#ZJET-&cRoi~-HOh02aaAO{Kkf;fdS=l^NA5%TJgs3 zp_DVqN~4l=#hq=wi$n17j=eo{0*50K3(MK&7S3B@&CoSUH-N)q0s4vW^2C`d(c@1b zQ&_sA!>fAk*NeRSXcd(g3Cy;yMg?!w0wDw}6{DizMxn0jl&oy_iI-<4O-)iyPlwB% zKDoWJYvsN7la2Dp3Y3BfZ^JK!CKU;6+>PPUjtk@%DC`JKhDQYhYHMj=U<5!z$&FEy zv;-ifV#2m(S^RcDS^Lge=mR<+U#0jh$QgA{tQ7`q%L8sYUCgyud?TJFV2ST4uY*o3 zVf28a69EQ~qG@ToC765TgHBSqRo;sb)2JPcXQ|s0LWYEvKhhb#C|0^WUZmX|jI_~O zw`=eD*jlMpV?m_VP_^360%yykw|={q!j&2XD0fBqm7n7#n|?Q~+39TqG5x{)Lqmpu z_Rxg<#l+zEC_KRHGT98IL3m3-;`Qx{%abR%u8tD5CDS~A70zbK5zTb`FHE7z@rUZ8 z)pjhtiyl9l)xhcg$pvHORU<9ZK8|sO(vAU2(zSOK|s&}eYK-$sq1pM0|ZFX z$OB3VC{7V^e=Jf=DK9U~$2;c^01+!y%|RSR;fABB!o(7Mat2C2zI-t&`_T(S$c~(& zag%2U+3Q;~00Go!ccPlgs1o6*)H>*7?n)fTar0F2CG8cfe-5eJ5d(R6HoLAtN7d?V zNHF*p!^_Iv95)ow=y;sLJ>NX!cJ+|vNJf7>#-Bg~Yz|;ttb3im6J-9pHMgdtgNo3B zAexE?zW17nt+)Wm(ps+*&#M zyhB2Xyo??jzQ-{vw`rX4h0JNX1&#Oiq9hNGI7>q=`g?;vFxDt)9ySE(dU>r;P3KRd zef(aok1(0b^yB(47D%92I5=V}$@AJ%XJI6~Xl?z`dy94#l~grUB+`!09{1^t7E4J& z7KeIGO+$QFmv3&Gy&3VM{R=-1Hmt34%ZcwL?vQye_Vh)1Ch|H{oisQ4(Db6J?BKkx z=HRWPqy8(BBwN5hfDU5LGjgAVlsv7XuVG;DgIeA-ie2!Cn(>(V>`Xi`lz(U-b^&^tIqu@7t;vao zls~ddW5winytBE3XLna+zF(Tx*K@X!uT=UCT2bS;#4`=zav@I?3-udZ-06!{1hA0l zY_a)fS>&X}GQfmz007#*Zq>9F+_qH>7_;3Ze+w0k12H(w~>^gBQ{{ScQ|zUA}t)A~ut&+iY9cJKxW(&>Vq z*tqKDDV(v_p~2v977xd3i^6@~BL@1*ML z(l9E`A)jxQ*131YLiQo0!;_=vddG{(4EK7$Jmmp>hPs5j$Mt78w5wZ=h9wQ>%(S^$ z1#JUf4=0WIKm-f}zBqS8g2+$~SC01?oOpQ7>d8qb+;huoYis$L;w4!rDR{L+*V$l_ z+dH*@xvKcXLk}RwCOkoJW{=P-D$belLqA!3_21lNJDjtRxV*FgG!Po$%sasIP4gBQ z`UywFjE_2$Se^3lKyiMKY`RiH`(l0;b8fSE-m#wBUG$HP?Ci`6TP$Mx=f)>uQr@qx z*{%uH8jpU~zagfm(njNPvN%R2j(wbqw~3(?SKxA41)vlv^JKuH`DpKD1JO4OubOn9# zPFA~^WQ2wFIAu-*valfJFlxhiET5tuP zOfUftAzv%EfT!}Gym`{hHkp6n$aUzXpujp?%M=L6h4nqt9M5~|5YK(|8)fZS`g>rR zV4|W&SG)Sboo-QaN&W49G;`%Y1drx9w*W_^g4>8GFtFEjHV+-xD~6-Ta49^oBl(JZ zKmwPI&dEsOar{H+FCUfQlA_)GH(#mW-}62yzzjz$v&RuIUQ<#I`SkU#&nNe_4p-7d zq66Nf60-%EO~ZO$qzoVnf{NAsqZ1Q{OSBN?DyUL{s0K0`wbkxK6d=5`U7pxd6!0+Yl83}N@~_I%W|ZWF2|4pvI8do#H861| zWJiCJo^sp?6KQ!g`iOu*9SsJDmzHu~?N4U`u4z?R7>N9@=?B0Wmh1fE_*CZRaX+jM ztoK;0H4)*bqRtX90^^|V&Ho^Imr>!y=^V z0dw}Lxpon)UgiHB5W5>P!f?9`fNis?t@M7j)z@F2tK4~$=6UuBFz>X-?~VZCV4Rj1 z1StF;xki9r+XpOLm&Lra#o@bUmkXGrPtAV-(`GzNvRSZ*)Tz-=6a@k|W-YWc=ynLe zU(4au(QH2VWDcX@!Hnm7Dlj!y-JK!-Q%*{7lNc?D`fGW_MaDSD_9HP&QKL8jyUA<(TR<&4f75tvyc*e8MvlfVJ#-dt{7|KtQk06nGu!~(rk)aX>N>Ax8r#JvK!U0HQ! zhNIaC3VJV_MTKQY#H%9KOQ9>z9PvCW_LSHF;gU&i_?3|0 z0ss#ZdP&bIX0z#Jj%4Cdm3^<*#H78+#QPY)+>V;z_XLi6^X=5sOYzvK842 zuggk6z;S5^@G{q%M}H&~6x(XWA?=>$6}(@6W&qJ-w>u^Y5)%{%%*zvNx*TY3)e!XW z><20jYUOskfC#_6b-@In+Wn+4ap3b4pqm<;LJgiC(!MvF(Kw@>II7*t0ynE&aSZ@a zv*p<<*eg;*pX)J6Tw6f^7zBRWK<1WJ{Effxi5 z=F8wgj{8bm)vCs9#v@X0mloXsGy#uV57-pPn1nilD1alfD_{k1KeMSk8KBf52bRh; z8YbsR#d|d{QR)0?q4wm*#rCUI?_)MTKABGz-;)6mFARGdBue$wr3%eJqdm#N61o+K@fU;O^iv zEODUJn5R)oSlYrGj5Aovb~!?Nc|7?CdZAKjbwnz*wQV zrT;H}+i-JrQ<;KR_Zn|WaHe#jbC7A+LGWqR-nn7n+dmabspaL=hx@e!DZZPfC|Qr& zwOSc)MU3-`_h!@O`Cz!*PAU2(gQtc6q}sN`NC?58TCEc&K-!~$Y4W@i*;v!N8+My1 zR1J2!GPBhXgt)~s0Tdgc8GsPG^IhBVf3`GQn&$_QVzmX~N!iR@Lc&o@(AGWwI`4yD z`_}@35%(Yg&xIFP5MWt;{U#5?ktOFdOso|w(Ybv6Noh`4c+Ls8x0!&FFxnb8^0*y0 z0c@a=!jkr4?fqbd)uqZ$%uc>;?%}~!3aW~VvZ7${p142dG76&Umh?)fRJAU zYSd!Gkik*gIm9HLmk|$}@9sH>0~<+}uuwy#K6k89G^`pqIc4xw)YObg$jwn*1Ys z@J6q^BKL>%p$+IARMQ6A@cC)uI1S@{gI9?%eIGyJRLNsCP~t8Js19IIC>3mKY^ziG zcYZs*qoPTB$Ic%8&e;i}&FrJm5LPq*dF{YGO=j~Lxs4ZkMn~<&J==PF?|l`6U!K@r znU10Ayxm==k<8d&y9@|#5YNgOtj|xl#xic|)Y4Y%U@VBz6sJz4`;dON<9bl-|Q=E^rfFQ-Tj zN($wHvjhf4+km4+0a|JhjZ?cW=G;+KGN5E&NB}A=vvu##ed0b60cSa+3RObqYAgkr zWFMkbiaTzP0iI1NyOwC|uaGIWHy*m$P$Fcszg7x&OAez|$! zVM3B$=JIeZ2zucSyISK#A{F8QPL@g*JfY!nlF8Kw&s!;8iIK;xbhYQJU3TLSB}9k@ z$M*Is?g8}{MD%>yni5*7j;XDf$W*_jV{&@>6N|5dJytK}rNFdS(bn%Ty3$*VL&G*u zxgJ^Bd1{8<>x`(S=HkVr_Zpo#MywU{<*??$f*me@I((iU=VDs*RL8)nF2z)MoXcll_a_g*H6kMF)EgtNLQ$!1Lz2 zDqH_FQhuq_izB$kh?p3ue`M^i!vW@1;}sE`6?2-L*1+9WIfE9VAHdvXlYg;VxG>#( zP+c4?Y2Yw*8#X-QlowzLjHa1f3Mcd0<%hVYHc$Hi2A(Y{ZT{<#AZ$cLW}V0&H}|AV zD{yRL;^3)XsqQC2sNHtu?fw13)`3MWZ{5L2yrlf7-n{XK)Y8Gujm2_P{27zSXAu!Z z>(={IGHo6I&h{neW-&7}YQ>z;`a@1`Y|Kb{@f&D;P>KM!A203sofRf)@W6=?*UNIRV_u`@NWgCM z;5p(2Qr@CYkFpg|nRlG5`Y-?T<@nrxa?JHoo$+3J|8)t6PHd9T>}4V1(Fn% L6Dj$k>-&EI3#xgX literal 0 HcmV?d00001 diff --git a/ui/src/components/style/EntriesList.module.sass b/ui/src/components/style/EntriesList.module.sass index 453f46081..c787799ab 100644 --- a/ui/src/components/style/EntriesList.module.sass +++ b/ui/src/components/style/EntriesList.module.sass @@ -38,15 +38,6 @@ border: 1px solid #627ef7 background-color: rgba(255, 255, 255, 0.06) -.spinnerContainer - display: flex - justify-content: center - margin-bottom: 10px - -.noMoreDataAvailable - text-align: center - font-weight: 600 - color: $secondary-font-color .fetchButtonContainer width: 100% display: flex diff --git a/ui/src/components/style/Filters.module.sass b/ui/src/components/style/Filters.module.sass index ae8af8224..b58795bad 100644 --- a/ui/src/components/style/Filters.module.sass +++ b/ui/src/components/style/Filters.module.sass @@ -4,9 +4,6 @@ display: flex flex-direction: row align-items: center - min-height: 3rem - overflow-y: hidden - overflow-x: auto padding: .5rem 0 border-bottom: 1px solid #BCC6DD margin-right: 20px @@ -29,8 +26,8 @@ input padding: 4px 12px background: $main-background-color - border-radius: 12px - font-size: 12px + border-radius: 4px + font-size: 14px border: 1px solid #BCC6DD fieldset border: none diff --git a/ui/src/components/style/TrafficPage.sass b/ui/src/components/style/TrafficPage.sass index ceeefe128..3a3c558f5 100644 --- a/ui/src/components/style/TrafficPage.sass +++ b/ui/src/components/style/TrafficPage.sass @@ -110,3 +110,8 @@ align-items: center height: 17px font-size: 16px + +.playPauseIcon + cursor: pointer + margin-right: 15px + height: 30px \ No newline at end of file diff --git a/ui/src/helpers/api.js b/ui/src/helpers/api.js index 3e6d1de65..2c92eb279 100644 --- a/ui/src/helpers/api.js +++ b/ui/src/helpers/api.js @@ -29,13 +29,8 @@ export default class Api { return response.data; } - getEntry = async (entryId) => { - const response = await this.client.get(`/entries/${entryId}`); - return response.data; - } - - fetchEntries = async (operator, timestamp) => { - const response = await this.client.get(`/entries?limit=50&operator=${operator}×tamp=${timestamp}`); + getEntry = async (id) => { + const response = await this.client.get(`/entries/${id}`); return response.data; } @@ -48,4 +43,11 @@ export default class Api { const response = await this.client.get("/status/auth"); return response.data; } + + validateQuery = async (query) => { + const form = new FormData(); + form.append('query', query) + const response = await this.client.post(`/query/validate`, form); + return response.data; + } } diff --git a/ui/src/index.sass b/ui/src/index.sass index 2cb79f24b..256f72501 100644 --- a/ui/src/index.sass +++ b/ui/src/index.sass @@ -22,6 +22,11 @@ code .uppercase text-transform: uppercase +.queryable + cursor: pointer + &:hover + text-decoration: underline + /**** * Button ***/ @@ -31,11 +36,13 @@ button &:not(.MuiFab-root) &.MuiButtonBase-root box-sizing: border-box - font-weight: 500 + font-weight: 600 line-height: 1 - border-radius: 20px + border-radius: 4px letter-spacing: 0.02857em - text-transform: uppercase + background-color: $blue-color + color: #fff + text-transform: none img:not(.custom) max-width: 13px max-height: 13px