github/kubeshark
1
0
Fork 0
mirror of https://github.com/kubeshark/kubeshark.git synced 2025-09-01 18:47:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

Migrate from SQLite to Basenine and introduce a new filtering syntax (#279)

Browse Source 
* Fix the OOMKilled error by calling `debug.FreeOSMemory` periodically

* Remove `MAX_NUMBER_OF_GOROUTINES` environment variable

* Change the line

* Increase the default value of `TCP_STREAM_CHANNEL_TIMEOUT_MS` to `10000`

* Write the client and integrate to the new real-time database

* Refactor the WebSocket implementaiton for `/ws`

* Adapt the UI to the new filtering system

* Fix the rest of the issues in the UI

* Increase the buffer of the scanner

* Implement accessing single records

* Increase the buffer of another scanner

* Populate `Request` and `Response` fields of `MizuEntry`

* Add syntax highlighting for the query

* Add database to `Dockerfile`

* Fix some issues

* Update the `realtime_dbms` Git module commit hash

* Upgrade Gin version and print the query string

* Revert "Upgrade Gin version and print the query string"

This reverts commit aa09f904ee.

* Use WebSocket's itself to query instead of the query string

* Fix some errors related to conversion to HAR

* Fix the issues caused by the latest merge

* Fix the build error

* Fix PR validation GitHub workflow

* Replace the git submodule with latest Basenine version `0.1.0`

Remove `realtime_client.go` and use the official client library `github.com/up9inc/basenine/client/go` instead.

* Move Basenine host and port constants to `shared` module

* Reliably execute and wait for Basenine to become available

* Upgrade Basenine version

* Properly close WebSocket and data channel

* Fix the issues caused by the recent merge commit

* Clean up the TypeScript code

* Update `.gitignore`

* Limit the database size

* Add `Macros` method signature to `Dissector` interface and set the macros provided by the protocol extensions

* Run `go mod tidy` on `agent`

* Upgrade `github.com/up9inc/basenine/client/go` version

* Implement a mechanism to update the query using click events in the UI and use it for protocol macros

* Update the query on click to timestamps

* Fix some issues in the WebSocket and channel handling

* Update the query on clicks to status code

* Update the query on clicks to method, path and service

* Update the query on clicks to is outgoing, source and destination ports

* Add an API endpoint to validate the query against syntax errors

* Move the query background color state into `TrafficPage`

* Fix the logic in `setQuery`

* Display a toast message in case of a syntax error in the query

* Remove a call to `fmt.Printf`

* Upgrade Basenine version to `0.1.3`

* Fix an issue related to getting `MAX_ENTRIES_DB_BYTES` environment variable

* Have the `path` key in request details, in HTTP

* Rearrange the HTTP headers for the querying

* Do the same thing for `cookies` and `queryString`

* Update the query on click to table elements

Add the selectors for `TABLE` type representations in HTTP extension.

* Update the query on click to `bodySize` and `elapsedTime` in `EntryTitle`

* Add the selectors for `TABLE` type representations in AMQP extension

* Add the selectors for `TABLE` type representations in Kafka extension

* Add the selectors for `TABLE` type representations in Redis extension

* Define a struct in `tap/api.go` for the section representation data

* Add the selectors for `BODY` type representations

* Add `request.path` to the HTTP request details

* Change the summary string's field name from `path` to `summary`

* Introduce `queryable` CSS class for queryable UI elements and underline them on hover

* Instead of `N requests` at the bottom, make it `Displaying N results (queried X/Y)` and live update the values

Upgrade Basenine version to `0.2.0`.

* Verify the sha256sum of Basenine executable inside `Dockerfile`

* Pass the start time to web UI through WebSocket and always show the `EntriesList` footer

* Pipe the `stderr` of Basenine as well

* Fix the layout issues related to `CodeEditor` in the UI

* Use the correct `shasum` command in `Dockerfile`

* Upgrade Basenine version to `0.2.1`

* Limit the height of `CodeEditor` container

* Remove `Paused` enum `ConnectionStatus` in UI

* Fix the issue caused by the recent merge

* Add the filtering guide (cheatsheet)

* Update open cheatsheet button's title

* Update cheatsheet content

* Remove the old SQLite code, adapt the `--analyze` related code to Basenine

* Change the method signature of `NewEntry`

* Change the method signature of `Represent`

* Introduce `HTTPPair` field in `MizuEntry` specific to HTTP

* Remove `Entry`, `EntryId` and `EstimatedSizeBytes` fields from `MizuEntry`

Also remove the `getEstimatedEntrySizeBytes` method.

* Remove `gorm.io/gorm` dependency

* Remove unused `sensitiveDataFiltering` folder

* Increase the left margin of open cheatsheet button

* Add `overflow: auto` to the cheatsheet `Modal`

* Fix `GetEntry` method

* Fix the macro for gRPC

* Fix an interface conversion in case of AMQP

* Fix two more interface conversion errors in AMQP

* Make the `syncEntriesImpl` method blocking

* Fix a grammar mistake in the cheatsheet

* Adapt to the changes in the recent merge commit

* Improve the cheatsheet text

* Always display the timestamp in `en-US`

* Upgrade Basenine version to `0.2.2`

* Fix the order of closing Basenine connections and channels

* Don't close the Basenine channels at all

* Upgrade Basenine version to `0.2.3`

* Set the initial filter to `rlimit(100)`

* Make Basenine persistent

* Upgrade Basenine version to `0.2.4`

* Update `debug.Dockerfile`

* Fix a failing test

* Upgrade Basenine version to `0.2.5`

* Revert "Do not show play icon when disconnected (#428)"

This reverts commit 8af2e562f8.

* Upgrade Basenine version to `0.2.6`

* Make all non-informative things informative

* Make `100` a constant

* Use `===` in JavaScript no matter what

* Remove a forgotten `console.log`

* Add a comment and update the `query` in `syncEntriesImpl`

* Don't call `panic` in `GetEntry`

* Replace `panic` calls in `startBasenineServer` with `logger.Log.Panicf`

* Remove unnecessary `\n` characters in the logs
This commit is contained in:
M. Mert Yıldıran
2021-11-09 19:54:48 +03:00
committed by GitHub
parent 31d95c6557
commit d2fe3f6620
62 changed files with 3077 additions and 2327 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
tap/api/api.go
View File

@@ -18,7 +18,8 @@ import (
type Protocol struct {
Name string `json:"name"`
LongName string `json:"longName"`
Abbreviation string `json:"abbreviation"`
Abbreviation string `json:"abbr"`
Macro string `json:"macro"`
Version string `json:"version"`
BackgroundColor string `json:"backgroundColor"`
ForegroundColor string `json:"foregroundColor"`
@@ -28,6 +29,12 @@ type Protocol struct {
Priority uint8 `json:"priority"`
}
type TCP struct {
IP string `json:"ip"`
Port string `json:"port"`
Name string `json:"name"`
}
type Extension struct {
Protocol *Protocol
Path string
@@ -74,6 +81,7 @@ type OutputChannelItem struct {
Timestamp int64
ConnectionInfo *ConnectionInfo
Pair *RequestResponsePair
Summary *BaseEntryDetails
}
type SuperTimer struct {
@@ -89,9 +97,10 @@ type Dissector interface {
Register(*Extension)
Ping()
Dissect(b *bufio.Reader, isClient bool, tcpID *TcpID, counterPair *CounterPair, superTimer *SuperTimer, superIdentifier *SuperIdentifier, emitter Emitter, options *TrafficFilteringOptions) error
Analyze(item *OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *MizuEntry
Analyze(item *OutputChannelItem, resolvedSource string, resolvedDestination string) *MizuEntry
Summarize(entry *MizuEntry) *BaseEntryDetails
Represent(entry *MizuEntry) (protocol Protocol, object []byte, bodySize int64, err error)
Represent(pIn Protocol, request map[string]interface{}, response map[string]interface{}) (pOut Protocol, object []byte, bodySize int64, err error)
Macros() map[string]string
}
type Emitting struct {
@@ -109,39 +118,36 @@ func (e *Emitting) Emit(item *OutputChannelItem) {
}
type MizuEntry struct {
ID uint `gorm:"primarykey"`
CreatedAt time.Time
UpdatedAt time.Time
ProtocolName string `json:"protocolName" gorm:"column:protocolName"`
ProtocolLongName string `json:"protocolLongName" gorm:"column:protocolLongName"`
ProtocolAbbreviation string `json:"protocolAbbreviation" gorm:"column:protocolAbbreviation"`
ProtocolVersion string `json:"protocolVersion" gorm:"column:protocolVersion"`
ProtocolBackgroundColor string `json:"protocolBackgroundColor" gorm:"column:protocolBackgroundColor"`
ProtocolForegroundColor string `json:"protocolForegroundColor" gorm:"column:protocolForegroundColor"`
ProtocolFontSize int8 `json:"protocolFontSize" gorm:"column:protocolFontSize"`
ProtocolReferenceLink string `json:"protocolReferenceLink" gorm:"column:protocolReferenceLink"`
Entry string `json:"entry,omitempty" gorm:"column:entry"`
EntryId string `json:"entryId" gorm:"column:entryId"`
Url string `json:"url" gorm:"column:url"`
Method string `json:"method" gorm:"column:method"`
Status int `json:"status" gorm:"column:status"`
RequestSenderIp string `json:"requestSenderIp" gorm:"column:requestSenderIp"`
Service string `json:"service" gorm:"column:service"`
Timestamp int64 `json:"timestamp" gorm:"column:timestamp"`
ElapsedTime int64 `json:"elapsedTime" gorm:"column:elapsedTime"`
Path string `json:"path" gorm:"column:path"`
ResolvedSource string `json:"resolvedSource,omitempty" gorm:"column:resolvedSource"`
ResolvedDestination string `json:"resolvedDestination,omitempty" gorm:"column:resolvedDestination"`
SourceIp string `json:"sourceIp,omitempty" gorm:"column:sourceIp"`
DestinationIp string `json:"destinationIp,omitempty" gorm:"column:destinationIp"`
SourcePort string `json:"sourcePort,omitempty" gorm:"column:sourcePort"`
DestinationPort string `json:"destinationPort,omitempty" gorm:"column:destinationPort"`
IsOutgoing bool `json:"isOutgoing,omitempty" gorm:"column:isOutgoing"`
ContractStatus ContractStatus `json:"contractStatus,omitempty" gorm:"column:contractStatus"`
ContractRequestReason string `json:"contractRequestReason,omitempty" gorm:"column:contractRequestReason"`
ContractResponseReason string `json:"contractResponseReason,omitempty" gorm:"column:contractResponseReason"`
ContractContent string `json:"contractContent,omitempty" gorm:"column:contractContent"`
EstimatedSizeBytes int `json:"-" gorm:"column:estimatedSizeBytes"`
Id uint `json:"id"`
Protocol Protocol `json:"proto"`
Source *TCP `json:"src"`
Destination *TCP `json:"dst"`
Outgoing bool `json:"outgoing"`
Timestamp int64 `json:"timestamp"`
StartTime time.Time `json:"startTime"`
Request map[string]interface{} `json:"request"`
Response map[string]interface{} `json:"response"`
Base *BaseEntryDetails `json:"base"`
Summary string `json:"summary"`
Url string `json:"url"`
Method string `json:"method"`
Status int `json:"status"`
RequestSenderIp string `json:"requestSenderIp"`
Service string `json:"service"`
ElapsedTime int64 `json:"elapsedTime"`
Path string `json:"path"`
ResolvedSource string `json:"resolvedSource,omitempty"`
ResolvedDestination string `json:"resolvedDestination,omitempty"`
SourceIp string `json:"sourceIp,omitempty"`
DestinationIp string `json:"destinationIp,omitempty"`
SourcePort string `json:"sourcePort,omitempty"`
DestinationPort string `json:"destinationPort,omitempty"`
IsOutgoing bool `json:"isOutgoing,omitempty"`
ContractStatus ContractStatus `json:"contractStatus,omitempty"`
ContractRequestReason string `json:"contractRequestReason,omitempty"`
ContractResponseReason string `json:"contractResponseReason,omitempty"`
ContractContent string `json:"contractContent,omitempty"`
HTTPPair string `json:"httpPair,omitempty"`
}
type MizuEntryWrapper struct {
@@ -154,7 +160,7 @@ type MizuEntryWrapper struct {
}
type BaseEntryDetails struct {
Id string `json:"id,omitempty"`
Id uint `json:"id"`
Protocol Protocol `json:"protocol,omitempty"`
Url string `json:"url,omitempty"`
RequestSenderIp string `json:"requestSenderIp,omitempty"`
@@ -194,17 +200,8 @@ type DataUnmarshaler interface {
}
func (bed *BaseEntryDetails) UnmarshalData(entry *MizuEntry) error {
bed.Protocol = Protocol{
Name: entry.ProtocolName,
LongName: entry.ProtocolLongName,
Abbreviation: entry.ProtocolAbbreviation,
Version: entry.ProtocolVersion,
BackgroundColor: entry.ProtocolBackgroundColor,
ForegroundColor: entry.ProtocolForegroundColor,
FontSize: entry.ProtocolFontSize,
ReferenceLink: entry.ProtocolReferenceLink,
}
bed.Id = entry.EntryId
bed.Protocol = entry.Protocol
bed.Id = entry.Id
bed.Url = entry.Url
bed.RequestSenderIp = entry.RequestSenderIp
bed.Service = entry.Service
@@ -228,6 +225,21 @@ const (
BODY string = "body"
)
type SectionData struct {
Type string `json:"type"`
Title string `json:"title"`
Data string `json:"data"`
Encoding string `json:"encoding,omitempty"`
MimeType string `json:"mimeType,omitempty"`
Selector string `json:"selector,omitempty"`
}
type TableData struct {
Name string `json:"name"`
Value interface{} `json:"value"`
Selector string `json:"selector"`
}
const (
TypeHttpRequest = iota
TypeHttpResponse

591
tap/extensions/amqp/helpers.go
View File

@@ -131,97 +131,109 @@ func representProperties(properties map[string]interface{}, rep []interface{}) (
userId := ""
appId := ""
if properties["ContentType"] != nil {
contentType = properties["ContentType"].(string)
if properties["contentType"] != nil {
contentType = properties["contentType"].(string)
}
if properties["ContentEncoding"] != nil {
contentEncoding = properties["ContentEncoding"].(string)
if properties["contentEncoding"] != nil {
contentEncoding = properties["contentEncoding"].(string)
}
if properties["Delivery Mode"] != nil {
deliveryMode = fmt.Sprintf("%g", properties["DeliveryMode"].(float64))
if properties["deliveryMode"] != nil {
deliveryMode = fmt.Sprintf("%g", properties["deliveryMode"].(float64))
}
if properties["Priority"] != nil {
priority = fmt.Sprintf("%g", properties["Priority"].(float64))
if properties["priority"] != nil {
priority = fmt.Sprintf("%g", properties["priority"].(float64))
}
if properties["CorrelationId"] != nil {
correlationId = properties["CorrelationId"].(string)
if properties["correlationId"] != nil {
correlationId = properties["correlationId"].(string)
}
if properties["ReplyTo"] != nil {
replyTo = properties["ReplyTo"].(string)
if properties["replyTo"] != nil {
replyTo = properties["replyTo"].(string)
}
if properties["Expiration"] != nil {
expiration = properties["Expiration"].(string)
if properties["expiration"] != nil {
expiration = properties["expiration"].(string)
}
if properties["MessageId"] != nil {
messageId = properties["MessageId"].(string)
if properties["messageId"] != nil {
messageId = properties["messageId"].(string)
}
if properties["Timestamp"] != nil {
timestamp = properties["Timestamp"].(string)
if properties["timestamp"] != nil {
timestamp = properties["timestamp"].(string)
}
if properties["Type"] != nil {
_type = properties["Type"].(string)
if properties["type"] != nil {
_type = properties["type"].(string)
}
if properties["UserId"] != nil {
userId = properties["UserId"].(string)
if properties["userId"] != nil {
userId = properties["userId"].(string)
}
if properties["AppId"] != nil {
appId = properties["AppId"].(string)
if properties["appId"] != nil {
appId = properties["appId"].(string)
}
props, _ := json.Marshal([]map[string]string{
props, _ := json.Marshal([]api.TableData{
{
"name": "Content Type",
"value": contentType,
Name: "Content Type",
Value: contentType,
Selector: `request.properties.contentType`,
},
{
"name": "Content Encoding",
"value": contentEncoding,
Name: "Content Encoding",
Value: contentEncoding,
Selector: `request.properties.contentEncoding`,
},
{
"name": "Delivery Mode",
"value": deliveryMode,
Name: "Delivery Mode",
Value: deliveryMode,
Selector: `request.properties.deliveryMode`,
},
{
"name": "Priority",
"value": priority,
Name: "Priority",
Value: priority,
Selector: `request.properties.priority`,
},
{
"name": "Correlation ID",
"value": correlationId,
Name: "Correlation ID",
Value: correlationId,
Selector: `request.properties.correlationId`,
},
{
"name": "Reply To",
"value": replyTo,
Name: "Reply To",
Value: replyTo,
Selector: `request.properties.replyTo`,
},
{
"name": "Expiration",
"value": expiration,
Name: "Expiration",
Value: expiration,
Selector: `request.properties.expiration`,
},
{
"name": "Message ID",
"value": messageId,
Name: "Message ID",
Value: messageId,
Selector: `request.properties.messageId`,
},
{
"name": "Timestamp",
"value": timestamp,
Name: "Timestamp",
Value: timestamp,
Selector: `request.properties.timestamp`,
},
{
"name": "Type",
"value": _type,
Name: "Type",
Value: _type,
Selector: `request.properties.type`,
},
{
"name": "User ID",
"value": userId,
Name: "User ID",
Value: userId,
Selector: `request.properties.userId`,
},
{
"name": "App ID",
"value": appId,
Name: "App ID",
Value: appId,
Selector: `request.properties.appId`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Properties",
"data": string(props),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Properties",
Data: string(props),
})
return rep, contentType, contentEncoding
@@ -230,56 +242,62 @@ func representProperties(properties map[string]interface{}, rep []interface{}) (
func representBasicPublish(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Exchange",
"value": event["Exchange"].(string),
Name: "Exchange",
Value: event["exchange"].(string),
Selector: `request.exchange`,
},
{
"name": "Routing Key",
"value": event["RoutingKey"].(string),
Name: "Routing Key",
Value: event["routingKey"].(string),
Selector: `request.routingKey`,
},
{
"name": "Mandatory",
"value": strconv.FormatBool(event["Mandatory"].(bool)),
Name: "Mandatory",
Value: strconv.FormatBool(event["mandatory"].(bool)),
Selector: `request.mandatory`,
},
{
"name": "Immediate",
"value": strconv.FormatBool(event["Immediate"].(bool)),
Name: "Immediate",
Value: strconv.FormatBool(event["immediate"].(bool)),
Selector: `request.immediate`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
properties := event["Properties"].(map[string]interface{})
properties := event["properties"].(map[string]interface{})
rep, contentType, _ := representProperties(properties, rep)
if properties["Headers"] != nil {
headers := make([]map[string]string, 0)
for name, value := range properties["Headers"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if properties["headers"] != nil {
headers := make([]api.TableData, 0)
for name, value := range properties["headers"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.properties.headers["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Headers",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Headers",
Data: string(headersMarshaled),
})
}
if event["Body"] != nil {
rep = append(rep, map[string]string{
"type": api.BODY,
"title": "Body",
"encoding": "base64",
"mime_type": contentType,
"data": event["Body"].(string),
if event["body"] != nil {
rep = append(rep, api.SectionData{
Type: api.BODY,
Title: "Body",
Encoding: "base64",
MimeType: contentType,
Data: event["body"].(string),
Selector: `request.body`,
})
}
@@ -293,70 +311,77 @@ func representBasicDeliver(event map[string]interface{}) []interface{} {
deliveryTag := ""
redelivered := ""
if event["ConsumerTag"] != nil {
consumerTag = event["ConsumerTag"].(string)
if event["consumerTag"] != nil {
consumerTag = event["consumerTag"].(string)
}
if event["DeliveryTag"] != nil {
deliveryTag = fmt.Sprintf("%g", event["DeliveryTag"].(float64))
if event["deliveryTag"] != nil {
deliveryTag = fmt.Sprintf("%g", event["deliveryTag"].(float64))
}
if event["Redelivered"] != nil {
redelivered = strconv.FormatBool(event["Redelivered"].(bool))
if event["redelivered"] != nil {
redelivered = strconv.FormatBool(event["redelivered"].(bool))
}
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Consumer Tag",
"value": consumerTag,
Name: "Consumer Tag",
Value: consumerTag,
Selector: `request.consumerTag`,
},
{
"name": "Delivery Tag",
"value": deliveryTag,
Name: "Delivery Tag",
Value: deliveryTag,
Selector: `request.deliveryTag`,
},
{
"name": "Redelivered",
"value": redelivered,
Name: "Redelivered",
Value: redelivered,
Selector: `request.redelivered`,
},
{
"name": "Exchange",
"value": event["Exchange"].(string),
Name: "Exchange",
Value: event["exchange"].(string),
Selector: `request.exchange`,
},
{
"name": "Routing Key",
"value": event["RoutingKey"].(string),
Name: "Routing Key",
Value: event["routingKey"].(string),
Selector: `request.routingKey`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
properties := event["Properties"].(map[string]interface{})
properties := event["properties"].(map[string]interface{})
rep, contentType, _ := representProperties(properties, rep)
if properties["Headers"] != nil {
headers := make([]map[string]string, 0)
for name, value := range properties["Headers"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if properties["headers"] != nil {
headers := make([]api.TableData, 0)
for name, value := range properties["headers"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.properties.headers["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Headers",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Headers",
Data: string(headersMarshaled),
})
}
if event["Body"] != nil {
rep = append(rep, map[string]string{
"type": api.BODY,
"title": "Body",
"encoding": "base64",
"mime_type": contentType,
"data": event["Body"].(string),
if event["body"] != nil {
rep = append(rep, api.SectionData{
Type: api.BODY,
Title: "Body",
Encoding: "base64",
MimeType: contentType,
Data: event["body"].(string),
Selector: `request.body`,
})
}
@@ -366,51 +391,58 @@ func representBasicDeliver(event map[string]interface{}) []interface{} {
func representQueueDeclare(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Queue",
"value": event["Queue"].(string),
Name: "Queue",
Value: event["queue"].(string),
Selector: `request.queue`,
},
{
"name": "Passive",
"value": strconv.FormatBool(event["Passive"].(bool)),
Name: "Passive",
Value: strconv.FormatBool(event["passive"].(bool)),
Selector: `request.queue`,
},
{
"name": "Durable",
"value": strconv.FormatBool(event["Durable"].(bool)),
Name: "Durable",
Value: strconv.FormatBool(event["durable"].(bool)),
Selector: `request.durable`,
},
{
"name": "Exclusive",
"value": strconv.FormatBool(event["Exclusive"].(bool)),
Name: "Exclusive",
Value: strconv.FormatBool(event["exclusive"].(bool)),
Selector: `request.exclusive`,
},
{
"name": "Auto Delete",
"value": strconv.FormatBool(event["AutoDelete"].(bool)),
Name: "Auto Delete",
Value: strconv.FormatBool(event["autoDelete"].(bool)),
Selector: `request.autoDelete`,
},
{
"name": "NoWait",
"value": strconv.FormatBool(event["NoWait"].(bool)),
Name: "NoWait",
Value: strconv.FormatBool(event["noWait"].(bool)),
Selector: `request.noWait`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
if event["Arguments"] != nil {
headers := make([]map[string]string, 0)
for name, value := range event["Arguments"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if event["arguments"] != nil {
headers := make([]api.TableData, 0)
for name, value := range event["arguments"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.arguments["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Arguments",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Arguments",
Data: string(headersMarshaled),
})
}
@@ -420,55 +452,63 @@ func representQueueDeclare(event map[string]interface{}) []interface{} {
func representExchangeDeclare(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Exchange",
"value": event["Exchange"].(string),
Name: "Exchange",
Value: event["exchange"].(string),
Selector: `request.exchange`,
},
{
"name": "Type",
"value": event["Type"].(string),
Name: "Type",
Value: event["type"].(string),
Selector: `request.type`,
},
{
"name": "Passive",
"value": strconv.FormatBool(event["Passive"].(bool)),
Name: "Passive",
Value: strconv.FormatBool(event["passive"].(bool)),
Selector: `request.passive`,
},
{
"name": "Durable",
"value": strconv.FormatBool(event["Durable"].(bool)),
Name: "Durable",
Value: strconv.FormatBool(event["durable"].(bool)),
Selector: `request.durable`,
},
{
"name": "Auto Delete",
"value": strconv.FormatBool(event["AutoDelete"].(bool)),
Name: "Auto Delete",
Value: strconv.FormatBool(event["autoDelete"].(bool)),
Selector: `request.autoDelete`,
},
{
"name": "Internal",
"value": strconv.FormatBool(event["Internal"].(bool)),
Name: "Internal",
Value: strconv.FormatBool(event["internal"].(bool)),
Selector: `request.internal`,
},
{
"name": "NoWait",
"value": strconv.FormatBool(event["NoWait"].(bool)),
Name: "NoWait",
Value: strconv.FormatBool(event["noWait"].(bool)),
Selector: `request.noWait`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
if event["Arguments"] != nil {
headers := make([]map[string]string, 0)
for name, value := range event["Arguments"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if event["arguments"] != nil {
headers := make([]api.TableData, 0)
for name, value := range event["arguments"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.arguments["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Arguments",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Arguments",
Data: string(headersMarshaled),
})
}
@@ -478,33 +518,37 @@ func representExchangeDeclare(event map[string]interface{}) []interface{} {
func representConnectionStart(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Version Major",
"value": fmt.Sprintf("%g", event["VersionMajor"].(float64)),
Name: "Version Major",
Value: fmt.Sprintf("%g", event["versionMajor"].(float64)),
Selector: `request.versionMajor`,
},
{
"name": "Version Minor",
"value": fmt.Sprintf("%g", event["VersionMinor"].(float64)),
Name: "Version Minor",
Value: fmt.Sprintf("%g", event["versionMinor"].(float64)),
Selector: `request.versionMinor`,
},
{
"name": "Mechanisms",
"value": event["Mechanisms"].(string),
Name: "Mechanisms",
Value: event["mechanisms"].(string),
Selector: `request.mechanisms`,
},
{
"name": "Locales",
"value": event["Locales"].(string),
Name: "Locales",
Value: event["locales"].(string),
Selector: `request.locales`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
if event["ServerProperties"] != nil {
headers := make([]map[string]string, 0)
for name, value := range event["ServerProperties"].(map[string]interface{}) {
if event["serverProperties"] != nil {
headers := make([]api.TableData, 0)
for name, value := range event["serverProperties"].(map[string]interface{}) {
var outcome string
switch value.(type) {
case string:
@@ -517,16 +561,17 @@ func representConnectionStart(event map[string]interface{}) []interface{} {
default:
panic("Unknown data type for the server property!")
}
headers = append(headers, map[string]string{
"name": name,
"value": outcome,
headers = append(headers, api.TableData{
Name: name,
Value: outcome,
Selector: fmt.Sprintf(`request.serverProperties["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Server Properties",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Server Properties",
Data: string(headersMarshaled),
})
}
@@ -536,28 +581,32 @@ func representConnectionStart(event map[string]interface{}) []interface{} {
func representConnectionClose(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Reply Code",
"value": fmt.Sprintf("%g", event["ReplyCode"].(float64)),
Name: "Reply Code",
Value: fmt.Sprintf("%g", event["replyCode"].(float64)),
Selector: `request.replyCode`,
},
{
"name": "Reply Text",
"value": event["ReplyText"].(string),
Name: "Reply Text",
Value: event["replyText"].(string),
Selector: `request.replyText`,
},
{
"name": "Class ID",
"value": fmt.Sprintf("%g", event["ClassId"].(float64)),
Name: "Class ID",
Value: fmt.Sprintf("%g", event["classId"].(float64)),
Selector: `request.classId`,
},
{
"name": "Method ID",
"value": fmt.Sprintf("%g", event["MethodId"].(float64)),
Name: "Method ID",
Value: fmt.Sprintf("%g", event["methodId"].(float64)),
Selector: `request.methodId`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
return rep
@@ -566,43 +615,48 @@ func representConnectionClose(event map[string]interface{}) []interface{} {
func representQueueBind(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Queue",
"value": event["Queue"].(string),
Name: "Queue",
Value: event["queue"].(string),
Selector: `request.queue`,
},
{
"name": "Exchange",
"value": event["Exchange"].(string),
Name: "Exchange",
Value: event["exchange"].(string),
Selector: `request.exchange`,
},
{
"name": "RoutingKey",
"value": event["RoutingKey"].(string),
Name: "RoutingKey",
Value: event["routingKey"].(string),
Selector: `request.routingKey`,
},
{
"name": "NoWait",
"value": strconv.FormatBool(event["NoWait"].(bool)),
Name: "NoWait",
Value: strconv.FormatBool(event["noWait"].(bool)),
Selector: `request.noWait`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
if event["Arguments"] != nil {
headers := make([]map[string]string, 0)
for name, value := range event["Arguments"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if event["arguments"] != nil {
headers := make([]api.TableData, 0)
for name, value := range event["arguments"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.arguments["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Arguments",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Arguments",
Data: string(headersMarshaled),
})
}
@@ -612,51 +666,58 @@ func representQueueBind(event map[string]interface{}) []interface{} {
func representBasicConsume(event map[string]interface{}) []interface{} {
rep := make([]interface{}, 0)
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Queue",
"value": event["Queue"].(string),
Name: "Queue",
Value: event["queue"].(string),
Selector: `request.queue`,
},
{
"name": "Consumer Tag",
"value": event["ConsumerTag"].(string),
Name: "Consumer Tag",
Value: event["consumerTag"].(string),
Selector: `request.consumerTag`,
},
{
"name": "No Local",
"value": strconv.FormatBool(event["NoLocal"].(bool)),
Name: "No Local",
Value: strconv.FormatBool(event["noLocal"].(bool)),
Selector: `request.noLocal`,
},
{
"name": "No Ack",
"value": strconv.FormatBool(event["NoAck"].(bool)),
Name: "No Ack",
Value: strconv.FormatBool(event["noAck"].(bool)),
Selector: `request.noAck`,
},
{
"name": "Exclusive",
"value": strconv.FormatBool(event["Exclusive"].(bool)),
Name: "Exclusive",
Value: strconv.FormatBool(event["exclusive"].(bool)),
Selector: `request.exclusive`,
},
{
"name": "NoWait",
"value": strconv.FormatBool(event["NoWait"].(bool)),
Name: "NoWait",
Value: strconv.FormatBool(event["noWait"].(bool)),
Selector: `request.noWait`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
if event["Arguments"] != nil {
headers := make([]map[string]string, 0)
for name, value := range event["Arguments"].(map[string]interface{}) {
headers = append(headers, map[string]string{
"name": name,
"value": value.(string),
if event["arguments"] != nil {
headers := make([]api.TableData, 0)
for name, value := range event["arguments"].(map[string]interface{}) {
headers = append(headers, api.TableData{
Name: name,
Value: value.(string),
Selector: fmt.Sprintf(`request.arguments["%s"]`, name),
})
}
headersMarshaled, _ := json.Marshal(headers)
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Arguments",
"data": string(headersMarshaled),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Arguments",
Data: string(headersMarshaled),
})
}

111
tap/extensions/amqp/main.go
View File

@@ -16,6 +16,7 @@ var protocol api.Protocol = api.Protocol{
Name: "amqp",
LongName: "Advanced Message Queuing Protocol 0-9-1",
Abbreviation: "AMQP",
Macro: "amqp",
Version: "0-9-1",
BackgroundColor: "#ff6600",
ForegroundColor: "#ffffff",
@@ -222,7 +223,7 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co
}
}
func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry {
func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry {
request := item.Pair.Request.Payload.(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
service := "amqp"
@@ -235,75 +236,79 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
summary := ""
switch request["method"] {
case basicMethodMap[40]:
summary = reqDetails["Exchange"].(string)
summary = reqDetails["exchange"].(string)
break
case basicMethodMap[60]:
summary = reqDetails["Exchange"].(string)
summary = reqDetails["exchange"].(string)
break
case exchangeMethodMap[10]:
summary = reqDetails["Exchange"].(string)
summary = reqDetails["exchange"].(string)
break
case queueMethodMap[10]:
summary = reqDetails["Queue"].(string)
summary = reqDetails["queue"].(string)
break
case connectionMethodMap[10]:
summary = fmt.Sprintf(
"%s.%s",
strconv.Itoa(int(reqDetails["VersionMajor"].(float64))),
strconv.Itoa(int(reqDetails["VersionMinor"].(float64))),
strconv.Itoa(int(reqDetails["versionMajor"].(float64))),
strconv.Itoa(int(reqDetails["versionMinor"].(float64))),
)
break
case connectionMethodMap[50]:
summary = reqDetails["ReplyText"].(string)
summary = reqDetails["replyText"].(string)
break
case queueMethodMap[20]:
summary = reqDetails["Queue"].(string)
summary = reqDetails["queue"].(string)
break
case basicMethodMap[20]:
summary = reqDetails["Queue"].(string)
summary = reqDetails["queue"].(string)
break
}
request["url"] = summary
entryBytes, _ := json.Marshal(item.Pair)
reqDetails["method"] = request["method"]
return &api.MizuEntry{
ProtocolName: protocol.Name,
ProtocolLongName: protocol.LongName,
ProtocolAbbreviation: protocol.Abbreviation,
ProtocolVersion: protocol.Version,
ProtocolBackgroundColor: protocol.BackgroundColor,
ProtocolForegroundColor: protocol.ForegroundColor,
ProtocolFontSize: protocol.FontSize,
ProtocolReferenceLink: protocol.ReferenceLink,
EntryId: entryId,
Entry: string(entryBytes),
Url: fmt.Sprintf("%s%s", service, summary),
Method: request["method"].(string),
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
ElapsedTime: 0,
Path: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
Protocol: protocol,
Source: &api.TCP{
Name: resolvedSource,
IP: item.ConnectionInfo.ClientIP,
Port: item.ConnectionInfo.ClientPort,
},
Destination: &api.TCP{
Name: resolvedDestination,
IP: item.ConnectionInfo.ServerIP,
Port: item.ConnectionInfo.ServerPort,
},
Outgoing: item.ConnectionInfo.IsOutgoing,
Request: reqDetails,
Url: fmt.Sprintf("%s%s", service, summary),
Method: request["method"].(string),
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
StartTime: item.Pair.Request.CaptureTime,
ElapsedTime: 0,
Summary: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
}
}
func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
return &api.BaseEntryDetails{
Id: entry.EntryId,
Id: entry.Id,
Protocol: protocol,
Url: entry.Url,
RequestSenderIp: entry.RequestSenderIp,
Service: entry.Service,
Summary: entry.Path,
Summary: entry.Summary,
StatusCode: entry.Status,
Method: entry.Method,
Timestamp: entry.Timestamp,
@@ -320,39 +325,35 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
}
}
func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) {
p = protocol
func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) {
protoOut = protocol
bodySize = 0
var root map[string]interface{}
json.Unmarshal([]byte(entry.Entry), &root)
representation := make(map[string]interface{}, 0)
request := root["request"].(map[string]interface{})["payload"].(map[string]interface{})
var repRequest []interface{}
details := request["details"].(map[string]interface{})
switch request["method"].(string) {
case basicMethodMap[40]:
repRequest = representBasicPublish(details)
repRequest = representBasicPublish(request)
break
case basicMethodMap[60]:
repRequest = representBasicDeliver(details)
repRequest = representBasicDeliver(request)
break
case queueMethodMap[10]:
repRequest = representQueueDeclare(details)
repRequest = representQueueDeclare(request)
break
case exchangeMethodMap[10]:
repRequest = representExchangeDeclare(details)
repRequest = representExchangeDeclare(request)
break
case connectionMethodMap[10]:
repRequest = representConnectionStart(details)
repRequest = representConnectionStart(request)
break
case connectionMethodMap[50]:
repRequest = representConnectionClose(details)
repRequest = representConnectionClose(request)
break
case queueMethodMap[20]:
repRequest = representQueueBind(details)
repRequest = representQueueBind(request)
break
case basicMethodMap[20]:
repRequest = representBasicConsume(details)
repRequest = representBasicConsume(request)
break
}
representation["request"] = repRequest
@@ -360,4 +361,10 @@ func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []by
return
}
func (d dissecting) Macros() map[string]string {
return map[string]string{
`amqp`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation),
}
}
var Dissector dissecting

98
tap/extensions/amqp/spec091.go
View File

@@ -71,11 +71,11 @@ func isSoftExceptionCode(code int) bool {
}
type ConnectionStart struct {
VersionMajor byte
VersionMinor byte
ServerProperties Table
Mechanisms string
Locales string
VersionMajor byte `json:"versionMajor"`
VersionMinor byte `json:"versionMinor"`
ServerProperties Table `json:"serverProperties"`
Mechanisms string `json:"mechanisms"`
Locales string `json:"locales"`
}
func (msg *ConnectionStart) id() (uint16, uint16) {
@@ -429,10 +429,10 @@ func (msg *connectionOpenOk) read(r io.Reader) (err error) {
}
type ConnectionClose struct {
ReplyCode uint16
ReplyText string
ClassId uint16
MethodId uint16
ReplyCode uint16 `json:"relyCode"`
ReplyText string `json:"replyText"`
ClassId uint16 `json:"classId"`
MethodId uint16 `json:"methodId"`
}
func (msg *ConnectionClose) id() (uint16, uint16) {
@@ -767,14 +767,14 @@ func (msg *channelCloseOk) read(r io.Reader) (err error) {
type ExchangeDeclare struct {
reserved1 uint16
Exchange string
Type string
Passive bool
Durable bool
AutoDelete bool
Internal bool
NoWait bool
Arguments Table
Exchange string `json:"exchange"`
Type string `json:"type"`
Passive bool `json:"passive"`
Durable bool `json:"durable"`
AutoDelete bool `json:"autoDelete"`
Internal bool `json:"internal"`
NoWait bool `json:"noWait"`
Arguments Table `json:"arguments"`
}
func (msg *ExchangeDeclare) id() (uint16, uint16) {
@@ -1163,13 +1163,13 @@ func (msg *exchangeUnbindOk) read(r io.Reader) (err error) {
type QueueDeclare struct {
reserved1 uint16
Queue string
Passive bool
Durable bool
Exclusive bool
AutoDelete bool
NoWait bool
Arguments Table
Queue string `json:"queue"`
Passive bool `json:"passive"`
Durable bool `json:"durable"`
Exclusive bool `json:"exclusive"`
AutoDelete bool `json:"autoDelete"`
NoWait bool `json:"noWait"`
Arguments Table `json:"arguments"`
}
func (msg *QueueDeclare) id() (uint16, uint16) {
@@ -1297,11 +1297,11 @@ func (msg *QueueDeclareOk) read(r io.Reader) (err error) {
type QueueBind struct {
reserved1 uint16
Queue string
Exchange string
RoutingKey string
NoWait bool
Arguments Table
Queue string `json:"queue"`
Exchange string `json:"exchange"`
RoutingKey string `json:"routingKey"`
NoWait bool `json:"noWait"`
Arguments Table `json:"arguments"`
}
func (msg *QueueBind) id() (uint16, uint16) {
@@ -1737,13 +1737,13 @@ func (msg *basicQosOk) read(r io.Reader) (err error) {
type BasicConsume struct {
reserved1 uint16
Queue string
ConsumerTag string
NoLocal bool
NoAck bool
Exclusive bool
NoWait bool
Arguments Table
Queue string `json:"queue"`
ConsumerTag string `json:"consumerTag"`
NoLocal bool `json:"noLocal"`
NoAck bool `json:"noAck"`
Exclusive bool `json:"exclusive"`
NoWait bool `json:"noWait"`
Arguments Table `json:"arguments"`
}
func (msg *BasicConsume) id() (uint16, uint16) {
@@ -1932,12 +1932,12 @@ func (msg *basicCancelOk) read(r io.Reader) (err error) {
type BasicPublish struct {
reserved1 uint16
Exchange string
RoutingKey string
Mandatory bool
Immediate bool
Properties Properties
Body []byte
Exchange string `json:"exchange"`
RoutingKey string `json:"routingKey"`
Mandatory bool `json:"mandatory"`
Immediate bool `json:"immediate"`
Properties Properties `json:"properties"`
Body []byte `json:"body"`
}
func (msg *BasicPublish) id() (uint16, uint16) {
@@ -2072,13 +2072,13 @@ func (msg *basicReturn) read(r io.Reader) (err error) {
}
type BasicDeliver struct {
ConsumerTag string
DeliveryTag uint64
Redelivered bool
Exchange string
RoutingKey string
Properties Properties
Body []byte
ConsumerTag string `json:"consumerTag"`
DeliveryTag uint64 `json:"deliveryTag"`
Redelivered bool `json:"redelivered"`
Exchange string `json:"exchange"`
RoutingKey string `json:"routingKey"`
Properties Properties `json:"properties"`
Body []byte `json:"body"`
}
func (msg *BasicDeliver) id() (uint16, uint16) {

26
tap/extensions/amqp/types.go
View File

@@ -93,19 +93,19 @@ func (e Error) Error() string {
// Used by header frames to capture routing and header information
type Properties struct {
ContentType string // MIME content type
ContentEncoding string // MIME content encoding
Headers Table // Application or header exchange table
DeliveryMode uint8 // queue implementation use - Transient (1) or Persistent (2)
Priority uint8 // queue implementation use - 0 to 9
CorrelationId string // application use - correlation identifier
ReplyTo string // application use - address to to reply to (ex: RPC)
Expiration string // implementation use - message expiration spec
MessageId string // application use - message identifier
Timestamp time.Time // application use - message timestamp
Type string // application use - message type name
UserId string // application use - creating user id
AppId string // application use - creating application
ContentType string `json:"contentType"` // MIME content type
ContentEncoding string `json:"contentEncoding"` // MIME content encoding
Headers Table `json:"headers"` // Application or header exchange table
DeliveryMode uint8 `json:"deliveryMode"` // queue implementation use - Transient (1) or Persistent (2)
Priority uint8 `json:"priority"` // queue implementation use - 0 to 9
CorrelationId string `json:"correlationId"` // application use - correlation identifier
ReplyTo string `json:"replyTo"` // application use - address to to reply to (ex: RPC)
Expiration string `json:"expiration"` // implementation use - message expiration spec
MessageId string `json:"messageId"` // application use - message identifier
Timestamp time.Time `json:"timestamp"` // application use - message timestamp
Type string `json:"type"` // application use - message type name
UserId string `json:"userId"` // application use - creating user id
AppId string `json:"appId"` // application use - creating application
reserved1 string // was cluster-id - process for buffer consumption
}

35
tap/extensions/http/helpers.go Normal file
View File

@@ -0,0 +1,35 @@
package main
import (
"encoding/json"
"fmt"
"github.com/up9inc/mizu/tap/api"
)
func mapSliceRebuildAsMap(mapSlice []interface{}) (newMap map[string]interface{}) {
newMap = make(map[string]interface{})
for _, header := range mapSlice {
h := header.(map[string]interface{})
newMap[h["name"].(string)] = h["value"]
}
return
}
func representMapSliceAsTable(mapSlice []interface{}, selectorPrefix string) (representation string) {
var table []api.TableData
for _, header := range mapSlice {
h := header.(map[string]interface{})
selector := fmt.Sprintf("%s[\"%s\"]", selectorPrefix, h["name"].(string))
table = append(table, api.TableData{
Name: h["name"].(string),
Value: h["value"],
Selector: selector,
})
}
obj, _ := json.Marshal(table)
representation = string(obj)
return
}

263
tap/extensions/http/main.go
View File

@@ -17,6 +17,7 @@ var protocol api.Protocol = api.Protocol{
Name: "http",
LongName: "Hypertext Transfer Protocol -- HTTP/1.1",
Abbreviation: "HTTP",
Macro: "http",
Version: "1.1",
BackgroundColor: "#205cf5",
ForegroundColor: "#ffffff",
@@ -30,6 +31,7 @@ var http2Protocol api.Protocol = api.Protocol{
Name: "http",
LongName: "Hypertext Transfer Protocol Version 2 (HTTP/2) (gRPC)",
Abbreviation: "HTTP/2",
Macro: "grpc",
Version: "2.0",
BackgroundColor: "#244c5a",
ForegroundColor: "#ffffff",
@@ -117,7 +119,7 @@ func SetHostname(address, newHostname string) string {
return replacedUrl.String()
}
func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry {
func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry {
var host, scheme, authority, path, service string
request := item.Pair.Request.Payload.(map[string]interface{})
@@ -145,10 +147,32 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
service = fmt.Sprintf("%s://%s", scheme, authority)
} else {
service = fmt.Sprintf("http://%s", host)
path = reqDetails["url"].(string)
u, err := url.Parse(reqDetails["url"].(string))
if err != nil {
path = reqDetails["url"].(string)
} else {
path = u.Path
}
}
request["url"] = path
request["url"] = reqDetails["url"].(string)
reqDetails["path"] = path
reqDetails["summary"] = path
// Rearrange the maps for the querying
reqDetails["_headers"] = reqDetails["headers"]
reqDetails["headers"] = mapSliceRebuildAsMap(reqDetails["_headers"].([]interface{}))
resDetails["_headers"] = resDetails["headers"]
resDetails["headers"] = mapSliceRebuildAsMap(resDetails["_headers"].([]interface{}))
reqDetails["_cookies"] = reqDetails["cookies"]
reqDetails["cookies"] = mapSliceRebuildAsMap(reqDetails["_cookies"].([]interface{}))
resDetails["_cookies"] = resDetails["cookies"]
resDetails["cookies"] = mapSliceRebuildAsMap(resDetails["_cookies"].([]interface{}))
reqDetails["_queryString"] = reqDetails["queryString"]
reqDetails["queryString"] = mapSliceRebuildAsMap(reqDetails["_queryString"].([]interface{}))
if resolvedDestination != "" {
service = SetHostname(service, resolvedDestination)
} else if resolvedSource != "" {
@@ -156,51 +180,59 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
}
elapsedTime := item.Pair.Response.CaptureTime.Sub(item.Pair.Request.CaptureTime).Round(time.Millisecond).Milliseconds()
entryBytes, _ := json.Marshal(item.Pair)
httpPair, _ := json.Marshal(item.Pair)
_protocol := protocol
_protocol.Version = item.Protocol.Version
return &api.MizuEntry{
ProtocolName: protocol.Name,
ProtocolLongName: protocol.LongName,
ProtocolAbbreviation: protocol.Abbreviation,
ProtocolVersion: item.Protocol.Version,
ProtocolBackgroundColor: protocol.BackgroundColor,
ProtocolForegroundColor: protocol.ForegroundColor,
ProtocolFontSize: protocol.FontSize,
ProtocolReferenceLink: protocol.ReferenceLink,
EntryId: entryId,
Entry: string(entryBytes),
Url: fmt.Sprintf("%s%s", service, path),
Method: reqDetails["method"].(string),
Status: int(resDetails["status"].(float64)),
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
ElapsedTime: elapsedTime,
Path: path,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
Protocol: _protocol,
Source: &api.TCP{
Name: resolvedSource,
IP: item.ConnectionInfo.ClientIP,
Port: item.ConnectionInfo.ClientPort,
},
Destination: &api.TCP{
Name: resolvedDestination,
IP: item.ConnectionInfo.ServerIP,
Port: item.ConnectionInfo.ServerPort,
},
Outgoing: item.ConnectionInfo.IsOutgoing,
Request: reqDetails,
Response: resDetails,
Url: fmt.Sprintf("%s%s", service, path),
Method: reqDetails["method"].(string),
Status: int(resDetails["status"].(float64)),
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
StartTime: item.Pair.Request.CaptureTime,
ElapsedTime: elapsedTime,
Summary: path,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
HTTPPair: string(httpPair),
}
}
func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
var p api.Protocol
if entry.ProtocolVersion == "2.0" {
if entry.Protocol.Version == "2.0" {
p = http2Protocol
} else {
p = protocol
}
return &api.BaseEntryDetails{
Id: entry.EntryId,
Id: entry.Id,
Protocol: p,
Url: entry.Url,
RequestSenderIp: entry.RequestSenderIp,
Service: entry.Service,
Path: entry.Path,
Summary: entry.Path,
Summary: entry.Summary,
StatusCode: entry.Status,
Method: entry.Method,
Timestamp: entry.Timestamp,
@@ -218,45 +250,50 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
}
func representRequest(request map[string]interface{}) (repRequest []interface{}) {
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Method",
"value": request["method"].(string),
Name: "Method",
Value: request["method"].(string),
Selector: `request.method`,
},
{
"name": "URL",
"value": request["url"].(string),
Name: "URL",
Value: request["url"].(string),
Selector: `request.url`,
},
{
"name": "Body Size",
"value": fmt.Sprintf("%g bytes", request["bodySize"].(float64)),
Name: "Path",
Value: request["path"].(string),
Selector: `request.path`,
},
{
Name: "Body Size (bytes)",
Value: int64(request["bodySize"].(float64)),
Selector: `request.bodySize`,
},
})
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
headers, _ := json.Marshal(request["headers"].([]interface{}))
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "Headers",
"data": string(headers),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "Headers",
Data: representMapSliceAsTable(request["_headers"].([]interface{}), `request.headers`),
})
cookies, _ := json.Marshal(request["cookies"].([]interface{}))
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "Cookies",
"data": string(cookies),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "Cookies",
Data: representMapSliceAsTable(request["_cookies"].([]interface{}), `request.cookies`),
})
queryString, _ := json.Marshal(request["queryString"].([]interface{}))
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "Query String",
"data": string(queryString),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "Query String",
Data: representMapSliceAsTable(request["_queryString"].([]interface{}), `request.queryString`),
})
postData, _ := request["postData"].(map[string]interface{})
@@ -266,12 +303,12 @@ func representRequest(request map[string]interface{}) (repRequest []interface{})
}
text, _ := postData["text"]
if text != nil {
repRequest = append(repRequest, map[string]string{
"type": api.BODY,
"title": "POST Data (text/plain)",
"encoding": "",
"mime_type": mimeType.(string),
"data": text.(string),
repRequest = append(repRequest, api.SectionData{
Type: api.BODY,
Title: "POST Data (text/plain)",
MimeType: mimeType.(string),
Data: text.(string),
Selector: `request.postData.text`,
})
}
@@ -285,16 +322,16 @@ func representRequest(request map[string]interface{}) (repRequest []interface{})
"value": string(params),
},
})
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "POST Data (multipart/form-data)",
"data": string(multipart),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "POST Data (multipart/form-data)",
Data: string(multipart),
})
} else {
repRequest = append(repRequest, map[string]string{
"type": api.TABLE,
"title": "POST Data (application/x-www-form-urlencoded)",
"data": string(params),
repRequest = append(repRequest, api.SectionData{
Type: api.TABLE,
Title: "POST Data (application/x-www-form-urlencoded)",
Data: representMapSliceAsTable(postData["params"].([]interface{}), `request.postData.params`),
})
}
}
@@ -308,38 +345,39 @@ func representResponse(response map[string]interface{}) (repResponse []interface
bodySize = int64(response["bodySize"].(float64))
details, _ := json.Marshal([]map[string]string{
details, _ := json.Marshal([]api.TableData{
{
"name": "Status",
"value": fmt.Sprintf("%g", response["status"].(float64)),
Name: "Status",
Value: int64(response["status"].(float64)),
Selector: `response.status`,
},
{
"name": "Status Text",
"value": response["statusText"].(string),
Name: "Status Text",
Value: response["statusText"].(string),
Selector: `response.statusText`,
},
{
"name": "Body Size",
"value": fmt.Sprintf("%d bytes", bodySize),
Name: "Body Size (bytes)",
Value: bodySize,
Selector: `response.bodySize`,
},
})
repResponse = append(repResponse, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
repResponse = append(repResponse, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
headers, _ := json.Marshal(response["headers"].([]interface{}))
repResponse = append(repResponse, map[string]string{
"type": api.TABLE,
"title": "Headers",
"data": string(headers),
repResponse = append(repResponse, api.SectionData{
Type: api.TABLE,
Title: "Headers",
Data: representMapSliceAsTable(response["_headers"].([]interface{}), `response.headers`),
})
cookies, _ := json.Marshal(response["cookies"].([]interface{}))
repResponse = append(repResponse, map[string]string{
"type": api.TABLE,
"title": "Cookies",
"data": string(cookies),
repResponse = append(repResponse, api.SectionData{
Type: api.TABLE,
Title: "Cookies",
Data: representMapSliceAsTable(response["_cookies"].([]interface{}), `response.cookies`),
})
content, _ := response["content"].(map[string]interface{})
@@ -350,37 +388,40 @@ func representResponse(response map[string]interface{}) (repResponse []interface
encoding, _ := content["encoding"]
text, _ := content["text"]
if text != nil {
repResponse = append(repResponse, map[string]string{
"type": api.BODY,
"title": "Body",
"encoding": encoding.(string),
"mime_type": mimeType.(string),
"data": text.(string),
repResponse = append(repResponse, api.SectionData{
Type: api.BODY,
Title: "Body",
Encoding: encoding.(string),
MimeType: mimeType.(string),
Data: text.(string),
Selector: `response.content.text`,
})
}
return
}
func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) {
if entry.ProtocolVersion == "2.0" {
p = http2Protocol
func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) {
if protoIn.Version == "2.0" {
protoOut = http2Protocol
} else {
p = protocol
protoOut = protocol
}
var root map[string]interface{}
json.Unmarshal([]byte(entry.Entry), &root)
representation := make(map[string]interface{}, 0)
request := root["request"].(map[string]interface{})["payload"].(map[string]interface{})
response := root["response"].(map[string]interface{})["payload"].(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
resDetails := response["details"].(map[string]interface{})
repRequest := representRequest(reqDetails)
repResponse, bodySize := representResponse(resDetails)
repRequest := representRequest(request)
repResponse, bodySize := representResponse(response)
representation["request"] = repRequest
representation["response"] = repResponse
object, err = json.Marshal(representation)
return
}
func (d dissecting) Macros() map[string]string {
return map[string]string{
`http`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation),
`grpc`: fmt.Sprintf(`proto.abbr == "%s" and proto.version == "%s"`, protocol.Abbreviation, http2Protocol.Version),
`http2`: fmt.Sprintf(`proto.abbr == "%s" and proto.version == "%s"`, protocol.Abbreviation, http2Protocol.Version),
}
}
var Dissector dissecting

617
tap/extensions/kafka/helpers.go
View File

@@ -27,48 +27,54 @@ type KafkaWrapper struct {
}
func representRequestHeader(data map[string]interface{}, rep []interface{}) []interface{} {
requestHeader, _ := json.Marshal([]map[string]string{
requestHeader, _ := json.Marshal([]api.TableData{
{
"name": "ApiKey",
"value": apiNames[int(data["ApiKey"].(float64))],
Name: "ApiKey",
Value: apiNames[int(data["apiKey"].(float64))],
Selector: `request.apiKey`,
},
{
"name": "ApiVersion",
"value": fmt.Sprintf("%d", int(data["ApiVersion"].(float64))),
Name: "ApiVersion",
Value: fmt.Sprintf("%d", int(data["apiVersion"].(float64))),
Selector: `request.apiVersion`,
},
{
"name": "Client ID",
"value": data["ClientID"].(string),
Name: "Client ID",
Value: data["clientID"].(string),
Selector: `request.clientID`,
},
{
"name": "Correlation ID",
"value": fmt.Sprintf("%d", int(data["CorrelationID"].(float64))),
Name: "Correlation ID",
Value: fmt.Sprintf("%d", int(data["correlationID"].(float64))),
Selector: `request.correlationID`,
},
{
"name": "Size",
"value": fmt.Sprintf("%d", int(data["Size"].(float64))),
Name: "Size",
Value: fmt.Sprintf("%d", int(data["size"].(float64))),
Selector: `request.size`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Request Header",
"data": string(requestHeader),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Request Header",
Data: string(requestHeader),
})
return rep
}
func representResponseHeader(data map[string]interface{}, rep []interface{}) []interface{} {
requestHeader, _ := json.Marshal([]map[string]string{
requestHeader, _ := json.Marshal([]api.TableData{
{
"name": "Correlation ID",
"value": fmt.Sprintf("%d", int(data["CorrelationID"].(float64))),
Name: "Correlation ID",
Value: fmt.Sprintf("%d", int(data["correlationID"].(float64))),
Selector: `response.correlationID`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Response Header",
"data": string(requestHeader),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Response Header",
Data: string(requestHeader),
})
return rep
@@ -79,46 +85,50 @@ func representMetadataRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topics := ""
allowAutoTopicCreation := ""
includeClusterAuthorizedOperations := ""
includeTopicAuthorizedOperations := ""
if payload["Topics"] != nil {
x, _ := json.Marshal(payload["Topics"].([]interface{}))
if payload["topics"] != nil {
x, _ := json.Marshal(payload["topics"].([]interface{}))
topics = string(x)
}
if payload["AllowAutoTopicCreation"] != nil {
allowAutoTopicCreation = strconv.FormatBool(payload["AllowAutoTopicCreation"].(bool))
if payload["allowAutoTopicCreation"] != nil {
allowAutoTopicCreation = strconv.FormatBool(payload["allowAutoTopicCreation"].(bool))
}
if payload["IncludeClusterAuthorizedOperations"] != nil {
includeClusterAuthorizedOperations = strconv.FormatBool(payload["IncludeClusterAuthorizedOperations"].(bool))
if payload["includeClusterAuthorizedOperations"] != nil {
includeClusterAuthorizedOperations = strconv.FormatBool(payload["includeClusterAuthorizedOperations"].(bool))
}
if payload["IncludeTopicAuthorizedOperations"] != nil {
includeTopicAuthorizedOperations = strconv.FormatBool(payload["IncludeTopicAuthorizedOperations"].(bool))
if payload["includeTopicAuthorizedOperations"] != nil {
includeTopicAuthorizedOperations = strconv.FormatBool(payload["includeTopicAuthorizedOperations"].(bool))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Topics",
"value": topics,
Name: "Topics",
Value: topics,
Selector: `request.payload.topics`,
},
{
"name": "Allow Auto Topic Creation",
"value": allowAutoTopicCreation,
Name: "Allow Auto Topic Creation",
Value: allowAutoTopicCreation,
Selector: `request.payload.allowAutoTopicCreation`,
},
{
"name": "Include Cluster Authorized Operations",
"value": includeClusterAuthorizedOperations,
Name: "Include Cluster Authorized Operations",
Value: includeClusterAuthorizedOperations,
Selector: `request.payload.includeClusterAuthorizedOperations`,
},
{
"name": "Include Topic Authorized Operations",
"value": includeTopicAuthorizedOperations,
Name: "Include Topic Authorized Operations",
Value: includeTopicAuthorizedOperations,
Selector: `request.payload.includeTopicAuthorizedOperations`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -129,63 +139,69 @@ func representMetadataResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topics := ""
if payload["Topics"] != nil {
_topics, _ := json.Marshal(payload["Topics"].([]interface{}))
if payload["topics"] != nil {
_topics, _ := json.Marshal(payload["topics"].([]interface{}))
topics = string(_topics)
}
brokers := ""
if payload["Brokers"] != nil {
_brokers, _ := json.Marshal(payload["Brokers"].([]interface{}))
if payload["brokers"] != nil {
_brokers, _ := json.Marshal(payload["brokers"].([]interface{}))
brokers = string(_brokers)
}
controllerID := ""
clusterID := ""
throttleTimeMs := ""
clusterAuthorizedOperations := ""
if payload["ControllerID"] != nil {
controllerID = fmt.Sprintf("%d", int(payload["ControllerID"].(float64)))
if payload["controllerID"] != nil {
controllerID = fmt.Sprintf("%d", int(payload["controllerID"].(float64)))
}
if payload["ClusterID"] != nil {
clusterID = payload["ClusterID"].(string)
if payload["clusterID"] != nil {
clusterID = payload["clusterID"].(string)
}
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
if payload["ClusterAuthorizedOperations"] != nil {
clusterAuthorizedOperations = fmt.Sprintf("%d", int(payload["ClusterAuthorizedOperations"].(float64)))
if payload["clusterAuthorizedOperations"] != nil {
clusterAuthorizedOperations = fmt.Sprintf("%d", int(payload["clusterAuthorizedOperations"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
{
"name": "Brokers",
"value": brokers,
Name: "Brokers",
Value: brokers,
Selector: `response.payload.brokers`,
},
{
"name": "Cluster ID",
"value": clusterID,
Name: "Cluster ID",
Value: clusterID,
Selector: `response.payload.clusterID`,
},
{
"name": "Controller ID",
"value": controllerID,
Name: "Controller ID",
Value: controllerID,
Selector: `response.payload.controllerID`,
},
{
"name": "Topics",
"value": topics,
Name: "Topics",
Value: topics,
Selector: `response.payload.topics`,
},
{
"name": "Cluster Authorized Operations",
"value": clusterAuthorizedOperations,
Name: "Cluster Authorized Operations",
Value: clusterAuthorizedOperations,
Selector: `response.payload.clusterAuthorizedOperations`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -196,29 +212,31 @@ func representApiVersionsRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
clientSoftwareName := ""
clientSoftwareVersion := ""
if payload["ClientSoftwareName"] != nil {
clientSoftwareName = payload["ClientSoftwareName"].(string)
if payload["clientSoftwareName"] != nil {
clientSoftwareName = payload["clientSoftwareName"].(string)
}
if payload["ClientSoftwareVersion"] != nil {
clientSoftwareVersion = payload["ClientSoftwareVersion"].(string)
if payload["clientSoftwareVersion"] != nil {
clientSoftwareVersion = payload["clientSoftwareVersion"].(string)
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Client Software Name",
"value": clientSoftwareName,
Name: "Client Software Name",
Value: clientSoftwareName,
Selector: `request.payload.clientSoftwareName`,
},
{
"name": "Client Software Version",
"value": clientSoftwareVersion,
Name: "Client Software Version",
Value: clientSoftwareVersion,
Selector: `request.payload.clientSoftwareVersion`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -229,34 +247,37 @@ func representApiVersionsResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
apiKeys := ""
if payload["TopicNames"] != nil {
x, _ := json.Marshal(payload["ApiKeys"].([]interface{}))
if payload["apiKeys"] != nil {
x, _ := json.Marshal(payload["apiKeys"].([]interface{}))
apiKeys = string(x)
}
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Error Code",
"value": fmt.Sprintf("%d", int(payload["ErrorCode"].(float64))),
Name: "Error Code",
Value: fmt.Sprintf("%d", int(payload["errorCode"].(float64))),
Selector: `response.payload.errorCode`,
},
{
"name": "ApiKeys",
"value": apiKeys,
Name: "ApiKeys",
Value: apiKeys,
Selector: `response.payload.apiKeys`,
},
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -267,39 +288,43 @@ func representProduceRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topicData := ""
_topicData := payload["TopicData"]
_topicData := payload["topicData"]
if _topicData != nil {
x, _ := json.Marshal(_topicData.([]interface{}))
topicData = string(x)
}
transactionalID := ""
if payload["TransactionalID"] != nil {
transactionalID = payload["TransactionalID"].(string)
if payload["transactionalID"] != nil {
transactionalID = payload["transactionalID"].(string)
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Transactional ID",
"value": transactionalID,
Name: "Transactional ID",
Value: transactionalID,
Selector: `request.payload.transactionalID`,
},
{
"name": "Required Acknowledgements",
"value": fmt.Sprintf("%d", int(payload["RequiredAcks"].(float64))),
Name: "Required Acknowledgements",
Value: fmt.Sprintf("%d", int(payload["requiredAcks"].(float64))),
Selector: `request.payload.requiredAcks`,
},
{
"name": "Timeout",
"value": fmt.Sprintf("%d", int(payload["Timeout"].(float64))),
Name: "Timeout",
Value: fmt.Sprintf("%d", int(payload["timeout"].(float64))),
Selector: `request.payload.timeout`,
},
{
"name": "Topic Data",
"value": topicData,
Name: "Topic Data",
Value: topicData,
Selector: `request.payload.topicData`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -310,30 +335,32 @@ func representProduceResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
responses := ""
if payload["Responses"] != nil {
_responses, _ := json.Marshal(payload["Responses"].([]interface{}))
if payload["responses"] != nil {
_responses, _ := json.Marshal(payload["responses"].([]interface{}))
responses = string(_responses)
}
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Responses",
"value": string(responses),
Name: "Responses",
Value: string(responses),
Selector: `response.payload.responses`,
},
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -344,87 +371,97 @@ func representFetchRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topics := ""
if payload["Topics"] != nil {
_topics, _ := json.Marshal(payload["Topics"].([]interface{}))
if payload["topics"] != nil {
_topics, _ := json.Marshal(payload["topics"].([]interface{}))
topics = string(_topics)
}
replicaId := ""
if payload["ReplicaId"] != nil {
replicaId = fmt.Sprintf("%d", int(payload["ReplicaId"].(float64)))
if payload["replicaId"] != nil {
replicaId = fmt.Sprintf("%d", int(payload["replicaId"].(float64)))
}
maxBytes := ""
if payload["MaxBytes"] != nil {
maxBytes = fmt.Sprintf("%d", int(payload["MaxBytes"].(float64)))
if payload["maxBytes"] != nil {
maxBytes = fmt.Sprintf("%d", int(payload["maxBytes"].(float64)))
}
isolationLevel := ""
if payload["IsolationLevel"] != nil {
isolationLevel = fmt.Sprintf("%d", int(payload["IsolationLevel"].(float64)))
if payload["isolationLevel"] != nil {
isolationLevel = fmt.Sprintf("%d", int(payload["isolationLevel"].(float64)))
}
sessionId := ""
if payload["SessionId"] != nil {
sessionId = fmt.Sprintf("%d", int(payload["SessionId"].(float64)))
if payload["sessionId"] != nil {
sessionId = fmt.Sprintf("%d", int(payload["sessionId"].(float64)))
}
sessionEpoch := ""
if payload["SessionEpoch"] != nil {
sessionEpoch = fmt.Sprintf("%d", int(payload["SessionEpoch"].(float64)))
if payload["sessionEpoch"] != nil {
sessionEpoch = fmt.Sprintf("%d", int(payload["sessionEpoch"].(float64)))
}
forgottenTopicsData := ""
if payload["ForgottenTopicsData"] != nil {
x, _ := json.Marshal(payload["ForgottenTopicsData"].(map[string]interface{}))
if payload["forgottenTopicsData"] != nil {
x, _ := json.Marshal(payload["forgottenTopicsData"].(map[string]interface{}))
forgottenTopicsData = string(x)
}
rackId := ""
if payload["RackId"] != nil {
rackId = payload["RackId"].(string)
if payload["rackId"] != nil {
rackId = payload["rackId"].(string)
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Replica ID",
"value": replicaId,
Name: "Replica ID",
Value: replicaId,
Selector: `request.payload.replicaId`,
},
{
"name": "Maximum Wait (ms)",
"value": fmt.Sprintf("%d", int(payload["MaxWaitMs"].(float64))),
Name: "Maximum Wait (ms)",
Value: fmt.Sprintf("%d", int(payload["maxWaitMs"].(float64))),
Selector: `request.payload.maxWaitMs`,
},
{
"name": "Minimum Bytes",
"value": fmt.Sprintf("%d", int(payload["MinBytes"].(float64))),
Name: "Minimum Bytes",
Value: fmt.Sprintf("%d", int(payload["minBytes"].(float64))),
Selector: `request.payload.minBytes`,
},
{
"name": "Maximum Bytes",
"value": maxBytes,
Name: "Maximum Bytes",
Value: maxBytes,
Selector: `request.payload.maxBytes`,
},
{
"name": "Isolation Level",
"value": isolationLevel,
Name: "Isolation Level",
Value: isolationLevel,
Selector: `request.payload.isolationLevel`,
},
{
"name": "Session ID",
"value": sessionId,
Name: "Session ID",
Value: sessionId,
Selector: `request.payload.sessionId`,
},
{
"name": "Session Epoch",
"value": sessionEpoch,
Name: "Session Epoch",
Value: sessionEpoch,
Selector: `request.payload.sessionEpoch`,
},
{
"name": "Topics",
"value": topics,
Name: "Topics",
Value: topics,
Selector: `request.payload.topics`,
},
{
"name": "Forgotten Topics Data",
"value": forgottenTopicsData,
Name: "Forgotten Topics Data",
Value: forgottenTopicsData,
Selector: `request.payload.forgottenTopicsData`,
},
{
"name": "Rack ID",
"value": rackId,
Name: "Rack ID",
Value: rackId,
Selector: `request.payload.rackId`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -435,46 +472,50 @@ func representFetchResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
responses := ""
if payload["Responses"] != nil {
_responses, _ := json.Marshal(payload["Responses"].([]interface{}))
if payload["responses"] != nil {
_responses, _ := json.Marshal(payload["responses"].([]interface{}))
responses = string(_responses)
}
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
errorCode := ""
if payload["ErrorCode"] != nil {
errorCode = fmt.Sprintf("%d", int(payload["ErrorCode"].(float64)))
if payload["errorCode"] != nil {
errorCode = fmt.Sprintf("%d", int(payload["errorCode"].(float64)))
}
sessionId := ""
if payload["SessionId"] != nil {
sessionId = fmt.Sprintf("%d", int(payload["SessionId"].(float64)))
if payload["sessionId"] != nil {
sessionId = fmt.Sprintf("%d", int(payload["sessionId"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
{
"name": "Error Code",
"value": errorCode,
Name: "Error Code",
Value: errorCode,
Selector: `response.payload.errorCode`,
},
{
"name": "Session ID",
"value": sessionId,
Name: "Session ID",
Value: sessionId,
Selector: `response.payload.sessionId`,
},
{
"name": "Responses",
"value": responses,
Name: "Responses",
Value: responses,
Selector: `response.payload.responses`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -485,26 +526,28 @@ func representListOffsetsRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topics := ""
if payload["Topics"] != nil {
_topics, _ := json.Marshal(payload["Topics"].([]interface{}))
if payload["topics"] != nil {
_topics, _ := json.Marshal(payload["topics"].([]interface{}))
topics = string(_topics)
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Replica ID",
"value": fmt.Sprintf("%d", int(payload["ReplicaId"].(float64))),
Name: "Replica ID",
Value: fmt.Sprintf("%d", int(payload["replicaId"].(float64))),
Selector: `request.payload.replicaId`,
},
{
"name": "Topics",
"value": topics,
Name: "Topics",
Value: topics,
Selector: `request.payload.topics`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -515,26 +558,28 @@ func representListOffsetsResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["Topics"].([]interface{}))
payload := data["payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["topics"].([]interface{}))
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
{
"name": "Topics",
"value": string(topics),
Name: "Topics",
Value: string(topics),
Selector: `response.payload.topics`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -545,30 +590,33 @@ func representCreateTopicsRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["Topics"].([]interface{}))
payload := data["payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["topics"].([]interface{}))
validateOnly := ""
if payload["ValidateOnly"] != nil {
validateOnly = strconv.FormatBool(payload["ValidateOnly"].(bool))
if payload["validateOnly"] != nil {
validateOnly = strconv.FormatBool(payload["validateOnly"].(bool))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Topics",
"value": string(topics),
Name: "Topics",
Value: string(topics),
Selector: `request.payload.topics`,
},
{
"name": "Timeout (ms)",
"value": fmt.Sprintf("%d", int(payload["TimeoutMs"].(float64))),
Name: "Timeout (ms)",
Value: fmt.Sprintf("%d", int(payload["timeoutMs"].(float64))),
Selector: `request.payload.timeoutMs`,
},
{
"name": "Validate Only",
"value": validateOnly,
Name: "Validate Only",
Value: validateOnly,
Selector: `request.payload.validateOnly`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -579,26 +627,28 @@ func representCreateTopicsResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["Topics"].([]interface{}))
payload := data["payload"].(map[string]interface{})
topics, _ := json.Marshal(payload["topics"].([]interface{}))
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
{
"name": "Topics",
"value": string(topics),
Name: "Topics",
Value: string(topics),
Selector: `response.payload.topics`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -609,35 +659,38 @@ func representDeleteTopicsRequest(data map[string]interface{}) []interface{} {
rep = representRequestHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
payload := data["payload"].(map[string]interface{})
topics := ""
if payload["Topics"] != nil {
x, _ := json.Marshal(payload["Topics"].([]interface{}))
if payload["topics"] != nil {
x, _ := json.Marshal(payload["topics"].([]interface{}))
topics = string(x)
}
topicNames := ""
if payload["TopicNames"] != nil {
x, _ := json.Marshal(payload["TopicNames"].([]interface{}))
if payload["topicNames"] != nil {
x, _ := json.Marshal(payload["topicNames"].([]interface{}))
topicNames = string(x)
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "TopicNames",
"value": string(topicNames),
Name: "TopicNames",
Value: string(topicNames),
Selector: `request.payload.topicNames`,
},
{
"name": "Topics",
"value": string(topics),
Name: "Topics",
Value: string(topics),
Selector: `request.payload.topics`,
},
{
"name": "Timeout (ms)",
"value": fmt.Sprintf("%d", int(payload["TimeoutMs"].(float64))),
Name: "Timeout (ms)",
Value: fmt.Sprintf("%d", int(payload["timeoutMs"].(float64))),
Selector: `request.payload.timeoutMs`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep
@@ -648,26 +701,28 @@ func representDeleteTopicsResponse(data map[string]interface{}) []interface{} {
rep = representResponseHeader(data, rep)
payload := data["Payload"].(map[string]interface{})
responses, _ := json.Marshal(payload["Responses"].([]interface{}))
payload := data["payload"].(map[string]interface{})
responses, _ := json.Marshal(payload["responses"].([]interface{}))
throttleTimeMs := ""
if payload["ThrottleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["ThrottleTimeMs"].(float64)))
if payload["throttleTimeMs"] != nil {
throttleTimeMs = fmt.Sprintf("%d", int(payload["throttleTimeMs"].(float64)))
}
repPayload, _ := json.Marshal([]map[string]string{
repPayload, _ := json.Marshal([]api.TableData{
{
"name": "Throttle Time (ms)",
"value": throttleTimeMs,
Name: "Throttle Time (ms)",
Value: throttleTimeMs,
Selector: `response.payload.throttleTimeMs`,
},
{
"name": "Responses",
"value": string(responses),
Name: "Responses",
Value: string(responses),
Selector: `response.payload.responses`,
},
})
rep = append(rep, map[string]string{
"type": api.TABLE,
"title": "Payload",
"data": string(repPayload),
rep = append(rep, api.SectionData{
Type: api.TABLE,
Title: "Payload",
Data: string(repPayload),
})
return rep

135
tap/extensions/kafka/main.go
View File

@@ -15,6 +15,7 @@ var _protocol api.Protocol = api.Protocol{
Name: "kafka",
LongName: "Apache Kafka Protocol",
Abbreviation: "KAFKA",
Macro: "kafka",
Version: "12",
BackgroundColor: "#000000",
ForegroundColor: "#ffffff",
@@ -61,7 +62,7 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co
}
}
func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry {
func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry {
request := item.Pair.Request.Payload.(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
service := "kafka"
@@ -70,76 +71,76 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
} else if resolvedSource != "" {
service = resolvedSource
}
apiKey := ApiKey(reqDetails["ApiKey"].(float64))
apiKey := ApiKey(reqDetails["apiKey"].(float64))
summary := ""
switch apiKey {
case Metadata:
_topics := reqDetails["Payload"].(map[string]interface{})["Topics"]
_topics := reqDetails["payload"].(map[string]interface{})["topics"]
if _topics == nil {
break
}
topics := _topics.([]interface{})
for _, topic := range topics {
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string))
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string))
}
if len(summary) > 0 {
summary = summary[:len(summary)-2]
}
break
case ApiVersions:
summary = reqDetails["ClientID"].(string)
summary = reqDetails["clientID"].(string)
break
case Produce:
_topics := reqDetails["Payload"].(map[string]interface{})["TopicData"]
_topics := reqDetails["payload"].(map[string]interface{})["topicData"]
if _topics == nil {
break
}
topics := _topics.([]interface{})
for _, topic := range topics {
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Topic"].(string))
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["topic"].(string))
}
if len(summary) > 0 {
summary = summary[:len(summary)-2]
}
break
case Fetch:
_topics := reqDetails["Payload"].(map[string]interface{})["Topics"]
_topics := reqDetails["payload"].(map[string]interface{})["topics"]
if _topics == nil {
break
}
topics := _topics.([]interface{})
for _, topic := range topics {
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Topic"].(string))
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["topic"].(string))
}
if len(summary) > 0 {
summary = summary[:len(summary)-2]
}
break
case ListOffsets:
_topics := reqDetails["Payload"].(map[string]interface{})["Topics"]
_topics := reqDetails["payload"].(map[string]interface{})["topics"]
if _topics == nil {
break
}
topics := _topics.([]interface{})
for _, topic := range topics {
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string))
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string))
}
if len(summary) > 0 {
summary = summary[:len(summary)-2]
}
break
case CreateTopics:
topics := reqDetails["Payload"].(map[string]interface{})["Topics"].([]interface{})
topics := reqDetails["payload"].(map[string]interface{})["topics"].([]interface{})
for _, topic := range topics {
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["Name"].(string))
summary += fmt.Sprintf("%s, ", topic.(map[string]interface{})["name"].(string))
}
if len(summary) > 0 {
summary = summary[:len(summary)-2]
}
break
case DeleteTopics:
topicNames := reqDetails["TopicNames"].([]string)
topicNames := reqDetails["topicNames"].([]string)
for _, name := range topicNames {
summary += fmt.Sprintf("%s, ", name)
}
@@ -148,44 +149,48 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
request["url"] = summary
elapsedTime := item.Pair.Response.CaptureTime.Sub(item.Pair.Request.CaptureTime).Round(time.Millisecond).Milliseconds()
entryBytes, _ := json.Marshal(item.Pair)
return &api.MizuEntry{
ProtocolName: _protocol.Name,
ProtocolLongName: _protocol.LongName,
ProtocolAbbreviation: _protocol.Abbreviation,
ProtocolVersion: _protocol.Version,
ProtocolBackgroundColor: _protocol.BackgroundColor,
ProtocolForegroundColor: _protocol.ForegroundColor,
ProtocolFontSize: _protocol.FontSize,
ProtocolReferenceLink: _protocol.ReferenceLink,
EntryId: entryId,
Entry: string(entryBytes),
Url: fmt.Sprintf("%s%s", service, summary),
Method: apiNames[apiKey],
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
ElapsedTime: elapsedTime,
Path: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
Protocol: _protocol,
Source: &api.TCP{
Name: resolvedSource,
IP: item.ConnectionInfo.ClientIP,
Port: item.ConnectionInfo.ClientPort,
},
Destination: &api.TCP{
Name: resolvedDestination,
IP: item.ConnectionInfo.ServerIP,
Port: item.ConnectionInfo.ServerPort,
},
Outgoing: item.ConnectionInfo.IsOutgoing,
Request: reqDetails,
Response: item.Pair.Response.Payload.(map[string]interface{})["details"].(map[string]interface{}),
Url: fmt.Sprintf("%s%s", service, summary),
Method: apiNames[apiKey],
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
StartTime: item.Pair.Request.CaptureTime,
ElapsedTime: elapsedTime,
Summary: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
}
}
func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
return &api.BaseEntryDetails{
Id: entry.EntryId,
Id: entry.Id,
Protocol: _protocol,
Url: entry.Url,
RequestSenderIp: entry.RequestSenderIp,
Service: entry.Service,
Summary: entry.Path,
Summary: entry.Summary,
StatusCode: entry.Status,
Method: entry.Method,
Timestamp: entry.Timestamp,
@@ -202,49 +207,43 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
}
}
func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) {
p = _protocol
func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) {
protoOut = _protocol
bodySize = 0
var root map[string]interface{}
json.Unmarshal([]byte(entry.Entry), &root)
representation := make(map[string]interface{}, 0)
request := root["request"].(map[string]interface{})["payload"].(map[string]interface{})
response := root["response"].(map[string]interface{})["payload"].(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
resDetails := response["details"].(map[string]interface{})
apiKey := ApiKey(reqDetails["ApiKey"].(float64))
apiKey := ApiKey(request["apiKey"].(float64))
var repRequest []interface{}
var repResponse []interface{}
switch apiKey {
case Metadata:
repRequest = representMetadataRequest(reqDetails)
repResponse = representMetadataResponse(resDetails)
repRequest = representMetadataRequest(request)
repResponse = representMetadataResponse(response)
break
case ApiVersions:
repRequest = representApiVersionsRequest(reqDetails)
repResponse = representApiVersionsResponse(resDetails)
repRequest = representApiVersionsRequest(request)
repResponse = representApiVersionsResponse(response)
break
case Produce:
repRequest = representProduceRequest(reqDetails)
repResponse = representProduceResponse(resDetails)
repRequest = representProduceRequest(request)
repResponse = representProduceResponse(response)
break
case Fetch:
repRequest = representFetchRequest(reqDetails)
repResponse = representFetchResponse(resDetails)
repRequest = representFetchRequest(request)
repResponse = representFetchResponse(response)
break
case ListOffsets:
repRequest = representListOffsetsRequest(reqDetails)
repResponse = representListOffsetsResponse(resDetails)
repRequest = representListOffsetsRequest(request)
repResponse = representListOffsetsResponse(response)
break
case CreateTopics:
repRequest = representCreateTopicsRequest(reqDetails)
repResponse = representCreateTopicsResponse(resDetails)
repRequest = representCreateTopicsRequest(request)
repResponse = representCreateTopicsResponse(response)
break
case DeleteTopics:
repRequest = representDeleteTopicsRequest(reqDetails)
repResponse = representDeleteTopicsResponse(resDetails)
repRequest = representDeleteTopicsRequest(request)
repResponse = representDeleteTopicsResponse(response)
break
}
@@ -254,4 +253,10 @@ func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []by
return
}
func (d dissecting) Macros() map[string]string {
return map[string]string{
`kafka`: fmt.Sprintf(`proto.abbr == "%s"`, _protocol.Abbreviation),
}
}
var Dissector dissecting

14
tap/extensions/kafka/request.go
View File

@@ -10,13 +10,13 @@ import (
)
type Request struct {
Size int32
ApiKey ApiKey
ApiVersion int16
CorrelationID int32
ClientID string
Payload interface{}
CaptureTime time.Time
Size int32 `json:"size"`
ApiKey ApiKey `json:"apiKey"`
ApiVersion int16 `json:"apiVersion"`
CorrelationID int32 `json:"correlationID"`
ClientID string `json:"clientID"`
Payload interface{} `json:"payload"`
CaptureTime time.Time `json:"captureTime"`
}
func ReadRequest(r io.Reader, tcpID *api.TcpID, superTimer *api.SuperTimer) (apiKey ApiKey, apiVersion int16, err error) {

8
tap/extensions/kafka/response.go
View File

@@ -10,10 +10,10 @@ import (
)
type Response struct {
Size int32
CorrelationID int32
Payload interface{}
CaptureTime time.Time
Size int32 `json:"size"`
CorrelationID int32 `json:"correlationID"`
Payload interface{} `json:"payload"`
CaptureTime time.Time `json:"captureTime"`
}
func ReadResponse(r io.Reader, tcpID *api.TcpID, superTimer *api.SuperTimer, emitter api.Emitter) (err error) {

894
tap/extensions/kafka/structs.go
View File

File diff suppressed because it is too large Load Diff

38
tap/extensions/redis/helpers.go
View File

@@ -2,6 +2,7 @@ package main
import (
"encoding/json"
"fmt"
"github.com/up9inc/mizu/tap/api"
)
@@ -24,33 +25,38 @@ type RedisWrapper struct {
Details interface{} `json:"details"`
}
func representGeneric(generic map[string]interface{}) (representation []interface{}) {
details, _ := json.Marshal([]map[string]string{
func representGeneric(generic map[string]interface{}, selectorPrefix string) (representation []interface{}) {
details, _ := json.Marshal([]api.TableData{
{
"name": "Type",
"value": generic["type"].(string),
Name: "Type",
Value: generic["type"].(string),
Selector: fmt.Sprintf("%stype", selectorPrefix),
},
{
"name": "Command",
"value": generic["command"].(string),
Name: "Command",
Value: generic["command"].(string),
Selector: fmt.Sprintf("%scommand", selectorPrefix),
},
{
"name": "Key",
"value": generic["key"].(string),
Name: "Key",
Value: generic["key"].(string),
Selector: fmt.Sprintf("%skey", selectorPrefix),
},
{
"name": "Value",
"value": generic["value"].(string),
Name: "Value",
Value: generic["value"].(string),
Selector: fmt.Sprintf("%svalue", selectorPrefix),
},
{
"name": "Keyword",
"value": generic["keyword"].(string),
Name: "Keyword",
Value: generic["keyword"].(string),
Selector: fmt.Sprintf("%skeyword", selectorPrefix),
},
})
representation = append(representation, map[string]string{
"type": api.TABLE,
"title": "Details",
"data": string(details),
representation = append(representation, api.SectionData{
Type: api.TABLE,
Title: "Details",
Data: string(details),
})
return

86
tap/extensions/redis/main.go
View File

@@ -13,6 +13,7 @@ var protocol api.Protocol = api.Protocol{
Name: "redis",
LongName: "Redis Serialization Protocol",
Abbreviation: "REDIS",
Macro: "redis",
Version: "3.x",
BackgroundColor: "#a41e11",
ForegroundColor: "#ffffff",
@@ -57,9 +58,12 @@ func (d dissecting) Dissect(b *bufio.Reader, isClient bool, tcpID *api.TcpID, co
}
}
func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolvedSource string, resolvedDestination string) *api.MizuEntry {
func (d dissecting) Analyze(item *api.OutputChannelItem, resolvedSource string, resolvedDestination string) *api.MizuEntry {
request := item.Pair.Request.Payload.(map[string]interface{})
response := item.Pair.Response.Payload.(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
resDetails := response["details"].(map[string]interface{})
service := "redis"
if resolvedDestination != "" {
service = resolvedDestination
@@ -78,45 +82,49 @@ func (d dissecting) Analyze(item *api.OutputChannelItem, entryId string, resolve
}
request["url"] = summary
entryBytes, _ := json.Marshal(item.Pair)
return &api.MizuEntry{
ProtocolName: protocol.Name,
ProtocolLongName: protocol.LongName,
ProtocolAbbreviation: protocol.Abbreviation,
ProtocolVersion: protocol.Version,
ProtocolBackgroundColor: protocol.BackgroundColor,
ProtocolForegroundColor: protocol.ForegroundColor,
ProtocolFontSize: protocol.FontSize,
ProtocolReferenceLink: protocol.ReferenceLink,
EntryId: entryId,
Entry: string(entryBytes),
Url: fmt.Sprintf("%s%s", service, summary),
Method: method,
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
ElapsedTime: 0,
Path: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
Protocol: protocol,
Source: &api.TCP{
Name: resolvedSource,
IP: item.ConnectionInfo.ClientIP,
Port: item.ConnectionInfo.ClientPort,
},
Destination: &api.TCP{
Name: resolvedDestination,
IP: item.ConnectionInfo.ServerIP,
Port: item.ConnectionInfo.ServerPort,
},
Outgoing: item.ConnectionInfo.IsOutgoing,
Request: reqDetails,
Response: resDetails,
Url: fmt.Sprintf("%s%s", service, summary),
Method: method,
Status: 0,
RequestSenderIp: item.ConnectionInfo.ClientIP,
Service: service,
Timestamp: item.Timestamp,
StartTime: item.Pair.Request.CaptureTime,
ElapsedTime: 0,
Summary: summary,
ResolvedSource: resolvedSource,
ResolvedDestination: resolvedDestination,
SourceIp: item.ConnectionInfo.ClientIP,
DestinationIp: item.ConnectionInfo.ServerIP,
SourcePort: item.ConnectionInfo.ClientPort,
DestinationPort: item.ConnectionInfo.ServerPort,
IsOutgoing: item.ConnectionInfo.IsOutgoing,
}
}
func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
return &api.BaseEntryDetails{
Id: entry.EntryId,
Id: entry.Id,
Protocol: protocol,
Url: entry.Url,
RequestSenderIp: entry.RequestSenderIp,
Service: entry.Service,
Summary: entry.Path,
Summary: entry.Summary,
StatusCode: entry.Status,
Method: entry.Method,
Timestamp: entry.Timestamp,
@@ -133,22 +141,22 @@ func (d dissecting) Summarize(entry *api.MizuEntry) *api.BaseEntryDetails {
}
}
func (d dissecting) Represent(entry *api.MizuEntry) (p api.Protocol, object []byte, bodySize int64, err error) {
p = protocol
func (d dissecting) Represent(protoIn api.Protocol, request map[string]interface{}, response map[string]interface{}) (protoOut api.Protocol, object []byte, bodySize int64, err error) {
protoOut = protocol
bodySize = 0
var root map[string]interface{}
json.Unmarshal([]byte(entry.Entry), &root)
representation := make(map[string]interface{}, 0)
request := root["request"].(map[string]interface{})["payload"].(map[string]interface{})
response := root["response"].(map[string]interface{})["payload"].(map[string]interface{})
reqDetails := request["details"].(map[string]interface{})
resDetails := response["details"].(map[string]interface{})
repRequest := representGeneric(reqDetails)
repResponse := representGeneric(resDetails)
repRequest := representGeneric(request, `request.`)
repResponse := representGeneric(response, `response.`)
representation["request"] = repRequest
representation["response"] = repResponse
object, err = json.Marshal(representation)
return
}
func (d dissecting) Macros() map[string]string {
return map[string]string{
`redis`: fmt.Sprintf(`proto.abbr == "%s"`, protocol.Abbreviation),
}
}
var Dissector dissecting