System
Microsoft Graph Security API Add-On for Splunk v1.1.0
Splunk Enterprise 7.1.10
Problem
recommendedActions node for Network_TrafficFromUnrecommendedIP events are not valid JSON
Details
The data contained within the recommendedActions node isn’t JSON, which prevents Splunk from being able to properly apply the JSON indexed_extraction. Thus it shows up as a large text string and the contained fields are not reportable.
Using any JSON formatting tool, the highlighted row is are the fields in question for correction:
"azureSubscriptionId": "48381800-1ec9-4a2f-bbda-a1a1a188a2b1",
"recommendedActions": [
"{\"kind\":\"openBlade\",\"displayValue\":\"Enforce rule\",\"extension\":\"Microsoft_Azure_Security_R3\",\"detailBlade\":\"AdaptiveNetworkControlsResourceBlade\",\"detailBladeInputs\":\"protectedResourceId=/subscriptions/48381800-1ec9-4a2f-bbda-a1a1a188a2b1/resourcegroups/freddata-rg/providers/microsoft.compute/virtualmachines/fredcomputebdc1111111\"}"
],
"title": "Traffic detected from IP addresses recommended for blocking",
After minor regex to remove the invalid characters, that same section above would look like the following:
"azureSubscriptionId": "48381800-1ec9-4a2f-bbda-a1a1a188a2b1",
"recommendedActions": [
{
"kind": "openBlade",
"displayValue": "Enforce rule",
"extension": "Microsoft_Azure_Security_R3",
"detailBlade": "AdaptiveNetworkControlsResourceBlade",
"detailBladeInputs": "protectedResourceId=/subscriptions/48381800-1ec9-4a2f-bbda-a1a1a188a2b1/resourcegroups/freddata-rg/providers/microsoft.compute/virtualmachines/fredcomputebdc947035"
}
],
"title": "Traffic detected from IP addresses recommended for blocking",
| makeresults
|eval _raw="\"azureSubscriptionId\": \"48381800-1ec9-4a2f-bbda-a1a1a188a2b1\",
\"recommendedActions\": [
\"{\\\"kind\\\":\\\"openBlade\\\",\\\"displayValue\\\":\\\"Enforce rule\\\",\\\"extension\\\":\\\"Microsoft_Azure_Security_R3\\\",\\\"detailBlade\\\":\\\"AdaptiveNetworkControlsResourceBlade\\\",\\\"detailBladeInputs\\\":\\\"protectedResourceId=/subscriptions/48381800-1ec9-4a2f-bbda-a1a1a188a2b1/resourcegroups/freddata-rg/providers/microsoft.compute/virtualmachines/fredcomputebdc1111111\\\"}\"
],
\"title\": \"Traffic detected from IP addresses recommended for blocking\","
| rex mode=sed "s/(?s)^.*?(\{.*}).*$/\1/g"
| rex mode=sed "s$\\\$$g"
| spath
This query works well.
props.conf
[msgraph_json]
SEDCMD-backslash = s/\\\//g
EXTRACT-azureSubscriptionId = azureSubscriptionId\"\:\s+\"(?<azureSubscriptionId>\S+)\",
EXTRACT-title = title\"\:\s+\"(?<title>[^\"]+)\"
SEDCMD-trim = s/(?s)^.*?(\{.*}).*$/\1/g
KV_MODE = json
| makeresults
|eval _raw="\"azureSubscriptionId\": \"48381800-1ec9-4a2f-bbda-a1a1a188a2b1\",
\"recommendedActions\": [
\"{\\\"kind\\\":\\\"openBlade\\\",\\\"displayValue\\\":\\\"Enforce rule\\\",\\\"extension\\\":\\\"Microsoft_Azure_Security_R3\\\",\\\"detailBlade\\\":\\\"AdaptiveNetworkControlsResourceBlade\\\",\\\"detailBladeInputs\\\":\\\"protectedResourceId=/subscriptions/48381800-1ec9-4a2f-bbda-a1a1a188a2b1/resourcegroups/freddata-rg/providers/microsoft.compute/virtualmachines/fredcomputebdc1111111\\\"}\"
],
\"title\": \"Traffic detected from IP addresses recommended for blocking\","
| eval _raw=split(_raw,",")
| stats count by _raw
| fields - count
| rex max_match=0 "(?m)(?<keyvalue>\".*:.*\")"
| eval keyvalue=replace(keyvalue,"[{\\\\\"}]","")
| eval _raw=replace(keyvalue,"(\w+):(.*)","\1=\"\2\"")
| kv
| fields - keyvalue
| stats values(*) as *
| foreach * [eval <<FIELD>> = trim('<<FIELD>>')]
Thanks for the code.
Yup, that can work for an adhoc fix. Still would like to see MS implement the fix in their addon or in the API data.
my props.conf example is useless? `
what do you want to do?
Have the addon (either the python input code) or props, implement a fix.
None of the other categories of data have the backslash quoted fields either, so not sure if its a problem upstream with the API or how the input handles the data.