You recognize that this is a Splunk forum where volunteers offer help related to Splunk, right?  As I said, Splunk does not "color" search results. (The only coloring function in Splunk is provided in dashboard visualization.)  If you want to color text, you will need to develop something external to Splunk.  As you suggested, you can possibly achieve this by modifying sendmail.py (not recommended).  Alternatively, you can develop a custom command for this.  Either way, this is not the right forum. ... View more

mvzip, mvexpand and mvindex are simply wrong tools for your data structure. (Well, mvexpand will be needed, but only after you properly handle the array in your data.)  As everybody in this post has pointed out: You need to post sample or precise mock data to reveal the structure. (In text, never screenshot.) This is extremely important when asking question about data analytics in a forum.  When you force volunteers to read your mind, not only will they get FRUSTRATED, but even if they are willing, most of the time their mind reading will be incorrect. This said, based on your code, I kind of picture together a rough structure of your data.  I will use JSON to illustrate.  Something like   { "userActions": [ { "application": "app1", "name": "action1", "targetUrl": "url1", "duration": 1234, "type": "actiontype1", "apdexCategory": "SATISFIED" }, { "application": "app1", "name": "action2", "targetUrl": "url1", "duration": 2345, "type": "actiontype1", "apdexCategory": "DISATISFIED" }, { "application": "app1", "name": "action3", "targetUrl": "url2", "duration": 3456, "type": "actiontype2", "apdexCategory": "FRUSTRATED" } ], "userExperienceScore": "FRUSTRATED", "events": [ {"application": "xxx", "irrelevant": "aaa"}, {"application": "yyy", "irrelevant": "bbb"} ] }     Your event could be in JSON or it could be in XML, but it contains at least two arrays, events[] and userActions[].  Is this correct?  The array events[] is not what frustrates you because its elements and components are no longer needed after initial search.  Your end goal from the above three elements of userActions[] is to pick out   { "application": "app1", "name": "action3", "targetUrl": "url2", "duration": 3456, "type": "actiontype2", "apdexCategory": "FRUSTRATED" }   and display it in this format: _time Application Action Target_URL Duration_in_Mins User_Action_Type useractions_experience_score 2024-04-18 22:45:22 app1 action3 url2 0.06 actiontype2 FRUSTRATED If the above looks close, the first thing you need to do is to forget all about Splunk's flattened fields userActions{}.*; in fact, discard them all.  Use spath to reach elements of this array, then mvexpand over the elements, no funny mvzip business.  After that, everything becomes trivial. Using my speculated data, I can reconstruct your SPL into the following to obtain my illustrated output:     index="xxx" sourcetype="xxx" source=xxx events{}.application="xxx" userExperienceScore=FRUSTRATED | fields - userActions{}.* | spath path=userActions{} | mvexpand userActions{} | spath input=userActions{} | dedup application name targetUrl | search apdexCategory = FRUSTRATED application = * name = * | sort - _time | rename application as Application, name as Action, targetUrl as Target_URL, type as User_Action_Type, apdexCategory as useractions_experience_score | eval Duration_in_Mins = round(duration / 60000, 2) | table _time, Application, Action, Target_URL,Duration_in_Mins,User_Action_Type,useractions_experience_score   Hope this helps. Here is an emulation of my speculated data.  Play with it and compare with real data | makeresults | eval _raw = "{ \"userActions\": [ { \"application\": \"app1\", \"name\": \"action1\", \"targetUrl\": \"url1\", \"duration\": 1234, \"type\": \"actiontype1\", \"apdexCategory\": \"SATISFIED\" }, { \"application\": \"app1\", \"name\": \"action2\", \"targetUrl\": \"url1\", \"duration\": 2345, \"type\": \"actiontype1\", \"apdexCategory\": \"DISATISFIED\" }, { \"application\": \"app1\", \"name\": \"action3\", \"targetUrl\": \"url2\", \"duration\": 3456, \"type\": \"actiontype2\", \"apdexCategory\": \"FRUSTRATED\" } ], \"userExperienceScore\": \"FRUSTRATED\", \"events\": [ {\"application\": \"xxx\", \"irrelevant\": \"aaa\"}, {\"application\": \"yyy\", \"irrelevant\": \"bbb\"} ] }" | spath ``` data speculation for index="xxx" sourcetype="xxx" source=xxx events{}.application="xxx" userExperienceScore=FRUSTRATED ``` ... View more

Have you tried my previous code? | eval route = if(match(request_path, "^/orders/\d+"), "/order/{orderID}", null()) This does exactly what you ask: create a new field named route that has a fixed pattern "/order/{orderID}".  Is there anything wrong with this? In fact, because you really only care about first segment of the path - that fixed string "{orderID}" is just a decoration, the command could be simplified to slightly less expensive | eval route = "/" . mvindex(split(request_path, "/"), 1) . "/{orderID}" You can do whatever analysis against this field. ... View more

04-18-2024 13:36:06.590 ERROR EvalCommand [102993 searchOrchestrator] - The 'bit_shift_left' function is unsupported or undefined. I believe the function requires 9.2.0+ Thanks for noticing!  I always assumed that bitwise operations had been part of SPL from day one but no.  The document has this footer: "This documentation applies to the following versions of Splunk® Enterprise: 9.2.0, 9.2.1." (Searching in previous versions results in the same pointers to 9.2.) For the above, should the second set have been given a different value for the field? Those are really bad copy-and-paste errors.  Corrected. ... View more

I do not see why you needed to do that extra extraction because Splunk should have given you a field named "request_path" already. (See emulation below.)  All you need to do is to assign a new field based on match.   | eval route = if(match(request_path, "^/orders/\d+"), "/order/{orderID}", null())   The sample data should give you something like level request_elapsed request_id request_method request_path response_status route info 100 2ca011b5-ad34-4f32-a95c-78e8b5b1a270 GET /orders/123456 500 /order/{orderID} Is this what you wanted? Here is a data emulation you can play with and compare with real data.   | makeresults | eval _raw = "level=info request.elapsed=100 request.method=GET request.path=/orders/123456 request_id=2ca011b5-ad34-4f32-a95c-78e8b5b1a270 response.status=500" | extract ``` data emulation above ```   Of course, if for unknown reasons Splunk doesn't give you request_path, simply add an extract command and skip all the rex which is expensive. ... View more

Let me give this a semantic makeover using bit_shift_left😃 (9.2 and above - thanks @jason_hotchkiss for noticing) because semantic code is easier to understand and maintain.   | eval offset = mvappend("24", "16", "8") | eval segment_rev = mvrange(0, 3) | foreach *_ip [eval <<FIELD>> = split(<<FIELD>>, "."), <<FIELD>>_dec = sum(mvmap(segment_rev, bit_shift_left(tonumber(mvindex(<<FIELD>>, segment_rev)), tonumber(mvindex(offset, segment_rev)))), tonumber(mvindex(<<FIELD>>, 3))), <<FIELD>> = mvjoin(<<FIELD>>, ".") ``` this last part for display only ```] | fields - offset segment_rev   The sample data gives dst_ip dst_ip_dec src_ip src_ip_dec 192.168.1.100 3232235876 192.168.1.1 3232235777 Here is an emulation you can play with and compare with real data     | makeresults format=csv data="src_ip, dst_ip 192.168.1.1, 192.168.1.100" ``` data emulation above ```     Note: If it helps readability., you can skip foreach and spell the two operations separately.   | eval offset = mvappend("24", "16", "8") | eval segment_rev = mvrange(0, 3) | eval src_ip = split(src_ip, ".") | eval dst_ip = split(dst_ip, ".") | eval src_ip_dec = sum(mvmap(segment_rev, bit_shift_left(tonumber(mvindex(src_ip, segment_rev)), tonumber(mvindex(offset, segment_rev)))), tonumber(mvindex(src_ip, 3))) | eval dst_ip_dec = sum(mvmap(segment_rev, bit_shift_left(tonumber(mvindex(dst_ip, segment_rev)), tonumber(mvindex(offset, segment_rev)))), tonumber(mvindex(dst_ip, 3))) | eval src_ip = mvjoin(src_ip, "."), dst_ip = mvjoin(dst_ip, ".") ``` for display only ``` | fields - offset segment_rev         ... View more

Let me clarify: When you say "color", you are talking about converting percentage text to a string representation of color code, not to color the percentage text in E-mail alert.  Correct?  In other words, you want something like Name color percentage A red 71 B amber 90 C red 44 D amber 88 E red 78 Because while potentially doable (and likely will involve a custom command you need to develop externally), Splunk doesn't provide such a function to color text used in E-mail alerts.  If this is the correct requirement, look up documentation for case I further assume that your "percentage" field doesn't come with a percent sign (%); if you want that % in E-mail, you can always add it after color mapping.   | eval color = case(percentage < 80, "red", percentage < 95, "amber", true(), "green")   Here is a data emulation you can play with and compare with real data   | makeresults format=csv data="Name, percentage A, 71% B, 90% C, 44% D, 88% E, 78%" | eval percentage = replace(percentage, "%", "") ``` data emulation above ```   Hope this helps. ... View more

Before I rant, thank you for sharing valid mock data in text.  This said, this is the second time in as many consecutive days I feel like screaming at lazy developers who makes some terrible use of JSON arrays. (The developer might be you.  But the rant stands😏)  Your data would have much cleaner, self-evidenced semantics had the developer simply use this:   [ { "attributes": {"host.name":{"stringValue":"myname1"},"telemetry.sdk.name":{"stringValue":"my_sdk"}}, "metrics": {"hw.host.energy":{"dataPoints":[{"timeUnixNano":"1712951030986039000","asDouble":359}]},"hw.host.power":{"dataPoints":[{"timeUnixNano":"1712951030986039000","asDouble":26}]}} }, { "attributes": {"host.name":{"stringValue":"myname2"},"telemetry.sdk.name":{"stringValue":"my_sdk"}}, "metrics": {"hw.host.energy":{"dataPoints":[{"timeUnixNano":"1712951030987780000","asDouble":211}]}} ]   In other words, only two JSON arrays in the original data are used correctly.  resourceMetrics.resource.attributes[] and resourceMetrics.scopeMetrics.metrics[] are total abomination of the intent of JSON arrays.  Speak to your developers to see if they could change the data structure not just for Splunk, but for future maintainers of their own code and any other downstream team as well. Now that this is off my chest, I understand that it will take more than one day for developers to change code even if you convince them on day one.  Here is the SPL that I use to tabulate your data like the following: host.name.stringValue hw.host.energy{}.asDouble hw.host.energy{}.timeUnixNano hw.host.power{}.asDouble hw.host.power{}.timeUnixNano sdk.name.stringValue myname1 359 1712951030986039000 26 1712951030986039000 my_sdk myname2 211 1712951030987780000     my_sdk In this form, I have assumed that dataPoints[] is the only node of interest under resourceMetrics[].scopeMetrics[].metrics.gauge  | spath path=resourceMetrics{} | fields - _* resourceMetrics{}.* | mvexpand resourceMetrics{} | spath input=resourceMetrics{} path=resource.attributes{} | spath input=resourceMetrics{} path=scopeMetrics{} | spath input=scopeMetrics{} path=metrics{} | fields - resourceMetrics{} scopeMetrics{} | foreach resource.attributes{} mode=multivalue [eval key = mvappend(key, json_extract(<<ITEM>>, "key"))] | eval idx = mvrange(0, mvcount(key)) | eval attributes_good = json_object() | foreach idx mode=multivalue [eval attribute = mvindex('resource.attributes{}', <<ITEM>>), attributes_good = json_set_exact(attributes_good, json_extract(attribute, "key"), json_extract(attribute, "value"))] | fields - key attribute resource.attributes{} | foreach metrics{} mode=multivalue [eval name = mvappend(name, json_extract(<<ITEM>>, "name"))] | eval name = if(isnull(name), json_extract('metrics{}', "name"), name) | eval idx = mvrange(0, mvcount(name)) | eval metrics_good = json_object() | foreach idx mode=multivalue [eval metric = mvindex('metrics{}', <<ITEM>>), metrics_good = json_set_exact(metrics_good, json_extract(metric, "name"), json_extract(metric, "gauge.dataPoints"))] ``` the above assumes that gauge.dataPoints is the only subnode of interest ``` | fields - idx name metric metrics{} ``` the above transforms array-laden JSON into easily understandable JSON ``` | spath input=attributes_good | spath input=metrics_good | fields - *_good ``` the following is only needed if dataPoints[] actually contain multiple values. This is the only code requiring prior knowledge about data fields ``` | mvexpand hw.host.energy{}.timeUnixNano | mvexpand hw.host.power{}.timeUnixNano (The fields - xxx commands are not essential; they just declutter view.)  Hope this helps. This is an emulation you can play with and compare with real data: | makeresults | eval _raw = "{ \"resourceMetrics\": [ { \"resource\": { \"attributes\": [ { \"key\": \"host.name\", \"value\": { \"stringValue\": \"myname1\" } }, { \"key\": \"telemetry.sdk.name\", \"value\": { \"stringValue\": \"my_sdk\" } } ] }, \"scopeMetrics\": [ { \"metrics\": [ { \"name\": \"hw.host.energy\", \"gauge\": { \"dataPoints\": [ { \"timeUnixNano\": \"1712951030986039000\", \"asDouble\": 359 } ] } }, { \"name\": \"hw.host.power\", \"gauge\": { \"dataPoints\": [ { \"timeUnixNano\": \"1712951030986039000\", \"asDouble\": 26 } ] } } ] } ] }, { \"resource\": { \"attributes\": [ { \"key\": \"host.name\", \"value\": { \"stringValue\": \"myname2\" } }, { \"key\": \"telemetry.sdk.name\", \"value\": { \"stringValue\": \"my_sdk\" } } ] }, \"scopeMetrics\": [ { \"metrics\": [ { \"name\": \"hw.host.energy\", \"gauge\": { \"dataPoints\": [ { \"timeUnixNano\": \"1712951030987780000\", \"asDouble\": 211 } ] } } ] } ] } ] }" | spath ``` data emulation above ``` Final thoughts about data structure with self-evidence semantics: If my speculation about dataPoints[] being the only node of interest under resourceMetrics[].scopeMetrics[].metrics.gauge stands, good data could be further simplified to [ { "attributes": {"host.name":{"stringValue":"myname1"},"telemetry.sdk.name":{"stringValue":"my_sdk"}}, "metrics": {"hw.host.energy":[{"timeUnixNano":"1712951030986039000","asDouble":359}],"hw.host.power":[{"timeUnixNano":"1712951030986039000","asDouble":26}]} }, { "attributes": {"host.name":{"stringValue":"myname2"},"telemetry.sdk.name":{"stringValue":"my_sdk"}}, "metrics": {"hw.host.energy":[{"timeUnixNano":"1712951030987780000","asDouble":211}]} ] I do understand that listing hw.host.energy and hw.host.power as coexisting columns is different from your illustrated output and may not suite your needs.  But presentation can easily be adapted.  Bad data structure remains bad. ... View more

Do not treat structured data such as JSON as text and be tempted to use rex for extraction.  Use QA tested Splunk command such as spath to extract from structure, then mvexpand to handle array.   | spath path=message{} | mvexpand message{} | spath input=message{}   Your sample will give you ERROR ERROR_1 ERROR_IND FUNCTION_NAME PROCESSED REMAINING SKIPPED TARGET_SYSTEM TOTAL id severity message{} 0 (0%) 0 0 CPW_02170 121257 0 35 (0%) SEQ 121257 0 Information {"TARGET_SYSTEM":"SEQ","FUNCTION_NAME":"CPW_02170","TOTAL":"121257","PROCESSED":"121257","REMAINING":"0","ERROR":"0 (0%)","SKIPPED":"35 (0%)","ERROR_IND":"0","ERROR_1":"0"} 0 (0%) 0 0 CPW_02171 26434 0 19 (0%) CPW 26434 0 Information {"TARGET_SYSTEM":"CPW","FUNCTION_NAME":"CPW_02171","TOTAL":"26434","PROCESSED":"26434","REMAINING":"0","ERROR":"0 (0%)","SKIPPED":"19 (0%)","ERROR_IND":"0","ERROR_1":"0"} 0 (0%) 0 0 CPW_02172 2647812 0 19 (0%) SEQ 23343 0 Information {"TARGET_SYSTEM":"SEQ","FUNCTION_NAME":"CPW_02172","TOTAL":"23343","PROCESSED":"2647812","REMAINING":"0","ERROR":"0 (0%)","SKIPPED":"19 (0%)","ERROR_IND":"0","ERROR_1":"0"} Here is a data emulation.  Play with it and compare with real data   | makeresults | eval _raw="{\"id\":\"0\",\"severity\":\"Information\",\"message\":[{\"TARGET_SYSTEM\":\"SEQ\",\"FUNCTION_NAME\":\"CPW_02170\",\"TOTAL\":\"121257\",\"PROCESSED\":\"121257\",\"REMAINING\":\"0\",\"ERROR\":\"0 (0%)\",\"SKIPPED\":\"35 (0%)\",\"ERROR_IND\":\"0\",\"ERROR_1\":\"0\"},{\"TARGET_SYSTEM\":\"CPW\",\"FUNCTION_NAME\":\"CPW_02171\",\"TOTAL\":\"26434\",\"PROCESSED\":\"26434\",\"REMAINING\":\"0\",\"ERROR\":\"0 (0%)\",\"SKIPPED\":\"19 (0%)\",\"ERROR_IND\":\"0\",\"ERROR_1\":\"0\"},{\"TARGET_SYSTEM\":\"SEQ\",\"FUNCTION_NAME\":\"CPW_02172\",\"TOTAL\":\"23343\",\"PROCESSED\":\"2647812\",\"REMAINING\":\"0\",\"ERROR\":\"0 (0%)\",\"SKIPPED\":\"19 (0%)\",\"ERROR_IND\":\"0\",\"ERROR_1\":\"0\"}]}" | spath ``` data emulation above ```     ... View more

Simply put: Don't.  Do not treat structured data as text and use rex to extract fields, use rex to extract the structured data.  In this case, the structure is in JSON. Not only that.  Your data contains very different JSON nodes that both have "LastModifiedBy", "RecordId", etc.  Your result table must distinguish between JSON nodes "servicechannel" and "omnisupervisionconfig", etc.  Does this make sense? Further, each of these nodes uses an array.  You may need to distinguish between each element in the arrays.  Because your illustration does not include multiple elements in the array, I cannot speculate what your developers' intention (semantics) is to use array for three distinct nodes.  It is possible that they committed the ultimate JSON data sin by assuming an implied semantic meaning in the arrays.   In light of this, I will not introduce the more intricate part of mixed kv-array data processing and just assume that all your data come with a single element in every of the three arrays. (When your developers give this type data, it is even more dangerous to use rex to extract data elements because no regex is compatible with the inherent flexibility afforded by the data structure.) Here is my suggestion:   | rex "^[^{]+(?<jsondata>.+)" | spath input=jsondata   Your sample data gives 3 sets of the 4 columns you desired, for a total of 12 columns.  That's too wide for display, so I will show a transposed table: field_name field_value livechatbutton{}.ComponentName LiveChatButton livechatbutton{}.LastmodifiedBy XYZ livechatbutton{}.ModifiedDate 2024-04-16T16:31:35.000Z livechatbutton{}.RecordId 5638X000000Xw55QAC omnisupervisorconfig{}.ComponentName OmniSupervisorConfig omnisupervisorconfig{}.LastmodifiedBy XYZ omnisupervisorconfig{}.ModifiedDate 2024-04-16T16:17:37.000Z omnisupervisorconfig{}.RecordId 0Q27X000000KyrESAS servicechannel{}.ComponentName ServiceChannel servicechannel{}.LastmodifiedBy XYZ servicechannel{}.ModifiedDate 2024-04-15T17:20:09.000Z servicechannel{}.RecordId 0N98X001200Gvv3SAC You must then decide how you want to present such data.  I do notice that each JSON node key and each ComponentName have an apparent semantic relationship.  If key name servicechannel and ComponentName ServiceChannel, etc., are indeed semantically related, your developers also committed a different type of data structure sin: that of duplicating semantic notation (without declaration).  The data could easily be presented without losing resolution but in a much simpler and more comprehensible form:   [ { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-15T17:20:09.000Z", "ComponentName": "ServiceChannel", "RecordId": "0N98X001200Gvv3SAC" }, { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-16T16:17:37.000Z", "ComponentName": "OmniSupervisorConfig", "RecordId": "0Q27X000000KyrESAS" }, { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-16T16:31:35.000Z", "ComponentName": "LiveChatButton", "RecordId": "5638X000000Xw55QAC" } ]   If you have any influence on developers, discuss data structure with them, ask them to clarify the intention/semantics of the structure and help improve structure.  This is better for everybody in the long run. If you have no influence, one possible way to deal with this mess is to ignore all the key names and treat them like single name, i.e., by assuming the data to be   [ {"array": [ { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-15T17:20:09.000Z", "ComponentName": "ServiceChannel", "RecordId": "0N98X001200Gvv3SAC" } ]}, {"array": [ { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-16T16:17:37.000Z", "ComponentName": "OmniSupervisorConfig", "RecordId": "0Q27X000000KyrESAS" } ]}, {"array": [ { "LastmodifiedBy": "XYZ", "ModifiedDate": "2024-04-16T16:31:35.000Z", "ComponentName": "LiveChatButton", "RecordId": "5638X000000Xw55QAC" } ]} ]   To do the equivalent in SPL,  (and also handle potential multiple array elements in absence of semantic knowledge)   | rex "^[^{]+(?<jsondata>.+)" | eval jsonnode = json_keys(jsondata) | foreach jsonnode mode=json_array [eval newjson = mvappend(newjson, json_object("array", json_extract(jsondata, <<ITEM>>)))] | mvexpand newjson | spath input=newjson path=array{} | mvexpand array{} ``` potential multiple elements ``` | spath input=array{}    Your sample data will give ComponentName LastmodiefiedBy ModifiedDate RecordId newjson ServiceChannel XYZ 2024-04-15T17:20:09.000Z 0N98X001200Gvv3SAC {"array":[{"LastmodifiedBy":"XYZ","ModifiedDate":"2024-04-15T17:20:09.000Z","ComponentName":"ServiceChannel","RecordId":"0N98X001200Gvv3SAC"}]} OmniSupervisorConfig XYZ 2024-04-16T16:17:37.000Z 0Q27X000000KyrESAS {"array":[{"LastmodifiedBy":"XYZ","ModifiedDate":"2024-04-16T16:17:37.000Z","ComponentName":"OmniSupervisorConfig","RecordId":"0Q27X000000KyrESAS"}]} LiveChatButton XYZ 2024-04-16T16:31:35.000Z 5638X000000Xw55QAC {"array":[{"LastmodifiedBy":"XYZ","ModifiedDate":"2024-04-16T16:31:35.000Z","ComponentName":"LiveChatButton","RecordId":"5638X000000Xw55QAC"}]} Here is an emulation for you to play with and compare with real data   | makeresults | eval _raw="message: Updated Components { \"servicechannel\": [ { \"LastmodifiedBy\": \"XYZ\", \"ModifiedDate\": \"2024-04-15T17:20:09.000Z\", \"ComponentName\": \"ServiceChannel\", \"RecordId\": \"0N98X001200Gvv3SAC\" } ], \"omnisupervisorconfig\": [ { \"LastmodifiedBy\": \"XYZ\", \"ModifiedDate\": \"2024-04-16T16:17:37.000Z\", \"ComponentName\": \"OmniSupervisorConfig\", \"RecordId\": \"0Q27X000000KyrESAS\" } ], \"livechatbutton\": [ { \"LastmodifiedBy\": \"XYZ\", \"ModifiedDate\": \"2024-04-16T16:31:35.000Z\", \"ComponentName\": \"LiveChatButton\", \"RecordId\": \"5638X000000Xw55QAC\" } ] }"   ... View more

Assuming Invetory is spelled (in)correctly, you could try this - the rex at the end is required because this date has an embedded space and it is the last field in the message If the fields were re-ordered or an extra field was in the message (without an embedded space),  then the rex would not be required The problem is less embedded space, more lack of embedded quotation marks/proper field separator.  It is semantically more pleasing to fix structure with rex than using rex to extract one data snippet when most are extracted with extract command. (But if you have any influence on developers, beg them to add quotation marks - more on this later.)   | rex field=message mode=sed "s/Date=/&\"/ s/$/\"/" | rename message as _raw | extract   It would give you the same result like CPWRemaining CPWTotal EASRemaining EAStatal InvetoryDate SEQRemaining SEQTotal VRSRemaining VRSTotal id severity 5612 749860 15 1062804 4/16/2024 7:34:25 PM 32746 1026137 0 238 0 Information About feedback to developers.  @ITWhisperer gave one option by taking advantage of a side effect/gem feature) from Splunk's extract command by adding a comma at the end of every key-value pair.  They do not have to swap order, but also by simply adding a literal comma after each value, like this:   {"id":"0","severity":"Information","message":"CPWTotal=749860, SEQTotal=1026137, EASTotal=1062804, VRSTotal=238, CPWRemaining=5612, SEQRemaining=32746, EASRemaining=15, VRSRemaining=0, InvetoryDate=4/16/2024 7:34:25 PM,"}   A more robust fix (that does not rely on Splunk's "generosity") is to properly quote the value.  Any language can extract that without the programmer's attention.   {"id":"0","severity":"Information","message":"CPWTotal=749860, SEQTotal=1026137, EASTotal=1062804, VRSTotal=238, CPWRemaining=5612, SEQRemaining=32746, EASRemaining=15, VRSRemaining=0, InvetoryDate=\"4/16/2024 7:34:25 PM\""}   The logic should be simple enough: Numeric data, no quote, string data, quote.   ... View more

I understand look and feel.  You can search https://ideas.splunk.com/ to see if someone is asking for this parity; if not, you can submit an idea for it. ... View more

This ask could have two interpretations.  The simple one is extremely simple.  Let me give you the formula first. | inputlookup pod_name_lookup where NOT [search index=abc sourcetype=kubectl | eval pod_name = mvindex(split(pod_name, "-"), 0) | stats values(pod_name) as pod_name] | stats dc(pod_name) as count values(pod_name) as pod_name by importance Your mock data will give you something like pod_name importance podc critical   Now, my interpretations of your use case.  First, I think your lookup table actually look like this, with pod_name as column name instead of pod_name_lookup.  Is this correct? pod_name importance poda non-critical podb critical podc critical I call the lookup name "pod_name_lookup".  Second, I interpret the "pod_name" column in the lookup table, mocked up as "poda", "podb", "podc", to be the first part of running pod names (mocked up as "poda-284489-cs834" and "podb-834hgv8-cn28s") that does not contain a dash.  If this is not how the two names match, you will need to either make the transformation, or come up with more accurate mockups. Now, I am assuming that 'importance" in lookup and events match exactly.  If you want to detect the discrepancies in "importance" as well, the search will be more complicated. ... View more

addtotals is the wrong tool for what you wanted. (It is really bad to use screenshot to illustrate text output.  Always use text unless you are illustrating a visual effect.)  addcoltotals is what you need. | addcoltotals labelfield=MOP ... View more

Note you need to place source=OMITTED host="SERVER1" OR host="SERVER2" in parentheses; alternatively use IN operator.  Finding difference should not be that complicated. index=_internal earliest=-15mindex=OMITTED source=OMITTED host IN ("SERVER1", "SERVER2") | stats max(Value) as Value by host | stats max(Value) as max_of_two min(Value) as min_of_two | where max_of_two / min_of_two > 0.75 However, your OP says you want timechart.  That's why @richgalloway includes _time in groupby in that first stats.  But you can substitute the first stats with timechart to simplify this, then use the same technique in every row to find percent deviation. index=_internal earliest=-15mindex=OMITTED source=OMITTED host IN ("SERVER1", "SERVER2") | timechart span=1d max(Value) as Value by host | eventstats max(Value) as max_of_two min(Value) as min_of_two | where max_of_two / min_of_two > 0.75   ... View more

In addition to possible additions such as "Timeout Errors", you case requires an additional consideration.  Using case function (or use timechart command) will count each 5-minute interval separately in disagreement with the semantics of "last 10 min vol" and "last 15min vol".  These terms are cumulative.  Any event in "last 5min" must also be counted in "last 10min" and "last 15min". Here is a semantic implementation; mvappend satisfies both considerations.   | foreach 5min 10min 15min [eval header = mvappend(header, if(_time - relative_time(now(), "-<<FIELD>>") > 0, "Last <<FIELD>> Vol", null()))] | eval header = mvappend(header, if(log_level == "ERROR", "Timeout Errors", null()), ``` This is error emulation. Use real condition(s) ``` if(someother > 0, "Some other count", null())) | chart count OVER Transaction BY header | table Transaction "Last 5min Vol" "Last 10min Vol" "Last 15min Vol" "Timeout Errors"   Note the last mvappend evaluation is emulation.  Use your real condition.  Here is a data emulation I use to test the above code; one emulated error condition gives non-zero output.   index=_internal earliest=-15m | rename sourcetype AS Transaction ``` data emulation above; some events have leg_level "ERROR" ```   This emulation gives the counts like the following: Transaction Last 5min Vol Last 10min Vol Last 15min Vol Timeout Errors dbx_health_metrics 1370 2055 2740 0 dbx_server 0 0 4 0 splunk_python 10 15 20 0 splunk_search_messages 4 2 2 2 splunkd 4736 7390 9833 2779 splunkd_access 388 600 787 0 splunkd_ui_access 244 134 148 0 ... View more

@marnall is correct.  I don't believe that Dashboard Studio supports checkbox but aside from look and feel, multiselect provides all functionality that checkbox does. You can also ask in this dedicated forum Dashboards & Visualizations. ... View more

Have you compared emulation with real data?  Also, really get rid of that table command which can be in the way. (You can add some formatting after you verify that outputs are satisfactory.)  Is there some real data that you can share? (Anonymize as needed but take care to preserve precise structure.)  Using emulation, the output is not zero.  Clearly, actual data is different from what you posted above. Run this:   | makeresults | eval _raw = "{\"date\": \"1/2/2022 00:12:22,124\", \"DATA\": \"[http:nio-12567-exec-44] DIP: [675478-7655a-56778d-655de45565] Data: [7665-56767ed-5454656] MIM: [483748348-632637f-38648266257d] FLOW: [NEW] { SERVICE: AAP | Applicationid: iis-675456 | ACTION: START | REQ: GET data published/data/ui } DADTA -:TIME:<TIMESTAMP> (0) 1712721546785 to 1712721546885 ms GET /v8/wi/data/*, GET data/ui/wi/load/success\", \"tags\": {\"host\": \"GTU5656\", \"insuranceid\": \"8786578896667\", \"lib\": \"app\"}}" | spath | eval _time = strptime(date, "%d/%m/%Y %H:%M:%S,%f") ``` the above emulates index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") ``` | rex field=DATA mode=sed "s/ *[\|}\]]/\"/g s/: *\[*/=\"/g" | rename DATA AS _raw | kv |search ACTION= start OR ACTION=done NOT SERVICE="null" |eval split=SERVICE.":".ACTION |timechart span=1d count by split |eval _time=strftime(_time, "%d/%m/%Y") | table _time *START *DONE   Do you get the same results as I did in the previous comment? (I do not encourage use of screenshot to show search or results, but I had already shared them in text previously. So, here you go for a screenshot.) ... View more

This is exactly what I speculated in your previous question: that your developers have left a compliant JSON, while having some structure within DATA field.  Instead of rex individual elements as if DATA is made of random text, you should utilize the structure your developers intended.  Have you tried my suggestion yesterday?   index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") | rex field=DATA mode=sed "s/ *[\|}\]]/\"/g s/: *\[*/=\"/g" | rename DATA AS _raw | kv |search ACTION= start OR ACTION=done NOT SERVICE="null" |eval split=SERVICE.":".ACTION |timechart span=1d count by split |eval _time=strftime(_time, "%d/%m/%Y") | table _time *START *DONE   (Since you are running timechart, there is no need to preserver _raw, so I omitted that.  I also don't see how your last table command could give you the result you illustrated because START and DONE are capitalized.) Your sample data (only one event) gives _time AAP:START 01/02/2022 1 11/04/2024 0 This is the data emulation including _time conversion   | makeresults | eval _raw = "{\"date\": \"1/2/2022 00:12:22,124\", \"DATA\": \"[http:nio-12567-exec-44] DIP: [675478-7655a-56778d-655de45565] Data: [7665-56767ed-5454656] MIM: [483748348-632637f-38648266257d] FLOW: [NEW] { SERVICE: AAP | Applicationid: iis-675456 | ACTION: START | REQ: GET data published/data/ui } DADTA -:TIME:<TIMESTAMP> (0) 1712721546785 to 1712721546885 ms GET /v8/wi/data/*, GET data/ui/wi/load/success\", \"tags\": {\"host\": \"GTU5656\", \"insuranceid\": \"8786578896667\", \"lib\": \"app\"}}" | spath | eval _time = strptime(date, "%d/%m/%Y %H:%M:%S,%f") ``` the above emulates index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") ```   Play with it and compare to real data.  If this doesn't work for select events, you need to post samples of those events.   ... View more

Treating structured data as pure text is doomed to be unstable.  Have you tried my suggestion of reconstructing events based on inherent structure? ... View more

Are you sure that your raw event is not a valid JSON closer to   {"date": "1/2/2022 00:12:22,124", "DATA": "[http:nio-12567-exec-44] DIP: [675478-7655a-56778d-655de45565] Data: [7665-56767ed-5454656] MIM: [483748348-632637f-38648266257d] FLOW: [NEW] { SERVICE: AAP | Applicationid: iis-675456 | ACTION: START | REQ: GET data published/data/ui } DADTA -:TIME:<TIMESTAMP> (0) 1712721546785 to 1712721546885 ms GET /v8/wi/data/*, GET data/ui/wi/load/success", "tags": {"host": "GTU5656", "insuranceid": "8786578896667", "lib": "app"}}   instead?  In other words, do you not have a field  named "DATA" already? Because the overall structure of your illustration is very much compliant. Assuming you have a field named DATA, a better strategy is trying to reconstruct a structure as your developers intended, instead of trying to extract individual tidbits as random text because your developers have clearly put in thoughts about data structure within DATA.  I would propose something like   index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") | rex field=DATA mode=sed "s/ *[\|}\]]/\"/g s/: *\[*/=\"/g" | rename _raw as temp | rename DATA AS _raw | kv | rename temp as _raw   Your sample data should give you ACTION Applicationid DIP Data FLOW MIM REQ SERVICE date http tags.host tags.insuranceid tags.lib START iis-675456 675478-7655a-56778d-655de45565 7665-56767ed-5454656 NEW 483748348-632637f-38648266257d GET data published/data/ui AAP 1/2/2022 00:12:22,124 nio-12567-exec-44 GTU5656 8786578896667 app Here is an emulation that results in my hypothesized raw log:   | makeresults | eval _raw = "{\"date\": \"1/2/2022 00:12:22,124\", \"DATA\": \"[http:nio-12567-exec-44] DIP: [675478-7655a-56778d-655de45565] Data: [7665-56767ed-5454656] MIM: [483748348-632637f-38648266257d] FLOW: [NEW] { SERVICE: AAP | Applicationid: iis-675456 | ACTION: START | REQ: GET data published/data/ui } DADTA -:TIME:<TIMESTAMP> (0) 1712721546785 to 1712721546885 ms GET /v8/wi/data/*, GET data/ui/wi/load/success\", \"tags\": {\"host\": \"GTU5656\", \"insuranceid\": \"8786578896667\", \"lib\": \"app\"}}" | spath ``` the above emulates index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") ```   Play with the emulation and compare with real data. Note: In the unimaginable case where your developers try really hard to mess up everybody's mind and inject semblance of JSON compliance while violating common sense, you can still apply the same principle against _raw.  Like this:   index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") ``` | rex mode=sed "s/ *[\|}\]]/\"/g s/: *\[*/=\"/g" | kv   This is what the output would look like: ACTION Applicationid DATA DIP Data FLOW MIM REQ SERVICE host START iis-675456 http= 675478-7655a-56778d-655de45565 7665-56767ed-5454656 NEW 483748348-632637f-38648266257d GET data published/data/ui AAP   Without a better structure, you won't get subnodes embedded in tags; but your original question does not seem to care about tags. Here is an emulation that resembles the actual sample you posted:   | makeresults | eval _raw = "{\"date\": \"1/2/2022 00:12:22,124\", DATA: [http:nio-12567-exec-44] DIP: [675478-7655a-56778d-655de45565] Data: [7665-56767ed-5454656] MIM: [483748348-632637f-38648266257d] FLOW: [NEW] { SERVICE: AAP | Applicationid: iis-675456 | ACTION: START | REQ: GET data published/data/ui } DADTA -:TIME:<TIMESTAMP> (0) 1712721546785 to 1712721546885 ms GET /v8/wi/data/*, GET data/ui/wi/load/success\", \"tags\": {\"host\": \"GTU5656\", \"insuranceid\": \"8786578896667\", \"lib\": \"app\"}}" ``` the above emulates index=test-index (data loaded) OR ("GET data published/data/ui" OR "GET /v8/wi/data/*" OR "GET data/ui/wi/load/success") ```   ... View more

Instead of dealing with the messiness of a natural language, it might be better to use standard notation of duration, like | fieldformat max(event.Properties.duration) = tostring('max(event.Properties.duration)', "duration") Instead of 40 mins , 30 mins or 1 hrs, you get 00:40:00, 00:30:00, 01:00:00, and so on. ... View more

| chart max('event.Properties.duration') by event.Properties.endpoint Something like this? ... View more