Do you mean to say that some periods have no data about pods? (Or rather, no data about pods with importance value "non-important".)  My initial suggestion was based on the assumption that during any given interval, there are some pods.  Now that I think about it, it is possible that that assumption is still true but some intervals may only get important pods and all non-important ones are missing. Try this index=abc sourcetype=kubectl importance=non-critical | lookup pod_list pod_name_lookup as pod_name OUTPUT pod_name_lookup | dedup pod_name | where sourcetype == "kubectl" | timechart span=1m@m values(pod_name_lookup) as pod_name_lookup values(pod_name_all) as pod_name_all | append [makeresults format=csv data="namespace, pod_name_lookup, importance ns1, kafka-*, critical ns1, apache-*, critical ns2, grafana-backup-*, non-critical ns2, someapp-*, non-critical" | where importance = "non-critical" ``` subsearch thus far emulates | inputlookup pod_list where importance = non-critical ``` | rename pod_name_lookup as pod_name_all] | eventstats values(pod_name_all) as pod_name_all | eval missing = if(isnull(pod_name_all), pod_name_all, mvappend(missing, mvmap(pod_name_all, if(pod_name_all IN (pod_name_lookup), null(), pod_name_all)))) | where isnotnull(missing) | timechart span=1m@m count by missing Exactly the same idea, just fill intervals with no non-important pod groups.  Those intervals will see all pod groups marked as missing. ... View more

@ITWhisperer and @bowesmana gave some good ideas for obtaining streams of events.  I want to note that " if event has "PROD*" in field name I need to get the value" can have different meanings depending on what you want to do with the keys and values.  If all you want is to list all values of each key, it can be as simple as   index=myIndex sourcetype=mySourceType | stats values(Prod*) as Prod*   ... View more

Thanks, how can we club the both into one to show count based on the two conditions What do you mean "based on the two conditions?"  Your original question simply says  to show in pie chart. But i am getting values as other The answer to this is: You cannot have a pie chart with two columns.  If you "getting values as other" is not the problem, what is?  Illustrate your data - in text (anonymize as needed), illustrate desired result - normally I'd say in text but in this case, a mockup graphic piechart could work, then, explain the logic to derive the desired results from illustrated data in plain language without SPL.  These are three essential ingredients of an answerable question. ... View more

This is a data analytics forum.  So, you cannot just say "I know is missing" without data to substantiate.  My mock data actually includes conditions where a group of pods are missing in both current interval and previous intervals. They are shown as missing in all intervals in which they are missing in the chart screenshot.  If you need concrete help, always post sample data that will demonstrate all features necessary. (Anonymize as needed.) Speaking of pod groups, you still haven't confirmed whether it is the pod groups you are trying to mark.  As I said, there is no logic that will support detecting missing of individual instances of any pod by using lookup table with wildcards. ... View more

Your last stats command outputs two columns LastRunTime_Count and NA_Count.  Pie chart can only use one column.  Can you illustrate your intentions with column output and describe how a pie chart can depict both? ... View more

As I always say, what search is best data analytics solution depends on data.  While this is as true as what @PickleRick explained in general, it is even more true with an ambiguous case as yours.  A discussion last year has showed me possibilities that I hadn't known before.  But whether it will help your use case depends on a lot of things.  So, let me first put out some qualifiers that immediately come to mind.  There can be many others. Does every field of interest appear in every event in which sourcetype_1_primary, sourcetype_2_primary, or sourcetype_3_primary is present? Are sourcetype_1_primary, sourcetype_2_primary, and sourcetype_3_primary already extracted at search time, i.e., your <initial search> does not have to extract any of them? Gain from such optimization also depends on how many calculations are to be performed between index search and stats. This is not to say that failing these qualifiers will preclude potential benefits from similar strategies, but the following is based on them. The idea is to limit search intervals using subsearches.  For this to work, of course, employed subsearches must be extremely light.  Hence tstats.  Here is a little demonstration. original with time filters index=_introspection component=* earliest=-4h | stats latest(*) as * by component index=_introspection component=* earliest=-4h     [tstats max(_time) as latest where index=_introspection earliest=-4h     by component index     | eval earliest = latest - 0.1, latest = latest + 0.1] | stats latest(*) as * by component I tested them on a standalone instance on my laptop.  That is to say there are few events (only 10 components); instead of 0.1s shifts, I use 1s. Even so, the baseline is extremely unstable, ranging from 0.76s to 1.8s.  The biggest gain I saw was from 1.8s to 0.6s.  Smaller gains were like from 0.75s to 0.68s. Back to your correlation search.  Assuming your <initial search> is a combined search, try something like this:   (sourcetype=sourcetype_1 sourcetype_1_primary=* [tstats max(_time) as latest where sourcetype=sourcetype_1 by sourcetype_1_primary | eval earliest = latest - 0.1, latest = latest + 0.1]) OR (sourcetype=sourcetype_2 sourcetype_2_primary [tstats max(_time) as latest where sourcetype=sourcetype_2 by sourcetype_2_primary | eval earliest = latest - 0.1, latest = latest + 0.1]) OR (sourcetype=sourcetype_3 sourcetype_3_primary=* [tstats max(_time) as latest where sourcetype=sourcetype_3 by sourcetype_3_primary | eval earliest = latest - 0.1, latest = latest + 0.1]) | fields _time, xxx, xxx, <pick your required fields> | eval coalesced_primary_key=coalesce(sourcetype_1_primary, sourcetype_2_primary, sourcetype_3_primary) | stats latest(*) AS * by coalesced_primary_key   ... View more

The problem is not in use of case, but in regex you applied. (I think this very same problem was discussed recently.  Is this another homework question?)  There is an unnecessary asterisk (*) at the end of several expressions.  But that's not necessarily a real problem.  There is also a code choice of case vs if; the latter would be more expressive and concise in your use case.  But that's not a problem, either. The problem is that the regex's probably do not match data.  For volunteers to help you, you need to post output from index=mulesoft environment=DEV applicationName="Test" |stats values(content.FileName) as Filename1 values(content.ErrorMsg) as errormsg values(content.Error) as error values(message) as message values(priority) as priority min(timestamp) AS Logon_Time, max(timestamp) AS Logoff_Time BY correlationId (Anonymize as needed.)  If you ask a data analytics question, you need to illustrate data. ... View more

You haven't answered my key questions about data.  Is there is a data ingestion problem that causes corrupt JSON snippet? (The data in your original illustration is NOT compliant.)  Do you have an "event" field from Splunk?  If yes, can you post an example? (Anonymize as needed.)  Can you post corrected raw event? (Anonymize as needed.) Without correct data, you cannot expect any good result. ... View more

Maybe you can clarify the use case more?  For example, how do data and model enter Splunk?  Assuming that data are in one set of ingested events (and that your model is about time series), are the predictions also in some ingested events?  Or are predictions in some sort of data table?  Or is the model a prescribed mathematical formula from which Splunk is expected to calculate predictions? R2 is nothing but mathematics.  Splunk is not bad at math.  But no, Splunk doesn't have built-in function or command for this. Another possible route is Splunk Machine Learning Tool Kit.  Even though your problem is perhaps not machine learning, mathematics are similar enough. ... View more

Let this be a lesson for all who ask questions: Illustrate/explain your data (anonymize as needed), your desired result, and explain the logic between data and desired result in plain language without SPL.  SPL should be after all the explanations, before illustrating the actual result from SPL, then explain why that result is different from desired one if it is not painfully obvious. Secondly, posting SPL without formatting discourages volunteers.  Third, SPL (and raw data) is best illustrated in code box.  Let me help so other volunteers do not have to do the hard work.     index=hum_stg_app "msg.OM_MsgType"=REQUEST msg.OM_Body.header.transactionId=* "msg.service_name"="fai-np-notification" "msg.OM_Body.header.templateType"=vsf_device_auth_otp_template "msg.OM_Body.header.channelType{}"=sms "msg.OM_Body.header.organization"=VSF | rename msg.OM_Body.header.transactionId as transactionId | eval lenth=len(transactionId) | sort 1000000 _time | dedup transactionId _time | search lenth=40 | rename _time as Time1 | eval Request_time=strftime(Time1,"%y-%m-%d %H:%M:%S") | stats count by Time1 transactionId Request_time | appendcols [| search index=hum_stg_app earliest=-30d fcr-np-sms-gateway "msg.service_name"="fcr-np-sms-gateway" "msg.TransactionId"=* "msg.NowSMSResponse"="{*Success\"}" | rename "msg.TransactionId" as transactionId_request | sort 1000000 _time | dedup transactionId_request _time | eval Time=case(like(_raw,"%fcr-np-sms-gateway%"),_time) | eval lenth=len(transactionId_request) | search lenth=40 | dedup transactionId_request | stats count by transactionId_request Time ] | eval Transaction_Completed_time=strftime(Time,"%y-%m-%d %H:%M:%S") | eval Time_dif=Time-Time1 | eval Time_diff=(Time_dif)/3600 | fields transactionId transactionId_request Request_time Transaction_Completed_time count Time_diff Request_time Time Time1     I took the pain to reverse engineer your intentions.  One thing I cannot understand is why you expect appendcols to not misalign transactionId between request and response. (I am quite convinced that Time_diff is only meaningful only when transactionId and transactionId_request match when there is such a field name such as transactionId.)  Additionally, using subsearch in the same dataset should be used only as a last resort.  This type of transaction-based calculations do not warrant such use. Let me try mind-reading a bit and state the goal of your search: Calculate the difference between the time request is send and the time response indicates completion for the same transactionId.  To do this, simply search both the request event and completion event in one search, then do a stats to find time range, the earliest (request) time and the latest (completion) time.  Like this index=hum_stg_app (("msg.OM_MsgType"=REQUEST msg.OM_Body.header.transactionId=* "msg.service_name"="fai-np-notification" "msg.OM_Body.header.templateType"=vsf_device_auth_otp_template "msg.OM_Body.header.channelType{}"=sms "msg.OM_Body.header.organization"=VSF) OR (fcr-np-sms-gateway "msg.service_name"="fcr-np-sms-gateway" "msg.TransactionId"=* "msg.NowSMSResponse"="{*Success\"}")) | eval transactionId = coalesce('msg.OM_Body.header.transactionId', 'msg.transactionId') | eval lenth=len(transactionId) | sort 1000000 _time | dedup transactionId _time | search lenth=40 | stats range(_time) as Time_diff min(_time) as Request_time max(_time) as Transaction_Completed_time by transactionId | eval Time_diff=Time_diff/3600 Two notes: I see you inserted earliest=-30d in the sub search (for completion message).  I do not know how that value is relative to earliest in the main search (for request message), so the above didn't adjust for that.  If anything, I assume that the request message has to be earlier, so the search window would necessarily be larger (or equal). Between Request_time (min) and Transaction_Completed_time (max), only one is necessary because Time_diff is already calculated by range.  I put both there to validate that range is not negative. (range function will always return positive number, so before taking one of those times out, do some testing.) ... View more

Let me first point out that you can only determine if a group of pods as denoted in pod_name_lookup is completely absent (missing), not any individual pod_name.  As such, your "timechart" can only have values 1 and 0 for each missing pod_name_lookup.  Second, I want to note that calculations to fill null importance values is irrelevant to the problem in hand, therefore I will ignore them. The way to think through a solution is as follows: You want to populate a field that contains all non-critical pod_name_lookup values in every event so you can compare with running ones in each time interval. (Hint: eventstats.)  In other words, if you have these pods _time pod_name sourcetype 2024-05-08 01:42:10 apache-12 kubectl 2024-05-08 01:41:58 apache-2 kubectl 2024-05-08 01:41:46 kakfa-8 kubectl 2024-05-08 01:41:00 apache-13 kubectl 2024-05-08 01:40:52 someapp-6 kubectl 2024-05-08 01:39:40 grafana-backup-11 kubectl 2024-05-08 01:39:34 apache-4 kubectl 2024-05-08 01:39:32 kafka-6 kubectl 2024-05-08 01:39:26 someapp-2 kubectl 2024-05-08 01:38:16 apache-12 kubectl 2024-05-08 01:38:10 grafana-backup-6 kubectl and pod_list lookup contains the following importance namespace pod_name_lookup critical ns1 kafka-* critical ns1 apache-* non-critical ns2 grafana-backup-* non-critical ns2 someapp-* (As you can see, I added "someapp-*" because in your illustration, only one app is "non-critical".  This makes data nontrivial.) You will want to produce an intermediate table like this (please ignore the time interval differences just focus on material fields): _time pod_name_lookup pod_name_all 2024-05-08 01:35:00     2024-05-08 01:36:00 apache-* grafana-backup-* grafana-backup-* someapp-* 2024-05-08 01:37:00 kafka-* someapp-* grafana-backup-* someapp-* 2024-05-08 01:38:00 apache-* grafana-backup-* grafana-backup-* someapp-* 2024-05-08 01:39:00 apache-* someapp-* grafana-backup-* someapp-* 2024-05-08 01:40:00 apache-* kakfa-* grafana-backup-* someapp-* (This illustration assumes that you are looking for missing pods in each calendar minute; I know this is ridiculous, but it is easier to emulate.)  From this table, you can calculate which value(s) in pod_name_all is/are missing from pod_name_lookup. (Hint: mvmap can be an easy method.) In SPL, this thought process can be implemented as   index=abc sourcetype=kubectl importance=non-critical | lookup pod_list pod_name_lookup as pod_name OUTPUT pod_name_lookup | dedup pod_name | append [inputlookup pod_list where importance = non-critical | rename pod_name_lookup as pod_name_all] | eventstats values(pod_name_all) as pod_name_all | where sourcetype == "kubectl" | timechart span=1h@h values(pod_name_lookup) as pod_name_lookup values(pod_name_all) as pod_name_all | eval missing = mvappend(missing, mvmap(pod_name_all, if(pod_name_all IN (pod_name_lookup), null(), pod_name_all))) | where isnotnull(missing) | timechart span=1h@h count by missing   In the above, I changed time bucket to 1h@h  (as opposed to 1m@m used in illustrations).  You need to change that to whatever suits your needs. Here is an emulation used to produce the above tables and this chart:   | makeresults format=csv data="_time, pod_name 10,apache-12 22,apache-2 34,kakfa-8 80,apache-13 88,someapp-6 160,grafana-backup-11 166,apache-4 168,kafka-6 174,someapp-2 244,apache-12 250,grafana-backup-6" | eval _time = now() - _time | eval sourcetype = "kubectl", importance = "non-critical" | eval pod_name_lookup = replace(pod_name, "\d+", "*") ``` the above emulates index=abc sourcetype=kubectl importance=non-critical | lookup pod_list pod_name_lookup as pod_name OUTPUT pod_name_lookup | dedup pod_name ``` | append [makeresults format=csv data="namespace, pod_name_lookup, importance ns1, kafka-*, critical ns1, apache-*, critical ns2, grafana-backup-*, non-critical ns2, someapp-*, non-critical" | where importance = "non-critical" ``` subsearch thus far emulates | inputlookup pod_list where importance = non-critical ``` | rename pod_name_lookup as pod_name_all] | eventstats values(pod_name_all) as pod_name_all | where sourcetype == "kubectl" | timechart span=1m@m values(pod_name_lookup) as pod_name_lookup values(pod_name_all) as pod_name_all | eval missing = mvappend(missing, mvmap(pod_name_all, if(pod_name_all IN (pod_name_lookup), null(), pod_name_all))) | where isnotnull(missing) | timechart span=1m@m count by missing     ... View more

First, thanks for posting data in text.  Second, it's a huge risk posting text data without code box.  See how many smily faces you sprinkled all over.  Let me clean up for you here:     event": "{\"eventVersion\":\"1.08\",\"userIdentity\":{\"type\":\"AssumedRole\",\"principalId\":\"AROAXYKJUXCU7M4FXD7ZZ:redlock\",\"arn\":\"arn:aws:sts::533267265705:assumed-role/PrismaCloudRole-804603675133320192/redlock\",\"accountId\":\"533267265705\",\"accessKeyId\":\"ASIAXYKJUXCUSTP25SUE\",\"sessionContext\":{\"sessionIssuer\":{\"type\":\"Role\",\"principalId\":\"AROAXYKJUXCU7M4FXD7ZZ\",\"arn\":\"arn:aws:iam::533267265705:role/PrismaCloudRole-804603675133320192\",\"accountId\":\"533267265705\",\"userName\":\"PrismaCloudRole-804603675133320192\"},\"webIdFederationData\":{},\"attributes\":{\"creationDate\":\"2024-05-03T00:53:45Z\",\"mfaAuthenticated\":\"false\"}}},\"eventTime\":\"2024-05-03T04:09:07Z\",\"eventSource\":\"autoscaling.amazonaws.com\",\"eventName\":\"DescribeScalingPolicies\",\"awsRegion\":\"us-west-2\",\"sourceIPAddress\":\"13.52.105.217\",\"userAgent\":\"Vert.x-WebClient/4.4.6\",\"requestParameters\":{\"maxResults\":10,\"serviceNamespace\":\"cassandra\"},\"responseElements\":null,\"additionalEventData\":{\"service\":\"application-autoscaling\"},\"requestID\":\"ef12925d-0e9a-4913-8da5-1022cfd15964\",\"eventID\":\"a1799eeb-1323-46b6-a964-efd9b2c30a8a\",\"readOnly\":true,\"eventType\":\"AwsApiCall\",\"managementEvent\":true,\"recipientAccountId\":\"533267265705\",\"eventCategory\":\"Management\",\"tlsDetails\":{\"tlsVersion\":\"TLSv1.3\",\"cipherSuite\":\"TLS_AES_128_GCM_SHA256\",\"clientProvidedHostHeader\":\"application-autoscaling.us-west-2.amazonaws.com\"}}"}     Third, and this is key.  Are you sure that's the true form of a complete event?  For one thing, it seems that there is a missing opening curly bracket ({) and a missing double quotation mark (") before the entire snippet.   If I am correct that you just forget to include the opening bracket and opening question mark, i.e., your real events look like     {"event": "{\"eventVersion\":\"1.08\",\"userIdentity\":{\"type\":\"AssumedRole\",\"principalId\":\"AROAXYKJUXCU7M4FXD7ZZ:redlock\",\"arn\":\"arn:aws:sts::533267265705:assumed-role/PrismaCloudRole-804603675133320192/redlock\",\"accountId\":\"533267265705\",\"accessKeyId\":\"ASIAXYKJUXCUSTP25SUE\",\"sessionContext\":{\"sessionIssuer\":{\"type\":\"Role\",\"principalId\":\"AROAXYKJUXCU7M4FXD7ZZ\",\"arn\":\"arn:aws:iam::533267265705:role/PrismaCloudRole-804603675133320192\",\"accountId\":\"533267265705\",\"userName\":\"PrismaCloudRole-804603675133320192\"},\"webIdFederationData\":{},\"attributes\":{\"creationDate\":\"2024-05-03T00:53:45Z\",\"mfaAuthenticated\":\"false\"}}},\"eventTime\":\"2024-05-03T04:09:07Z\",\"eventSource\":\"autoscaling.amazonaws.com\",\"eventName\":\"DescribeScalingPolicies\",\"awsRegion\":\"us-west-2\",\"sourceIPAddress\":\"13.52.105.217\",\"userAgent\":\"Vert.x-WebClient/4.4.6\",\"requestParameters\":{\"maxResults\":10,\"serviceNamespace\":\"cassandra\"},\"responseElements\":null,\"additionalEventData\":{\"service\":\"application-autoscaling\"},\"requestID\":\"ef12925d-0e9a-4913-8da5-1022cfd15964\",\"eventID\":\"a1799eeb-1323-46b6-a964-efd9b2c30a8a\",\"readOnly\":true,\"eventType\":\"AwsApiCall\",\"managementEvent\":true,\"recipientAccountId\":\"533267265705\",\"eventCategory\":\"Management\",\"tlsDetails\":{\"tlsVersion\":\"TLSv1.3\",\"cipherSuite\":\"TLS_AES_128_GCM_SHA256\",\"clientProvidedHostHeader\":\"application-autoscaling.us-west-2.amazonaws.com\"}}"}     you would have gotten a field "event" containing the following value     {"eventVersion":"1.08","userIdentity":{"type":"AssumedRole","principalId":"AROAXYKJUXCU7M4FXD7ZZ:redlock","arn":"arn:aws:sts::533267265705:assumed-role/PrismaCloudRole-804603675133320192/redlock","accountId":"533267265705","accessKeyId":"ASIAXYKJUXCUSTP25SUE","sessionContext":{"sessionIssuer":{"type":"Role","principalId":"AROAXYKJUXCU7M4FXD7ZZ","arn":"arn:aws:iam::533267265705:role/PrismaCloudRole-804603675133320192","accountId":"533267265705","userName":"PrismaCloudRole-804603675133320192"},"webIdFederationData":{},"attributes":{"creationDate":"2024-05-03T00:53:45Z","mfaAuthenticated":"false"}}},"eventTime":"2024-05-03T04:09:07Z","eventSource":"autoscaling.amazonaws.com","eventName":"DescribeScalingPolicies","awsRegion":"us-west-2","sourceIPAddress":"13.52.105.217","userAgent":"Vert.x-WebClient/4.4.6","requestParameters":{"maxResults":10,"serviceNamespace":"cassandra"},"responseElements":null,"additionalEventData":{"service":"application-autoscaling"},"requestID":"ef12925d-0e9a-4913-8da5-1022cfd15964","eventID":"a1799eeb-1323-46b6-a964-efd9b2c30a8a","readOnly":true,"eventType":"AwsApiCall","managementEvent":true,"recipientAccountId":"533267265705","eventCategory":"Management","tlsDetails":{"tlsVersion":"TLSv1.3","cipherSuite":"TLS_AES_128_GCM_SHA256","clientProvidedHostHeader":"application-autoscaling.us-west-2.amazonaws.com"}}     (By the way, event should be available whether or not you have KV_MODE=json, whether or not you have index_extraction=JSON.)  As you can see, this value is a compliant JSON.  All you need to do is to feed this field to spath.     | spath input=event     This way, if my speculation about missing bracket and quotation mark is correct, the sample you posted should give the following fields and values field name field value additionalEventData.service application-autoscaling awsRegion us-west-2 eventCategory Management eventID a1799eeb-1323-46b6-a964-efd9b2c30a8a eventName DescribeScalingPolicies eventSource autoscaling.amazonaws.com eventTime 2024-05-03T04:09:07Z eventType AwsApiCall eventVersion 1.08 managementEvent true readOnly true recipientAccountId 533267265705 requestID ef12925d-0e9a-4913-8da5-1022cfd15964 requestParameters.maxResults 10 requestParameters.serviceNamespace cassandra responseElements null sourceIPAddress 13.52.105.217 tlsDetails.cipherSuite TLS_AES_128_GCM_SHA256 tlsDetails.clientProvidedHostHeader application-autoscaling.us-west-2.amazonaws.com tlsDetails.tlsVersion TLSv1.3 userAgent Vert.x-WebClient/4.4.6 userIdentity.accessKeyId ASIAXYKJUXCUSTP25SUE userIdentity.accountId 533267265705 userIdentity.arn arn:aws:sts::533267265705:assumed-role/PrismaCloudRole-804603675133320192/redlock userIdentity.principalId AROAXYKJUXCU7M4FXD7ZZ:redlock userIdentity.sessionContext.attributes.creationDate 2024-05-03T00:53:45Z userIdentity.sessionContext.attributes.mfaAuthenticated false userIdentity.sessionContext.sessionIssuer.accountId 533267265705 userIdentity.sessionContext.sessionIssuer.arn arn:aws:iam::533267265705:role/PrismaCloudRole-804603675133320192 userIdentity.sessionContext.sessionIssuer.principalId AROAXYKJUXCU7M4FXD7ZZ userIdentity.sessionContext.sessionIssuer.type Role userIdentity.sessionContext.sessionIssuer.userName PrismaCloudRole-804603675133320192 userIdentity.type AssumedRole However, if your raw events truly miss the opening bracket and opening quotation mark, you need to examine your ingestion process and fix that.  No developer will knowingly omit those.   Temporarily, you can use SPL to "fix" the omission and extract data, like     | eval _raw = "{\"" . _raw | spath | spath input=event     But this is not a real solution.  Bad ingestion can do many other damage. Lastly, here is an emulation you can play with an compare with real data     | makeresults | eval _raw = "{\"event\": \"{\\\"eventVersion\\\":\\\"1.08\\\",\\\"userIdentity\\\":{\\\"type\\\":\\\"AssumedRole\\\",\\\"principalId\\\":\\\"AROAXYKJUXCU7M4FXD7ZZ:redlock\\\",\\\"arn\\\":\\\"arn:aws:sts::533267265705:assumed-role/PrismaCloudRole-804603675133320192/redlock\\\",\\\"accountId\\\":\\\"533267265705\\\",\\\"accessKeyId\\\":\\\"ASIAXYKJUXCUSTP25SUE\\\",\\\"sessionContext\\\":{\\\"sessionIssuer\\\":{\\\"type\\\":\\\"Role\\\",\\\"principalId\\\":\\\"AROAXYKJUXCU7M4FXD7ZZ\\\",\\\"arn\\\":\\\"arn:aws:iam::533267265705:role/PrismaCloudRole-804603675133320192\\\",\\\"accountId\\\":\\\"533267265705\\\",\\\"userName\\\":\\\"PrismaCloudRole-804603675133320192\\\"},\\\"webIdFederationData\\\":{},\\\"attributes\\\":{\\\"creationDate\\\":\\\"2024-05-03T00:53:45Z\\\",\\\"mfaAuthenticated\\\":\\\"false\\\"}}},\\\"eventTime\\\":\\\"2024-05-03T04:09:07Z\\\",\\\"eventSource\\\":\\\"autoscaling.amazonaws.com\\\",\\\"eventName\\\":\\\"DescribeScalingPolicies\\\",\\\"awsRegion\\\":\\\"us-west-2\\\",\\\"sourceIPAddress\\\":\\\"13.52.105.217\\\",\\\"userAgent\\\":\\\"Vert.x-WebClient/4.4.6\\\",\\\"requestParameters\\\":{\\\"maxResults\\\":10,\\\"serviceNamespace\\\":\\\"cassandra\\\"},\\\"responseElements\\\":null,\\\"additionalEventData\\\":{\\\"service\\\":\\\"application-autoscaling\\\"},\\\"requestID\\\":\\\"ef12925d-0e9a-4913-8da5-1022cfd15964\\\",\\\"eventID\\\":\\\"a1799eeb-1323-46b6-a964-efd9b2c30a8a\\\",\\\"readOnly\\\":true,\\\"eventType\\\":\\\"AwsApiCall\\\",\\\"managementEvent\\\":true,\\\"recipientAccountId\\\":\\\"533267265705\\\",\\\"eventCategory\\\":\\\"Management\\\",\\\"tlsDetails\\\":{\\\"tlsVersion\\\":\\\"TLSv1.3\\\",\\\"cipherSuite\\\":\\\"TLS_AES_128_GCM_SHA256\\\",\\\"clientProvidedHostHeader\\\":\\\"application-autoscaling.us-west-2.amazonaws.com\\\"}}\"}" | spath ``` data emulation above ``` | spath input=event       ... View more

It is very unclear what you mean by "the first one that shows".  Your screenshot shows that your input contains several JSON arrays data.events[], data.initiate_state[], data.initiate_state[].community[], data.initiate_state[].path[], etc. (It is important to illustrate raw JSON data, not Splunk's "beautified view", much less screenshot of "beautified view".  You can reveal raw data by clicking "Show as raw text" in search window.  Anonymize as needed.) I am also curious what is the use case to only wanting/needing "the first one that shows" from a data structure that is meant to contain multiple values?  Are other elements in the array not meaningful?  In a JSON array, every element is assumed to be equally weighed semantically.  How do you determine that "the first" is significant and the rest is not?  If there is truly some semantic insignificance of the rest of an array, you should exert every bit of your influence on developers to restructure data so you don't have bad semantics.  If you are uncertain, you should consult developers/manuals to clarify how data should be used. This much said, it is still unclear what is the meaning of "first one that shows."  Array data.initiate_state[].path[] is nested in array data.initiate_state[].  Do you want "first one that shows" in every element of data.initiate_state[]?  Of do you want "first one that shows" in data.initiate_state[].path[] in the "first one that shows" in data.initiate_state[]? ... View more

is there any other way of handling json content for using rex command which would be much easier. although my request is not completely in a json format. You must understand why @PickleRick and I keep telling you not to try using rex to handle structured data like JSON: rex is the wrong tool because syntax is not bound by format in JSON.  The same semantics can be expressed by a million variants of format while conforming to the same syntax. {"ka":"va","kb":"vb"} is exactly the same as {"kb":"vb","ka":"va"}.  Any rex you develop will always be instable.  By insisting on using regex, i.e., treating structured data as pure text, you are just reinforcing some bad habit that will inhibit your abilities in the future. ... View more

The way I read your premise, this sounds like a transaction logic.  So, let me first clarify your use case. You data look like _time id message state 1969-12-31 16:00:00 101 executed started 1969-12-31 16:00:04 102 activity printed started 1969-12-31 16:00:09 101 null in progress 1969-12-31 16:00:10 102 null in progress 1969-12-31 16:00:18 102 none completed 1969-12-31 16:00:24 101 none completed Note I added some time interleave between 101 and 102 to make the transaction nature more obvious. (Never mind the date is from 1969; that is just for ease of emulation.)  You want to use some results like _time duration eventcount id message state 1969-12-31 16:00:04 14 3 102 activity printed completed<-in progress<-started 1969-12-31 16:00:00 24 3 101 executed completed<-in progress<-started Here, I ignored the format of the expected output in your earlier comment, just want to clarify that "state" goes through "started", "in progress", and "completed" to form a transaction for each unique "id".  Your material requirement is to obtain a single value for "message" that is NEITHER "null" nor "none".  Is this correct?  The result as illustrated here can be obtained with   | transaction id startswith="state=started" endswith="state=completed" | eval message = mvfilter(NOT message IN ("none", "null")) | eval state = mvjoin(state, "<-")   The first two commands literally implements my interpretation of your intentions.  The third line is just a visual element to make state transition obvious for each . In my mind, the above results table is sufficient, and is more representative of the problem.  But if you really want to list each event, like _time id message state 1969-12-31 16:00:00 101 executed started 1969-12-31 16:00:04 102 activity printed started 1969-12-31 16:00:09 101 executed in progress 1969-12-31 16:00:10 102 activity printed in progress 1969-12-31 16:00:18 102 activity printed completed 1969-12-31 16:00:24 101 executed completed You can either use eventstats   | eventstats values(message) as message by id| eval message = mvfilter(NOT message IN ("none", "null")) | eval message = mvfilter(NOT message IN ("none", "null"))   or streamstats as @bowesmana suggested   | streamstats values(message) as message by id| eval message = mvfilter(NOT message IN ("none", "null")) | eval message = mvfilter(NOT message IN ("none", "null"))   To emulate input, I added _time into @bowesmana's formula because it's just simpler.   | makeresults format=csv data="id,message,state,_time 101,executed,started,0 102,activity printed,started,4 101,null,in progress,9 102,null,in progress,10 102,none,completed,18 101,none,completed,24" | eval _raw = "doesn't matter" ``` mock field _raw is important for transaction ``` ``` data mockup above ```         ... View more

The First Law of asking an answerable question states: Present your dataset (anonymize as needed), illustrate desired output from illustrated dataset, explain the logic between illustrated dataset and desired output. (Without SPL.) If attempted SPL does not give desired output, also illustrate actual output (anonymize as needed), then explain its difference from desired results if it is not painfully clear. I am able to pull my AD users account information successfully except for their email addresses.  Can you explain from which source are you pulling AD info?  Your SPL only uses a lookup file.  Do you mean lookup table AD_Obj_User contains email addresses but the illustrated SPL does not output them, or your effort to populate AD_Obj_User fails to obtain email addresses from a legitimate AD source (as @deepakc speculated)? If former, what is the purpose of the SPL?  What is the content of AD_Obj_User?  What is the desired output and the logic between the content and desired output? If latter, what is the purpose of showing SPL? ... View more

Now you see the importance of illustrating data accurately.  My could only give you channel because the only data snippet I could see has channel.  Now, you can see that accountNumber is a subnode in REQUEST.body.customer, serialNumber is a subnode in REQUEST.body.equipment, while redemptionEquipmentMemory and transactionReferenceNumber are those in RESPONSE.body.model.  Your initial data snippet already established that Channel is a subnode in REQUEST.headers. All this is to say that to write the correct SPL, you need to understand data.  Before trying to render results, use SPL to help analyze data. Now that you know where in the JSON structure each of those fields lies, you can just extract each node.  But doing so usually is too laborious and not good for maintenance and enhancement.  So, I will give a more flexible code   index="wireless_retail" source="CREATE_FREEDOM.transactionlog" OPERATION="/FPC/Redemption/Redeem" | rex "REQUEST=\"(?<REQUEST>.+)\", RESPONSE=\"(?<RESPONSE>.+)\", RETRYNO" | spath input=REQUEST path=headers | spath input=REQUEST path=body output=REQUEST | spath input=RESPONSE path=body output=RESPONSE | foreach headers REQUEST RESPONSE [spath input=<<FIELD>>] ```| spath input=RESPONSE path=headers.set-cookie{} | mvexpand headers.set-cookie{}``` | foreach customer equipment model [rename <<FIELD>>.* AS *] |table accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber   This is an emulation of your sample data   | makeresults | eval _raw = "2024-05-02 23:40:22.000, ID=\"5e2276d3-7f02-7984-ad4b-e11507580872\", ACCOUNTID=\"5\", ACCOUNTNAME=\"prd\", APPLICATIONID=\"6\", APPLICATIONNAME=\"ws\", REQUEST=\"{\"body\":{\"customer\":{\"accountNumber\":\"DBC00089571590\",\"lineNumber\":\"8604338\"},\"equipment\":{\"serialNumber\":\"359938615394762\",\"grade\":\"A\"},\"redemptionDetails\":{\"redemptionDate\":\"20240502\",\"user\":\"WVMSKaul\",\"storeNumber\":\"WD227907\",\"dealerNumber\":\"2279\"}},\"headers\":{\"content-type\":\"application/json;charset=UTF-8\",\"Accept\":\"application/json;charset=UTF-8\",\"Channel\":\"6\",\"Locale\":\"en-US\",\"TransactionID\":\"65E5519B-F170-4367-AA03-54A33BA29B4E\",\"ApplicationID\":\"00000411\",\"Authorization\":\"Basic ZnJlZWRvbWNyZWF0ZTpDd0t4dGlmbGZ3ZnFaQVYydWhtUg==\"}}\", RESPONSE=\"{\"body\":{\"model\":{\"isRedeemed\":true,\"transactionReferenceNumber\":\"6200753992\",\"redeemType\":\"Original\",\"redemptionFailureReasonType\":null,\"redemptionEquipmentMake\":\"Samsung\",\"redemptionEquipmentModel\":\"Galaxy S21 FE 128GB Graphite\",\"redemptionEquipmentMemory\":\"128 GB\",\"committedPrice\":1,\"additionalFees\":0},\"code\":200,\"messages\":null,\"isSuccess\":true},\"headers\":{\"connection\":\"close\",\"content-type\":\"application/json;charset=utf-8\",\"set-cookie\":[\"AWSELB=B3A9CDE108B7A1C9F0AFA19D2F1D801BC5EA2DB758E049CA400C049FE7C310DF0BB906899FF431BCEF2EF75D94E40E95B107D7A5B122F6844BA88CEC0D864FC12E75279814;PATH=/\",\"AWSELBCORS=B3A9CDE108B7A1C9F0AFA19D2F1D801BC5EA2DB758E049CA400C049FE7C310DF0BB906899FF431BCEF2EF75D94E40E95B107D7A5B122F6844BA88CEC0D864FC12E75279814;PATH=/;SECURE;SAMESITE=None\",\"visid_incap_968152=gpkNFRF6QtKeSmDdY/9FWWUkNGYAAAAAQUIPAAAAAABmisXXPd3Y2+ulqGUibHZU; expires=Fri, 02 May 2025 07:12:03 GMT; HttpOnly; path=/; Domain=.likewize.com\",\"nlbi_968152=FnwQGi3rMWk+u+PCILjsZwAAAACniSzzxzSlwTCqfbP87/10; path=/; Domain=.likewize.com\",\"incap_ses_677_968152=2ZElDA77lnjppwgU8y9lCWUkNGYAAAAArXuktDctGDMtVtCwqfe5bw==; path=/; Domain=.likewize.com\"],\"content-length\":\"349\",\"server\":\"Jetty(9.4.45.v20220203)\"}}\", RETRYNO=\"0\", ENDPOINT=\"https://apptium.freedommobile.ca/Activation.TradeUp\", OPERATION=\"/FPC/Redemption/Redeem\", METHOD=\"POST\", CONNECTORID=\"0748a993-4566-48ae-9885-2a4dce9de585\", CONNECTORNAME=\"Likewize\", CONNECTORTYPE=\"Application\", CONNECTORSUBTYPE=\"REST\", STARTTIME=\"1714693218282\", ENDTIME=\"1714693222213\", RESPONSETIME=\"3931\", SUCCESS=\"1\", CLIENT=\"eportal-services\", CREATEDDATE=\"2024-05-02 23:40:22\", USERNAME=\"WVMSKaul@wmbd.local\", SESSIONID=\"_027c735b-30ed-472c-99e8-6d0748e5a7d9\", ACTIONID=\"5c0a6f88-5a1e-4fdc-a454-01c53fdc0b9b\", TRACKID=\"674e1eed-ba9e-429f-87fc-3b4773b7dd06\"" ``` the above emulates index="wireless_retail" source="CREATE_FREEDOM.transactionlog" OPERATION="/FPC/Redemption/Redeem" ```   The output from emulated data is accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber DBC00089571590 359938615394762 6 128 GB 6200753992 Finally, I want to illustrate the most inflexible implementation, custom extraction of the needed fields directly   index="wireless_retail" source="CREATE_FREEDOM.transactionlog" OPERATION="/FPC/Redemption/Redeem" | rex "REQUEST=\"(?<REQUEST>.+)\", RESPONSE=\"(?<RESPONSE>.+)\", RETRYNO" | spath input=REQUEST path=headers.Channel output=Channel | spath input=REQUEST path=body.customer.accountNumber output=accountNumber | spath input=REQUEST path=body.equipment.serialNumber output=serialNumber | spath input=RESPONSE path=body.model.redemptionEquipmentMemory output=redemptionEquipmentMemory | spath input=RESPONSE path=body.model.transactionReferenceNumber output=transactionReferenceNumber | table accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber   Since 8.1, you can also implement these one-to-one extractions using json_extract.   index="wireless_retail" source="CREATE_FREEDOM.transactionlog" OPERATION="/FPC/Redemption/Redeem" | rex "REQUEST=\"(?<REQUEST>.+)\", RESPONSE=\"(?<RESPONSE>.+)\", RETRYNO" | eval Channel = json_extract(REQUEST, "headers.Channel") | eval accountNumber = json_extract(REQUEST, "body.customer.accountNumber") | eval serialNumber = json_extract(REQUEST, "body.equipment.serialNumber") | eval redemptionEquipmentMemory = json_extract(RESPONSE, "body.model.redemptionEquipmentMemory") | eval transactionReferenceNumber = json_extract(RESPONSE, "body.model.transactionReferenceNumber") | table accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber     ... View more

Do you mean you have a field named REQUEST with JSON data as illustrated, and want to have data like this: field name field value body.customer.accountNumber DBC50012225699 body.customer.lineNumber 5000654224 body.equipment.grade A body.equipment.serialNumber 351643935649535 body.redemptionDetails.dealerNumber GW_STORE body.redemptionDetails.redemptionDate 20240502 body.redemptionDetails.storeNumber WCCA0105 body.redemptionDetails.user BMashiana headers.Accept application/json;charset=UTF-8 headers.ApplicationID 00000411 headers.Authorization Basic ZnJlZWRvbWNyZWF0ZTpDd0t4dGlmbGZ3ZnFaQVYydWhtUg== headers.Channel 6 headers.Locale en-US headers.TransactionID E86B7D59-B3CC-401D-977F-65218248367E headers.content-type application/json;charset=UTF-8 where header.Channel has value 6? REQUEST does not contain any array, why the complicated path?  All you need is   | spath input=REQUEST | rename headers.* AS *   Here is an emulation based on your sample data.   | makeresults | eval REQUEST="{\"body\":{\"customer\":{\"accountNumber\":\"DBC50012225699\",\"lineNumber\":\"5000654224\"},\"equipment\":{\"serialNumber\":\"351643935649535\",\"grade\":\"A\"},\"redemptionDetails\":{\"redemptionDate\":\"20240502\",\"user\":\"BMashiana\",\"storeNumber\":\"WCCA0105\",\"dealerNumber\":\"GW_STORE\"}},\"headers\":{\"content-type\":\"application/json;charset=UTF-8\",\"Accept\":\"application/json;charset=UTF-8\",\"Channel\":\"6\",\"Locale\":\"en-US\",\"TransactionID\":\"E86B7D59-B3CC-401D-977F-65218248367E\",\"ApplicationID\":\"00000411\",\"Authorization\":\"Basic ZnJlZWRvbWNyZWF0ZTpDd0t4dGlmbGZ3ZnFaQVYydWhtUg==\"}}" ``` the above emulates index="wireless_retail" source="CREATE_FREEDOM.transactionlog" OPERATION="/FPC/Redemption/Redeem" | rex "REQUEST=\"(?<REQUEST>.+)\", RESPONSE=\"(?<RESPONSE>.+)\", RETRYNO" ``` | spath input=REQUEST | rename headers.* AS * | table accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber   The output is accountNumber serialNumber Channel redemptionEquipmentMemory transactionReferenceNumber     6     Obviously I do not have RESPONSE data.  But play with it and compare with real REQUEST data. ... View more

Do you mean to obtain something like this?   field name field value Action Name Read Input Files GDrive Start Record Id Type Invoice Run Reference xx-0645-11ef-xx-xx Task Name Cash Apps PAPI applicationName ct-fin-abc-apps-papi-v1-uw2-ut applicationType PAPI applicationVersion 1.0.7 batchSize 0 businessRecordId   businessRecordType   detailText Start Reading Input Files from G drive domain CR C4E endpointSystem   environment ct-app-UAT region us-ne-2 remainingRetries 0 stage MILESTONE status SUCCESS threadName [MuleRuntime].uber.05: [ct-fin-aps-apps-papi-v1-uw2-ut].abc-apps-schedular-main-flow.BLOCKING @68f82333 timestamp 2024-04-29 16:30:08.455 totalRecords 0 tx.fileName implementation.xml tx.flow read-input-files-sub-flow txlineNumber 71 worker 0 x-transaction-id xxxx-e691-xx-91bf-xxx As @gcusello suggested, you should use built-in JSON capability to do the job, not regex.  Use rex only to extract the JSON part.  Like this   | rex "eilog.EILog:\s*(?<eilog>{.+})" | spath input=eilog | spath input=jsonRecord   This is an emulation of your sample data.  Play with it and compare with real data.   | makeresults | eval _raw = "INFO 2024-04-29 16:30:08,456 [[MuleRuntime].uber.05: [ct-fin-abc-apps-papi-v1-uw2-ut].abc-apps-schedular-main-flow.BLOCKING @68f82333] com.sfdc.it.ei.mule4.eilog.EILog: {\"worker\":\"0\",\"region\":\"us-ne-2\",\"applicationName\":\"ct-fin-abc-apps-papi-v1-uw2-ut\",\"applicationVersion\":\"1.0.7\",\"applicationType\":\"PAPI\",\"environment\":\"ct-app-UAT\",\"domain\":\"CR C4E\",\"x-transaction-id\":\"xxxx-e691-xx-91bf-xxx\",\"tx.flow\":\"read-input-files-sub-flow\",\"tx.fileName\":\"implementation.xml\",\"txlineNumber\":\"71\",\"stage\":\"MILESTONE\",\"status\":\"SUCCESS\",\"endpointSystem\":\"\",\"jsonRecord\":\"{\\n \\\"Task Name\\\": \\\"Cash Apps PAPI\\\",\\n \\\"Action Name\\\": \\\"Read Input Files GDrive Start\\\",\\n \\\"Run Reference\\\": \\\"xx-0645-11ef-xx-xx\\\",\\n \\\"Record Id Type\\\": \\\"Invoice\\\"\\n}\",\"detailText\":\"Start Reading Input Files from G drive\",\"businessRecordId\":\"\",\"businessRecordType\":\"\",\"batchSize\":\"0\",\"totalRecords\":\"0\",\"remainingRetries\":\"0\",\"timestamp\":\"2024-04-29 16:30:08.455\",\"threadName\":\"[MuleRuntime].uber.05: [ct-fin-aps-apps-papi-v1-uw2-ut].abc-apps-schedular-main-flow.BLOCKING @68f82333\"}" ``` data emulation above ```   ... View more

Also, is the "/my_search" in the example mentioned below the title of the Knowledge Object ? Not quite.  @deepakc only gave saved searches (aka reports) as an example.  "my_search" is a URL encoded string of the title.  In the example, "https://<MY_CLOUD_STACK>splunkcloud.com:8089/servicesNS/nobody/YOU_APP/saved/searches/my_search"  is one property internally known as id. Is there anyway to reassign all the Knowledge Objects owner by a specific user ? instead of transferring one Knowledge object at a time ? Yes.  To continue the example with saved searches, you can use this search to find all id's owned by the old user "old_user".     | rest /servicesNS/-/-/saved/searches/ | search eai:acl.owner = "old_user" | fields id     Example output could be (taken from owner nobody on a standard deployment) id https://127.0.0.1:8089/servicesNS/nobody/search/saved/searches/Bucket%20Merge%20Retrieve%20Conf%20Settings https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerAppEvents https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerClients https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerClientsBasic https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerClientsByMachineType https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerEventsWithError https://127.0.0.1:8089/servicesNS/nobody/SplunkDeploymentServerConfig/saved/searches/DeploymentServerGetClientsUri Then, program a script using these values to update these saved searches to new user. To update other knowledge objects, consult REST API Reference Manual, especially Knowledge endpoint descriptions to find out how to retrieve their id's by owner. (Note saved searches is described in Search endpoint descriptions instead.) Hope this helps. ... View more

Splunk ingests all input as UTF8 characters - which is a superset of ASCII.  So, the boiler-plate answer is Yes, there is a way.  Is there something you know that is in your events but you are having difficulty with?  Are you concerned about ASCII-7, ASCII-8, or something totally different?  To get specific help, you need to illustrate your input and illustrate desired output. ... View more