How to fetch the count with details.

[avii7326]

Hi All,

I have one log that is ABC and it is present in sl-sfdc api and have another log EFG that is present in sl-gcdm api now I want to see the properties and error code fields which is present in EFG log but it has many other logs coming from different apis also . I only want the log which is having the correlationId same in ABC then it should check the other log .And then I will use this regular expression to get the fields, like spath.

Currently I am using this query 

index=whcrm (

    sourcetype=xl-sfdcapi ("Create / Update Consents for gcid" OR "Failure while Create / Update Consents for gcid" OR "Create / Update Consents done")

) OR (

    sourcetype=sl-gcdm-api ("Error in sync-consent-dataFlow:")

)

| rename properties.correlationId as correlationId

| rex field=_raw "correlationId: (?<correlationId>[^\s]+)"

| eval is_success=if(match(_raw, "Create / Update Consents done"), 1, 0)

| eval is_failed=if(match(_raw, "Failure while Create / Update Consents for gcid"), 1, 0)

| eval is_error=if(match(_raw, "Error in sync-consent-dataFlow:"), 1, 0)

| stats sum(is_success) as Success_Count, sum(is_failed) as Failed_Count,

| eval Total_Consents = Success_Count + Failed_Count

| table Total_Consents, Success_Count, Failed_Count

first one is the ABC log and second is the EFG also I want to use this regular expression in between to get the details 

| rex field=message "(?<json_ext>\{[\w\W]*\})"

    | spath input=json_ext

Or there can be any other way to write the query and get the counts please help .

Thanks in Advance

[avi7326]

@ITWhisperer Can I please get you guidance.

[marnall]

If I understand correctly, you have two different log types ABC and EFG in the same index, and you want to count how many success, fail, and error events occur, but only for correlation IDs that occur in both ABC and EFG?

Assuming the field names are correct, your current query should work to count success, fail, and error events from both, though it will count events that only occur in one of the two types.

It is not clear how you would like the details (json_ext of message) to be displayed with the count of success, fail, and error events. You could do stats ... by json_ext to see the counts by json_ext, but this would only be practical if the json_ext messages are not very different.

[avi7326]

Yes you  understand correctly, I have two different log types ABC and EFG in the same index, but the sourcetype is different in both logs so the condition is when there will be error it will be calculated from the ABC log but the details which it is containing it is in EFG log that is in other sourcetype and I will also fetch the details of that log but what I want is when I got total error is ABC is 5 then when I should search the ABC and EFG together it should show me 5 errors only related to the correlationid.

I hope you understand my query from this .

[marnall]

So if there is an error seen in the ABC log, then you would like to find the details for that error in the EFG log. You would like to count the number of errors for each correlationId, so that you can later search for that correlation Id and list all of the errors that occurred along with the details message for that correlationId. Is that correct?

E.g.:

CorrelationId	Errors	Details
abcd-0001	0
abcd-0002	4	Error msg 1 Error msg 2 Error msg 3 Error msg 4
abcd-0003	1	Error msg 1
abcd-0004	2	Error msg 1 Error msg 2