Splunk to search and analyse it in logs after one hour

[bsaujla131984]

I have a requirement to search and analyse result of searches in same log file after one hour.

For example ,

Search a keyword payment with ID at 12:00 PM in log X
Search the same payment ID at 1:00 PM in log X to check if acknowledgment has been received or not.

Please if anyone has done similar thing , then kindly share.

[bsaujla131984 · Accepted solution]

I used bit differently. Declared a response type as a field instead of using steam stats and used as below:-

host=hostname source="/Log Location/" Payment ID AND JobName | rex "Payment ID[\\": ](?[^\\":])" | dedup PaymentID | table PaymentID MyField

MyField contains a response type which can be success, null, error or any.

Based on this , I will create alerts.

[woodcock]

If so, then you should click Accept here and close the question. Don't forget to UpVote anybody that helped you.

[bsaujla131984]

I was able to extract payment ID as below:-

host=hostname source="//Logfilelocation/xyz.log" Jobname AND Queuename AND Payment ID earliest=-1h@h | rex "Payment ID\\": " |

Now I want to check if there is ACKNOWLEDGEMENT OR NOACKNOWLEDGEMENT in the search for extract PaymentID in last 5 minutes.

Tried to search steamstats, however it is throwing an error.

@renjith.nair

[bsaujla131984]

@renjith.nair Can you please assist with this ?

[woodcock]

Did you try my answer below @bsaujla131984?

[bsaujla131984]

Yes I tried, however not producing results as I need...

[woodcock]

Something like this:

index=YouShouldAlwaysSpecifyAnIndex AND sourcetype=AndSourceTypeToo
| streamstats count(eval(searchmatch("your acknowledgment string here"))) AS sessionID BY paymentID
| stats count list(_raw) BY sessionID paymentID
| where count=1

[bsaujla131984]

Thanks @woodcock and all for your assistance.

[bsaujla131984]

Thanks all for your suggestions..

I need a bit more assistance as I am a basic splunk user...

My search is as below:-

host=hostname source="Log location" Jobname AND QueueName AND Payment ID | rex "{(?.+?)," | stats count as Corr ID

I would like to fetch the value of Payment ID and then check if corresponding acknowledgement has been received or not.

Please guide for this...

[bsaujla131984]

I was able to extract payment ID as below:-

host=hostname source="//Logfilelocation/xyz.log" Jobname AND Queuename AND Payment ID earliest=-1h@h | rex "Payment ID[\\": ](?[^\\":])" |

Now I want to check if there is ACKNOWLEDGEMENT OR NOACKNOWLEDGEMENT in the search for extract PaymentID in last 5 minutes.

Tried to search steamstats, however it is throwing an error.

[woodcock]

First of all, stop using field names with spaces, that is what is giving you the error. Try this (and do not change anything that you do not have to change):

host=hostname source="//Logfilelocation/xyz.log" Jobname AND Queuename AND Payment ID earliest=-1h@h
| rex "Payment ID[\\\": ](?<PaymentID>[^\\\":])"
| streamstats count(eval(searchmatch("your acknowledgment string here"))) AS sessionID BY paymentID
| stats count list(_raw) BY sessionID paymentID
| where count=1

The only thing that you should change is the "your acknowledgment string here".

[bsaujla131984]

@woodcock I tried again, it is listing down results, irrespective of search string. Means it is not taking search string in to count.

[pkarpushin]

Hi @bsaujla131984 ,

Probably you should consider your log as a lookup, if the data inside of it is changing from time to time.
Could you please provide a sample of your log X, I might be able to give you a solution.

[Sukisen1981]

look here - https://answers.splunk.com/answers/633423/streamstats-and-delta.html
and here
https://answers.splunk.com/answers/612806/how-to-use-streamstats-to-display-the-last-current.html

use something like
|stats values(_raw) by paymentid
This should give you the 2 values of the raw events for the same payment id. use streamstats, if needed and compare