Splunk Search

files are not picked within 30 minutes

samani27
Observer

Hi All,

I have scenario. File will placed by one applicationA on below folder , Same file will be picked by another applicationB.
When ApplciationA place file i have entry with value b s i r Same file will be downlaoded with log entry b s o r.
Now question is

i want search and generate alert after 30 minutes files are not picked by ApplciationB (whatever ApplciationA placed files),
how i can achive this one ?

Wed Jul 11 15:50:06 2018 0 10.81.193.254 20 /export/Apps/splunksftest/test_email_alert.txt b s o r sam1 ssh 0 *

Wed Jul 11 15:49:34 2018 0 10.81.193.254 20 /export/Apps/splunksftest/test_email_alert.txt b s i r sam2 ssh 0 *

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

This looks like a transaction question.

You can use the transaction command
see http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

mysearch source=applicationA OR source=applicationB 
| rex "pseudo field extraction to extract the file name in a field names myfilename"
| transaction myfilename maxspan=3600 keeporphans=true 
| table _time duration myfilename source closed_txn  eventcount _raw

if you find a transaction with a beginning but without an end, it should be the ones you are looking for
look at the options keeporphans and maxspan to control the transaction duration. you also can use the options startswith and endswith to if the termination exists (look at the field closed_txn)

Of course, there is always a way to create a more efficient search using the "stats" command.

example :

  mysearch source=applicationA OR source=applicationB 
  | rex "pseudo field extraction to extract the file name in a field names myfilename" 
  |   stats min(_time) AS timeSTART max(_time) AS timeEND earliest(source) AS sourceSTART latest(source) As sourceEND count dc(source) AS distinctSources by myfilename 
  | eval _time=timeSTART
  | eval duration=timeEND-timeSTART | eval isin2sources=if(distinctSources>1,"yes","no")
  |   eval durationmorethan30min=if(duration<60*30,"no","yes")
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...