Splunk Search

files are not picked within 30 minutes

samani27
Observer

Hi All,

I have scenario. File will placed by one applicationA on below folder , Same file will be picked by another applicationB.
When ApplciationA place file i have entry with value b s i r Same file will be downlaoded with log entry b s o r.
Now question is

i want search and generate alert after 30 minutes files are not picked by ApplciationB (whatever ApplciationA placed files),
how i can achive this one ?

Wed Jul 11 15:50:06 2018 0 10.81.193.254 20 /export/Apps/splunksftest/test_email_alert.txt b s o r sam1 ssh 0 *

Wed Jul 11 15:49:34 2018 0 10.81.193.254 20 /export/Apps/splunksftest/test_email_alert.txt b s i r sam2 ssh 0 *

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

This looks like a transaction question.

You can use the transaction command
see http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

mysearch source=applicationA OR source=applicationB 
| rex "pseudo field extraction to extract the file name in a field names myfilename"
| transaction myfilename maxspan=3600 keeporphans=true 
| table _time duration myfilename source closed_txn  eventcount _raw

if you find a transaction with a beginning but without an end, it should be the ones you are looking for
look at the options keeporphans and maxspan to control the transaction duration. you also can use the options startswith and endswith to if the termination exists (look at the field closed_txn)

Of course, there is always a way to create a more efficient search using the "stats" command.

example :

  mysearch source=applicationA OR source=applicationB 
  | rex "pseudo field extraction to extract the file name in a field names myfilename" 
  |   stats min(_time) AS timeSTART max(_time) AS timeEND earliest(source) AS sourceSTART latest(source) As sourceEND count dc(source) AS distinctSources by myfilename 
  | eval _time=timeSTART
  | eval duration=timeEND-timeSTART | eval isin2sources=if(distinctSources>1,"yes","no")
  |   eval durationmorethan30min=if(duration<60*30,"no","yes")
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...