Alerting

Set up splunk Alert - a complicated one

loveforsplunk
Explorer

Here is my log file having a key word "error":

My search is : index=abc host="123" "error" source="efg/*"

My search returns results as below (check out the timings ) , suppose below is the event section:

Time Event
5/20/17 1:00:45.000 AM Completed at Sat May 20 03:00:45 2017

Under the _time section , the time which is displayed is 2hrs less than the time that is displayed in the logs(as you can see from the event section).

Now , suppose there is a failure in my log which I came to know right now. I go to splunk and check I do not see any result for the last 15 minutes even but when I do last 2 hours , I get to see the result.

Please tell me how do i set this alert. if I am setting to check every 5 minutes, I do not get any alert . When I did -2h as start time and now as finish time , I still did not get any alert . Now I did it -2h@h which I am sure will work or not until there is a failure.

Also, I have selected Run every minute while setting the alert.

Tags (1)
0 Karma
1 Solution

dineshraj9
Builder

You need to fix the timezone of your logs first. The 2hr difference is because of the logs being set to incorrect timezone.

https://docs.splunk.com/Documentation/Splunk/6.6.0/Data/Applytimezoneoffsetstotimestamps

I suggest you get the TZ attribute in props.conf for the sourcetype.
This will fix the timezone issue and logs will appear real time.

View solution in original post

0 Karma

dineshraj9
Builder

You need to fix the timezone of your logs first. The 2hr difference is because of the logs being set to incorrect timezone.

https://docs.splunk.com/Documentation/Splunk/6.6.0/Data/Applytimezoneoffsetstotimestamps

I suggest you get the TZ attribute in props.conf for the sourcetype.
This will fix the timezone issue and logs will appear real time.

0 Karma

loveforsplunk
Explorer

But for the same server , I have alert set up for other logs as well. For those I get the expected output . Only for this particular log I do not get, not sure why.

Do u mean , there is a way to set up timezone for particular logs ??

In my Splunk user settings , my timezone set is correct and the logs timezone in its server as I see is also the same as mine , then why do I get something else in _time ?

0 Karma

dineshraj9
Builder

Yes, you can setup for timezone for each log in different ways.

In props.conf, you can set TZ attribute for the particular sourcetype.

In inputs.conf, you can set _tzhint field for the particular log monitor stanza.

0 Karma

loveforsplunk
Explorer

oh ok. I will work on this. Thank you so much Dinesh.

0 Karma

adonio
Ultra Champion

it has nothing to do with your users timezone,
check the link in the answer

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...