Splunk Search

postfix_syslog time extraction inaccuracies

adamw
Communicator

We seem to be having an issue with the postfix_syslog sourcetype (that came as a default sourcetype in Splunk) and its date extractions.

I posted this at 8:20am on 9/29, and did a search of events that take place between 15:00 and 23:59 on 9/29 and come back with the following results.

As you can see, the date_hour is set up as 18 on one of these events, which translates to 6pm, but the original event actually took place at 5am.

I am not overriding any of the default postfix_syslog stuff, and these events are getting sourcetyped properly, as shown below.

[postfix_syslog]
TIME_FORMAT = %b %d %H:%M:%S

According to the docs, %H is the 24 hour time, even though Splunk seems to believe it is not.

Any help is appreciated. Thanks,

--adam (EDIT) This is Splunk 4.1.4, data is coming in with syslog-ng. alt text

0 Karma

Lowell
Super Champion

One more possibly helpful resource:

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Seems likely that you have some conflicting configuration. Splunk does not appear to be looking at your event timestamps at all, but using CURRENT or the file modification time. This may be because of changes or redefinitions of DATETIME_CONFIG or the file it points to. Probably using btool http://www.splunk.com/base/Documentation/4.1.5/Admin/Troubleshootingconfigurations may help show if this is so, of if there are other configurations conflicting (e.g., both source:: and host:: configurations will override sourcetype props.conf configurations.)

Lowell
Super Champion

I agree, a rogue DATETIME_CONFIG = CURRENT entry does seem to be the most logical explication for what's going on here.

0 Karma

southeringtonp
Motivator

Check to make sure that there isn't another rule overriding TIME_FORMAT or other timestamping options. If you have conflicting entries in props.conf, configuration settings applied to [host::myhost] or '[source::mysource] will take precedence over those applied to the sourcetype.

If that isn't the problem, then more information is always helpful, such as:

  • How are you bringing the syslog data in? (Looks like syslog-ng?)
  • How is the input configured in inputs.conf?
  • 0 Karma

    Lowell
    Super Champion

    That's very weird. Some additional details may help. Please "edit" your post and provide: The version of splunk you are running. Have you ever made any changes to datetime.xml? Does the syslog host contain any digits it or is it an IP address?

    0 Karma
    Get Updates on the Splunk Community!

    Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

    March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

    What’s New in Splunk App for PCI Compliance 5.3.1?

    The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

    Extending Observability Content to Splunk Cloud

    Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...