Splunk Search

Index Time VS Actual Occurs Time

jackyc
Explorer

Hi all,

I have a month (2010-Nov) SAR reports (30 copies) for my host which I want to import them to the Splunk server. For testing purpose, I first import one SAR report to the Splunk and it can be successfully imported. But the event time is today not two months ago, can I change the index time back to the actual occurs time? Since I need to search for (2010-Nov)'s SAR report. I found maillog didn't have this issue..

Many thx!

BR, Jacky.

Tags (2)
0 Karma

cyndiback
Path Finder

Hi Jacky,
I encountered the same issue today when indexing old data into Splunk but wanting to preserve the actual time as index time.

Copy of the logs I'm indexing:

  • ....change_time: 2011-11-04 10:30:27, view_rfc_status, 1803, 17, Approve, 137, John Doe, 2243
  • ....change_time: 2011-11-04 10:30:47, view_rfc_status, 1803, 17, Approve, 137, John Norris, 2243
  • ....change_time: 2011-11-04 10:40:13, view_rfc_status, 1806, 17, Approve, 142, Chuck Norris, 2246
  • ....change_time: 2011-11-04 12:17:39, view_rfc_status, 1807, 16, Pending Approval, 148, Chuck Norris, 2247

The correct timestamp should be the after change_time: 2011-01-04 10:30:27 but if I indexed these today Splunk would mark them as 2012-01-07 12:10:00 PM

To always use the time in the log I made the following changes:

  • On the Splunk indexer edit the local props.conf (if linux server file is in /opt/splunk/etc/system/local/props.conf)
  • Create a stanza for the specific source
  • Tell Splunk what comes before the timestamp you want to use - In my case the timestamp is after "change_time: "
  • Tell Splunk what format the datetime is in

Copy of stanza in Props.conf

[source::/opt/splunk/bin/scripts/rfc_status.sh]  #specific source
     TIMEPREFIX="changetime:  "     #look for time after this text
     TIME_FORMAT=%Y-%m-%d %H:%M:%S  #this is how time is formatted

Followed Splunk Docs: http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

I wanted to clean up the logs I had already indexed incorrectly so my whole process was to (NOTE depending on your setup this process may not work for you):

  • Disable indexing for the specific source while making the props.conf changes
  • Delete the old data for the specific source *****Careful you do not delete ALL logs from host.
  • Save the props.conf changes
  • Reload config changes in props.conf by typing the following search string in Splunk Web:

    | extract reload=T

  • Enabled indexing for the specific source

This is what I did I don't know if there are easier ways to do this.

Drainy
Champion

One note, as this is an index time change you will need to restart Splunk to reload the relevant changes in props.conf. The extract reload=T command will only reload search time extractions.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...