Getting Data In

How to index a log that was missing for a specific date in the past?

niwebadmin
New Member

Hey Guys,

We have a log for a specific index that was missing during an outage and we got it recovered. Obviously this log was not indexed with all the rest in the inputs.conf as it was not generated that day.

How can I index this specific log for this specific day in the proper index and make it appear as it was index that day?

Thanks all for your help in advance.

0 Karma

adonio
Ultra Champion

if im not mistaken, as long its a full file that is missing, you are very safe monitor it and the forwarder will pick it up and if it has correct time stamps, splunk will do the work for you.
if you are dealing with portions of a file, e.g. time you can use the ignoreOlderThan in your inputs.conf
read here:
http://docs.splunk.com/Documentation/Splunk/6.6.0/admin/Inputsconf

0 Karma

niwebadmin
New Member

So could just create a monitor in the index.conf and point to the file and that would be it? Or we are talking about a different approach here?

Also the file is not a portion or segment is the file for the whole day.

Thanks a lot for your help.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Why create a new one? Don't you already have a monitor for the directory that this file would have been in if you didn't have the outage? Just copy the file into that same directory and you should be OK.
Important notes:

  • This only works if the event timestamps are extracted from the file, i.e. you are not using indexing time as your event timestamps
  • there is no way of faking the internal _indextime field, it will always be the time the event was written to the index
0 Karma

niwebadmin
New Member

So I placed the entire log inside the folder of the current monitor. I just renamed it as the log rotates daily, let's see what happens. Will answer if that works.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

OK, please accept niwebadmin's answer if you were successful!

0 Karma

adonio
Ultra Champion

yes, create monitor in inputs.conf (not indexes.conf)
you are all set

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...