Possible to add a complete file (oneshot/sinkhole) AND followTail

[nocostk]

I'd like to start monitoring a file that has been around for a while. I need to get all the older data in the file AND start tailing it (or inversely start tailing it and then realize I need the older data, too). What is the easiest way to do this?

Here is the inputs.conf

[monitor:///opt/scripts/sendShipConf/sendShipConf_cron.log]
disabled = false
followTail = 1
sourcetype = sendshipconf

I tried to oneshot it but the older info isn't there:

/opt/splunk/bin/splunk add oneshot /opt/scripts/sendShipConf/sendShipConf_cron.log

I could use a sinkhole but the sinkhole is defined globally (which means that I can't [or rather, don't want to] apply specific sourcetypes to it). This is from my "common" (global) app pushed to each forwarder:

[batch:///opt/splunk/var/spool/splunk]
move_policy = sinkhole

[mayler]

I'd agree with Mick. Wouldn't nocostk just need to add a new data input, file, source: monitor file or directory, /path/to/file, make sure "follow tail" is de-selected.

[mayler]

if you set it to 1, wouldn't it only index the new data? setting it to 0 would index the old data and new data. no?

[nocostk]

Right. I could just set followTail = 0, but for the sake of consistency across my environments I always just enable that value. I'd prefer to be able to add retroactive events on the fly.

[Mick]

I think you may be making this a bit over-complicated, it sounds like you just need to add a regular monitor input, that will capture all of the old data and any new incoming data going forward.

Or is there some detail that I'm missing here?

[nocostk]

Well, maybe so. Here's a scenario: A developer asks me to start indexing a log. I get that setup and Splunk is happily chugging away at it (followTail). Developer emails me later and asks "is this all the older data or just the new data? How can I get the older data, too?".