- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- how to 'tail' the input that is being sent to ind...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
how to 'tail' the input that is being sent to indexer
Hi
Is it possible to tail the log files and send to indexer rather than sending one file as such.
To make it clear - We have some log files of huge size for which we cannot do log rotation for some reason and the file has got entry from last year which I dont want to monitor through splunk.
So is it possible to tail the last 600-700 lines of that file alone and send it to indexer -not the whole file?
Every update to those 700 lines should be sent to indexer on regular basis
Thanks
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You must have your reasons but I think it would be much easier to just index the whole file and then have Splunk follow the tail. You can always use time range picker or earliest|latest in your search to filter out older results. Or you could "delete" events older than your selected age.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It will cross your limit and register a violation but on an Enterprise licence Splunk will allow you up to 5 violations in a 30 day rolling window. So as long as you then spend the next 30 days without a single violation you will have a clean licence again 🙂 Splunk uses this model as it allows for scenarios like this and for companys to do monthly dumps of batch files and the like.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The reason I ruled out option of indexing the whole file is due to license limitation.We are allowed to index till 10GB perday
these files as a whole had grown up to 13GB now
so if I index whole file then ,at least for the first time when the whole file is indexed it will cross 10 GB?(as per my understanding)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Your best bet would be to use a scripted input to run tail manually for the last 700 lines, something like tail blah -n 700 (double check the syntax).
The only issue would be duplication if you don't have 700 new events in between readings.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That's just based on the modification time of the files. Monitor statements are all about the file, props are about the content 🙂
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How about the ignoreOlderThan option? Or will this just work for the files itself and not the content of it?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Syntax is correct :-)... or you could do tail -n 700 blah
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
Splunk Custom Visualizations App End of Life
