- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Why is splunk not indexing all the data?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why is splunk not indexing all the data?
Hi, lately we've been checking how many files our splunk is indexing, and we noticed that it "skips" some files... We checked by searching:
index=our_index | stats dc(source)
index = _internal group = per_so* series=*.ourfile | stats count
And both ways we got the same results, which are not all the files we indexed.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A cheerful place to start at I can't find my data!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I didn't find any help in this page....
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also, check internal logs from yoru forwarder(s) to see if there are any warnings/error for your files. ( index=_itnernal sourcetype=splunkd host=yourFwd *filename.ext* )
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've already looked there... Zero warnings or errors....
Also we've tried both monitor and batch input. Both had the same problem...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try running following on your forwarder instance. See if Splunk is monitoring all the files you've configured for monitoring (will prompt for admin credentials for that Splunk instance)
$SPLUNK_HOME/bin/splunk list monitor
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even if there are no errors/warnings, do you see any entry for your log file that's missing?
index=_internal sourcetype=splunkd host=yourFwd adding watch
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
well the forwarder doesnt write log when it monitors, only with batch input for some reason. Today we indexed some logs, and we saw one of the files in the forwarders log, but we could not search it...
I wonder if it has something to do with the fact that we added a few indexers recently.
Is there something i should update in the search head when i add indexers?
ICYMI - Check out the latest releases of Splunk Edge Processor
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
