Getting Data In

How to troubleshoot why an indexer stopped receiving data on one index after I fixed a different index?

antessima
Explorer

We are working on configuring Splunk for the first time in advance of buying it, and I am having problems with the indexer. Or maybe the forwarder....not sure which is guilty here.

Here's what I'm doing: I have three indexes: one that monitors a specific file, one that monitors a different specific file, and one that monitors a group of files with similar names. Initially, index #1 was all I had set up, and it was working fine. I added the second and third indexes, and after setting them up in the inputs.conf and props.conf files on the forwarder, the second index started working just fine, but the first index stopped working. (The third one hasn't yet worked but that's not the question here.)

So, the question is, why did the first index stop working? These are the entries in the two files for the three indices and their associated sourcetypes:

inputs.conf

[monitor:///usr/local/ourstuff/logs]    <== This is index #3
disabled = false
index = transactions
sourcetype = translog
crcSalt = <SOURCE>
whitelist = *_transLog

[monitor:///usr/local/ourstuff/logs]     <== This is index #1
disabled = false
index = server_all
sourcetype = server_dblog
crcSalt = <SOURCE>
whitelist = yellowboxSync

[monitor:///usr/local/ourstuff/logs]    <== This is index #2
disabled = false
index = performance
sourcetype = httplog
crcSalt = <SOURCE>
whitelist = HttpLog

props.conf

[default]
maxDist = 500

[translog]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999
MAX_TIMESTAMP_LOOKAHEAD = 28
BREAK_ONLY_BEFORE = "xmlInquery"

[server_dblog]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999
MAX_TIMESTAMP_LOOKAHEAD = 28

[httplog]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999
MAX_TIMESTAMP_LOOKAHEAD = 28

Any ideas would be appreciated.

0 Karma
1 Solution

somesoni2
Revered Legend

First of all, these are data monitors, not Index (index is where data is stored like transactions in this case).
Second, you say you're monitoring a single file but if you see the monitoring stanza, you're monitoring the same directory, thus only one will work.

This is how I'll write inputs.conf. Please update the full file names as per your situation.

 [monitor:///usr/local/ourstuff/logs/*_transLog]    <== This is monitoring #3
 disabled = false
 index = transactions
 sourcetype = translog
 crcSalt = <SOURCE>

 [monitor:///usr/local/ourstuff/logs/yellowboxSync]     <== This is monitoring #1
 disabled = false
 index = server_all
 sourcetype = server_dblog
 crcSalt = <SOURCE>

 [monitor:///usr/local/ourstuff/logs/HttpLog]    <== This is monitoring #2
 disabled = false
 index = performance
 sourcetype = httplog
 crcSalt = <SOURCE>

View solution in original post

somesoni2
Revered Legend

First of all, these are data monitors, not Index (index is where data is stored like transactions in this case).
Second, you say you're monitoring a single file but if you see the monitoring stanza, you're monitoring the same directory, thus only one will work.

This is how I'll write inputs.conf. Please update the full file names as per your situation.

 [monitor:///usr/local/ourstuff/logs/*_transLog]    <== This is monitoring #3
 disabled = false
 index = transactions
 sourcetype = translog
 crcSalt = <SOURCE>

 [monitor:///usr/local/ourstuff/logs/yellowboxSync]     <== This is monitoring #1
 disabled = false
 index = server_all
 sourcetype = server_dblog
 crcSalt = <SOURCE>

 [monitor:///usr/local/ourstuff/logs/HttpLog]    <== This is monitoring #2
 disabled = false
 index = performance
 sourcetype = httplog
 crcSalt = <SOURCE>

antessima
Explorer

I thought I had tried that previously, but I did as you indicated and now both #1 and #2 work simultaneously. Thank you for your help!

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...