log does not forward to indexer

[SplunkCSIT]

i did not configure the indexer server properly initially hence the log is indexed locally. After i configured the indexer server correctly, the forwarder did not forward the data over to indexer and was also not index locally. What happen and how to resolve this?

[Ayn]

@SplunkCSIT, I agree - I feel this was not that time though. I'm more than happy to help you, but that's very hard without getting details on your case. Splunk is a complex product and depending on how exactly you've configured it it can behave very differently, so it's crucial that you provide us with details.

[SplunkCSIT]

Ayn, sometime it is best not to post anything instead of posting negative comment.

[MuS]

Hi SplunkCSIT,

I don't know what your problem is or why things are happening, but here is something you must know about re-indexing data. Splunk keeps track of what was indexed and therefor does not re-index the same data again .... unless you do some work.

The radical method is to clean the fishbucket index. That will remove the memory of every files, be warned it will re-index all data.

• on an indexer splunk clean eventdata -index _fishbucket
• on a forwarder by removing the folder $SPLUNK_HOME/var/lib/splunk/fishbucket

You could selectively forgot a single file from the fish bucket like this:

splunk cmd btprobe -d $SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file $FILE --reset

You could manually re-index each file with the oneshot option

splunk add oneshot "/path/to/my/file.log" -sourcetype mysourcetype

You could modify the first line of the files to re-index, by default Splunk checks the first 256 chars of a file to differentiate them. If you had a simple comment on the first line it wil reindex it

You could change the crcSalt, create a new input for a new folder, add all the correct sourcetypes, etc... add the option crcSalt=<SOURCE> then move or copy the files to be re-index to the folder.

hope this helps ...

cheers, MuS

[Ayn]

With this level of details, the only thing we can say is "do things right instead of wrong". 😛

[linu1988]

post your configuration? i think it's a heavy forwarder and if output is set for another indexer and not set to indexandforward then it will not index anything locally.