CSV File Ingestion problem

[rajyah]

Good day sirs, would you be so kind to please help me regarding csv file ingestion? Here's the scenario:

When I try to upload the csv manually from remote server, the data within it are ingested. But if I monitor it, it wont. This csv was rsynced from remote server to local, I assumed it was because of permission but I checked it wasnt the case. Are there still anything I need to check? Using btool? How?

Any thoughts will do. Please help.

Thank you.

[gcusello · Accepted solution]

Hi rajyah,
when you say "I monitor" do you mean using a Universal Forwarder?
If yes, at first check if it's correctly configured:

• firewall ports are open (check them with telnet <Indexer_Ip_Address> 9997)
• Splunk Universal Forwarder is up and running (/opt/splunkforwarder/bin/splunk status)
• grants of the csv files are ok (644)
• Splunk Forwarder's outputs.conf it's OK (there a room for the Indexer), you can check it searching on Indexer: index=_internal host=your_host
• Splunk Forwarder's inputs.conf it's OK (the csv path in the monitor command is correct)

After these checks, you can analyze your situation.

Bye.
Giuseppe

[rajyah]

Oh, I forgot to mention sir that Im only running single instance without forwarders. I see, so it must be permission. Seems weird though, other CSVs has same grants but was still ingested. Ill double check sir. Thank you and Ill add an update sir after confirming.

[gcusello]

Hi rajyah,
in this case, check if the path you configured in your inputs.conf is correct, then be sure that this file isn't the copy of another file because Splunk doesn't index twice a file (you can do it only when you manually index).
To force to reindex, you can put in your inputs.conf the option

crcsalt = <SOURCE>

Anf then change the filename.

Bye.
Giuseppe

[rajyah]

Sir, I just found an oddity behind this problem. If I use 'index once' instead of 'continous monitoring', the data in csv are indexed but when I choose the latter it doesn't. Please enlighten me.

[gcusello]

Using "once" it's the same thing that manually index.
did you tried crcSalt?

Bye.
Giuseppe

[rajyah]

Tried crcSalt. Thank you!

[rajyah]

Is there a way sir to check the splunkd.log? How?

[gcusello]

you can see splunkd.log directly on you Splunk server at

/opt/splunk/var/log/splunk 

or using Splunk with this search:

index=_internal sourcetype=splunkd

Bye.
Giuseppe