All Apps and Add-ons

splunk monitoring inputs.conf (after deleting file how to add the file that has a same file name as deleted file)

jenny_life
Path Finder

hello,
every one

I wrote like this in inputs.conf.

[monitor:///home/ec2-user/aaa/]
host = ip-10-0-0-xxx
index = aaa
whitelist = aaa[^/]*\.csv$
sourcetype = csv
crcSalt = <SOURCE>

[monitor:///home/ec2-user/bbb]
host = ip-10-0-0-xxx
index = bbb
whitelist=bbb[^/]*\.csv$
sourcetype = csv
crcSalt = <SOURCE>

I can put the new data to 'aaa' index, but If i do as below process, I can't put the data to 'aaa' index.

step 1. put the ' aaa_20181004.csv' to the aaa folder. This step is ok. (I can check the data in the aaa index)
step 2. delete the 'aaa_20181004.csv' on the splunk . This step is ok.
of course i deleted 'aaa_20181004.csv' on the aaa folder.
step 3. put the ' aaa_20181004.csv' again to the aaa folder. - This step has problem.
I can't read 'aaa_20181004.csv'' again even though there is ' aaa_20181004.csv' in the aaa folder.
after step 3, There isn't aaa_20181004.csv' in the aaa index.
how should I do.

But If I do that process to bbb. all steps are fine.

Just different thing is the contents of aaa and bbb.

I'll put the file to folder daily.(aaa_20181003.csv , aaa_20181004.csv , aaa_20181005.csv ...)
I'd like to know daily data.

thank you in advance.

0 Karma

stephenoleary
Explorer

If you want to re-index the same file, you can use btprobe to reset the fishbucket for that file so that it is re-sent by the forwarder:

E.g.

  • Stop splunk on the forwarder
  • Run: ./splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /home/ec2-user/aaa/aaa_20181004.csv --reset
  • Start Splunk

You should find that the forwarder will re-send the file for indexing.

0 Karma

christianhuber
Path Finder

how do you delete the data in splunk ? (step 2)

with crcSalt = , splunk remembers the file name, so if you like to reindex that file, take another filename (e.g. filename_v2).

What is the exacte use case why you need to reindex this files ? maybe with a better understandig I can provide you some ideas how to solve your problem.

0 Karma

jenny_life
Path Finder

Thank you for your attention.

how do you delete the data in splunk ? (step 2)
->I used the 'delete' command on the splunk.

I'd like the file to be entered automatically to Splunk, if i place the daily file in specific folder.

0 Karma
Get Updates on the Splunk Community!

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

A Guide To Cloud Migration Success

As enterprises’ rapid expansion to the cloud continues, IT leaders are continuously looking for ways to focus ...