Failed to remove indexed log files from search

[sumanth_isac]

Dear all,

I am not able to remove old log files from my search.
I tried all possibilities.
I tried
1. ./splunk stop
2. ./splunk clean eventdata
3. ./splunk start

I also removed everything from inputs.conf file

but old log files are not removed please help

[sumanth_isac]

I tried it
./splunk clean eventdata
This action will permanently erase all events from ALL indexes; it cannot be undone.
Are you sure you want to continue [y/n]? y
Cleaning database _audit.Cleaning database _blocksignature.Cleaning database _internal.Cleaning database _thefishbucket.Cleaning database history.Cleaning database main.Cleaning database os.Cleaning database splunklogger.Cleaning database summary.Cleaning database wi_summary_daily.Cleaning database wi_summary_fivemin.
Cleaning database wi_summary_hourly.If i start splunk and see in search still all old files are present

[sumanth_isac]

Dear DaveSavage,
Here log files i mean are iis files from web analytics tool. I want the yesterday added files not to appear in search today. I want the search to be fresh without any files.

I knew about ./splunk clean eventdata -f
this command but i dont know what my index name is ?
I went to Manager>Indexes
there are many like os(size 7mb),main(it shows current sizein mb is 1802) all others like _audit,internal are less than 5 MB

Please help me in identifying my index name.

[Drainy]

You can just use clean eventdata to clear data from all indexes.

[DaveSavage]

Sumanth_isac, do you mean log files because Splunk doesn't delete or clean up old files, you have to write your own scripts for that. Are the old log files still around somewhere - and therefore being re-indexed again? Are you qualifying your statement splunk clean eventdata indexname?

[Drainy · Accepted solution]

Did any of those commands throw an error? Did you by any chance run the clean eventdata, start splunk and then remove the inputs.conf detail?

Running clean eventdata will clear all indexes, this includes the fishbucket where Splunk tracks what files it has already consumed and where it is up to in those files, if you then restart it will just re-read the contents of the files in the inputs.conf.

Disable your inputs, stop Splunk, run clean eventdata, start Splunk and nothing should remain.

[sumanth_isac]

Dear DaveSavage
Thanks Drainy solved my problem. Thanks for the guidance.
I will explore it deeply. Thanks for wishes.
Thank you
Sumanth

[sumanth_isac]

Thank You Very much Drainy.
I followed your advice on how to disable data input and other steps and succeeded my requirement. Now i dont have my old files in my search. I had a wonderful start of new year as my problem is solved. I appreciate your concern for new bie's and helping them.
Thank You.
Cheers
Sumanth

[DaveSavage]

I'd encourage you to get familiar with the indexes in the system, and know where your data is coming from / going to. The default is main but if you are using a plug-in such as '*.nix' for example, you will find yourself addressing and searching across other indexes, OS in that case. If you are using forwarders then clearly either you, or someone chose where the data was to go to. If IIS is your need today, consider how you'd deal with Win Performance Data next week. You may have already thought this through, so forgive if that is the case. Good luck though.

[DaveSavage]

Sumanth_isac,
Drainy has covered it all, so don't vote this response up, am merely closing off your question above.
Your 'business requirement' or driving needs are to see just today's files, not previous ones. Well, Splunk does that very elegantly in the Search parameters as it is, plus if you save previous IIS logs then you can start getting cute looking for trends over time?
Re which index name should you be looking at, well - the blanket delete all covers everything, but you may find over time that you no longer have that luxury i.e. there is other data in your system.

[Drainy]

The error about the logger index isn't anything to worry about, you can leave it disabled. The easiest way to ensure you disable your inputs is via the GUI to go to manager in the top right, then to Data Inputs, then go through to whichever category covers the data input you have running and select the option to disable it. Then shutdown Splunk, clean eventdata and restart.

[sumanth_isac]

Dear Drainy,
Thanks for reply.

Yes i had run clean eventdata and as your prediction i did all the combinations in cleaning starting and stopping and editing inputs.conf .

Later followed the procedure of stopping splunk running clean command later starting the splunk.

Currently my inputs.conf file is empty

Before when i run clean eventdata it was throwing me
"could not clean splunk logger"
Later i enabled splunklogger from Manager>Indexes.
Now clean command runs smoothly.

But i only know how to enable index.
How to disable inputs ? Please let me know.