Getting Data In

Add monitor file with new index not working

tanglong
Engager

Hello,

I have problem with Splunk Forwarder.

Currently, i monitor a dir (/var/log/httpd/*) but it automatic delete old log and insert new log. Index default is "main".

I want to store with new index name "weblog", i have do:
- Step 1: delete all log httpd with search query: "source=httpd | delete"
- Step 2: remove old monitoring and add new: ./splunk add monitor "/var/log/httpd/*" -index weblog -sourcetype newsource

But it not working.

I use command below it working but not define new index: ./splunk add monitor "/var/log/httpd/*" -sourcetype newsource

Can you help me resolve my problem?
Thanks.

Tags (2)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi tanglong,
when you already ingested logs from a file you cannot ingest them again, also renaming the file.
To do this you have to use a workaround:

  • delete the old logs from the main index using the delete command, remember that only special users (can_delete role) can delete logs, this step isn't mandatory;
  • create the new index weblog on Indexer
  • modify inputs.conf on Forwarder adding the option index = weblog to inputs.conf;
  • modify inputs.conf on Forwarder adding the option crcSalt = <SOURCE> to inputs.conf;
  • restart Forwarder;
  • after the logs are reindexed delete the crcSalt = <SOURCE> row from inputs.conf and restart Forwarder.

In this way logs are reindexed and sent to the new index.

Remember that if you add the can_delete role to you admin user, it's a good idea disable this feature as soon as after the deletion, it isn't a good idea to have this feature enabled.

Bye.
Giuseppe

View solution in original post

tanglong
Engager

Thanks for all comment, its helpful with me. Very important keyword.

  1. Its show on Indexer.

alt text

  1. This is config inputs.conf and "weblog" sourcetype not show in Splunk main page.

alt text

Anyway to help me? Thanks so much!

0 Karma

woodcock
Esteemed Legend

The fishbucket on the forwarder will remember that this file has already been forwarded and will refuse to forward it again. You can either:
1: Clear the fishbucket for this file.
2: Manually send the file using splunk add oneshot.
Also, do not use the CLI to configure ANYTHING in Splunk. You should create your own forwarder app and put your configurations in that.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi tanglong,
when you already ingested logs from a file you cannot ingest them again, also renaming the file.
To do this you have to use a workaround:

  • delete the old logs from the main index using the delete command, remember that only special users (can_delete role) can delete logs, this step isn't mandatory;
  • create the new index weblog on Indexer
  • modify inputs.conf on Forwarder adding the option index = weblog to inputs.conf;
  • modify inputs.conf on Forwarder adding the option crcSalt = <SOURCE> to inputs.conf;
  • restart Forwarder;
  • after the logs are reindexed delete the crcSalt = <SOURCE> row from inputs.conf and restart Forwarder.

In this way logs are reindexed and sent to the new index.

Remember that if you add the can_delete role to you admin user, it's a good idea disable this feature as soon as after the deletion, it isn't a good idea to have this feature enabled.

Bye.
Giuseppe

DavidHourani
Super Champion

Are you trying to re-ingest the same data ?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...