Adding more forwarders when licenses are almost maxed out

[skoelpin]

If I have a 100GB licenses and my current indexing rate is around 90GB/day and I turned a forwarder on which had 15GB of log files total but only produced around 1GB/daily. Would this cause me to go over my current 100GB licenses since it will retroactively index all the logs or will it only index up to my license amount of 100GB then grab the rest later when space is available?

[ddrillic]

-- .. or will it only index up to my license amount of 100GB then grab the rest later when space is available

It doesn't seem that such logic exists in the product.

As a workaround you can control it via the ignoreOlderThan parameter in the inputs.conf file -

[default]
index=xxxx
ignoreOlderThan = 4d

You can set it up for 4 days, for example, and increase it on a daily basis and go further back in time while monitoring the license usage.

[javiergn · Accepted solution]

It all depends on how you configure your inputs.
If you want your forwarder to backfill all the existing logs and not just real-time ones, then it is likely going to blow your license that day based on what you are saying, but because you can violate the license up to 5 times within a 30 day period before it stops searching, I wouldn't worry too much.

Worst case contact your account manager and request a license violation reset explaining that you are backfilling old logs. It shouldn't be a problem.

[skoelpin]

By default it will backfill retroactively right? What .conf file would I modify to prevent this?

[javiergn]

If you have 15GB of log files but only want to read 1GB/daily then configure your inputs.conf to read just today's file.

For example, assuming there's 15GB of *.log files under /var/log that you want to read:

[monitor:///var/log/*.log] -> this will tell Splunk to read them all

[monitor:///var/log/*.log] -> this will tell Splunk to ignore files older than 1 day
ignoreOlderThan = 1d

If you still need those 15GB of logs then go for it as you will only violate the license one day out of the 5 you have on any 30 day period.

Or maybe you want to configure Splunk to read the 15GB logs on a weekend when the license usage is lower.

There are plenty of options and none of them is wrong