Splunk went loco... reporting it indexed 250+ GB in half an hour when it didn't

[juriggs]

Here's the long and short of it. My Splunk instance went nuts and said it indexed 250+ GB in a very short time. I started looking into it, and the two big culprits (according to the Splunk License Usage app) were the Windows Event Logs (System and Application). But when I actually look at what's been indexed, it's nothin. For example, over the past 24 hours there are 9,085 events from the System logs, but Splunk is reporting that that sourcetype has indexed 180,914 KB of data.

What's going on, and how do I fix it?

As an aside, this isn't the first time I've seen something like this with Splunk. It did something very similar with a custom log, but in that case I just deleted the input and recreated it. I'd rather not have to do that with the event logs - I'd rather the app just work as it should.

Thanks.

[Drainy]

So I have a few thoughts here, firstly have you had a look at the size of your index on disk to see if it reflects any issues?
Secondly, what version are you running?

Finally, if you're pretty certain that you've checked everything and it really can't possibly be any crazy log data or wild whitespacing in a custom log - raise a support ticket.
The licensing is fairly baked in so short of you actually consuming 250GB+, you'll be best putting your efforts into creating a diag and getting a support case rolling.

[piebob]

as Drainy points out, as a community user you're more likely to get a quicker answer here. another place you can try is the #splunk IRC channel on EFNET. it's not an official support channel, but a lot of clever people hang out there.

[Drainy]

Did Splunk fire a violation/warning for going over the limit?
Sure, you can raise a ticket but it will be treated as a "Community" ticket - no guaranteed response time and it will always be at the bottom of the support pile. But you can still raise one 🙂

[juriggs]

Drainy,

Thanks for the thoughts. Nothing seems to be wrong except that Splunk seems to think it's indexing a lot more data than it really is.

I'm using the free license; can I still open a ticket with support?

Thanks!