How do I filter the data I want to bring into Splunk and index?

[chjamey]

I'm doing a Splunk POC and I'm using the trial download. Thanks to a message I just got at the top of Splunk, I just learned that there are limits to the amount of data I can index daily.

How do I filter what I want to index so I can get only the information I need and stay under the limit? I'll be indexing windows event logs, performance data, snmp logs from hp blade systems, switch\firewall data, sharepoint logs, sql logs, iis logs and what ever else we can to get a good overview of our machine\network data.

Thanks,
Jamey

[satishsdange]

Free license supports data volume upto 500MB/day. As you are doing a POC, I would strongly recommend you engage Splunk Team to make it successful. Splunk Sale Rep can provide you evaluation license for POC purpose.

For information about license violations (as you are ingesting so many data sources with trial version), you may refer to http://docs.splunk.com/Documentation/Splunk/6.2.1/Admin/Aboutlicenseviolations

[markthompson]

Hi Jamey,

Once you download the trial, this is a 60 day enterprise trial, which allows full functionality and has an index limit of 500mb/day, after the 60 days you can purchase a license or downgrade to the free version, please see this link:
https://www.splunk.com/en_us/products/splunk-enterprise/free-vs-enterprise.html

On the free license there is a 500gb Limit / day, but you can obtain an enterprise license trial from Sales if you would like to see the benefits as the free license means you don't have to logon.

But some aspects of PPape's response are correct, such as using the config files to index only the data you want.

As PPape said;
inputs.conf
props.conf
transforms.conf

EDIT - Corrected facts based on http://www.splunk.com/en_us/download/splunk-enterprise.html - thanks to PPape for pointing this out

[PPape]

Hi markthompson,

where do you have your Information from?
As far as i know and found in the documentation I'm right.
There is a 60 Day trial. But it is not Volume free. In this 60 Days you have 500 MB per day and all enterprise features. Also the free license after the 60 Days is even 500 MB not GB.

Types of Splunk licenses

And the trial enterprise License with an higher Volume (obtained from a splunk Partner) can be placed from first day of POC on.

[markthompson]

Hi PPape, you're absolutely right, that was a typo, it is indeed 500MB/day.

I have used the trial before for my VM. Although, after checking it's actually a 60 day trial.

Yes sorry, last time I used the trial was a long time ago!!! The trial is now limited to 500mb/day.

Apologies for any confusion.

[PPape · Accepted solution]

Hi Jamey,

first of all - there is no limit in Splunk -

I'd suggest you contact a Splunk Partner in your near and ask them for an trial license.
They can help you to get a 30 Day trial license with the volume you need. So you can test and see how much Data you gather daily.
Find your Partner here

And when you want to do everything on your own, take a look here
inputs.conf
props.conf
transforms.conf

This Link should contain the Answer to your question 🙂
Route and Filter Data

[chjamey]

Thanks for your answers. The version I downloaded said 1GB limit per day. I'll reach out to Splunk and see about getting an Enterprise Trial. Thanks for the other helpful information about filtering data PPape.