Getting Data In

Regex in summary index

namrithadeepak
Path Finder

Hi,

I have a lot of searches like this:

Search 1: | common regex | stats using some fields extracted in regex

Search 2: | common regex | stats using some fields extracted in regex

...
...
...

Search 9: | eval | stats using some fields calculated in eval

Search 10: | eval | stats using some fields calculated in eval

Can I include the regex and eval in the summary index? How do I create a summary index for the above?

Thanks

Tags (1)
0 Karma

lloydknight
Builder

Hello namrithadeepak,

Yes, you can include regex and eval in creating summary index.

You may need to create an index first or use an existing index.

Below is the link on how to setup and schedule a summary index.
http://docs.splunk.com/Documentation/Splunk/6.5.3/Knowledge/Usesummaryindexing

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

REGULAR INDEXES:

You CAN create a data index with any data fields or calculated fields that you want. As with anything in technology, there is a tradeoff. The more calculations you do at index time, the slower the ingestion process, and the more index space you will eat up. That time is then (hopefully) saved back again at search time.

If you run certain regexes repeatedly, and/or if those searches make up a large percentage of the access against a certain type of records, then setting the regex up to run at index time may be a good idea. The less frequent the searches are used, the less you benefit and the more you lose from extracting them at index time.

Also, for a field that you use occasionally or frequently, the higher the cardinality of that field, the more effective an index-time extraction will be. A field that has 10000 different values is going to be a better bet than a field that has 2 possible values.

SUMMARY INDEXES:

A summary index is made up (usually) of data that has been aggregated in such a way that it is useful without having to review the underlying data. If there are five dimensions to your data, plus three count/amount fields, with sparse combinations of keys, and if these dimensions and details are commonly needed for reporting, then pre-aggregating a data cube can save a ton of access and calculation time.

If you are thinking of a summary index as just a data holding pen to hold intermediate data that has been prechewed during a reporting process, you might want to look at CSVs or lookups instead.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...