Getting Data In

Heavy Forwarder configure to populate data to two separate indexes

mookiie2005
Communicator

We are running a heavy forwarder. We want to send the perfmon data that it is currently receiving to two separate indexes. the Perfmon:CPU Load to index perfmon_cpu_load, and the Perfmon: Available Memory to perfmon_memory. How would we configure a heavy forwarder to do this? Ity is currently sending the data into main on the indexer. This forwarder does not have indexing currently enabled.

This is our current props.conf stanza:
[perfmon_cpu_load]
TRANSFORMS-routing=perfmon_cpu_load

[perfmon_memory]
TRANSFORMS-routing=perfmon_memory

This is our current transforms.conf stanzas:

[perfmon_cpu_load]
SOURCETYPE=Perfmon:CPU Load
DEST_KEY=_MetaData:Index
FORMAT=perfmon_cpu_load

[perfmon_memory]
SOURCETYPE=Perfmon:Avalible Memory
DEST_KEY=_MetaData:Index
FORMAT=perfmon_memory

0 Karma

lguinn2
Legend

Transforms.conf does not have a sourcetype key. Your stanza needs to look like this

[perfmon_memory]
SOURCE_KEY = MetaData:Sourcetype
REGEX =Perfmon\:Available Memory
DEST_KEY=_MetaData:Index
FORMAT=perfmon_memory

[perfmon_cpu_load]
SOURCE_KEY = MetaData:Sourcetype
REGEX=Perfmon\:CPU Load
DEST_KEY=_MetaData:Index
FORMAT=perfmon_cpu_load

These settings are specified in transforms.conf.spec which can be found in your Splunk installation under $SPLUNK_HOME/etc/system/README or in the documentation on transforms.conf

0 Karma

mookiie2005
Communicator

I should point out that we have a universal forwarder sending perfmon data to the heavy forwarder. Could we not use indexing on the heavy forwarder to assign sourcetype to the data and than forward that data on to the indexers based on sourcetype since it does not seem to be defined in the raw data? The reason we have a universal forwarder sending data to the heavy forwarder is so we can load balance the data between the two indexers.

0 Karma

mookiie2005
Communicator

Ok I found the raw data:
07/11/2013 08:35:59.720
collection="CPU Load"
object=Processor
counter="% User Time"
instance=_Total
Value=0.46874700001919994
should I use the key value pair [collection="CPU Load"]?
something like this
[perfmon_cpu_load]
SOURCE_KEY = MetaData:collection
REGEX=CPU Load
DEST_KEY=_MetaData:Index
FORMAT=perfmon_cpu_load

0 Karma

mookiie2005
Communicator

Is their anyway to see the raw data as it comes to the heavy forwarder? You maybe right maybe these sourcetypes are being applied at/after the time of indexing.

0 Karma

lguinn2
Legend

The question that comes to mind is "when and how is the sourcetype assigned for Windows event logs?" Perhaps the sourcetyping is not complete at the time these transforms are run. So perhaps we need to find a different SOURCE_KEY and REGEX -- or look into altering the order of evaluation of the transformations...

0 Karma

lguinn2
Legend

You should not enable indexing on the forwarder - this will cause the data to be stored locally on the heavy forwarder, which you do not want.

0 Karma

mookiie2005
Communicator

Does indexing need to be enabled on the forwarder for this to work correctly?

0 Karma

mookiie2005
Communicator

This does not seem to be working the data is still only appearing in the "main" index. Anyone have any other ideas?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...