Getting Data In

How to can I configure dynamic sourcetype assignment on a Universal Forwarder or a Heavy Forwarder?

santosh_sshanbh
Path Finder

I have a folder which has multiple log files in format CalculationMgr-xxx(xx).log and EventMgr-xxx(xx).log where xx is a numeric value. I tried configuring 2 separate monitor stanza on UF to monitor these log files but it didn't work. So I have to configure a single stanza as below

# Monitors CalculationMgr & EventMgr Log File
[monitor://D:\Program Files (x86)\LogFiles\]
disabled = false
source = Log
recursive = false
queue = parsingQueue
whitelist = (?i)CalculationMgr-\d+\(\d+\)\.log$|(?i)EventMgr-\d+\(\d+\)\.log$
_TCP_ROUTING = development_hf
followTail = 0
ignoreOlderThan = 10d

Now, I want to set separate source type for these 2 log files. So I tried doing this at both location UF and HF as per below configuration. But getting no success.

On UF

props.conf

[source::.../LogFiles/EventMgr*.log]
sourcetype = EventMgr1

[source::.../LogFiles/CalculationMgr*.log]
sourcetype = CalculationMgr1

On HF

props.conf

[source::Log]
TRANSFORMS-changesourcetype = set_sourcetype_calculationmgr, set_sourcetype_eventmgr

transforms.conf

[set_sourcetype_calculationmgr]
REGEX = (?i)^CalculationMgr\S+
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::CalculationMgr1

[set_sourcetype_eventmgr]
REGEX = (?i)^EventMgr\S+
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::EventMgr1

Any comment on what is wrong in this configuration? How can I achieve the results on Windows platform?

0 Karma

FrankVl
Ultra Champion

Can you share the separate input stanzas you had in your first attempt? Because it should very well be possible to set it up like that, which would make your whole config a lot simpler I guess.

0 Karma

ansif
Motivator

-Try by removing source override @UF inputs.conf.
-Make use of HF to do props.conf and transforms.conf and remove it from UF
- Make necessary changes to your configuration files.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...