Splunk Search

direct files in /var/log/atpco to different indexes and sourcetypes

steveirogers
Communicator

I am running Splunk 4.2.3.

I have a directory called "/var/log/atpco" which contains numerous log files.

I have played with all types of coding for whitelist and blacklist configurations but nothing seems to work.
My goals are as follows:
I would like to direct these specific files to a separate index and sourcetype as index=rules, soucetype=RulesOffline:
fmgpjob01_RulesApplyBC01.log
fmgpjob01_RulesCopyBC01.log
fmgpjob01_RulesQueryBC01.log
fmgpjob01_RulesQueryBC02.log
fmgpjob01_RulesQuickJobsBC01.log
fmgpjob01_RulesQuickJobsBC02.log
fmgpjob01_RulesRBDTableSaveBC01.log

I would like to direct these specific files to a separate index and sourcetype as index=fares, soucetype=FaresOffline:
fmgpjob01_ApplyBC01.log
fmgpjob01_CriteriaSetBC01.log
fmgpjob01_InquiryBC01.log
fmgpjob01_LoadBC01.log
fmgpjob01_QuickJobsBC01.log
fmgpjob01_QuickJobsBC02.log
fmgpjob01_StrikeoverBC01.log
fmgpjob01_ValidationJobsBC01.log
fmgpjob01_ValidationJobsBC02.log
fmgpjob01_ValidationJobsBC03.log

I want to ignore any other files in the /var/log/atpco directory.

Could anyone please provide some guidance on how to accomplish the above? Currently I get nothing with the existing configuration. Do I need to do this in props.conf?

Here is what I have now:

[monitor:///var/log/atpco]
disabled = false
index = rules
sourcetype = RulesOffline
whitelist = fmgpjob01_Rules[^/]*.log$

[monitor:///var/log/atpco]
disabled = false
index = fares
sourcetype = FaresOffline
whitelist = fmgpjob01_Apply*.log$
whitelist = fmgpjob01_CriteriaSet*.log$
whitelist = fmgpjob01_DDSAllAdds*.log$
whitelist = fmgpjob01_Inquiry*.log$
whitelist = fmgpjob01_QuickJobs*.log$
whitelist = fmgpjob01_Strikeover*.log$
whitelist = fmgpjob01_ValidationJobs*.log$

Tags (2)
0 Karma

MuS
Legend

Hi steveirogers

with the monitor stanza you have to do this in inputs.conf.

props.conf and transforms.conf can also be used for route and filter into different indexes, but then this would be for any kind of input.

cheers

steveirogers
Communicator

Thanks. I will do some more research on overlapping monitor stanza's then.

0 Karma

MuS
Legend

basically: yes. you can run 'splunk cmd btool --debug inputs list monitor' to see what your monitor stanzas will look at the end. I think the problem is that you have two monitor stanzas for the same directory.

0 Karma

steveirogers
Communicator

Thanks MuS, but I do not quite understand. Should my monitor stanza work as coded?

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...