Getting Data In

How to seperate different index and sourcetype?

johnwyane
New Member

HI,
I have one Splunk server. I would like to receive data from some servers and network devices.
1 I would send F5 (10.10.10.10 and 10.10.10.11) log and cisco router (172.16.16.10 etc.) and firewall Fortigate (10.10.10.20 & 10.10.10.21)log to my Splunk server by UDP 514.
2 Also, I would use light universal forwarder to send data of windows & Unix servers to my splunk server.
However, I would like to use different sourcetype and index to identify the different devices.
For example, sourcetype=win_AD index=win_AD , sourcetype=sun index=sun , sourcetype=asm index=asm , sourcetype=cisco index=cisco , sourcetype=forti index=forti

After I study some manuals and some of splunkbase answers. I know I have to modified props.conf and transforms.conf .
1 How to configure above two files?
2 How to setup unix and window AD with universal forwarder configured file?
Any one could give me a complete method to set it up?
Thank you very much!

Anthony

Tags (2)
0 Karma

johnwyane
New Member

I know how to set index using Splunk web interface.
Also, I know how to receive the data from UDF514.

But I don't know about the meaning of "Format" and "$0" and "metadata" "_metadata" even I check the transforms.conf.sample file.
1 So anyone can explain it ?
2 Also, if possible , correct my configuration if anything is wrong!!
3 Should I assign different sourcetype for F5, cisco, fortigate? How? Which file?
Thank you very much!

Here is my configuration
props.conf
[F5_syslog]
TRANSFORMS-F5=asm_log

[cisco-router_syslog]
TRANSFORM-cisco=cisco-router

[fortigate_syslog]
TRANSFORM-forti=fortigate

transforms.conf
[asm_log]
REGEX = 10.10.10.10|10.10.10.11
DEST_KEY = _MetaData:Index
FORMAT = F5_ASM

[cisco-router]
REGEX = 172.16.16.*
DEST_KEY = _MetaData:Index
FORMAT = cisco_log

[Fortigate]
REGEX = 10.10.10.20|10.10.10.21
DEST_KEY = _MetaData:Index
FORMAT = Forti_log

For my 2nd question
at windows server (with forwarder):
I do nothing just using windows-ta-app.

at unix server(with forwarder):
[monitor:///var/log]
sourcetype=unix
index=unix

0 Karma

kristian_kolb
Ultra Champion

You'll need to ensure that the incoming data on port 514 will be separated into different sourcetypes before this configuration will work (since you are making transforms based on sourcetypes, right).

Also, quite separately, your REGEXes might be a bit too unspecific, since they will match on the existence of a string (in your case ip-addresses) regardless of where they appear in the message. Perhaps the addition of a SOURCE_KEY attribute in your transforms can help you with that (i.e. ensure that the REGEX is matching on the host and not on the whole _raw event.

0 Karma

Ayn
Legend

What information are you missing from the docs/splunkbase answers you found? We can help, but we can't do all the work for you - you need to understand the mechanisms before you can implement this.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...