Getting Data In

How to seperate different index and sourcetype?

johnwyane
New Member

HI,
I have one Splunk server. I would like to receive data from some servers and network devices.
1 I would send F5 (10.10.10.10 and 10.10.10.11) log and cisco router (172.16.16.10 etc.) and firewall Fortigate (10.10.10.20 & 10.10.10.21)log to my Splunk server by UDP 514.
2 Also, I would use light universal forwarder to send data of windows & Unix servers to my splunk server.
However, I would like to use different sourcetype and index to identify the different devices.
For example, sourcetype=win_AD index=win_AD , sourcetype=sun index=sun , sourcetype=asm index=asm , sourcetype=cisco index=cisco , sourcetype=forti index=forti

After I study some manuals and some of splunkbase answers. I know I have to modified props.conf and transforms.conf .
1 How to configure above two files?
2 How to setup unix and window AD with universal forwarder configured file?
Any one could give me a complete method to set it up?
Thank you very much!

Anthony

Tags (2)
0 Karma

johnwyane
New Member

I know how to set index using Splunk web interface.
Also, I know how to receive the data from UDF514.

But I don't know about the meaning of "Format" and "$0" and "metadata" "_metadata" even I check the transforms.conf.sample file.
1 So anyone can explain it ?
2 Also, if possible , correct my configuration if anything is wrong!!
3 Should I assign different sourcetype for F5, cisco, fortigate? How? Which file?
Thank you very much!

Here is my configuration
props.conf
[F5_syslog]
TRANSFORMS-F5=asm_log

[cisco-router_syslog]
TRANSFORM-cisco=cisco-router

[fortigate_syslog]
TRANSFORM-forti=fortigate

transforms.conf
[asm_log]
REGEX = 10.10.10.10|10.10.10.11
DEST_KEY = _MetaData:Index
FORMAT = F5_ASM

[cisco-router]
REGEX = 172.16.16.*
DEST_KEY = _MetaData:Index
FORMAT = cisco_log

[Fortigate]
REGEX = 10.10.10.20|10.10.10.21
DEST_KEY = _MetaData:Index
FORMAT = Forti_log

For my 2nd question
at windows server (with forwarder):
I do nothing just using windows-ta-app.

at unix server(with forwarder):
[monitor:///var/log]
sourcetype=unix
index=unix

0 Karma

kristian_kolb
Ultra Champion

You'll need to ensure that the incoming data on port 514 will be separated into different sourcetypes before this configuration will work (since you are making transforms based on sourcetypes, right).

Also, quite separately, your REGEXes might be a bit too unspecific, since they will match on the existence of a string (in your case ip-addresses) regardless of where they appear in the message. Perhaps the addition of a SOURCE_KEY attribute in your transforms can help you with that (i.e. ensure that the REGEX is matching on the host and not on the whole _raw event.

0 Karma

Ayn
Legend

What information are you missing from the docs/splunkbase answers you found? We can help, but we can't do all the work for you - you need to understand the mechanisms before you can implement this.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...