Getting Data In

How to seperate different index and sourcetype?

johnwyane
New Member

HI,
I have one Splunk server. I would like to receive data from some servers and network devices.
1 I would send F5 (10.10.10.10 and 10.10.10.11) log and cisco router (172.16.16.10 etc.) and firewall Fortigate (10.10.10.20 & 10.10.10.21)log to my Splunk server by UDP 514.
2 Also, I would use light universal forwarder to send data of windows & Unix servers to my splunk server.
However, I would like to use different sourcetype and index to identify the different devices.
For example, sourcetype=win_AD index=win_AD , sourcetype=sun index=sun , sourcetype=asm index=asm , sourcetype=cisco index=cisco , sourcetype=forti index=forti

After I study some manuals and some of splunkbase answers. I know I have to modified props.conf and transforms.conf .
1 How to configure above two files?
2 How to setup unix and window AD with universal forwarder configured file?
Any one could give me a complete method to set it up?
Thank you very much!

Anthony

Tags (2)
0 Karma

johnwyane
New Member

I know how to set index using Splunk web interface.
Also, I know how to receive the data from UDF514.

But I don't know about the meaning of "Format" and "$0" and "metadata" "_metadata" even I check the transforms.conf.sample file.
1 So anyone can explain it ?
2 Also, if possible , correct my configuration if anything is wrong!!
3 Should I assign different sourcetype for F5, cisco, fortigate? How? Which file?
Thank you very much!

Here is my configuration
props.conf
[F5_syslog]
TRANSFORMS-F5=asm_log

[cisco-router_syslog]
TRANSFORM-cisco=cisco-router

[fortigate_syslog]
TRANSFORM-forti=fortigate

transforms.conf
[asm_log]
REGEX = 10.10.10.10|10.10.10.11
DEST_KEY = _MetaData:Index
FORMAT = F5_ASM

[cisco-router]
REGEX = 172.16.16.*
DEST_KEY = _MetaData:Index
FORMAT = cisco_log

[Fortigate]
REGEX = 10.10.10.20|10.10.10.21
DEST_KEY = _MetaData:Index
FORMAT = Forti_log

For my 2nd question
at windows server (with forwarder):
I do nothing just using windows-ta-app.

at unix server(with forwarder):
[monitor:///var/log]
sourcetype=unix
index=unix

0 Karma

kristian_kolb
Ultra Champion

You'll need to ensure that the incoming data on port 514 will be separated into different sourcetypes before this configuration will work (since you are making transforms based on sourcetypes, right).

Also, quite separately, your REGEXes might be a bit too unspecific, since they will match on the existence of a string (in your case ip-addresses) regardless of where they appear in the message. Perhaps the addition of a SOURCE_KEY attribute in your transforms can help you with that (i.e. ensure that the REGEX is matching on the host and not on the whole _raw event.

0 Karma

Ayn
Legend

What information are you missing from the docs/splunkbase answers you found? We can help, but we can't do all the work for you - you need to understand the mechanisms before you can implement this.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...