Getting Data In

spliting multiple feed that use udp:514

jarjoh42
Path Finder

I have multiple systems reporting over UDP:514. I want to separate the iron port email, Cisco ASA's, iseries as400, and PaloAlto firewall feeds at the global level before sending the data to the separate apps. I currently have this stanza in my etc\system\local\inputs file

[udp://514]
connection_host = ip

index = index_syslog

sourcetype = syslog

I know I need to write a stanza in the props.conf to separate out the individual sources by ip address but am not sure how to do it.

Tags (1)
0 Karma
1 Solution

jarjoh42
Path Finder

Final solution

I had to do all this in the splunk/system/local. this separated the host out of the data and set them into their own source type. Please note that this is still all in the same indexed. The next goal would to send each sourcetype to its own index. Also when I was doiing the calls in the app I had to use [sourcetype::cisco_asa] for it to accept the parameter.

inputs.conf

[udp://514]<
connection_host = ip
index = index_syslog

props.conf

[source::udp:514]
TRANSFORMS-changesourcetype = cisco_esa_st, cisco_asa_st, as400_st

transforms.conf

[cisco_asa_st]
REGEX = 192.x.x.x|192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_asa
DEST_KEY = MetaData:Sourcetype

[cisco_esa_st]
REGEX = 192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_esa
DEST_KEY = MetaData:Sourcetype

[as400_st]
REGEX = 192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::as400
DEST_KEY = MetaData:Sourcetype

View solution in original post

0 Karma

jarjoh42
Path Finder

Final solution

I had to do all this in the splunk/system/local. this separated the host out of the data and set them into their own source type. Please note that this is still all in the same indexed. The next goal would to send each sourcetype to its own index. Also when I was doiing the calls in the app I had to use [sourcetype::cisco_asa] for it to accept the parameter.

inputs.conf

[udp://514]<
connection_host = ip
index = index_syslog

props.conf

[source::udp:514]
TRANSFORMS-changesourcetype = cisco_esa_st, cisco_asa_st, as400_st

transforms.conf

[cisco_asa_st]
REGEX = 192.x.x.x|192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_asa
DEST_KEY = MetaData:Sourcetype

[cisco_esa_st]
REGEX = 192.x.x.x|192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::cisco_esa
DEST_KEY = MetaData:Sourcetype

[as400_st]
REGEX = 192.x.x.x
SOURCE_KEY = MetaData:Host
FORMAT = sourcetype::as400
DEST_KEY = MetaData:Sourcetype

0 Karma

jarjoh42
Path Finder

I used your template changed the names of the stanzas to match what the default valuse are for each of the apps as you can see.

[cisco_asa]
REGEX = 192\.20\.9\.9|192\.20\.9\.10
SOURCE_KEY = MetaData:Host
DEFAULT_VALUE = cisco_asa
DEST_KEY = MetaData:Sourcetype

[dspjrn:5]
REGEX = 192\.16\.9\.20
SOURCE_KEY = MetaData:Host
DEFAULT_VALUE = as400
DEST_KEY = MetaData:Sourcetype

is this data still indexed to the index_syslog. is there a way to change the index for just the dspjrn:5 source(from props.conf in iseries app)to index_iseries to match the index.conf in the iseries app's index.conf file.

I would leave this as a comment but it will not let me post it as comment

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

You'll probably want an index-time transform to do this. You can chain the transforms, and use REGEX to match which IPs you want to have specific sourcetypes.

props.conf

[syslog]
TRANSFORMS-forcesourcetype = cisco_asa_st, iron_port, i_as400

transforms.conf

[cisco_asa_st]
REGEX = 192\.168\.0\.1|192\.168\.0\.2
SOURCE_KEY = MetaData:Host
DEFAULT_VALUE = cisco_asa
DEST_KEY = MetaData:Sourcetype

[iron_port]
REGEX = 192\.168\.0\.1|192\.168\.0\.2
SOURCE_KEY = MetaData:Host
DEFAULT_VALUE = iron_port
DEST_KEY = MetaData:Sourcetype

[i_as400]
REGEX = 192\.168\.0\.1|192\.168\.0\.2
SOURCE_KEY = MetaData:Host
DEFAULT_VALUE = as400
DEST_KEY = MetaData:Sourcetype

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf

The other, probably easier way, is to use a syslog-ng server, use specific destinations for each ip, and then index the resulting output. http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/overview

alacercogitatus
SplunkTrust
SplunkTrust

You could perform Double Transforms (add a second stanza for each sourcetype and set the _MetaData:Index to the index you want). But to better utilize the processing power, you probably should just collect the syslog onto the server, and index from there.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...