Getting Data In

How do you send Syslog from one source to different Paths?

bobbychanthongp
Explorer

I have a firewall index that is currently receiving syslog data. From that data I want to copy only certain logs from the firewall and send them to another folder so that the UF can send them on to a different index. I do not want to change the original firewall data, but merely make a copy of the data that I want to be sent to another path. An example is below.

Scenario:
I have an index=firewall. It ingests subnets 10.10.x.y, 10.20.x.y and 10.30.x.y. I want to take any data concerning to subnet 10.20.x.y and copy them into another index (index=firewall2).

Outcome: index=firewall has 10.10.x.y, 10.20.x.y, and 10.30.x.y. index=firewall2 has 10.20.x.y

  1. How can this be achieved?
  2. Will this increase my volume consumption?
  3. If it does increase my volume ingestion, is there a work around? (Could something be done post ingestion?)

vishaltaneja070
Motivator

Hello @bobbychanthongphio

As per me the best way to do is using collect command:

<search to filter events of 10.20.x.y> | collect index=firewall2 source=source sourcetype=sourcetype testmode=false
0 Karma

bobbychanthongp
Explorer

Does this search create a new index? My overall goal is to monitor the different subnet from the firewall and report on volume consumption for the index. Will this acheive that?

0 Karma

vishaltaneja070
Motivator

Index need to be already in place then only it will work. And also it will just copy what ever events there to new index and yes you can used to report volume.

0 Karma

whrg
Motivator

Hello @bobbychanthongphio,

If you are using Linux: I found this article Splunk Success with Syslog very helpful.

It explains how to setup rsyslog to create a separate folder for each incoming log source.

Since you are interested in IP addresses, take this modification into account:

...
$ModLoad imudp
$template RemoteHost, "/logs/%FROMHOST-IP%/syslog"
...

Now you can use wildcards in your inputs.conf:

[monitor:///logs/10.10.*/syslog]
index=firewall
...
[monitor:///logs/10.20.*/syslog]
index=firewall2
...

Regarding question 2 and 3: I don't think it will increase your Splunk license usaage, because the size of the indexed logs stays the same.

bobbychanthongp
Explorer

We currently have a syslog server that all logs are sent to. They are ingested in their respective folder. The firewall log is one source and they are needing to be copied into separate indexes.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...