Deployment Architecture

Filtering CISCO events on Forwarder

pfabrizi
Path Finder

I have a CISCO monitor file log on a lightweight forwarder. We want to blacklist specific events, like teardowns and builds. Is this down in the props.conf stanza or can I use a transform.conf and send those to nullque or does this go into the inputs.conf file?

Not sure where to do this.

Thanks!

Tags (1)
0 Karma
1 Solution

mayurr98
Super Champion

See the syntax in props.conf it should be

TRANSFORMS-null=cisco-setnull,cisco-extractions

Also in transforms.conf cisco-setnull regex remove .* only write the regex for the keyword present in an event. Also I am not sure if you have written correct regex. So give me sample event and tell me the keyword and i will write a regex for you!!

Let me know if this helps!

View solution in original post

0 Karma

mayurr98
Super Champion

See the syntax in props.conf it should be

TRANSFORMS-null=cisco-setnull,cisco-extractions

Also in transforms.conf cisco-setnull regex remove .* only write the regex for the keyword present in an event. Also I am not sure if you have written correct regex. So give me sample event and tell me the keyword and i will write a regex for you!!

Let me know if this helps!

0 Karma

pfabrizi
Path Finder

All I am looking for is the %ASA-?-302013.

I had this at first, but removed it when it didn't work, thought it needed to match the entire line.
so I originally had %ASA-\d-302013, I am using regex101.com to test.

The other question I had was on the [source::/trvapps/logs/cisco, can I use the index::sourcetype?

[network::cisco:asa]

I am also going to remove those extracts for the time being as they are not working wither.

Here is the event message:

Jan 25 06:36:26 ttxxxyyy %ASA-6-302013: Built inbound TCP connection 3984448672 for outside:10.xx.xx.xx/56724 (10.xx.xx.xx/56724)(LOCAL\Nrrrrr) to inside:10.xx.xx.xx/139 (10.xx.xx.xx/139) (Nrrrr)

the %ASA-6-302013 is what I am looking for. The number can be a range.

Thanks!

0 Karma

mayurr98
Super Champion

you do not need to match entire line only keyword is sufficient !
you can use sourcetype..suppose your sourcetype is cisco then write only [cisco]

0 Karma

pfabrizi
Path Finder

normally these are out on the deployment server, but I was changing it on the forwarder and just restarting splunk, this doesn't cause an issue does it?

Thanks!

0 Karma

mayurr98
Super Champion

you have to do this on the indexer or heavy forwarder. I am assuming that you are using universal forwarder so you have to do this on the indexer. It will not work if you do it on the forwarder.Cause this happens at parsing stage and parsing stage happens on the indexer.

0 Karma

pfabrizi
Path Finder

when I pushed the change using the below format from my deployment to server to my forwarder and indexer, it appeared to work.

props.cof:
[cisco:asa]
TRANSFORMS-null = cisco-setnul

transforms.conf
[cisco-setnull]
REGEX = 302013
DEST_KEY=queue
FORMAT = nullQueue

coincidence?

0 Karma

mayurr98
Super Champion

not coincidence as you have pushed the changes to indexer as well so that is why it is working,
also your regex seems to be incorrect as ^ means start of the event and your string appears to be in middle.
can you give all the 5 keywords that you want to remove?
and give me full keywords like %ASA-6-302013: till you encounter the space.
if you want to write by yourself then have a look at this regex
https://regex101.com/r/N9Lmnd/1

0 Karma

pfabrizi
Path Finder

I didn't create the regex another team did. These are what they sent:

^%ASA-[0-9]-302.$
^%ASA-[0-9]-304.
$
^%ASA-[0-9]-7220.$
^%ASA-[0-9]-106.
$
^%ASA-[0-9]-313.$
^%ASA-[0-9]-710.
$
^%ASA-[0-9]-725.*$
I think we don't know all the ranges, just how they start and we are hoping we don't need to write a transform for each, like 302013, 302014, 302015.

We use regex101 and these rules return what we want, but it seems like they don't work in SPLUNK as a transform?

Thanks for you assistance.

0 Karma

mayurr98
Super Champion

first of of it will never work if you put ^ in the regex as your event do not start with %ASA-[0-9]-302 and no you do not need to write transforms for everything.

just write one transform with regex

%ASA-[0-9]-(302|304|7220|106|313|710|725)\d{1,4}

this will cover only the ips you want and all the above ips given

0 Karma

pfabrizi
Path Finder

ok, Thanks, I will give this a try. One problem we are having is it creating a performance issue on the indexer, so I had to turn off rsyslog for a bit to let it catch up.

Thanks again.

0 Karma

mayurr98
Super Champion

no it will not create any performance issue. try and let me know if !!
put transforms and restart the indexer. If it does not work then provide some 4-5 different sample events that you want to exclude .

0 Karma

pfabrizi
Path Finder

so I thought it was working, but now I see the events I thought I was filtering out.

Here is an event:
Jan 26 08:14:38 aaaaaaaa %ASA-6-302014: Teardown TCP connection 4025971555 for outside:10.xx.xx.xxx/58118(LOCAL\Nzzz1) to inside:10.xx.xx.xxx/3163 duration 0:00:00 bytes 20 TCP Reset-I (Nzzzz1)

Here is my transform:
[cisco-setnull]
REGEX = %ASA-[0-9]-(302|304|313|106|7220|710|725)\d{1,4}
DEST_KEY = queue
FORMAT = nullQueue

Here is my props:
[cisco:asa]
TRANSFORMS-null = cisco-setnull

I validated that it is the same on my forwarder and indexers.

0 Karma

mayurr98
Super Champion

try this

%ASA-[0-9]-(302|304|313|106|7220|710|725)\d{1,4}:

also, after done with extractions give some time and then check...if you are checking past results they will not get filtered out only future events will get filter out.
are you checking historical events? historical events will not get filter out only the event which are gonna come will get filter out. i hope you are not searching for historical events!

0 Karma

pfabrizi
Path Finder

I made the change then waited a bit and then checked 15 minutes, however we are noticing that there is a backlog on the indexers (which is why we are trying to filter) so it is possible that is why I am seeing what I am seeing.

0 Karma

mayurr98
Super Champion

it does not matter do a real_time search as there my some events in the queue. check after half an hour

0 Karma

pfabrizi
Path Finder

I just checked and it appears it just took sometime. It does appear to be working, sorry I jumped the gun.

I am being asked to filter all the 7220* except the 722051, so I am guessing I need to add more to the transform?

0 Karma

mayurr98
Super Champion

okay then use this

 %ASA-[0-9]-(302|304|313|(7220(?!051))|106|710|725)\d{1,4}:

this must work now
check my work on
https://regex101.com/r/x5j5Vr/1

let me know if this helps!

kleber_silva
Engager

@mayurr98

It's work for me thanks

%ASA-[0-9]-(302|304|313|(7220(?!051))|106|710|725)\d{1,4}:

0 Karma

pfabrizi
Path Finder

can I use a transform to include the 722051?

so have a transforms:
[cisco-send722051]
REGEX = %ASA-[0-9]-(722051)

also my filtering out transform:
[cisco-setnull]
REGEX = %ASA-[0-9]-(304|303)\d{1,4}
DEST_KEY= queue
FORMAT = nullQueue

and then in props:

transforms-null = cisco-send722051,cisco-setnull

0 Karma

pfabrizi
Path Finder

can I ask a question about how REGEX works.

We want to this regex expression but doesn't appear to work.
REGEX = ^%ASA-[0-9]-72200.*$

this doesn't seem to work, We will have them for about 5 different series. Like 302.*
313.*

Thanks!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...