Getting Data In

Filtering events not working

remy06
Contributor

I've recently upgraded the forwarder to a universal forwarder on our app server.I'm collecting windows event logs as well as app logs on this server where previously parsing was done on it.

On the forwarder I've input for windows event logs and have configured to monitor the app log file. These are sent over to the indexer.Here's the configuration:

After the upgrade,I've moved the configuration to the indexer and it doesn't seem to work now.

app_log is the sourcetype for all logs collected from all app servers.I wish to filter the ip off on all app servers.
appserverA is the server that I wish to filter off eventcode 123,456,789,while the rest remain.

props.conf:

[app_log]  
TRANSFORMS-null = setnull 

 [host::appserverA]
TRANSFORMS-null = setnullappserverA

transforms.conf:

[setnull]
REGEX = (192\.168\.1\.10|192\.168\.1\.18)
DEST_KEY = queue
FORMAT = nullQueue

[setnullappserverA]
REGEX = EventCode=(123|456|789)
DEST_KEY = queue
FORMAT = nullQueue

An example of the event:

192.168.1.10 - - [09/Jan/2012:17:46:54] "message"   
192.168.1.18 - - [09/Jan/2012:17:46:55] "message"   

Filtering for the eventcode is successful,but not for the IP.Can anyone advise if I've missed something?

0 Karma

remy06
Contributor

Here's the output

search     [app_log]
system     ANNOTATE_PUNCT = True
system     BREAK_ONLY_BEFORE =
system     BREAK_ONLY_BEFORE_DATE = True
system     CHARSET = UTF-8
system     DATETIME_CONFIG = /etc/datetime.xml
search     EXTRACT-Time = (?i)^[^"]*"(?P<Time>[^ ]*)(?= )
system     HEADER_MODE =
system     LEARN_SOURCETYPE = true
system     LINE_BREAKER_LOOKBEHIND = 100
system     MAX_DAYS_AGO = 2000
system     MAX_DAYS_HENCE = 2
system     MAX_DIFF_SECS_AGO = 3600
system     MAX_DIFF_SECS_HENCE = 604800
system     MAX_EVENTS = 256
system     MAX_TIMESTAMP_LOOKAHEAD = 128
system     MUST_BREAK_AFTER =
system     MUST_NOT_BREAK_AFTER =
system     MUST_NOT_BREAK_BEFORE =
system     SEGMENTATION = indexing
system     SEGMENTATION-all = full
system     SEGMENTATION-inner = inner
system     SEGMENTATION-outer = outer
system     SEGMENTATION-raw = none
system     SEGMENTATION-standard = standard
system     SHOULD_LINEMERGE = True
system     TRANSFORMS =
system     TRANSFORMS-null = setnull
system     TRUNCATE = 10000
system     maxDist = 100
0 Karma

remy06
Contributor

any status?

0 Karma

_d_
Splunk Employee
Splunk Employee

What is the output of btool?

$SPLUNK_HOME/bin/splunk cmd btool --debug props list app_log

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi remy06

I think your props.conf stanza [app_log] does not match your sourcetype; had this kind of problem before with some WMI sourcetypes.

Also try to simplify your regex for the IP to this one (\d+\.){3}(10|18)

cheers,
MuS

remy06
Contributor

Hi MuS,
I've checked that [app_log] is the sourcetype.The sourcetype I've defined for application logs is app_log.Is there anything else I've missed?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...