Filter event data and send to nullqueue - DOES NOT WORK

[vuong]

Hi,

I am running the free version 4.2 and trying to follow the instruction here http://www.splunk.com/base/Documentation/4.2/Deploy/Routeandfilterdatad#Discard_specific_events_and_... to filter out unwanted log entries. Here is what I have in Splunk\etc\system\local\transforms.conf and props.conf

in props.conf

[source::\\st-w1833\c$\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS\ST-W1833-20110320-0412.log]
TRANSFORMS-null = setnull

in transforms.conf

[setnull]
REGEX = ....\s+(Verbose|Medium|High)
DEST_KEY = queue
FORMAT = nullQueue

However, this setup is not working. I have a sample log below and all entries are indexed despite my filtering to send Medium and High to nullqueue. Did I do something wrong?

Thanks!

03/20/2011 04:12:21.12  wsstracing.exe (0x033C)                     0x1E50  SharePoint Foundation           Unified Logging Service         b9wt    High        Log retention limit reached.  Log file 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS\ST-W1833-20110306-0341.log' has been deleted.   
03/20/2011 04:12:21.12  wsstracing.exe (0x033C)                     0x1E50  SharePoint Foundation           Tracing Controller Service      8096    Information Usage log retention limit reached.  Some old usage log files have been deleted.  
03/20/2011 04:12:31.82  OWSTIMER.EXE (0x0DC0)                       0x1040  SharePoint Foundation           Monitoring                      nasq    Medium      Entering monitored scope (Timer Job job-timer-locks)    6d33e3a8-e3aa-4a1e-954a-5232e8ec4fee
03/20/2011 04:12:31.82  OWSTIMER.EXE (0x0DC0)                       0x1040  SharePoint Foundation           Monitoring                      b4ly    Medium      Leaving Monitored Scope (Timer Job job-timer-locks). Execution Time=1.82062245341237    6d33e3a8-e3aa-4a1e-954a-5232e8ec4fee
03/20/2011 04:12:34.82  OWSTIMER.EXE (0x0DC0)                       0x1DD0  SharePoint Foundation           Monitoring                      nasq    Medium      Entering monitored scope (Timer Job job-application-server-admin-service)   1bb9266b-1f82-4767-8b8b-17cfdbbb9746
03/20/2011 04:12:34.82  OWSTIMER.EXE (0x0DC0)                       0x1DD0  SharePoint Server Search        Administration                  dkd5    High        synchronizing search service instance   1bb9266b-1f82-4767-8b8b-17cfdbbb9746
03/20/2011 04:12:34.82  OWSTIMER.EXE (0x0DC0)                       0x1DD0  SharePoint Server Search        Administration                  eff0    High        synchronizing search data access service instance   1bb9266b-1f82-4767-8b8b-17cfdbbb9746

[Stephen_Sorkin]

I'd suspect an escaping issue in the source:: path being treated as a regex.

Could you try a props.conf stanza like:

[source::...ST-W1833-20110320-0412.log]
TRANSFORMS-null = setnull

[vuong]

Thank you for your suggestions. I tried the suggestion but it still refuses to work. Any other suggestion is greatly appreciated.

[ftk]

Actually it looks like you are trying to send Verbose, Medium, and High to the nullQueue, not just Medium and High. I would define my regex differently, rather than use four dots, I would use \w{4} for four word characters:

[setnull]
REGEX = \w{4}\s+(Verbose|Medium|High)
DEST_KEY = queue
FORMAT = nullQueue

The rest of your setup looks fine. If after the regex change it still doesn't work, make sure that your source specification is correct. I personally prefer to assign nullQueue routing entries based on sourcetypes rather than specific sources. You'll want to make sure that the unfiltered entries you see indexed are actually coming from that very specific file you configured with source:: and not from some other file in the same directory.

[vuong]

Thank you for your suggestions. I tried the suggestion but it still refuses to work. Any other suggestion is greatly appreciated.