Update: problem with syslog output [Solved]filter and route Windows event logs

[Ludwig_MDC]

Hello,

I am trying to set up a heavy forwarder that filters the received Events from a universal forwarder and other sources, send them all to another logging solution per syslog, and only a subset of logs should be sent for further use to a Splunk indexer.

For now I am only experimenting with Windows event logs to get a hang of proper filtering, but it simply doesn't work.
Configs on the heavy forwarder in %Splunk_Home%\etc\system\local

Inputs.conf

#no code here

Because I defined the input port for the Splunk forwarder in the Splunk Web, but it would be nice to now in what .conf file Splunk saves this port.

Outputs.conf

[tcpout]
defaultGroup = nothing

[tcpout:OutSplunk]
server = <server>:<port>

[syslog:OutLogger]
server=<server>:<port>
type=tcp

props.conf

[WinEventLog:Security]
TRANSFORMS-route=WinSecEvent-Splunk,Everything

[WinEventLog:Setup]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

[WinEventLog:System]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

[WinEventLog:Application]
TRANSFORMS-route=AllWinEvent-Splunk,Everything

transforms.conf

[WinSecEvent-Splunk]
REGEX=(?msi)(.*Keywords=(?!Audit\sSuccess).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk

[AllWinEvent-Splunk]
REGEX=(?msi)(.*Type=(Error|Warning).*)
DEST_KEY=_TCP_ROUTING
FORMAT=OutSplunk

[Everything]
REGEX=.
DEST_KEY=_SYSLOG_ROUTING
FORMAT=OutLogger

The syslog output somehow works, but the filtering for the Windows events not. The indexer gets all events.
To check my regular expressions I use https://regex101.com . Since splunk also uses perl compatible regular expressions (pcre), it should work in theory, but in practice the events don't get filtered.

Update 03.08.2015
11:00 AM
Instead of creating a new question I will ask it here
according to this site
http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog...
the syslog processor of splunk creates rfc 3164 compliant output even with windows event logs but my third-party logging solution receives every line of a win event as a syslog message and not the the whole event as one message.
I will try to contact the support / a splunk partner about it but maybe someone here knows the answer

And by the way the filtering still doesn't work even after playing around a bit with the regex

@maciep
you were right, I found the config for the splunk input set in the web gui in an app
it is in %Splunk_Home%\etc\apps\search\local\input.conf

01:00 PM

@MuS
thanks for the answer that solved the problem with the filtering
I thought that you define a default group for events and messages where you don't have routing rules, so that everything that doesn't have a stanza in props.conf and transforms.conf still gets routed to an default indexer.
By the way all data has to be routed to the third party system and a copy of a subset to splunk

Ok now I still have the problem with the syslog output that is s clearly not rfc 3164 conform
I tested it now with two different third party systems and in both every line of an windows event is received as one syslog message.

Update 04.08.2015

9:00 AM

@maciep

that works by adding this line to every stanza of the windows events

SEDCMD-rmlines=s/[\n\r\t]/ /g

I get the events as one liner to the third party system but now I also get every thing as one line to splunk
is there maybe a way to remove the line breaks in the Everything stanza of transforms.conf or do I really need a another splunk instance to convert windows event logs to syslog before forwarding the message because the syslog processor doesn't create proper output

thanks in advance for the help
Ludwig_mdc

[campbellj1977]

I do not see an answer here. Why was it closed?

[maciep · Accepted solution]

We got this from a vendor who I think got it from the comment section here: http://blogs.splunk.com/2013/10/14/windows-event-logs-in-splunk-6/

In our props on the heavy forwarder for this sourcetype, we have the following. It seems to format the data in a way that the target syslog server expects.

SHOULD_LINEMERGE = true
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
TRANSFORMS-FIELDS = strip-winevt-linebreaker
SEDCMD-win=s/(?mis)(Token Elevation Type indicates|This event is generated|Application Id=).*$//g
SEDCMD-rmlines=s/[\n\r\t]/ /g

[Ludwig_MDC]

it works but you will also get single line windows events in the splunk indexer
therefore I am a bit disappointed that splunk an enterprise logging solution that claims in its documentation to be able to create rfc 3164 compliant syslog output even with event logs
´http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Forwarddatatothird-partysystemsd#Syslog...
or does that only work if you have the heavy forwarder on every system instead of an universal forwarder

[woodcock]

It is unnecessary (maybe even problematic) to update your question's subject-line to indicate a solution. Just add the solution as an "Answer" (not a "Comment") and then click "Accept" on your Answer to close out the question and let others know that there is a validated solution.

[MuS]

Hi Ludwig_MDC,

Your Splunk sends everything to the third party server because you use the defaultGroup in your outputs.conf see the docs for details

defaultGroup = <target_group>, <target_group>, ...
* The forwarder sends all data to the specified groups.

cheers, MuS

[maciep]

If you're asking about the receiving port for the heavy forwarder, check inputs.conf. But since you set it in the GUI, it will likely be in one of your apps - like launcher or search etc. It won't be in system/local.

For your config, I would think everything would only be getting sent to the syslog server, so I'm obviously no help there. I'm working on a similar project and have been struggling with routing from a heavy forwarder, so a lot to learn yet there for me. But I do have a ticket open with Splunk so if I gain a better understanding and can apply here as well, I'll follow at that time.