I am setting up heavy forwarder on multiple machine, out of that one of them have below requirement,
1) Heavy forwarder should forward sub-set of events which match with given pattern and not all the events.
I search over web, I found couple of Q&A regarding the same but it’s seem not working for me.
First solutions I tried:
On the above solution, I can see in forwarder log, it’s continuously saying could not connect with 0.0.0.0:0 and connection failing but nothing is getting forwarded at Indexer side.
Second solution:
I tried with forwarding everything into NullQueue and forward only event which match with REGX by setting up two stanza into transforms.conf.
Seems that above solution also does not work for me.
Pattern which I want to match is following,
oauth.googleusercontent.com, ssl.gstatic.com, fb.com, twitter.com
Heavy forwarder should only send events to indexer if events matched with any of the above patterns.
Please let me know, if you have any question regarding the same.
I have already spend one full day in exploring above but could not find solution.
Sorry to raise this question in that manner but i feel very sad after investigating over a day and could not find solution, Below are the configuration i am suing at my heavy forwarder side.
props.conf
[host::
TRANSFORMS-set =setnull, allowtheseevents
transform.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[allowtheseevents]
REGEX = (?m)ssl.gstatic.com
FORMAT = splunkindexer_9997
inputs.conf
[monitor:///usr/local/logfilename.log]
index = main
sourcetype = filtersplunkproxy
blacklist = .gz
output.conf
[tcpout]
defaultGroup = splunkindexer_9997
[tcpout:splunkindexer_9997]
server =
[tcpout-server://
Please correct me if I am doing something wrong here.
The essential problem lies in the [allowtheseevents]
transform stanza. The FORMAT
shall be indexQueue
, not a reference to some outputs group.
Then you'll need to make your regex include more patterns.
/K
REGEX = (oauth\.googleusercontent|ssl\.gstatic|fb|twitter)\.com
Should work.
/k
Thanks Kristian, seems to be working this for me. If you don't mind, Please help me to have generate REGX for below pattern for OR condition with any one of oauth.googleusercontent.com,
ssl.gstatic.com,
fb.com,
twitter.com in anywhere in the events.
oh, and you need a DEST_KEY as well there;
[allowtheseevents]
REGEX = your regex
FORMAT = indexQueue
DEST_KEY = queue
/k
I have updated questions with config.
Asking us to do all the work for you is in my eyes pretty rude. Provide us with info on your nullQueue setup with relevant sections from the conf files you were talking about yourself, then we can have a look and see where you might be going wrong.