I've been attempting to create a regex in transforms.conf
that will keep events that have Value1 or Value2 and send all others to the nullQueue. My current expression works except that it will also keep the event if Value1 or Value2 occurs later in the event. I only care about the value for the first field value pair.
Sample events:
Event1 outgoing Field:Value1 username:value text:value Field:Value1 Field:Value2
Event2 outgoing Field:Value2 username:value text:value Field:Value1 Field:Value2
Event3 outgoing Field:Value3 username:value text:value Field:Value1 Field:Value2
Current regex: ^(?!.*?Field.(Value1|Value2)\b).*$
In the above example, I would only want to keep the first two event. Is there a way to make this work?
EDIT: Updated to reflect the actual question that was asked.
Well, that's still not full events, but far better. suggestion below requires that there 'outgoing' comes directly before your field/value pair, and that 'username' comes directly after (with spaces in between):
props.conf
[your_sourcetype]
TRANSFORMS-blah = setnull, keep_val_1_2
transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[keep_val_1_2]
REGEX = outgoing\sField:(Value1|Value2)\susername:\S+
DEST_KEY = queue
FORMAT = indexQueue
/K
EDIT: Updated to reflect the actual question that was asked.
Well, that's still not full events, but far better. suggestion below requires that there 'outgoing' comes directly before your field/value pair, and that 'username' comes directly after (with spaces in between):
props.conf
[your_sourcetype]
TRANSFORMS-blah = setnull, keep_val_1_2
transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[keep_val_1_2]
REGEX = outgoing\sField:(Value1|Value2)\susername:\S+
DEST_KEY = queue
FORMAT = indexQueue
/K
Thanks. This works. I didn't really think of using the other fields around it. I was trying to get the regex to stop after looking at the the first field:value in case there were changes to the logs.
didn't really read the question right. see update above.
The string "outgoing" will always appear before the first field:value that I want to check and "username" will always appear after. I updated the sample events so they are bit more structured.
How structured/predictable is the text leading up to the first field:value
pair? Sample events, please.
Can you base your decision on a fixed number of non-space, space sequences, or is there a particular string that will occur before the value you want to match?
Just updated it.
What is your current regex?