Hello,
I'm currently creating a new sourcetype that has a TRANSFORMS-null
setting with value discardit
. Within my transforms.conf
file I have created the appropriate stanza:
[discardit]
REGEX=(^|[\r\n])(.+,.+,.+,.+,"",.+)
DEST_KEY = queue
FORMAT = nullQueue
The regex that I've written is currently not working and I would like some help in formulating it.
The events are coming from csv files with the following header:
"Server","Job","Status","Start","End","Run Machine","Command"
My objective is to not index the events that have no value for End
, so events that look like this:
"Server1","Job1","Running","2018-02-01 00:00:00","","Machine 1","Command 1"
The regex that I've written basically counts 4 commas and then checks to see whether the two characters after the fourth comma are ""
(meaning that the value for Run
is blank).
What am I missing?
Best regards,
Andrew
Where is this props/transforms in place? On the indexer, heavy forwarder, universal forwarder, or search head?
@micahkemp They are both in the $SPLUNK_HOME/etc/apps//local folder. It's a single server instance.
Do you really need that complex bit with the \r\n?
Shouldn't this regex suffice (https://regex101.com/r/rbEHqz/1):
REGEX=^.+,.+,.+,.+,"",.+
Apart from that: what does your Splunk environment look like? Single instance, or distributed? If distributed, where did you deploy this config (should be on the first heavy instance, either HF or Indexer)?
@FrankVl Thanks for the suggestion. This doesn't solve the problem, and I've no idea why. I will continue to make changes and test until something starts working.
My Splunk environment is a single instance. The config is in the $SPLUNK_HOME/etc/apps//local folder.
If you have any additional suggestions then please let me know!
Hi @andrewtrobec,
you need to escape quotes by using back slashes
try this:
[discardit]
REGEX=(^|[\r\n])(.+,.+,.+,.+,\"\",.+)
DEST_KEY = queue
FORMAT = nullQueue
Thanks for the suggestion. unfortunately adding the back slashes did not work. Do you have any other suggestions?
Is your props.conf and transforms.conf files are in indexer/HWF?
I tried regex in splunk, it works as expected..
|makeresults|eval _raw="\"Server\",\"Job\",\"Status\",\"Start\",\"End\",\"\",\"Command\""|rex "(?<nulldata>(^|[\r\n])(.+,.+,.+,.+,\"\",.+))"
Yes, I am currently working on a single server instance. To test I am using the Add data wizard to see whether the lines disappear in the preview, and they do not. Am I correct to expect them to get removed there, or do I need to go ahead and index?