Hi!
I have some different sourcetypes defined by me where I'm extracting some of the fields with stanzas in transforms.conf at search time (I'm using REPORT in props.conf). Here is one example of a stanza I'm using:
REGEX=(?im)[\r\n]+([^\r\n]*name)\: ([^\r\n]+)
FORMAT=$1::$2
MV_ADD=true
This extracts the fields I want, but since I extract the field name like this, the field name may be in uppercase, lowercase or a combination.
Creating new stanzas for each field is not an option since I have a lot of fields and most of my stanzas are of the form shown above, where I just define the ending of the field name, to be able to extract most of them.
I wonder if there is any way to "force" the field names to lowercase?
If you look at the answer to a slightly different question (Dealing with key/value pairs with inconsistent key case) the solution is, perhaps, to pre-process the log stream at input time to convert to lower case with sed commands.
It does seem right to convert all characters to lower case in transforms.conf, in the spirit of -
[syslog-header-stripper-ts]
REGEX = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s(.*)$
FORMAT = $1
DEST_KEY = _raw
Sorry, but if I'm right, this transforms just STRIPS out the syslog header. It doesn't convert it to lowercase.
If you look at the answer to a slightly different question (Dealing with key/value pairs with inconsistent key case) the solution is, perhaps, to pre-process the log stream at input time to convert to lower case with sed commands.
So - you got it to work in the end?
I tried that, but didn't get it to work, hence my confused comments on that answer.
I don't really understand the regex (s/\([A-Za-z0-9]*)/\1\L\2\g) either, I haven't found any information about it(I'm used to regexes so it is only this one that confuses me 🙂 )
However, I tried this approach again, by adding it to my default-clause in props.conf, restarting Splunk and index new files, but I can't get it to work.. :S
It is for SEDCMD in props.conf and it looks like the author got it wrong. Part of the string is regex and the other parts are code: s/regex/text/g
"s" means replace and "g" means global
He should have written:
SEDCMD-contacts = s/([A-Za-z]*)/\L\1/g
Keep in mind that this only changes the raw text at index time and not the field names after extraction at search time, but if you are extracting the field name from the raw text then search time extractions will extract all lower case.
My mistake - you're absoutely right, field names are case sensitive. I don't have a good solution right now though, sorry.
The field values are case insensitive, but the field names are not.
If I have a field called Machine_name, and run a search for Machine_name=* I get the events I'm looking for, but if I search fort machine_name=* instead, I get no results..
I guess this is mostly for cosmetic purposes? Because field names are case insensitive.
Fields names ARE case sensitive!!! Values aren't.
stats and where both care about case, to name a couple. I suggest using Calculated fields at search time.