I'm thinking this might required a custom search command which I'd like to try to avoid if possible.
I have about 10 bitwise fields in a dataset with each bit in each field representing an alarm condition.
This works just fine however I would like to create something more generic if at all possible
| eval bit0Value=if(floor(bitField)%2>0,"on","off")
| eval bit1Value=if(floor(bitField/2)%2>0,"on","off")
| eval bit2Value=if(floor(bitField/4)%2>0,"on","off")
| eval bit3Value=if(floor(bitField/8)%2>0,"on","off")
So the above takes the field 'bitField' and creates a new field for each bit that is set.
This works and I could implement it as is with 10 different macros for each of my 10 bitwise fields.
However ideally I would prefer to have one macro to do it all. Something along the lines of a call like this
index=blah sourcetype=blah
| `myBitDecodingMacro("bit0Value, bit1Value, bit2Value, bit3Value",bitField)`
Where the first string passed to the macro is the fieldnames I want to apply to each bit that is set in the bitwise field.
Is this even possible with a macro?
Am I going to have to suck it up and write a custom search command?
or more likely just have 10 different macros one for each bitwise field.
Come back and pick an answer and click Accept
to close the question and award your bounty!
I'd create a lookup:
| makeresults count=65536
| streamstats c
| eval value_dec=tostring(c-1)
| eval value_0xhex=tostring((c-1), "hex")
| eval value_hex=replace(value_0xhex, "^0x", "")
| eval value_bin=value_hex
| rex mode=sed field=value_bin "s/0/0000/g s/1/0001/g s/2/0010/g s/3/0011/g s/4/0100/g s/5/0101/g s/6/0110/g s/7/0111/g s/8/1000/g s/9/1001/g s/a|A/1010/g s/b|B/1011/g s/c|C/1100/g s/d|D/1101/g s/e|E/1110/g s/f|F/1111/g"
| eval value_bin=substr("0000000000000000", 0, max(16-len(value_bin), 0)) . value_bin
| rex field=value_bin "(?<bit__15>.)(?<bit__14>.)(?<bit__13>.)(?<bit__12>.)(?<bit__11>.)(?<bit__10>.)(?<bit__09>.)(?<bit__08>.)(?<bit__07>.)(?<bit__06>.)(?<bit__05>.)(?<bit__04>.)(?<bit__03>.)(?<bit__02>.)(?<bit__01>.)(?<bit__00>.)"
| table value_dec value_hex value_0xhex value_bin bit__*
| outputlookup convert_value.csv
And then use this lookup to convert any number to any radix and have all my bits automatically extracted:
| makeresults count=111 | streamstats c
| lookup convert_value.csv value_dec AS c | table c value_* bit__*
Resulted lookup (even for 65536 values) is <5MB and will work really fast. You can make it even faster by generating smaller lookup table (if you don't need all 16 bits)
No unfortunately this macro was meant to simplify the creation and population of them.
It's all good though don't try too hard. I solved it with 4 different macros as there was some commonality with some of the groupings.
I did a very similar method to your examples. Great minds think alike. Two examples below.
MacroSimple(fieldName,bitField)
eval bankA_$fieldName$=if(floor($bitField$)%2>0,1,0)
| eval bankB_$fieldName$=if(floor($bitField$/2)%2>0,1,0)
| eval bankC_$fieldName$=if(floor($bitField$/4)%2>0,1,0)
| eval bankD_$fieldName$=if(floor($bitField$/8)%2>0,1,0)
| eval bankE_$fieldName$=if(floor($bitField$/16)%2>0,1,0)
| eval bankF_$fieldName$=if(floor($bitField$/32)%2>0,1,0)
The most complicated one which actually spanned 3 integers and instead of making many fields populated a single field with a multivalue set of results based on which bit was set.
bitFieldGensetDecoding(fieldName,bitFieldA,bitFieldB,bitFieldC)
eval gen$fieldName$=if(floor($bitFieldA$)%2>0,"Gen01$fieldName$","")
| eval gen$fieldName$=if(floor($bitFieldA$/2)%2>0,mvappend(gen$fieldName$,"Gen02$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/4)%2>0,mvappend(gen$fieldName$,"Gen03$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/8)%2>0,mvappend(gen$fieldName$,"Gen04$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/16)%2>0,mvappend(gen$fieldName$,"Gen05$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/32)%2>0,mvappend(gen$fieldName$,"Gen06$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/64)%2>0,mvappend(gen$fieldName$,"Gen07$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/128)%2>0,mvappend(gen$fieldName$,"Gen08$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/256)%2>0,mvappend(gen$fieldName$,"Gen09$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/512)%2>0,mvappend(gen$fieldName$,"Gen10$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/1024)%2>0,mvappend(gen$fieldName$,"Gen11$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/2048)%2>0,mvappend(gen$fieldName$,"Gen12$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/4096)%2>0,mvappend(gen$fieldName$,"Gen13$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/8192)%2>0,mvappend(gen$fieldName$,"Gen14$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/16384)%2>0,mvappend(gen$fieldName$,"Gen15$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldA$/32768)%2>0,mvappend(gen$fieldName$,"Gen16$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$)%2>0,mvappend(gen$fieldName$,"Gen17$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/2)%2>0,mvappend(gen$fieldName$,"Gen18$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/4)%2>0,mvappend(gen$fieldName$,"Gen19$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/8)%2>0,mvappend(gen$fieldName$,"Gen20$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/16)%2>0,mvappend(gen$fieldName$,"Gen21$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/32)%2>0,mvappend(gen$fieldName$,"Gen22$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/64)%2>0,mvappend(gen$fieldName$,"Gen23$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/128)%2>0,mvappend(gen$fieldName$,"Gen24$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/256)%2>0,mvappend(gen$fieldName$,"Gen25$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/512)%2>0,mvappend(gen$fieldName$,"Gen26$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/1024)%2>0,mvappend(gen$fieldName$,"Gen27$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/2048)%2>0,mvappend(gen$fieldName$,"Gen28$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/4096)%2>0,mvappend(gen$fieldName$,"Gen29$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/8192)%2>0,mvappend(gen$fieldName$,"Gen30$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/16384)%2>0,mvappend(gen$fieldName$,"Gen31$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldB$/32768)%2>0,mvappend(gen$fieldName$,"Gen32$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldC$)%2>0,mvappend(gen$fieldName$,"Gen33$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldC$/2)%2>0,mvappend(gen$fieldName$,"Gen34$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldC$/4)%2>0,mvappend(gen$fieldName$,"Gen35$fieldName$"),gen$fieldName$)
| eval gen$fieldName$=if(floor($bitFieldC$/8)%2>0,mvappend(gen$fieldName$,"Gen36$fieldName$"),gen$fieldName$)
Good times 🙂
I really do not think anything better can be done with core SPL.
If the client wants to spend the dev time I could whip up a custom search command which would do it. If they do I'll post the code up here or package it up as an app for Splunkbase (if they allow custom search commands as an app???)
If you just need a macro, then this will do:
Name=myBitDecodingMacro(5)
Definition=eval $bit0Value$=if(floor($bitField$)%2>0,"on","off") \
| eval $bit1Value$=if(floor($bitField$/2)%2>0,"on","off") \
| eval $bit2Value$=if(floor($bitField$/4)%2>0,"on","off") \
| eval $bit3Value$=if(floor($bitField$/8)%2>0,"on","off")
Arguments=bit0Value, bit1Value, bit2Value, bit3Value, bitField
Tested A-OK; compare this:
| makeresults
| eval bitField=11
| eval bit0Value=if(floor(bitField)%2>0,"on","off")
| eval bit1Value=if(floor(bitField/2)%2>0,"on","off")
| eval bit2Value=if(floor(bitField/4)%2>0,"on","off")
| eval bit3Value=if(floor(bitField/8)%2>0,"on","off")
To this:
| makeresults
| eval bitField=11
| rename COMMENT AS "Everything above fakes your data; everything below is your solution"
| `myBitDecodingMacro(bit0Value, bit1Value, bit2Value, bit3Value ,bitField)`
Or even more simply, like this:
Name=myBitDecodingMacro(2)
Definition=eval $prefix$0Value=if(floor($bitField$)%2>0,"on","off") | eval $prefix$1Value=if(floor($bitField$/2)%2>0,"on","off") | eval $prefix$2Value=if(floor($bitField$/4)%2>0,"on","off") | eval $prefix$3Value=if(floor($bitField$/8)%2>0,"on","off")
Arguments=prefix, bitField
Used like this:
| makeresults
| eval bitField=11
| rename COMMENT AS "Everything above fakes your data; everything below is your solution"
| `myBitDecodingMacro(bit ,bitField)`
Thanks for the reply. That is essentially what I am doing already with macros but what I was hoping was something like this. Which could create anywhere from 1 - 32 fields depending on what was passed in for the first field.
myBitDecoding("alarm1, alarm2, doorSensor1, doorSensor4", bitField)
and then reuse the same macro for another decoding of completely different data with only maybe two bitfields to extract
myBitDecoding("motionDetectorHallway,motionDetectorFridge", bitField)
then again for a massive set of bits
myBitDecoding("highTempAlarm,lowTempAlarm,highRPM,lowRPM,noPower,myBrainWentOfflineTryingToMakeThisAGenericMacro", bitField)
Dynamically being able to provide the fieldnames is what I was aiming for.
If the fields exist with any non-null value already, then I think that I can make it work, but not if the fields do not already exist. Is mass field initialization to zero doable?