Splunk Search

How do I get Splunk to use meaningful variable name labels?

teejayvee
Explorer

I'm a Newish Splunk Power-user. I have indexed results from analyzed emails from the publicly available Enron /maildir, timestamp of the email and some various metadata along with 419 variables (var1-var419) whose value indicates a weight of a classifier(s) assigned to an email.

e.g.;

var1,Culture5:Emotion:T:9900:happy
var2,Culture5:Emotion:T:9910:humble
...
var418,Culture0:Ontology:T:262:bungee jumping
var419,Culture0:Ontology:T:244:antiquities

I have uploaded a lookup table in the above format (vname, rlabel) with an entry for each var###.

How do I get Splunk to use the "meaningful" value as the displayed fieldname whenever I apply it in tables, charts, etc?

sample SPL; "dummy" => "insert var33-label here"."insert var48-label here"

sourcetype=csv  | 
bin e_datetime as e_day span=1d |
eval e_worried=(var48 / var33) as "i" | where var33 * v44 > 0 |  
eventstats count as e_count avg("dummy" ) as "dummy" by e_day |
table e_day, e_count, "dummy"  | 
chart avg(e_count) as avg_count, avg("dummy" ) as "dummy" over e_datetime by e_day

I'll eventually be monitoring for anomalous values, hence lots of streamstats and eventstats to transform the varying measured incidence of var1-var419 frequency and varied weight calculations by user, time, "context", etc.

How can I use the notion that: anything that originates from var33 needs to carry the label "var1,Culture5:Emotion:T:9900:happy", or at least be used in eval expressions?

i.e.;

eval indexed_value = (var48 / var33)...

equates to "consequences / troubles" defined by me as "a worry index"...

eval indexed_value = (var48 / var33) as "label for var33"." / "."label for var48)

I cannot figure out how to "get to" the lookup value to harness it as a string I can then use to do other things with that help me label the results of expressions at least semi-meaningfully...

var48 / var33 => "consequence / trouble" => worried to a "human editor" but at least "consequence / trouble" has some meaning if not the exact meaning i'd give it offline when I had the time to study it! ;^D

Thanks!

0 Karma
1 Solution

teejayvee
Explorer

Hoping not to sound impatient, this is pretty simple. In SAS, where the indexed files originated after getting it from a different semantic parser,) it's called a variable-label. In Matlab, too. SPSs as well I think!

It's just a simple display substitution.

For any variable in a list of specific variables named var1-var419, for obvious reasons) that are infuriating and needlessly meaningless labels in any display, to be substituted with a text field in a one-off lookup using "var###" as the "key" in a 420 record lookup file (line one the lookup car names I configured in the lookup dialogue when I uploaded it.

; ^D

Thanks. I am all alone here. My own SysAd.
too. That can't be good!

var33

152

T149:Probe:5:Emotion:Troubles

152

from lookup.csv entry...

vname,rlabel
var1,"redacted"
var2,"redacted"

var33, T149:Probe:5:Emotion:Trouble
...
var48,T272:Issues:0:Consequences
...
var419,"redacted"

View solution in original post

0 Karma

teejayvee
Explorer

Hoping not to sound impatient, this is pretty simple. In SAS, where the indexed files originated after getting it from a different semantic parser,) it's called a variable-label. In Matlab, too. SPSs as well I think!

It's just a simple display substitution.

For any variable in a list of specific variables named var1-var419, for obvious reasons) that are infuriating and needlessly meaningless labels in any display, to be substituted with a text field in a one-off lookup using "var###" as the "key" in a 420 record lookup file (line one the lookup car names I configured in the lookup dialogue when I uploaded it.

; ^D

Thanks. I am all alone here. My own SysAd.
too. That can't be good!

var33

152

T149:Probe:5:Emotion:Troubles

152

from lookup.csv entry...

vname,rlabel
var1,"redacted"
var2,"redacted"

var33, T149:Probe:5:Emotion:Trouble
...
var48,T272:Issues:0:Consequences
...
var419,"redacted"

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Can we have some sample entries from sourcetype=csv ?

0 Karma

teejayvee
Explorer

one record; all var1-var419 are very sparse (mostly 0's);

1/11/17
11:04:21.000 AM 
04MAY2000:01:10:00.000000,zufferli-j,zufferli-j\all_documents\2,1,2000,Q2_2000,2,kwright@momentumcars,ohn.zufferli@enron.c,momentum motor cars,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,0,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
e_datetime =    04MAY2000:01:10:00.000000 e_from =  kwright@momentumcars e_original_timestamp = 1 e_quarter =   Q2_2000 e_sequence =    2 e_subject =   momentum motor cars e_to =  ohn.zufferli@enron.c e_user =   zufferli-j e_year = 2000 host = cosmos-MacBook-Pro.local index =    main linecount =    1 source =  /Users/cosmo/Rawdata/Enron/Scored/allmails_rw_scored_2012_sp.csv sourcetype =   enron splunk =  zufferli-j\all_documents\2 splunk_server =  cosmos-MacBook-Pro.local var1 = 2 var10 =   0 var100 =  0 var102 =  0 var109 =  0 var11 =   0 var114 =  0 var116 =  0 var118 =  0 var12 =   0 var120 =  0 var123 =  0 var125 =  0 var129 =  0 var13 =   0 var14 =   0 var145 =  0 var15 =   0 var16 =   0 var161 =  0 var164 =  0 var167 =  0 var169 =  0 var17 =   1 var172 =  0 var178 =  0 var18 =   0 var183 =  0 var19 =   0 var190 =  0 var2 =    2 var205 =  0 var207 =  0 var21 =   0 var211 =  0 var22 =   0 var221 =  0 var23 =   0 var24 =   0 var25 =   0 var256 =  0 var26 =   0 var27 =   0 var288 =  0 var29 =   0 var292 =  0 var3 =    0 var30 =   0 var31 =   0 var32 =   0 var33 =   0 var34 =   0 var36 =   0 var37 =   0 var39 =   0 var4 =    1 var40 =   0 var41 =   0 var42 =   0 var48 =   0 var49 =   0 var5 =    1 var51 =   0 var52 =   0 var53 =   0 var54 =   0 var59 =   0 var6 =    1 var61 =   0 var62 =   0 var63 =   0 var64 =   0 var65 =   1 var66 =   0 var7 =    0 var71 =   0 var73 =   0 var74 =   0 var75 =   0 var79 =   0 var8 =    1 var81 =   0 var86 =   0 var9 =    1 var90 =   0 var91 =   0 var92 =   0 var94 =   0 var97 =   0 var99 =   1
0 Karma

somesoni2
SplunkTrust
SplunkTrust

I'm assuming they are already (automatically) being extracted as fields. From the query that you've in question, what is the output that you get now and what you expect (sample)?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...