Splunk Search

Eval fields not making it to Summary index when stats is used.

richard_whiffen
Explorer

I'm still sifting through the 'realated questsions' proposed in "Ask a Question" (great feature btw), but I don't think my senario is covered.

I have a search set to run every 24 hours to sumarize the previous 24 hours stats.

host="HOSTSBLAH*"  (source="/usr/local/tvs/apache-tomcat/logs/qlogger/*" NOT source="*.gz") 
| lookup Market_by_Controller_ID Controller_ID as Controller_ID OUTPUT Market as Market 
| eval QueryFirstTwo=substr(TextQuery,1,2) 
| transaction MAC, QueryFirstTwo maxspan=5m maxpause=1m delim="," mvlist=TextQuery 
| eval LastQuery=mvindex(TextQuery, -1) 
| fillnull value=0 forward
| eval MAC="salt".MAC 
| eval MAC=md5(MAC) 
| stats count(LastQuery) by Market, Controller_ID, StreamingServerID, forward 
| fields count LastQuery, Controller_ID StreamingServerID Market forward MAC 
| collect addtime=true index=dashboard_summarize

Data is getting into the index, but none of the fields that have been 'EVAL'ed at some stage. So in this example, LastQuery, forward and MAC are NULL in the summary index. If I take the stats stanza out, the data is collected. It seems odd/bug-like to me that STATS would some how null out EVAL'ed fields.

I've taken the stats stanza out and am moving on with my life. I'll generate the stats in a separate query, but again, I'm puzzled why they'd be NULL in the summary. Are there other cases like this that I need to watch out for?

Cheers, Rich

Tags (3)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

It is not a bug, but is how stats is intended to work. The only things to come out after stats are the actual stats you ask for, aggregated by the by fields, so it will be exactly those fields. stats does not keep any other values because it is not otherwise told how to aggregate them (i.e., what to do with multiple values per by field combination). I'm guessing what you need is just ... | stats first(MAC) as MAC ... but whether that's correct or not depends on your data.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

It is not a bug, but is how stats is intended to work. The only things to come out after stats are the actual stats you ask for, aggregated by the by fields, so it will be exactly those fields. stats does not keep any other values because it is not otherwise told how to aggregate them (i.e., what to do with multiple values per by field combination). I'm guessing what you need is just ... | stats first(MAC) as MAC ... but whether that's correct or not depends on your data.

richard_whiffen
Explorer

Yeah, I found my mistake by adding and removing stanzas and then talking with Chris Olson from Splunk. In hind sight its obvious but didn't click when I was looking at it the last two days. I think what tripped me up the most is that 'LastQuery' wasn't going through and it was in the stats command, but was in the stats command (but as a count not the actual value DOH!). Anyway, Chris set me on the same path. Thanks gkanapathy

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...