Splunk Search

the search job terminated unexpectedly

srobinsonxtl
Path Finder

This search take only a few second to come back index=* sourcetype=* (source="/opt/data/-AA_.csv" OR source="/opt/data2/-AA_.csv") | fields - field1 and return 81,000 records but once I add below to the search I get the search job terminated unexpectedly.

index=* sourcetype=* (source="/opt/data/-AA_.csv" OR source="/opt/data2/-AA_.csv") | fields - field1 | timechart span=5min sum(field*) as AA* | addtotals | table _time,Total | timechart span=1h max(Total) as Total | eval Total = Total/1000 | timechart span=1mon sum(Total) as Total

Here is a sample of the data and is written every 5 minutes. When I was writing the data every 15 minutes, it seem to work ok.
2017-12-31 23:55:00.001+00:00,695,0,733,0,817,0,1078,0,987,0,1004,0,1983,0,1744,0,1236,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

Any help to help improve my search is very much appreciated.

Thanks,

Stephen Robinson

Tags (1)
0 Karma
1 Solution

srobinsonxtl
Path Finder

This appeared to improve my search dramatically, thank you for the help

Thanks,

Stephen Robinson

View solution in original post

0 Karma

srobinsonxtl
Path Finder

This appeared to improve my search dramatically, thank you for the help

Thanks,

Stephen Robinson

0 Karma

mayurr98
Super Champion

can you please write search query which is getting terminated in 101010 sample code for a better understanding of query. Also, i think you need to optimize your search query
give us sample input with the field name and what output you want?

0 Karma

srobinsonxtl
Path Finder

index= sourcetype= (source="/opt/data/-AA_.csv" OR source="/opt/data2/-AA_.csv") | fields - field1 | timechart span=5min sum(field*) as AA* | addtotals | table _time,Total | timechart span=1h max(Total) as Total | eval Total = Total/1000 | timechart span=1mon sum(Total) as Total

This provides me the output I want, but the job doesn't seem to efficient. The search creates fields AA1-121 per event and then does all the calculations it needs to do.

This is the sample of data that I have.
2017-12-31 23:55:00.001+00:00,695,0,733,0,817,0,1078,0,987,0,1004,0,1983,0,1744,0,1236,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
2018-01-01 00:00:00.001+00:00,695,0,733,0,817,0,1078,0,987,0,1004,0,1983,0,1744,0,1236,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
2018-01-01 00:05:00.001+00:00,695,0,733,0,817,0,1078,0,987,0,1004,0,1983,0,1744,0,1236,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
....
Thanks,

Stephen Robinson

0 Karma

somesoni2
SplunkTrust
SplunkTrust

See if this helps

index= sourcetype= (source="/opt/data/-AA_.csv" OR source="/opt/data2/-AA_.csv") | fields - field1 
| eval Total=0 | foreach field* [eval Total=Total+'<<FIELD>>' ]
| timechart span=5min sum(Total) as Total | timechart span=1h max(Total) as Total | eval Total = Total/1000 | timechart span=1mon sum(Total) as Total
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...