Security

Search ends before it gets to the data cap (500MB)

CarsonZa
Contributor

I have a user that is attempting to run a very large search. I raised the role srchDiskQuota but the search is still ending before it gets anywhere near the data cap. The role limit is set to 500MB and the search stops at 243.66MB. I've ensured that the user doesn't have any saved / back grounded jobs as well. I've tried the search on my account and it does the same. Any thoughts on what might be causing this?

index=iis c_ip!="10." c_ip!="192." session_nonce!="" cs_uri_stem="*.php" cs_uri_stem!="*piwik.php" cs_uri_stem!="*ee-atlogofetch.php"
| eval Platform=if(substr(cs_User_Agent, len(cs_User_Agent)-13)="Mobile-Wrapper","Mobile","Online")
| eval session_nonce=substr(session_nonce, 1, 32)."-".c_ip
| eval client=substr(client, 1, 5)
| stats first(client) as client, first(user) as user, first(Platform) as Platform, first(date) as date, max(_time) as MaxTime, min(_time) as MinTime by session_nonce
| lookup Employee_Lookup user as "user" OUTPUT hr_eename as ee_name, hr_eestatus as ee_status, LOS as LOS, hr_deptdesc as dept_desc, hr_position as position
| eval hr_eename=if(len(hr_eename) > 2, hr_eename, user)
| search client=08001
| eval session_duration=tostring((MaxTime-MinTime),"duration")
| convert ctime(MaxTime), ctime(MinTime)

0 Karma

Vijeta
Influencer

Check your limits.conf for data size

0 Karma

CarsonZa
Contributor

What stanza am I looking at?

0 Karma

pradeepkumarg
Influencer

How exactly do you know that the search has stopped?
What does the job inspector say? There are numerous other limits enforced like maxSearchTime and number of records you can pass in a join or subsearch.
The job inspector should clearly say what limit was enforced for that particular run.

0 Karma

CarsonZa
Contributor

I know it stops because i get an error that says "search auto-finalized after disk usage limit (500MB) was reached." but the size of the search only gets to ~243MB. A time limit enforcement is very plausible because of the amount of events that are being returned but i wouldn't expect to get an disk usage message.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...