Security

Search ends before it gets to the data cap (500MB)

CarsonZa
Contributor

I have a user that is attempting to run a very large search. I raised the role srchDiskQuota but the search is still ending before it gets anywhere near the data cap. The role limit is set to 500MB and the search stops at 243.66MB. I've ensured that the user doesn't have any saved / back grounded jobs as well. I've tried the search on my account and it does the same. Any thoughts on what might be causing this?

index=iis c_ip!="10." c_ip!="192." session_nonce!="" cs_uri_stem="*.php" cs_uri_stem!="*piwik.php" cs_uri_stem!="*ee-atlogofetch.php"
| eval Platform=if(substr(cs_User_Agent, len(cs_User_Agent)-13)="Mobile-Wrapper","Mobile","Online")
| eval session_nonce=substr(session_nonce, 1, 32)."-".c_ip
| eval client=substr(client, 1, 5)
| stats first(client) as client, first(user) as user, first(Platform) as Platform, first(date) as date, max(_time) as MaxTime, min(_time) as MinTime by session_nonce
| lookup Employee_Lookup user as "user" OUTPUT hr_eename as ee_name, hr_eestatus as ee_status, LOS as LOS, hr_deptdesc as dept_desc, hr_position as position
| eval hr_eename=if(len(hr_eename) > 2, hr_eename, user)
| search client=08001
| eval session_duration=tostring((MaxTime-MinTime),"duration")
| convert ctime(MaxTime), ctime(MinTime)

0 Karma

Vijeta
Influencer

Check your limits.conf for data size

0 Karma

CarsonZa
Contributor

What stanza am I looking at?

0 Karma

pradeepkumarg
Influencer

How exactly do you know that the search has stopped?
What does the job inspector say? There are numerous other limits enforced like maxSearchTime and number of records you can pass in a join or subsearch.
The job inspector should clearly say what limit was enforced for that particular run.

0 Karma

CarsonZa
Contributor

I know it stops because i get an error that says "search auto-finalized after disk usage limit (500MB) was reached." but the size of the search only gets to ~243MB. A time limit enforcement is very plausible because of the amount of events that are being returned but i wouldn't expect to get an disk usage message.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...