Searches appearing in search results (yo dawg)

[bbegyperkspot]

When I search in the search application, my search terms are starting to appear in subsequent searches. So search for "earliest=-m error" then do it again, and half of my results are the previous search.

[02/Jun/2014:15:49:51.737 -0500] "GET /en-US/splunkd/_raw/services/messages?output_mode=json&count=1000&=1401742191674 HTTP/1.1" 200 198 "[splunk url redacted]/en-US/app/search/search?q=search%20earliest%20%3D-m%20error&earliest=0&latest=&sid=1401741994.2" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36" - 538ce36fbc31f17d0 7ms

I must have my settings broken. But how?

[neiljpeterson · Accepted solution]

That data is most certainly coming from the _internal index which contains lots of things like searches ran.

The indexes that are available to you as a user are a function of what role(s) you have. You can change which indexes are searched by default under the access control settings for that role.

To prevent everyone with the user role from seeing results from _internal click on Settings > Access controls > Roles > user > Indexes searched by default Choose what indexes you want them to see results from by adding them to Selected Indexes under indexes searched by default.

The default indexes are the indexes that will be searched when index= is NOT specified in query. When someone (like you) would want to search the _internal index they would then need to specify it:index=_internal

[bbegyperkspot]

That did it, thanks. It got changed when we were debugging another issue.

[somesoni2]

Check the role users are in and for that role changes the property "Indexes searched by default" OR at search level, specify "NOT index=_* earliest=-m error"

[bbegyperkspot]

ok, how do I disable searching _internal and _audit by default? It is baffling the user's I'm trying to convert from "just log into prod and poke at the error log."

[somesoni2]

When you just search "earliest=-m error", its basically searching all your default indexes (all internal and non-internal indexes). The internal indexes like _internal and _audit also logs user search activities which is what is being included in your results at it matches your criteria.
Please include the indexes your really want to search.