In my search, I calculate some values, but if I reach the 10000 result limit, I get wrong results. I would like change the wrong results to something like "its bigger than.....". I don't care about the real value, but I would like to show that something did not finish getting calculated.
Thanks in advance
If you want to do this programatically focus on the messages you can extract from the Search/Jobs endpoint:
| rest /services/search/jobs
| table author, title, cursorTime, messages.*
For instance, if you run the following:
index=_internal | head 10001 | sort _time
That will trigger an Info message in your job inspector (icon becomes green next to the Job dropdown).
This is great when you are running manual queries but what if you are running scheduled ones.
The rest query above will return the following in that case:
author title cursorTime messages.error messages.fatal messages.info messages.warn
admin "search index=_internal | sort _time" 2016-01-18T00:00:00.000+00:00 The sort command is truncating output to 10000 rows
That you can parse later on and use it for your own needs.
mmmhhh, i got a list with a lot of records from my search query and i want to know if something is truncated for a value in every record separately!
If you just want to display those with a message that contains the word "truncating" do this instead:
| rest /services/search/jobs
| search
messages.info = "*truncating*" OR
messages.warn = "*truncating*" OR
messages.error = "*truncating*"
| table author, title, cursorTime, messages.*
Keep in mind there are lots of other fields you might find useful. The query above is just an example.
Which limit are you reaching, the subsearch limit? The limit when sorting? We need to know some more details.
Usually, you get a warning next to the "Job"-dropdown just next to the buttons where you pause or stop a search.
hi, this is what job inspector says: (first reaching is with mvexpand or is it just by happenstance?, then presort and so on...)
Execution costs
Duration (seconds) Component Invocations Input count Output count
0.00 command.dedup 3 1 1
0.46 command.eval 60 100,010 100,010
0.00 command.fields 10 16 16
0.14 command.mvexpand 3 1 10,000
0.00 command.prededup 2 7 1
0.08 command.presort 1 10,000 10,000
0.00 command.rename 3 10,000 10,000
0.01 command.search 2 - 7
0.00 command.search.index 2 - -
0.00 command.search.calcfields 1 7 7
0.00 command.search.fieldalias 1 7 7
0.00 command.search.filter 1 - -
0.00 command.search.index.usec_1_8 398 - -
0.00 command.search.index.usec_512_4096 1 - -
0.00 command.search.kv 1 - -
0.00 command.search.rawdata 1 - -
0.00 command.search.lookups 1 7 7
0.00 command.search.summary 2 - -
0.00 command.search.tags 1 7 7
0.00 command.search.typer 1 7 7
0.04 command.sort 1 10,000 10,000
0.16 command.table 1 10,000 20,000
0.00 dispatch.check_disk_usage 1 - -
0.00 dispatch.createdSearchResultInfrastructure 1 - -
0.10 dispatch.emit_prereport_files 1 - -
0.40 dispatch.evaluate 1 - -
0.40 dispatch.evaluate.search 1 - -
0.00 dispatch.evaluate.eval 20 - -
0.00 dispatch.evaluate.dedup 1 - -
0.00 dispatch.evaluate.fields 3 - -
0.00 dispatch.evaluate.mvexpand 1 - -
0.00 dispatch.evaluate.rename 1 - -
0.00 dispatch.evaluate.sort 1 - -
0.00 dispatch.evaluate.table 1 - -
0.45 dispatch.fetch 3 - -
0.01 dispatch.localSearch 1 - -
0.37 dispatch.preview 1 - -
0.17 dispatch.preview.command.table 1 10,000 20,000
0.08 dispatch.preview.command.presort 1 10,000 10,000
0.05 dispatch.preview.command.sort 1 10,000 10,000
0.01 dispatch.preview.write_results_to_disk 1 - -
0.00 dispatch.results_combiner 3 - -
0.01 dispatch.stream.local 2 - -
0.19 dispatch.timeline 3 - -
0.02 dispatch.writeStatus 7 - -
0.06 startup.configuration 1 - -
0.58 startup.handoff 1 - -
I don't see a limit hit there.