Hi...
Its been a while I have problems with searching in Google maps or geoip which the thread was going on here: geoip search results not correct
if u follow up the thread, u see it came to a point that we all realized there exists some sort of limit that does not let geoip or Google maps to display more than ten thousand...
today suddenly, I realized its not only geoip/Google maps, but it actually is any kind of advanced searches. for instance if you search for all the data in normal search using "*", and if you have huge number of indexed data, (i.e. billions of data), u'll probably see all in the search but if you change the view to "Advanced Charting View" then you'll only see partially few thousands of those results...
Im not sure whether this is a bug or if there is some sort of limitation in any file... but that definitely causes major problems.
Does anyone have any idea how to solve this issue..?
thanks hexx for detailed information and references you provide here.. but this is getting even weirder... 😄
although the command is used by all the users without the lookup command, I tried your way and received error:
[EventsViewer module] Error in 'lookup' command: The lookup table 'geoip' does not exist.
and trying the same command in Google Maps gives this error:
Rendering...
Error : Traceback:
Traceback (most recent call last):
File "/opt/splunk/etc/apps/maps/appserver/modules/GoogleMaps/GoogleMaps.py", line 53, in generateResults
for result in getattr(job, entity_name)[offset:end]:
File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 1280, in __getitem__
self.job.pushValidation()
File "/opt/splunk/lib/python2.6/site-packages/splunk/search/__init__.py", line 610, in pushValidation
raise splunk.SearchException, fatality
SearchException: Error in 'lookup' command: The lookup table 'geoip' does not exist.
besides, the SPP about page, located at .../app/maps/about is a help document with this search as example:
Perform a geolocation lookup for
values of the clientip field in
access_combined events:
sourcetype=access_combined | geoip
clientipSame as the previous example, but also
perform DNS lookups in case when the
value of the clientip field is a
hostname and not an IP:
sourcetype=access_combined | geoip
clientip resolve_hostnames=trueSame as the first example, but using
the geo lookup instead of the command
sourcetype=access_combined | lookup
geo ip as clientip
etc... etc... etc...
I even tried this:
SIP="*" | lookup geo SIP
and got the same error...
yup dmaislin, and yet no difference... although I knew problem is not from Google Maps permissions as the geoip command behaves the same both in Google Maps and the flashtimeline search.
thats why I pointed my problem to geoip, not google maps..
the snapshots that i provided here on geoip as well was done on the flashtimeline, not Google Maps...
Are you in the google maps app when you run this command? Or, did you change the permissions on the App and make it Global yet?
Alright... since this problem behaves exactly the same with any kind of searching I do, I start from a very simple search first...
I am applying field extraction using DELIMS, hence I have a field called SIP which stands for sourceip...
so now, what I want to show u is results for SIP=* for a normal search, then results for same search on advanced charting view, then SIP=* | geoip SIP to also have geoip behaviour...
as you can see, both geoip and advanced chart only retrieve 10000 results !
these are snapshots for the normal search where you can see is up to 7 millions and I had to actually stop it since there was already enough results and it was taking much time... but the point is, the data that exists is way more than 10000...
then here it is on advanced charting:
and last but not least, geoip!
as you can see, both advanced charting and geoip have only 10000 results! the matching event is different. which indicates 2 problems actually:
Looking at the MaxMind geoip app, I'm not sure that you are invoking the geoip command correctly. As far as I can tell, geoip.py is an external command used by the geoip lookup, not intended to be a search command. It should therefore be invoked with the lookup command. What if instead of :
... | geoip SourceIP
...you run :
... | lookup geoip SourceIP
thank you for the update.
if the user wishes to maximize that for any reasons of their own, how would that go..?
besides, would you explain as well geoip's behavior as geoip command entered in flashtimeline also stops after 10000 results...
Thank you, now I know what's going on. Please review my amended answer.
The display of result rows will be limited on a per-command basis and is typically configurable in limits.conf with maxresultsrows
for stanzas such as [searchresults]
or [stats]
. Now, even if the display of results rows is truncated, reporting commands such as stats will still show accurate aggregates, which take all input events/results into account.
For more information and details, I would recommend to read this Splunk Answer as well as this one.
Update: Adding the information below to clarify the purpose of the advanced charting view.
The important thing to understand is that the advanced charting view is designed to process the results (not the raw events!) produced by a reporting command. This is why you will see the exact same search which would show millions of events in the flashtimeline return an approximate maximum of 10,000 results in the advanced charting view. Now, if you take that same search and pipe it to | stats count
, you will see that the search powering the advanced charting view will indeed process all expected events into results before letting you decide how to render those into a chart.
To sum it up : Do not use the advanced charting view to render and view events, it is not its purpose. Instead, use it to experiment with different visualization methods to apply on the results of your reporting search.
@nina15 : Just to be sure, could you tell us exactly what search string you are feeding into the advanced charting view? Could you tell us how many events the search reports to have found? Ideally, I'd like to see a screenshot of the search job inspector output.
Could you run the command
./splunk cmd btool limits list --debug
and pastebin the results with a link here please?
thanks for your response hexx,
as I'd stated in my description the view causes the problem... which is in line with your say that using other search commands (stats, timechart, etc) gives more results... I explained in my other post (the link is available above), the problem seems to raise when it has to display more than that number of results, not counts and stats... (I've already explained this in detail in my other post)
and again, in other post I did mention that I tried all possible parameters in limits.conf which includes the ones you are saying, and yet it did not have any effects neither on advanced charts nor geoip!!!
besides, if thats the case, why geoip behaves the same way... its very unlikely its a coincidence...
I know I tried many sorts of searches that should have shown thousands of results... but on using stats or timechart I have to try first and get back to u... but if there is any limits anyways, wouldnt it prevent from all kinds of searches...?
What search are you running that seems to limit its output to 10,000 rows in the advanced charting view? I am fairly certain that this limit is imposed by the default rendering of the advanced charting view, but if you were to run your search through a reporting command such as stats or timechart, you would get your full set of rows.
Looking at the module references the is a limit within the XML that you can define although you want to be careful how you adjust this as it can have an impact on the performance. Hopefully with 4.3 this won't be as large an impact thanks to the HTML5 visualisations but I expect the calculations in the background also have an equally large impact;
<module name="FlashChart">
<param name="width">100%</param>
<param name="height">400px</param>
<param name="maxResultCount">10000</param>
</module>
Also here, http://splunk-base.splunk.com/answers/10349/chart-only-showing-1000-events , Nick has some ideas on how to expand the number of results in other ways (Nick is pretty much the expert on all things XML related)
thanks so much dmaislin for responding... i really was feeling im loosing it...
anyways, i've already opened a support ticket. the number is CASE [73624].
thanks alot for ur support 🙂
Nina,
Can you open a support ticket? Let me know the ticket is and I will escalate for you.
still no answers/opinions...???
from what I understand, that post is only about charting, time ranges and XML...
my problem I think resides in lower level of splunk, is not the matter of how it is represented, but the problem is that the results are not fetched at all... the poster of that thread had a problem that the results were shown in the table, but not on the chart, only.
mine is not displayed anywhere when I search in Advanced charting or when I use geoip in the normal search...