I would avoid eventstats if the search is over a lot of events. While the "for each" solution is seems overly complicated, it would be far more efficient performance wise (especially for a distributed deployment with many indexers) if dealing with more than a few hundred thousand events. ... View more

I believe the simplest solution to this is to simply filter out events where count_delta=0, e.g. source="jms-queue" queue_name="SNMPTrapsQueue*" count_delta>0 | timechart span=5m per_minute(count_delta) by queue_name ... View more

Your solution is more efficient that the accepted answer. However, note that expression for NULL in eval is actually a function call of null(). It probably seems to work because NULL is treated as a field with the name NULL, which probably doesn't exist in your environment. ... View more

Splunk doesn't monitor the hosts directly, but you can use the unix app, or simply configure a scripted input to run a shell command like df periodically to report the free disk space. ... View more

actually the last method is likely to be faster than using multireport. Multireport doesn't attempt to ask the indexers to pre-compute intermediate aggregates for the reports, which will make a big difference for data that is distributed across many indexers. another method for doing this is by appending an "ALL" value to each DestinationIP and HTTPStatus. E.g. sourcetype=myhttpdata | eval DestinationIP = mvappend(DestinationIP,"ALL") | eval HTTPStatus=mvappend(HTTPStatus,"ALL") | stats count by SourceIP DestinationIP HTTPStatus This has the additional benefit that if for some reason there are events that have a SourceIP but no DestinationIP or HTTPStatus, it will still get counted in the total for that SourceIP ... View more

Unfortunately, this approach means that the comment command ends up de-serializing and re-serializing every event from and to CSV in python. That's in general fairly costly. A macro based approach would be best. Basically define a comment macro that evaluates to the empty string regardless of the input argument. ... View more

A macro with the following definition would be the best option [comment(1)] iseval=1 definition="" args=text description=Throw away comment text This allow the comment to be inserted anywhere in the search where it will always be expanded into the empty string (without quotes). We use the eval based definition so that we can specify the empty string explicitly. ... View more

what is the actual name of the field that contains output? It can't actually be "unnamed". whatever the name is, you can copy it into _raw via eval ... | eval _raw = your_field_name ... View more

changing the max_rt_search_multiplier is the way to go. changing max_searches_per_cpu and base_max_searches will change also affect the real-time limit, but will alter the limit for historical searches too. max real-time searches = max_rt_search_multiplier x (max_searches_per_cpu * + base_max_searches) And to address woodcock's comment, there is a setting that you can tweak which will allow you to optionally trade performance for latency. It will run real-time searches with higher latency but generally use far less system resources. The setting is under [realtime] indexed_realtime_use_by_default = true/false (defaults to false. set to true for less resource usage but higher latency) ... View more

search is going to be slightly more efficient than where, but not enough that you would notice for any realistic search scenario. Both commands can comparing the value of a field to some static value, but that is where the commonality ends. 'where' can be used to compare 2 fields against each other, to compared complex functions of a field to other fields or static values. E.g. you can't do something like to compare fields x and y. | search x>y That search would actually search for cases where field x is greater (lexicographically) than the literal value "y" Note that this is also why the syntax for search and where are slightly different. Because search is designed to compare a field against a static value, it assumes the right hand side of any expression is a literal value. For where, the RHS can be a literal or a field, so literals need to be disambiguated by using double quotes. ... View more

oh, you need to quote it for eval if it's supposed to be a string literal. Or else it will treat it as a field name or a number, which is why $count$ without the quote works if the count is just a number. ... View more

the following worked for me: | stats count | map search="search index=_internal | head 10 | eval x=$count$" | table x You should see that you'll end up with 10 events all with x=0. ... View more

you can do mvexpand on the PresentationStopReason field before the stats if you want to count each value individually. ... View more

double quotes are used to quote values in CSV. See https://en.wikipedia.org/wiki/Comma-separated_values#Basic_rules_and_examples ... View more

newlines can appear in a quoted value, so it's not as simple as one logical row per line. ... View more

I'd suggest running a subsearch using either tstats or metadata to get the last 2 sources, and then doing a search like [<subsearch to get last 2 source names>] | stats dc(source) as dc by _raw | search dc=1 This will get you all the rows that appear in only 1 of the last 2 sources. Replace '_raw' with whatever other field(s) to fit your needs. ... View more

set diff will only work as a generating command (it has to be the first command) ... View more

that limits.conf setting does not affect inputlookup. It only affects the performance optimization for performing lookups. inputlookup is basically inputcsv, but from the lookup directories rather than the dispatch directory. ... View more

Does your lookup have more than 1 line per event? Splunk generated lookup files with multivalued fields often have this property. If your lookup file has a primary key, you can try to find the set difference between the lookup file and what inputlookup returns and see if there is any pattern as to which rows are missing. ... View more

metadata stores only information segregated by source, sourcetype, or host individually, so it will not have information necessary divided by both sourcetype and host. For that, you could use the metrics.log, or you can use tstats to query what is stored in the indexes. | tstats max(_time) WHERE index=* GROUPBY sourcetype host ... View more

I agree that when by is specified, wanting a global window is a minority use case. But I can think of cases where you might want something like the count by type for the last 10000 events. I mainly left the default as global=f because it's more efficient to have a global window so I'd rather force the user explicitly have to request a non-global window. ... View more

when there is no unique mode, it should emit the lexicographically first mode value, which in this case should be 1. It purposely does not return multiple values. ... View more