Splunk Enterprise

Verbose mode returns results as expect, but not Fast mode.

hagjos43
Contributor

We are having some odd search results. When the query is ran in Verbose mode all fields return as expected, however when the same query is ran in Smart or Fast mode only the default fields are returned and the two fields extracted at search time via the "| rex" command. Anyone run into this before?

EDIT
Query used:

(index=foo sourcetype=bar) OR (index=metrics sourcetype=stats
    [ search index=foo sourcetype=bar
    | mvexpand combine
    | rex field=combine (?<id>\d+)=(?<hits>\d+)
    | dedup id site
    | fields id site] earliest=-1y)
| fillnull combine
| mvexpand combine
| rex field=combine (?<id>\d+)=(?<hits>\d+)
| eventstats latest(ip) as IP latest(info_id) as Info_ID by id, site
| where isnotnull(hits)
| table _time hits id site IP Info_ID

When my search runs in Verbose mode I see all fields "_time hits id site IP Info_ID" However when I run it in Fast mode I am missing IP and Info_ID.

If I remove the "| table" section altogether and run it in fast mode the query runs and all interesting fields are populated on the left side of the events.

Tags (2)

marycordova
SplunkTrust
SplunkTrust

https://answers.splunk.com/answers/679070/search-in-fast-mode-comes-back-with-different-resu.html#an...

Splunk JIRA SPL-153269

A configuration added somewhere between Splunk Enterprise versions 6.4.? and 7.0.2 introduced an issue where using a macro with several lookups against the same lookup table results in only a single match attempt with subsequent matches against the lookup table being skipped.

Make the following configuration change to limits.conf:
[search_optimization::projection_elimination] cmds_black_list = lookup

There should not be a significant performance hit since this is just reverting this configuration to that in a previous version of Splunk.

Fix has been tested and confirmed in my environment, under these specific test conditions. I know the problem didn't exist under some version of 6.x and started in some version of 7.x, I just don't recall which upgrade specifically broke the macro/lookups. I am not sure if it resolves other similar behavior observed under different conditions.

@marycordova
0 Karma

marycordova
SplunkTrust
SplunkTrust

it would be interesting to see if testing shows this resolves the issue
it's possible that this specific config doesn't but another option in the stanza would

@marycordova
0 Karma

marycordova
SplunkTrust
SplunkTrust

@hagjos43 ping me on linkedin if you want to try to test this or identify some other possible config that looks promising

also...you could ping support on this and reference the JIRA, they might be able to get a faster answer to you on your issue if they are at least related and this config doesn't help

@marycordova
0 Karma

woodcock
Esteemed Legend

Try adding info_id=* ip=* to the appropriate place to force splunk to care about those fields early on, like this maybe:

(index=foo sourcetype=bar) OR (index=metrics sourcetype=stats info_id=* ip=*
     [ search index=foo sourcetype=bar
     | mvexpand combine
     | rex field=combine (?<id>\d+)=(?<hits>\d+)
     | dedup id site
     | fields id site] earliest=-1y)
 | fillnull combine
 | mvexpand combine
 | rex field=combine (?<id>\d+)=(?<hits>\d+)
 | eventstats latest(ip) as IP latest(info_id) as Info_ID by id, site
 | where isnotnull(hits)
 | table _time hits id site IP Info_ID
0 Karma

hagjos43
Contributor

Thanks, unfortunately this didn't do the trick.

0 Karma

jpolvino
Builder

Curious: Is any of your data summary-indexed?

0 Karma

hagjos43
Contributor

The search is actually what is populating a summary index.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You should pass the fields command to make your fields show up under "Interested Fields" while in fast mode

index=... sourcetype=... | fields + foo

hagjos43
Contributor

See updated original question. I've included the query.

0 Karma

L1_marrera
Explorer

I added "|fields *" and it gave me the results i wanted.

jconger
Splunk Employee
Splunk Employee

Your fields are still there, but may not be displayed under the "Interesting Fields" column. For instance, if you have a field named "foo" in your search results, but it doesn't show up under "Interesting Fields" due to being in Fast mode, you can still do a search like this:

index=main foo=*

Fast mode prioritizes the performance of the search and does not return nonessential field or event data. This means that the search returns what is essential and required.

Verbose mode returns all of the field and event data it possibly can, even if it means the search takes longer to complete, and even if the search includes reporting commands.

Smart mode tries to make a determination based on your search (reporting vs. transforming vs. generating etc.)

More information can be found on search modes here -> https://docs.splunk.com/Documentation/Splunk/latest/Search/Changethesearchmode

0 Karma

hagjos43
Contributor

Thanks for your reply, however that doesn't do the trick. I should elaborate....the interesting fields DO in-fact populate in fast mode. But when I pipe the output to a table it does not properly show the statistics table.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...