Splunk Search

Why is summary index search returning duplicate and incorrect counts of data?

smaran06
Path Finder

Hi Team,

I am populating the data in summary index using the following Splunk search

index=data"  | sistats count as total by  appName,trueclient, httpstatus,request_uri

when, I do stats over this

index=summary_ |stats count as total by appName.

We are getting lot of difference in counts.

When, I run the search directly the app counts are very low, then on summary index it's very high. Why summary index data is returning wrong data? is it because, I added trueclient, httpstatus,request_uri in sistats?

0 Karma

briancronrath
Contributor

When you search for just

index=summary_

How many different source values are you getting? Are you getting sources outside of the search you used to populate it? If so, limit down to just the name of the search as your source when you search and see if the numbers look better.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

When you use the si* command for summary index, you need to use the same aggregation command on the summary index data. Give this a try and see if the counts are matching.

index=summary | stats count as total by  appName,trueclient, httpstatus,request_uri | stats sum(total) as total by appName

compare with this.

index=data |stats count as total by appName
0 Karma

smaran06
Path Finder

Thanks, still the count is not matching, summary index is at very high when compare to data which is not is summary index

0 Karma

graa1005
Explorer

I have exactly the same problem. Multiple entries in the summary index for the same data. Only one value for info_search_time So it looks like it is one search. only outputs on multiple indexers.
If i deleted the summary data and re runtje job to add the results to the summary index.I get double data only different as the previous summary data. So only a part of the data is double.

0 Karma

graa1005
Explorer

Some data can exist multiple times. IN my case up to 5 times. (i have 5 indexers)

0 Karma

ddrillic
Ultra Champion

We normally put a safe-guard to avoid duplicates. A left join in the spirit of - | join type=left <field> [search index=<summary index name>| eval matched="Y"]

0 Karma

graa1005
Explorer

ddrillic my search does NOT generate duplicates. I execute the search ones a day to generate a summary of the records of the day before.

0 Karma

ddrillic
Ultra Champion

Sorry ; -)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...