Knowledge Management

Why is summary indexing creating duplicate records in the summary index?

vganjare
Builder

Hi,

I am have the following definition for summary indexing:

[Test_Summary_Index]
action.summary_index = 1
action.summary_index._name = main
action.summary_index.report = summary_indexing_test
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = */5 * * * *
description = Test_Summary_Index
enableSched = 1
realtime_schedule = 0
search = | gentimes start=1 end=2

Ideally, after every 5 mins, only one event should get added to the summary index, but when I search over the report, I get duplicate records in return. I have two indexers for searching.

We are using Splunk version:6.1.4 and splunk build:233537

Are there any changes I need to make in the summary indexing definition to avoid the duplicate entries?

Thanks!!!

1 Solution

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi vganjare,

check your outputs.conf on the search head. You probaply have something like this:

[tcpout]
indexAndForward=false

[tcpout:indexer1]
server=Y.Y.Y.Y:9997

[tcpout:indexer2]
server=X.X.X.X:6666

This will clone events to indexer1 and indexer2. Where as in this setting

[tcpout:group3]
server=Y.Y.Y.Y:9997,X.X.X.X:6666

the data will be distributed using AutoLB between these two receivers.

Hope this helps ...

cheers, MuS

brenthale
Explorer

This may be my problem. However, I don't have rights to the Splunk server to view the outputs.conf file. Is there a way via the UI to see the settings to determine if this is my problem or not? I've tried to look but nothing was obvious.

0 Karma

MuS
SplunkTrust
SplunkTrust

You can check it in the Splunk UI using the URI

http[s]://YourSplunkServer:YourPort/en-US/manager/launcher/data/outputs/

Obviously this will not be possible to do on an universal forwarder.

0 Karma

brenthale
Explorer

Just my luck...I'm on a "universal forwarder" and I get a big "Page not found!".

We have a big farm of Splunk Servers. They are managed by another team. I just want/need to be able to write valid reports. Is there any other way to determine this through the UI?

0 Karma

MuS
SplunkTrust
SplunkTrust

You could use curl to access the universal forwarder REST API and query the config like this:

curl -k -u admin:YourUFPassword https://YourUFIP:YourUFMGMTPort/servicesNS/nobody/search/configs/conf-outputs/
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...