Knowledge Management

Collecting to a summary index not working

jizzmaster
Path Finder

am unable to collect data into a summary index. Getting odd behavior.

This works:

index=security sourcetype=dbx2 source=ca_owned_resource inactive=0 status="In Service" |stats count(resource_name) as asset_mgmt.cmdb.active |collect index=summary

This does not:

index=security sourcetype=nmap_xml state=up earliest=-5d |stats first(state) as State by ip |stats count(State) as asset_mgmt.nmap_xml.ip.live |collect index=summary

Both searches, per se, work fine. But the collection part does not work on the latter query. Each search query provides a number value as expected. When I look at the summary index (index=summary asset_mgmt) right after running each query, only the first search comes up. It does not matter the user I run it as; typical user, power user, or admin. I have also tried this on different searchheads but have the same result. Overall, I have three search strings that will not record into the summary index, and one that will.

0 Karma
1 Solution

jizzmaster
Path Finder

Turns out that it was indexing, just with a completely wrong date and timestamp. I could not find much rhyme or reason to when it applied the timestamp, but it was not the first or the last timestamp of the data it was pulling from.

Anyway, adding "|eval _time=now()" worked perfectly. Timestamps and summary indexing now being applied as I expect it to be. Timestamps are from when the saved search begins, not dependent on timestamps within the logs it is searching on.

View solution in original post

jizzmaster
Path Finder

Turns out that it was indexing, just with a completely wrong date and timestamp. I could not find much rhyme or reason to when it applied the timestamp, but it was not the first or the last timestamp of the data it was pulling from.

Anyway, adding "|eval _time=now()" worked perfectly. Timestamps and summary indexing now being applied as I expect it to be. Timestamps are from when the saved search begins, not dependent on timestamps within the logs it is searching on.

ConnorG
Path Finder

There's a few paragraphs in the documentation that may shed some light:

If you apply the collect command to events that do not have timestamps, it designates a time for all of the events using the earliest (or minimum) time of the search range. For example, if you use collect over the past four hours (range: -4h to +0h), it assigns a timestamp four hours previous to the time the search was launched to all of the events without a timestamp.

If you use collect with an all-time search and the events do not have timestamps, Splunk Enterprise uses the current system time for the timestamps.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...