Reporting

populate_lookup creates a gziped (.gz) file?

Lowell
Super Champion

Anyone know why the populate_lookup saved search alert action creates a compressed .gz file even if the destination file doesn't end in ".gz"?

I have a savedsearches.conf entry like so:

[Metrics daily sourcetype - build lookup]
action.populate_lookup = 1
action.populate_lookup.dest = etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv
cron_schedule = 0 * * * *
dispatch.earliest_time = -27d@d
dispatch.latest_time = @d
displayview = flashtimeline
enableSched = 1
search = (earliest=-10d@d latest=@d date_hour::`this_hour`) OR (earliest=-27d@d latest=@d date_wday::`this_wday`) index=_internal tag::host=splunk source=*metrics.log "group=per_sourcetype_thruput" | .... and so on ...

When the savesearch runs, it actually generates a file named metrics_daily_sourcetype.csv.gz (note the ".gz" on the end). Is this normal behavior or is it a bug of some type?

I also see the following message in my splunkd log that corresponds to each time this saved search runs:

ERROR SearchOperator:copyresults - You must provide a search id.

And when I bumped up the logging on the copyresults operator, I got the following message too (but apparently only one time):

INFO  SearchOperator:copyresults - copy results.csv.gz to /opt/splunk/etc/apps/SplunkAdmin/lookups/metrics_daily_sourcetype.csv.gz, success=1
1 Solution

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

View solution in original post

hexx
Splunk Employee
Splunk Employee

This issue has been filed as a bug (SPL-40222) and will be fixed in release 4.2.3. In the meantime, be sure to reference the gzipped lookup in transforms.conf instead of the plain CSV, or even better : Reference the lookup name (from transforms.conf) in action.populate_lookup.dest.

Example :

  • savedsearches.conf :

[queue name to max_size_kb lookup]
search = index=_internal source=*metrics.log group=queue | dedup name | table name, max_size_kb
(...)
action.populate_lookup = 1
action.populate_lookup.dest = q2s

  • transforms.conf :

[q2s]
filename = qname2size.csv.gz

jeffa
Path Finder

I am seeing the same issue. Any update as to the cause?

0 Karma
Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...