Getting Data In

unarchive_cmd doesn't work for CSV file

dmillis
Splunk Employee
Splunk Employee

In 6.2.0, I have written a pre-processor script for a particular CSV log format, to produce a useable timestamp. The script is designed to be used as a 'unarchive_cmd' script for the input. I.e., from the CLI, it functions like this:
cat | convert.pl

The problem: no matter how much I play with inputs.conf and props.conf, the unarchive_cmd is ignored. Here is inputs.conf:
[monitor:///var/cool_csv_logs/file.csv]
disabled = 0
followTail = 0
host = some_host
index = scratch
sourcetype = cool_csv

and props.conf:
`[cool_csv]
SHOULD_LINEMERGE = false
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl`

Why is the unarchive_cmd not working?

Tags (2)
0 Karma
1 Solution

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

View solution in original post

dmillis
Splunk Employee
Splunk Employee

Here's a more readable version:

Here is inputs.conf:

[monitor:///var/cool_csv_logs/file.csv] 
disabled = 0 
followTail = 0 
host = some_host 
index = scratch 
sourcetype = cool_csv

and props.conf:

[cool_csv] 
SHOULD_LINEMERGE = false 
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl

... should be changed to file.log

0 Karma

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...