Splunk Search

how to ignore the titile line of the csv file in the result? and display in a KV format?

William
Path Finder

I try to add some csv files, which contain data like the followings

Time, ACTION,ORDER_NO, ...

2009-11-2 20:00:00.041,REQUEST,48613840, ...

2009-11-2 20:00:00.041,REQUEST,48613839, ...

2009-11-2 20:00:00.041,REQUEST_ACK,48613840, ...

2009-11-2 20:00:00.041,REQUEST_ACK,48613839, ...

2009-11-2 20:00:00.046,REQUEST,48613841, ...

when I set the input source type as "csv", then the input file can be recognized with an "AutoHeader-1" stanza and a "csv-2" stanza being added to "$Splunk\etc\apps\learned\local\transforms.conf" and "$Splunk\etc\apps\learned\local\props.conf" respectively.

But I still have two problem,

  1. the first line (title line "Time, ACTION,ORDER_NO, ...") will be take as an event also, as follows

    10-4-5 02:49:28.000 _time,ACTION,ORDER_NO, ...

    2009-11-2 20:00:00.074,REQUEST,48613844, ...

    2009-11-2 20:00:00.055,REQUEST_ACK,48613842, ...

    so, how can I remove the title line from the result?

  2. How can I display the result in an KV format?

1 Solution

BunnyHop
Contributor

You can add the CHECK_FOR_HEADER = true on the props.conf.

View solution in original post

mike_lebrun
Explorer

During my research into dealing with the header of a .csv, I've found that CHECK_FOR_HEADER is a deprecated feature and is no longer a best practice for dealing with the header of a .csv file.

http://docs.splunk.com/Documentation/Splunk/5.0.4/releasenotes/Deprecatedfeatures

0 Karma

cmeo
Contributor

Seriously folks, the solution in answer <1> should be the default behaviour. Why would you want the headers included in your indexed data???

0 Karma

Lowell
Super Champion

Do you really want Splunk choosing (own its own) to just drop certain events out of your log files?

BunnyHop
Contributor

in your props.conf also add this line:

[yoursourcetype]
TRANSFORMS-NoHeader = NoHeader

on your transforms.conf add this:

[NoHeader]
REGEX = Time, ACTION,ORDER_NO, ...
DEST_KEY = queue
FORMAT = nullQueue

bhawkins1
Communicator

This is the only solution that worked for me.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
  1. I would just leave it there and ignore it/exclude it in your searches, but if you really really want to, you can apply a regex TRANSFORM to strip it out.

  2. Probably would be better if you were more specific about what you are trying to show/find. Splunk can transform results, but it's probably not very useful to simply rewrite the raw event text in a different format. What are you trying do or show with the data itself?

BunnyHop
Contributor

You can add the CHECK_FOR_HEADER = true on the props.conf.

William
Path Finder

thanks, it works for the 2nd question.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...