Getting Data In

Why are only the first two lines of my CSV file getting indexed?

fredkaiser
Path Finder

Trying to index a CSV, but only the first two lines are indexing. I want to skip the first line and start indexing the data from the headers for the columns

Splunk Search output

TYPE Selected.Microsoft.ActiveDirectory.Management.ADAccount,,
LastLogonDate,Name,LockedOut

CSV input file

TYPE Selected.Microsoft.ActiveDirectory.Management.ADAccount,,
   LastLogonDate,Name,LockedOut
    25/05/2016 2:13,SPKTest3,TRUE
    25/05/2016 2:13,SPKTest4,TRUE

Props.conf

[ADAcount]
HEADER_FIELD_LINE_NUMBER=2
INDEXED_EXTRACTIONS = csv
FIELD_DELIMITER=,

input file

 [monitor:...//test.csv]
    disabled = false
    sourcetype =ADAcount
    index = test
0 Karma

gwobben
Communicator

I'm not entirely sure this is the issue but I've seen Splunk trip over timestamps like these before. Splunk seems to expect a leading zero in the hour field of the timestamp. e.g. 25/05/2016 02:13,SPKTest3,TRUE

Furthermore it's good practice to include the timestamp format in props.conf, as this is empty by default.

TIME_FORMAT = <strptime-style format>
* Specifies a strptime format string to extract the date.
* strptime is an industry standard for designating time formats.
* For more information on strptime, see "Configure timestamp recognition" in
  the online documentation.
* TIME_FORMAT starts reading after the TIME_PREFIX. If both are specified,
  the TIME_PREFIX regex must match up to and including the character before
  the TIME_FORMAT date.
* For good results, the <strptime-style format> should describe the day of
  the year and the time of day.
* Defaults to empty.

Extra tip:
Try uploading the CSV via the web interface. This will allow you to preview the data you're about to index. You can modify the time field extraction, headers, etc. from here.

0 Karma

sk314
Builder

Have you tried naming the second field in your header? Just a guess.

0 Karma

sk314
Builder

Also this from the docs could be an issue since you have > 3 header fields but only 3 fields in each row.

Splunk Enterprise only indexes header fields whose rows contain data

0 Karma

sk314
Builder

Did you check if splunk is automatically assigning the first field as the timestamp? try extending your search to "All Time" and see if you get more results.

0 Karma

fredkaiser
Path Finder

have tried that still no luck

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...