How to edit my props.conf to make sure CSV file data gets indexed?

[yanivdutt]

Hi,
I am using below props file for CSV but data is not getting indexed or sent into Splunk. Need help in updating props

[data_csv]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
HEADER_MODE = firstline
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true
PREAMBLE_REGEX = ^\tDate
FIELD_NAMES = [SiteID,Domain,TotalRequests,SavedRequests,TotalBytes,SavedBytes,HitTime,BitsPerSecond,Human requests per second,Bot requests per second,Blocked requests per second,BlockedRequests]

Below is data from CSV

SiteID  Domain  TotalRequests   SavedRequests   TotalBytes  SavedBytes  HitTime BitsPerSecond   Human requests per second   Bot requests per second Blocked requests per second BlockedRequests
1001602 yahoo.com   0   0   0   0                       
1028875 yahoo2.com  0   0   0   0                       
1033027 yahoo1.com  3088    656 52976733    23001447    10/12/16 0:00   963 0   0   0   0
1033027 yahoo.com                   10/12/16 0:10   1350    0   0   0   0

[mattymo]

Hi Yanivdutt!

I would strongly recommend you run one of your sample CSV through the Add Data wizard on one of your searchheads.

http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/Howdoyouwanttoadddata

It will allow you to test these settings and allow you to preview how your data looks once they have been applied!

http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/Setsourcetype

I think you may have a couple conflicting settings....

Is the sample you pasted exactly what the file looks like? Can you share one with the preamble? I can run it through the add data wiz and show you what I come up with.

[yanivdutt]

SiteID Domain TotalRequests SavedRequests TotalBytes SavedBytes HitTime BitsPerSecond Human requests per second Bot requests per second Blocked requests per second BlockedRequests
1001602 linelink.yahoo.com 0 0 0 0

1028875 npe-facade.api.yahoo.com 0 0 0 0

1033027 ma.my.yahoo.com 3088 656 52976733 23001447 10/12/16 0:00 963 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:10 1350 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:20 91 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:30 251 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:40 581 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 17:50 74 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:00 25109 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:10 21946 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:20 73558 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:30 7011 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:40 7781 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 18:50 27902 0 0 0 0
1036959 e2e.commerce.digital.yahoo.com 0 0 0 0

[mattymo]

Ah, just noticed...thats not a csv at all...appears semi space delimited, although I am not sure I'd even call it that....

You are going to have issues with this data in this format as there are spaces in the headers and there are fields that don't get populated and dont contain nulll values...

for example...your first 7 headers don't match all of your events...

SiteID Domain TotalRequests SavedRequests TotalBytes SavedBytes HitTime
1033027 ma.my.yahoo.com 10/12/16 0:10 1350 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:20 91 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:30 251 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 0:40 581 0 0 0 0
1033027 ma.my.yahoo.com 10/12/16 17:50 74 0 0 0 0

Is there anyway to clean up the format of this data??? What is the source of this data?

At this point index time csv fields are not going to work on this...you may be able to do some searchtime magic...but truly, this data needs to be formatted better

Even Excel doesn't like it lol