Getting Data In

How to manually Index Data in Splunk 6.2.5?

_gkollias
SplunkTrust
SplunkTrust

Our production environment just upgraded to 6.2.5 from 6.0.3. The new data inputs seem to be pretty straight forward, except the index and sourcetype options are fairly limited. I will explain what I mean.

Data inputs in 6.0.x had the functionality to manually add a sourcetype. This was a nice feature because we have some props configurations out there to specifically parse the CSV file that I want to manually index for that sourcetype. For the index, I am unable to find it in the list. The only index names I can see are those that come "out of the box" (index=main, index=os, index=Default, etc. ). I also tried changing around the app context, however the list of indexes stays the same.

I am going in to Data Inputs-->Files & directories(local)-->New-->. From here I want to index a file once, and provide the directory and file name. Next, manually add a sourcetype that goes back to the props we have. It looks like most of the configurations needed should be done right in the UI instead?

As a work around, we can add an inputs.conf that monitors the local directory and tie it back to props.conf , but it would help to know how we could continue to manually add a csv with the correct index and manually add sourcetype if possible. If it helps, here are the configs I have tied to the data in the CSV:

props.conf

[oms_invoice]
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=15
DATETIME_CONFIG=/etc/apps/contract_props/default/oms_invoice_data_datetime.xml
SHOULD_LINEMERGE=false
EXTRACT-oms_invoice=(?<Date>[^,]*),\s*(?<Time>[^,]*),\s*(?<Division>[^,]*),\s*(?<Customer>[^,]*),\s*(?<BillTo>[^,]*),\s*(?<DiversityCustomer>[^,]*),\s*(?<InvoiceTypes>[^,]*),\s*(?<EDI>[^,]*),\s*(?<PDV>[^,]*),\s*(?<PAPER>[^,]*),\s*(?<InvoiceNumber>[^,]*),\s*(?<InvoiceAmount>[^,]*),\s*(?<OrderNumber>[^,]*),\s*(?<OrderLink>.*)

Any insight would be appreciated.

Thanks in Advance!

0 Karma
1 Solution

sloshburch
Splunk Employee
Splunk Employee

What's happening is the search head is only populating the drop downs based on what is knows about. The index is likely defined on the indexers and not on the search head. Also, the sourcetype might not have been configured to be shown, which is a relatively new feature.

So....one approach is to define the indexes on the search head but never use them - that allows them to be populated in drop downs like that. Then the sourcetype might need to have the "pulldown_type" set to true for it to show up in that drop down.

View solution in original post

sloshburch
Splunk Employee
Splunk Employee

What's happening is the search head is only populating the drop downs based on what is knows about. The index is likely defined on the indexers and not on the search head. Also, the sourcetype might not have been configured to be shown, which is a relatively new feature.

So....one approach is to define the indexes on the search head but never use them - that allows them to be populated in drop downs like that. Then the sourcetype might need to have the "pulldown_type" set to true for it to show up in that drop down.

_gkollias
SplunkTrust
SplunkTrust

Awesome - thanks!

0 Karma

woodcock
Esteemed Legend

Don't forget to click "Accept".

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...