Splunk Search

Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

shariinPH
Contributor

Hi all,

I have here a log file with a header and I'm using transforms.conf to extract the fields, but I'm not getting the right results.

for reference:

my log file consist of:

ARU|Portion|AR Text Sched|From Date|To Date|
02000000|02_AG0|SCAL MRU 02_AG0|02/01/20|12/31/20|
02001000|02_AG1|SCAL MRU 02_AG1|02/01/20|12/31/20|
02002000|02_AG2|SCAL MRU 02_AG2|02/01/20|12/31/20|
02003000|02_AG3|SCAL MRU 02_AG3|02/01/20|12/31/20|

I put props.conf both on:
C:\Program Files\Splunk\etc\system\local\props.conf
C:\Program Files\Splunk\etc\app\Maynilad\local\props.conf

[rbil_mrsched]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true
INDEXED_EXTRACTIONS = PSV
REPORT-AutoHeader = rbil_mrsched_trans

and in my transforms.conf
C:\Program Files\Splunk\etc\system\local\transfoms.conf

[rbil_mrsched_trans]
DELIMS= "|"
FIELDS="RbillARU","|","RbillPortion","|","RbillARTextSched","|","RbillFromDate","|","RbillToDate","|"

Values should be
RbillARU:

02000000
02001000
02002000
02003000

RbillPortion:

02_AG0
02_AG1
02_AG2
02_AG3

RbillARTextSched:

SCAL MRU 02_AG0
SCAL MRU 02_AG1
SCAL MRU 02_AG2
SCAL MRU 02_AG3

RbillFromDate:

02/01/20

RbillToDate:

12/31/20

but the results are:
02000000 for RbillARU (correct)
no values for RbillPortion
SCAL MRU 02_AG0 for RbillPortion (wrong this should be the result for RbillARTextSched)
12/31/20 for RbillARTextSched (wrong this should be the result for RbillToDate)
no values/result for RbillFromDate
no values/result for RbillToDate

Please help me with this. thanks

s2_splunk
Splunk Employee
Splunk Employee

You have specified INDEXED_EXTRACTIONS = PSV, Splunk should do the right thing automatically.

You definitely don't need a transforms.conf (aside from it being incorrect), please review this documentation

0 Karma

shariinPH
Contributor

with this i have to remove the configs on my transforms.conf?

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Yes, although it doesn't matter if it doesn't get referenced.
I also wouldn't specify anything in ../etc/system/local but instead put all your configurations for this in a separate app context. Whatever you decide, definitely only have it ONE place.

I would just try:

[rbil_mrsched]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
INDEXED_EXTRACTIONS = PSV

You should see that your events show up with the field names as defined in the header row of the PSV input file, assuming you specified sourcetype=rbil_mrsched in your inputs.conf.

If you don't like those field names, you can create field aliases on your search head, or use the rename command in your searches.

0 Karma

stephane_cyrill
Builder

Hi , I have put your sample data in a text file that i indexed. you can use this regex to have your fields extracted as you like.

index=* sourcetype=txt | rex field=_raw "^(?P\\s+\\d+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)"|table RbillARU RbillPortion RbillARTextSched RbillFromDate
0 Karma

shariinPH
Contributor

hello, does it work on your machine?

0 Karma

stephane_cyrill
Builder

Hi, I want to try But i need your file.
is it a csv?
can you send it to
cyrilleko@gmail.com

0 Karma

shariinPH
Contributor

Its a txtfile .. i'll send you

0 Karma

stephane_cyrill
Builder

ok i will take a look.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...