Getting Data In

Importing Text file with DAT extension separated by |

hneuman
Engager

Good Morning

First off i been using Splunk for a year but mostly importing Logs files from Firewalls and Windows Servers.
Now i been ask to import information from a VoIP platform it comes in text files with DAT extensions and are separated by |

I been trying to import the folder containing the files but i get a triangle error handling this .. i try importing this format as a CSV and other but i just cant get splunk to imported or even read it.
Here is a simple of the data inside the DAT file

0|5558013|20150103 234659|5558888|11||11001100||634|0|201|2061||PRDCWR7B00||10||1112068888||106
0|5557815|20150103 235656|5551634|1||11001000||201|14||||PRDCWR7B00|1123011634|10||||8
0|5554908|20150103 235000|5551349|7||11001100||551|2|611|0||CS2KTOHUAWEI|1123051349|10||||68
0|5556438|20150103 235249|5555224|39||11001000||551|18||||PRDCWRJF7B00|1123995224|10||||383

Am sure its my lack of experience with importing files in splunk
Can anyone point me in the right direction
Thanks

0 Karma
1 Solution

kristian_kolb
Ultra Champion

Just use a regular [monitor] in for inputting the files in the directory (i.e. in inputs.conf)

[monitor:///my/dir/*.dat]
sourcetype=my_dat
index=my_index

In props.conf, you might need to specify TIME_FORMAT
For the field extraction, use a REPORT in props.conf, and use FIELDS and DELIMS in transforms.conf

props.conf

[my_dat]
TIME_FORMAT = %Y%m%d %H%M%S
REPORT-dat = dat_pipes

transforms.conf

[dat-pipes]
DELIMS = "|"
FIELDS = field1, field2, field3 ... field20

You should probably read the Getting Data In section of the docs, and check out the documentation on REPORT field extractions.

EDIT: typo

View solution in original post

kristian_kolb
Ultra Champion

Just use a regular [monitor] in for inputting the files in the directory (i.e. in inputs.conf)

[monitor:///my/dir/*.dat]
sourcetype=my_dat
index=my_index

In props.conf, you might need to specify TIME_FORMAT
For the field extraction, use a REPORT in props.conf, and use FIELDS and DELIMS in transforms.conf

props.conf

[my_dat]
TIME_FORMAT = %Y%m%d %H%M%S
REPORT-dat = dat_pipes

transforms.conf

[dat-pipes]
DELIMS = "|"
FIELDS = field1, field2, field3 ... field20

You should probably read the Getting Data In section of the docs, and check out the documentation on REPORT field extractions.

EDIT: typo

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...