Getting Data In

How to read logs in Tabbed text format

eritzman
New Member

OK - I'm a NUB here and experimenting with SPLUNK.
I have some log files that are saved in a TAB/Columned format.
[example: Logs_2013-05-17.txt

RecID       TimeStamp               ClientIPAddress  ServerName                                         ServerIPAddress  ServerPort Method     URIStem                                            URIQuery                                                                                                                                                                                                                                                         Status UserName                                           URLRoot
26405679    2013-05-17 01:35:05.000 67.190.82.148    HOMESITE                                                192.168.100.109  443        GET        /sites/ter/_layouts/listfeed.aspx                  List=%7BD94F33B2%2DB09D%2D4EEA%2D8D14%2D0B8A1ACAC8B1%7D                                                                                                                                                                                                          401    NULL                                               home3.mysite.com
26536158    2013-05-17 00:00:41.000 174.255.208.31   HOME-WWW                                            192.168.100.89   80         GET        /styles/default.css                                NULL                                                                                                                                                                                                                                                             200    NULL                                               www.mysite.com

]

When I import it to Splunk I am not able to get it to recognize the fields and information correctly (EventBreaks)
How do I apply a "sourcetype" that will read these logs correctly?

Tags (3)
0 Karma

lguinn2
Legend

The docs will help: Create source types

But here is one way to do what you want

inputs.conf

[monitor:///path/ Logs_*.txt]
sourcetype=mynewsourcetypename

props.conf

[mynewsourcetypename]
SHOULD_LINEMERGE=false
TRANSFORMS=extract_tabbed_fields

transforms.conf

[extract_tabbed_fields]
DELIMS="\t"
FIELDS=RecID,TimeStamp,ClientIPAddress,ServerName,ServerIPAddress,ServerPort Method,URIStem

HTH

jkat54
SplunkTrust
SplunkTrust

Splunk auto line breaks based on time stamps. You should first give the preview tool a try and see if you can figure it out there, but here's an example of a tab delimited input I have:

Props.conf on indexer:

 [SMTP]
 REPORT-smtp = getsmtpfields

Transforms.conf on indexer:

 [getsmtpfields]
 DELIMS = "\t"
 FIELDS = Type,Number,MSG_ID,Date,IP,MSG

Inputs.conf on universal forwarder:

 [monitor://c:\Program Files (x86)\hMailServer\Logs]
 disabled = 0
 host = hostname
 index = main
 sourcetype = SMTP
 crcSalt = <SOURCE>
 whitelist = \.log$

http://docs.splunk.com/Documentation/Splunk/5.0.2/admin/Transformsconf
http://docs.splunk.com/Documentation/Splunk/5.0.2/Admin/Propsconf

So yours might look like this:

Props.conf on indexer:

 [custominput]
 REPORT-custominput = custominputfields

Transforms.conf on indexer:

 [custominputfields]
 DELIMS = "\t"
 FIELDS = RecID,TimeStamp,ClientIPAddress,ServerName,ServerIPAddress,ServerPort,Method,URIStem,URIQuery,Status,UserName,URLRoot
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...