Splunk Search

Importing a file, how do I configure Splunk to parse fields from the raw logs in this format?

dperry
Communicator

I'm importing a file into Splunk and the file always has these fields:

Date (07/25/16 ) | Time (01:12:04) | Message (ANS1898I ***** Processed     8,000 files *****)

Here is the raw log:

07/25/16   01:12:04 ANS1898I ***** Processed     8,000 files *****
07/25/16   01:12:05 ANS1898I ***** Processed    12,000 files *****
07/25/16   01:12:06 ANS1898I ***** Processed    14,000 files *****
07/25/16   01:12:07 ANS1898I ***** Processed    16,500 files *****
07/25/16   01:12:08 ANS1898I ***** Processed    18,000 files *****
07/25/16   01:12:09 ANS1898I ***** Processed    20,000 files *****
07/25/16   01:12:10 ANS1898I ***** Processed    21,500 files *****
07/25/16   01:12:11 ANS1898I ***** Processed    23,500 files *****
07/25/16   01:12:12 ANS1898I ***** Processed    25,500 files *****
07/25/16   01:12:13 ANS1898I ***** Processed    27,500 files *****
07/25/16   01:12:14 ANS1898I ***** Processed    29,500 files *****
07/25/16   01:12:15 ANS1898I ***** Processed    32,000 files *****
07/25/16   01:12:16 ANS1898I ***** Processed    34,500 files *****
07/25/16   01:12:17 ANS1898I ***** Processed    37,000 files *****
07/25/16   01:12:18 ANS1898I ***** Processed    38,000 files *****
07/25/16   01:12:19 ANS1898I ***** Processed    40,000 files *****
07/25/16   01:12:20 ANS1898I ***** Processed    42,500 files *****
07/25/16   01:12:21 ANS1898I ***** Processed    45,000 files *****
07/25/16   01:12:22 ANS1898I ***** Processed    48,500 files *****
07/25/16   01:12:23 ANS1898I ***** Processed    51,500 files *****
07/25/16   01:12:24 ANS1898I ***** Processed    55,000 files *****

Right now I have Splunk parsing it like this:

     _time      07_25_16 01_11_30 Server Version 7     Level 3_0     Release 1
1   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
2   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
3   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
4   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
5   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
6   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
7   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 

How do I get Splunk to format it like this:

     _time            Message
1   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
2   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
3   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
4   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
5   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
6   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
7   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****           
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

I am not sure I understand what you are after.

Without any specific configuration, Splunk is trying to auto-detect the timestamp (which it does in your example) and will break every line into its own event.
Given your example, I don't know where the "Server Version 7 Level 3_0 Release 1" stuff would come from.

In the Search and Reporting App, you will always see the time column with the timestamp that Splunk extracted, and a column titled "Event", which shows you the raw event as it was indexed. Here is an example of an event from my _audit index
alt text

If you want to produce a different display, consider using the table command to do that. Or you can simply do:

<yoursearch> | table _time, _raw

But as I said, I may not understand what you are after.

0 Karma

dperry
Communicator

alt text

For some odd reason Splunk is adding these additional interesting fields: 07_25_16 01_01_12 and so on......

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Sorry, your question got reformatted to be more clear after I responded. I think my example search using the _raw field may help you. You can always insert a | rename _raw as Message to rename the field.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

And I still don't understand where this header comes from:
_time 07_25_16 01_11_30 Server Version 7 Level 3_0 Release 1

0 Karma

dperry
Communicator

Sorry for the confusion...the header is coming from the actual log file....for some reason when I upload this log file via Add Data/Upload data....Use the default splunk settings The header automatically fills in:

_time and then Server Version 7 Level 3_0 Release 1

0 Karma

ppablo
Retired

Tried my best to format everything for them to be more clear, but I was also confused by that header. Hopefully the original poster can clarify that part.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...