Splunk Search

Importing a file, how do I configure Splunk to parse fields from the raw logs in this format?

dperry
Communicator

I'm importing a file into Splunk and the file always has these fields:

Date (07/25/16 ) | Time (01:12:04) | Message (ANS1898I ***** Processed     8,000 files *****)

Here is the raw log:

07/25/16   01:12:04 ANS1898I ***** Processed     8,000 files *****
07/25/16   01:12:05 ANS1898I ***** Processed    12,000 files *****
07/25/16   01:12:06 ANS1898I ***** Processed    14,000 files *****
07/25/16   01:12:07 ANS1898I ***** Processed    16,500 files *****
07/25/16   01:12:08 ANS1898I ***** Processed    18,000 files *****
07/25/16   01:12:09 ANS1898I ***** Processed    20,000 files *****
07/25/16   01:12:10 ANS1898I ***** Processed    21,500 files *****
07/25/16   01:12:11 ANS1898I ***** Processed    23,500 files *****
07/25/16   01:12:12 ANS1898I ***** Processed    25,500 files *****
07/25/16   01:12:13 ANS1898I ***** Processed    27,500 files *****
07/25/16   01:12:14 ANS1898I ***** Processed    29,500 files *****
07/25/16   01:12:15 ANS1898I ***** Processed    32,000 files *****
07/25/16   01:12:16 ANS1898I ***** Processed    34,500 files *****
07/25/16   01:12:17 ANS1898I ***** Processed    37,000 files *****
07/25/16   01:12:18 ANS1898I ***** Processed    38,000 files *****
07/25/16   01:12:19 ANS1898I ***** Processed    40,000 files *****
07/25/16   01:12:20 ANS1898I ***** Processed    42,500 files *****
07/25/16   01:12:21 ANS1898I ***** Processed    45,000 files *****
07/25/16   01:12:22 ANS1898I ***** Processed    48,500 files *****
07/25/16   01:12:23 ANS1898I ***** Processed    51,500 files *****
07/25/16   01:12:24 ANS1898I ***** Processed    55,000 files *****

Right now I have Splunk parsing it like this:

     _time      07_25_16 01_11_30 Server Version 7     Level 3_0     Release 1
1   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
2   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
3   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
4   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
5   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
6   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 
7   7/25/16      07/25/16 01:12:13 ANS1898I ***** Processed 27    500 files *****
     1:12:13.000 AM 

How do I get Splunk to format it like this:

     _time            Message
1   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
2   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
3   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
4   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
5   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
6   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****
7   7/25/16    
     1:12:09.000 AM     07/25/16 01:12:09 ANS1898I ***** Processed 20,000 files *****           
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

I am not sure I understand what you are after.

Without any specific configuration, Splunk is trying to auto-detect the timestamp (which it does in your example) and will break every line into its own event.
Given your example, I don't know where the "Server Version 7 Level 3_0 Release 1" stuff would come from.

In the Search and Reporting App, you will always see the time column with the timestamp that Splunk extracted, and a column titled "Event", which shows you the raw event as it was indexed. Here is an example of an event from my _audit index
alt text

If you want to produce a different display, consider using the table command to do that. Or you can simply do:

<yoursearch> | table _time, _raw

But as I said, I may not understand what you are after.

0 Karma

dperry
Communicator

alt text

For some odd reason Splunk is adding these additional interesting fields: 07_25_16 01_01_12 and so on......

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Sorry, your question got reformatted to be more clear after I responded. I think my example search using the _raw field may help you. You can always insert a | rename _raw as Message to rename the field.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

And I still don't understand where this header comes from:
_time 07_25_16 01_11_30 Server Version 7 Level 3_0 Release 1

0 Karma

dperry
Communicator

Sorry for the confusion...the header is coming from the actual log file....for some reason when I upload this log file via Add Data/Upload data....Use the default splunk settings The header automatically fills in:

_time and then Server Version 7 Level 3_0 Release 1

0 Karma

ppablo
Retired

Tried my best to format everything for them to be more clear, but I was also confused by that header. Hopefully the original poster can clarify that part.

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...