Getting Data In

How to edit my current props and transforms.conf to eliminate the first 10 lines of my sample log?

TheJagoff
Communicator

I have a log that I want to throw the first 9 lines to the bit bucket, but I can’t seem to get the transforms.conf to do it.
It’s doing the opposite of what I want it to do – it’s eliminating everything below the time\tthread… line

Here’s the the first 12 lines of the log… named trace.log

traceVersion=4.3.0
hostName=server1
clusterName=ADS
cpuType=INTEL
osType=LINUXRDH
buildType=gold
initCause=normal
initTime=Tue Aug 16 19:34:24 2016 (1471376064) schema=
time\tthreadID\tlevel\tunitName\tmessage
Aug 16 19:34:24:133     0xB76D06C0      3       srkDefault      SrkInitTime = 1471376064
Aug 16 19:34:24:143     0xB76D06C0      1       srkDefault      Watchdog Sucessfully Started
Aug 16 19:34:35:123     0xB76D06C0      0       srkDefault      Changing Trace Unit Launcher to Level 3

Here’s the props.conf

[trace-sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S:%3N
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
TRUNCATE = 999999
TRANSFORMS-t1=eliminate_headers
category = Custom
pulldown_type = true

Here’s the transforms.conf

[eliminate_headers]
REGEX=^(?:trace|hostName|clusterName|cpuType|osType|buildType|initCause|initTime|time)\s
DEST_KEY=queue
FORMAT=nullQueue

As always, any and all help is much appreciated.

Thank you

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You can use SEDCMD command which is like sed..

You need to set SEDCMD in your props.conf since this needs to happen at index-time

[trace-sourcetype]
 TIME_PREFIX = ^
 TIME_FORMAT = %b %d %H:%M:%S:%3N
 MAX_TIMESTAMP_LOOKAHEAD = 19
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
 TRUNCATE = 999999
 TRANSFORMS-t1=eliminate_headers
 category = Custom
 pulldown_type = true
 SEDCMD-strip-detail-msg = time\tthreadID\tlevel\tunitName\tmessage

http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Anonymizedatausingconfigurationfiles

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...