Splunk Search

How do you edit props.conf to correctly parse data from a PowerShell script?

ckeller2791
Explorer

I have a powershell script which feeds data into Splunk via a UDP port. The output of the script is as follows:

AbatInstanceID=32107862
AbatBatchID=32107825
AbatPlanName=ABM - Partner Remittance Loader -ASOBilling
AbatJobName=Execute Java partnerRemittanceLoader
AbatJobPath=/HXSPRD01/Promotable Objects/Plans/Business Processes/ABM/ABM - Partner Remittance Loader -ASOBilling
AbatStatus=Successful
AbatQueue=ExecutionQ1 HXSPRD01
AbatStartTime=01/31/2019 11:00:10
AbatEndTime=01/31/2019 11:00:36
AbatElapsedTime=0:0:0:23
AbatLogFile=\\CO1-BAXPRD01\ASCI_ABATLOG\CO1BASPRD01\ExecutionQ1 HXSPRD01\Execute Java partnerRemittanceLoader_0032107862-31Jan2019-160001_001.log

When the data is ingested into Splunk, it treats that input as two separate events splitting on the line where AbatStartTime and AbatEndTime are located due to there being two different date/time stamps. I am having trouble nailing down the correct regex syntax for parsing AbatStartTime as the official timestamp for the event. I have the powershell script output a double carriage return to separate the event. My sourcetype config is as follows

CHARSET = ASCII
BREAK_ONLY_BERFORE = ([\r\n\r\n]+)
LINE_BREAK = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 150
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_PREFIX = ^AbatStartTime=\s
TZ = America/New_York
category = Custom
pulldown_type = true

Any assistance would be appreciated.

0 Karma

ckeller2791
Explorer

Thank you both for you advise, I did a little tweaking and the final setting that got it working were

[activebatch]
BREAK_ONLY_BEFORE = AbatInstanceID
SHOULD_LINEMERGE = true
CHARSET = ASCII
DATETIME_CONFIG =
LINE_BREAKER = AbatInstanceID
NO_BINARY_CHECK = true
TIME_FORMAT = %m/%d/%Y %H:%M:%S
TIME_PREFIX = AbatStartTime=
TZ = America/New_York
category = Custom
pulldown_type = true

0 Karma

woodcock
Esteemed Legend

You almost certainly have too many settings that are likely to cause problems. Try this ONLY:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n\r\n]+)
NO_BINARY_CHECK = true
TIME_PREFIX = [\r\n]+AbatStartTime\s*=\s*
TIME_FORMAT = %m/%d/%Y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
TZ = America/New_York
category = Custom
0 Karma

mayurr98
Super Champion

alt text

I think your sourcetype should be like this :

[ sourcetypename]
SHOULD_LINEMERGE=true
CHARSET=ASCII
BREAK_ONLY_BEFORE=AbatInstanceID
TIME_PREFIX=AbatStartTime=
TIME_FORMAT=%m/%d/%Y %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=150
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...