Knowledge Management

How do I change the event boundaries of a syslog file from the mainframe

usernamejpblais
Engager

Hi! I created a new sourcetype (syslog_sic) because I have a syslog file coming from the mainframe with multiple line event that I want to break at each timestamp. My timestamp defenition is "2019099 00:24:48.71" meanning 2019=year 099=number of day in the year. When the data get indexed, it reconnized the time but not the date. The event break is set to breaking at each timestamp but instead it is breaking at each line.

0 Karma
1 Solution

rmjharris
Path Finder

In props.conf

Simplest
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N

Better but with specific regex based on the small sample you provided.
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \w{5}\s\w{7}\s\w{4}\s
LINE_BREAKER = ([\r\n]+)(?=\w{5}\s\w{7}\s\w{4}\s\d{7})
SHOULD_LINEMERGE = false

View solution in original post

CurtisGannaway
Loves-to-Learn

Hi @usernamejpblais,

If you need to get mainframe data (security, database, CICS, FTP, TCPIP, master console messages and much more), please see dgtechllc.com/meas. Our Mainframe Event Acquisition System (MEAS) product will allow you to monitor, filter and forward - in real time - any/all events from the mainframe that you would like to see in Splunk. It take roughly 1/2 day to install and no IPL necessary. Let me know if this solution could help you out. 

 

Thanks!

0 Karma

rmjharris
Path Finder

In props.conf

Simplest
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N

Better but with specific regex based on the small sample you provided.
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \w{5}\s\w{7}\s\w{4}\s
LINE_BREAKER = ([\r\n]+)(?=\w{5}\s\w{7}\s\w{4}\s\d{7})
SHOULD_LINEMERGE = false

usernamejpblais
Engager

Super!!!

Thanks mjharris!

0 Karma

koshyk
Super Champion

please provide atleast 4-5 lines to see how the sample data looks like

0 Karma

usernamejpblais
Engager

Hello Koshyk!

Thanks for you're help!

H158N 4020000 H158 2019099 00:24:47.97 STC67273 00000080 XCOMM0780E Txpi 227: Socket received
H158S Last error: 167
H158N 4020000 H158 2019099 00:24:47.97 STC67273 00000080 XCOMM0805I TCP/IP CONNECTION END
H158N 0002000 H158 2019099 00:24:48.11 STC64107 00000090 PGTV1710E TCPERR 00050000 on READ
H158S CONNECTION CLOSED PREMATURELY
H158M 0000000 H158 2019099 00:24:48.33 STC66246 00000090 CECA0143I The subscription heartbeat
H158S 779
H158D 779 00000090 DATASRC=IMS SUBSTATE=REPLICATE
H158D 779 00000090 PE=Active/Standby LATENCYSTATE=No
H158E 779 00000090 COMMITS=0 ABSBOOKMARK=2019-04-
H158N FDE0000 H158 2019099 00:24:48.71 STC66280 00000281 HWSP1415E TCP/IP SOCKET FUNCTION
H158S , M=SDRC, ID=DELDUMMY,IPv4=10.250.1

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...