Splunk for monitoring Mainframe logs

[Bhavna]

Hi,

I am exploring how we can use splunk to monitor mainframe logs. I have no idea about the same. can anyone answer the below question

(1) Is it possible to monitor mainframe logs with splunk ?
If yes,
(2) Can we get the mainframe logs directly into splunk by installing forwarder on Z/os like we do for any other OS, or is there any indirect way of doing the same ?

[kchism]

BMC Software is a great option for moving mainframe log (and other data like SYSLOG) into Splunk or other analytics engines.  East to install and set up (data moving into Splunk within days) with Splunk formatting.  BMC also "enriches" the mainframe data to make it easier to correlate in Splunk for key data fields like USER ID.

[tldenney]

IBM Common Data Provider for z Systems (CDPz) is the best option for sending Mainframe logs to Splunk.

CDPz can send a wide variety of data including 140 data sources and 100+ SMF record types. More specifically, CDPz can support the following:

• SMF records
• SYSLOG (IBM z/OS System Log and USS SyslogD)
• JOBLOGs
• Application logs (IBM CICS Transaction Server logs and IBM WebSphere Application Server logs)

CDPz also has advanced filtering capabilities including RegEx and time filtering that can be set up using the built-in web configuration tool shown below.

More information on IBM Common Data Provider for z Systems can be found directly on Splunkbase.

[tldenney]

The following Splunk Blog outlines how Splunk and IBM are partnering to help customers integrate IBM Z (Mainframe) Data and Insights into Splunk software:

https://www.splunk.com/blog/2017/08/22/insane-in-the-mainframe-splunk-and-ibm-partner-to-provide-end...

[jreda]

Ironstream from Syncsort can do all of this work for you. It will handle all of the issues related to SYSLOG, z/OS SMF records, log4j and flat files. It deals with the compression, the triplets, the binary data and converts the data from EBCDIC to ASCII. It does this very efficiently, even offloading a lot of the work to a zIIP engine in order to keep the MSU cost of this work to an absolute minimum. This is all done in real time to give you the best data latency possible while not impacting the existing workload on your system.

[kristian_kolb]

1) splunk can index any log that is plain text. However, if the events themselves do not contain timestamps, or the correct information in general, you will not be helped much

2) there is no forwarder for Z/OS. Not having too much experience with mainframes, my best guess is that you should send your logs via syslog to a syslog-server, where a forwarder is installed.

/K