Splunk Search

Identify all transactions running when an error occurs

eoszej123
Engager

I am attempting to merge two datasources to find every transaction (not to be confused with a Splunk transaction!) that was running when the violation occurred, if any. Unfortunately, I have only been able to figure out how to do this as a join.

Is there a way to make the query perform better?

  • MFSOURCETYPE=SYSLOG - typical events
  • MFSOURCETYPE=SMF110 - each "event" is really an instance of a transaction that includes both START and STOP times.

index=mainframe sourcetype=syncsortMF MFSOURCETYPE=SMF110 TRAN!=C* SAPPLID=CICSP* 
| eval ABCODEO=if(ABCODEO="","!success!",ABCODEO)
| fields SAPPLID host TRAN START STOP ABCODEO
| join SAPPLID host [
  search index=mainframe sourcetype=syncsortMF MFSOURCETYPE=SYSLOG MSGNUM=DFHSM0102 JOBNAME=CICSP*
  | rename JOBNAME as SAPPLID
  | rex field=MSGTXT "violation \(code (?<dump_code>[\'\w]+)\) has been detected by module (?<module>\w+)"
  | eval syslogTime=DATETIME
  | fields SAPPLID host MSGNUM syslogTime dump_code module
]
| where START <= syslogTime AND syslogTime <= STOP
| table START STOP syslogTime SAPPLID MSGNUM dump_code module TRAN ABCODEO

The kind of output I'm looking for, where there were two transactions running at the time a single DFHSM0102 appeared in SYSLOG:

START                      STOP                       syslogTime                   SAPPLID  MSGNUM    dump_code module  TRAN ABCODEO
2019-02-24 02:38:47.189675 2019-02-24 02:38:53.161519 2019-02-24 02:38:47.47 -0600 CICSP001 DFHSM0102 X'0D11' DFHSMMF GYAH !success!
2019-02-24 02:38:47.273213 2019-02-24 02:38:54.841579 2019-02-24 02:38:47.47 -0600 CICSP001 DFHSM0102 X'0D11' DFHSMMF GYQH ASRA

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...