Splunk Search

Transaction with changing transitive field?

Wilcooley
Path Finder

I am trying to create transactions based on two fields where one changes and one is not always present. For example, I want single transaction of the following events:


event=1 pid=1
event=2 pid=1 qid=2
event=3 pid=2 qid=2

But what I get is 2 transactions, with "...|transaction qid pid":


event=1 pid=1
event=2 pid=1 qid=2

and


event=3 pid=2 qid=2

Is it possible to make transaction do this? The only solution I have been able to think of is to null the pid field for the "event=3" sort of events, but that seems ugly. I have tried the various parameters to transaction, such as connected, unifyends, but they have made no difference.

Just in case my simplified example is inconsistent with reality, the real events are Sendmail submission logs, where the name of the authenticated sender has no queue ID and can only be correlated with the actual messages sent by the PID (and host). (No idea what's going to happen if/when I get looking for multiple messages related to a single authentication event.)

Here's a scrub'd example of my results.

First transaction:


madonna sendmail[10102]: r2RPOYuJ011100: to=<Hildred_Joan@example.com>, delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=000404, relay=marget-h0.example.com. [178.115.54.001], dsn=2.0.0, stat=Sent (Ok: queued as 502VQ10L21)

Second transaction:


madonna sendmail[02104]: AUTH=server, relay=host-167-042.dina.ben.edu [031.002.167.042], authid=hpjmm1@ben.edu, mech=PLAIN, bits=0
madonna sendmail[02104]: r2RPOYuJ011100: from=<hpjmm1@ben.edu>, size=10405, class=0, nrcpts=1, msgid=<V2K3020H-7264-3WMX-632E-T4X004IM01A0@ben.edu>, proto=ESMTP, daemon=MSA, relay=host-167-042.dina.ben.edu [031.002.167.042]

FWIW, I'm still on 5.0.3.

Tags (2)

markthompson
Builder

I'm not too sure what you wish to achieve here, transaction groups multiple events into a single event, so I'm assuming you're trying to group all events with a certain QID

In any case, there is a command known as "fillnull" used in the format of:
fillnull fieldname 0

This would then set your not present fields to visible in all events, and then you can use transaction.

Alternatively, If I've misinterpreted the question, you can use the transaction command based on a field. i.e.
transaction fieldname
Which will group all events that have the same value in fieldname into a single event.

if this is not the question you meant, please comment on this answer and I'll see what I can come up with.

Thanks

0 Karma

waydenie
New Member

Hi,
Wondering if there is a followup to this? I have a very similar use case, also with sendmail and qid's that are transitive, but not involving joining authid's and pids... In any case, issue of grouping log entries into common transactions is very similar...

To sum up my 'flavor' of this issue; I'm trying to 'follow' or trace an email as it passes through multiple sendmail hops in our organization until it is delivered. The log file sources come in from multiple servers and the 'hand off' issues a new qid which is needed to trace the log lines on the new hop.

ie:
hop1 log line 1:
hop1 log line 2:
hop1 log line n: ( message accepted for delivery)
hop2 log line 1:
hop2 log line 2:
hop2 log line n: ( message accepted for delivery)

I have created an externally defined regex based field (I called: AcceptQID) which captures the 'next hop' qid in the handful of different formats that I've come accross... I would now like to colapse all these log lines into one 'email message transaction'...

0 Karma

markthompson
Builder

Waydenie, I've posted a reply seen as it's relevant to you.

Hope it helps.

0 Karma

waydenie
New Member

Hi MarkThompson,

I just noticed that my example had the qid parts eaten... probably because I used gt / lt signs which were treated as some type of htm tags... I've tried to lay out my question again below, hopefully it's clearer now!

I would like all the log events from an email message delivery through a number of email server hops to end up as one transaction showing the delivery of that email message as it goes from ingress at our border server through our core to it's final email server delivery. To do this, I think I need to make a transaction out of all the log events related to an email delivery.

The complication is that while each email has log lines related to each other by an assigned qid, each email server that handles that email during delivery assigns it's own qid. The string of qid's is maintained by the receiving sendmail server providing the sending sendmail server the new qid...

I hope that a) my description is clear and b) I'm not telling you too much of what you already know... For the benefit of all who might read this, I'm trying to be as (painfully) clear as possible! 🙂

In short, I would like all log events of the following format to be "made" into one transaction;
mailsrv1 log line 1: qid1
mailsrv1 log line 2: qid1
mailsrv1 log line n: qid1 ( qid2 message accepted for delivery)
mailsrv2 log line 1: qid2
mailsrv2 log line 2: qid2
mailsrv2 log line n: qid2 ( qid3 message accepted for delivery)

0 Karma

waydenie
New Member

Just thought I should add...

Splunk already parses out a field for qid. I added a custom field, I called AcceptQID, which parses out the new qid from the various forms of "(qidX message accepted for delivery)" that I've come across... I guess I'm really just looking for a search that can create a transaction out of all the log events that join up through the chain: qid == AcceptQID ...

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

So you could do an eval to make a unique identifier.

your_search | eval txn_id = event . pid . qid | transaction txn_id

so You'd then have this:

event=1 pid=1 txn_id=11
event=2 pid=1 qid=2 txn_id=212
event=3 pid=2 qid=2 txn_id=322

Wilcooley
Path Finder

Wouldn't these give me separate transactions?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...