Splunk Search

How do I keep startswith from evicting a previous transaction?

jswarren
New Member

Assume I have an input file like the following:

2015-07-28 12:00:01 Executing function a...
2015-07-28 12:00:02 debug1
2015-07-28 12:00:03 debug2
2015-07-28 12:00:04 Completing function a (value=-1)
2015-07-28 12:00:05 Executing function a...
2015-07-28 12:00:06 debug3
2015-07-28 12:00:07 debug4
2015-07-28 12:00:08 Completing function a (value=0)

I want to build a transaction object that begins with the first line and ends with the last. If I use startswith="Executing function a..." and endswith="Completing function a (value=0)", it appears that the second Executing function a... evicts the first, even though an endswith has not occurred. I end up with a single transaction that begins at 12:00:05 and ends at 12:00:08, and the earlier is an incomplete transaction.

Is there any way to get transaction not to evict the previous transaction when it encounters another startswith (prior to an endswith)?

0 Karma

emiller42
Motivator

If the logging is single-threaded (meaning you won't have processes interleaved with each other) then you can actually omit the 'startswith' and get what you want. Or, I do in testing at least.

If you may have multiple hosts running concurrently, then you want to include that in the transaction:

| transaction host endswith="(value=0)"

alt text

0 Karma

jswarren
New Member

Ahh...yes...that would work, if there weren't "noise" between functions....I'll update the example to reflect that noise.

0 Karma

jswarren
New Member

Or...I would post the update, if I had enough karma....

Assume that there are more events before the "Executing..." and after the "Completing" that should be excluded.

0 Karma

emiller42
Motivator

Any way to filter out that noise? It might be a good idea to shape the initial search to only grab the lines you really care about from these transactions.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Make sure that 'value' field is extracted, then try the search suggested by @hgrow and filter the incomplete result by checking "| where value=0". This will drop the transaction with value=-1 (incomplete)

0 Karma

jswarren
New Member

It's not the endswith that is the problem - I can successfully filter out the incorrect "ends". I can't filter out the incorrect "starts" because they are identical to each other. I need to somehow tell transaction to ignore repeated starts.

0 Karma

hgrow
Communicator

Hi jswarren

I'm not sure if thats the key but your endswith="Completing function a (value=0)" is explicit looking for value=0.

It might be enough to simplify your search to something like:

... | transaction startswith="Executing" endswith="Completing"

sincerely
hgrow

0 Karma

jswarren
New Member

If I do that, it results in two transactions, one from 12:00:01 - 12:00:04 and another from 12:00:05 - 12:00:08, which is not the desired outcome. The explicit "value=0" is requred.

0 Karma

hgrow
Communicator

Ah I see ... i've got that wrong. It's a tricky problem ...Im not sure if there is a simple way to not evict the first transaction.

If i get you right, you want all events from execution function a until Completing with value=0 in one transaction.It all depends on how your other events look like. Is it always function a? Are these events all in order? Maybe what you can try is to reduce your transaction to an endswith.
Something like ... | transaction endswith="(value=0)"

0 Karma

jswarren
New Member

Answers to your questions:

  1. No, it's not always "function a", it could be one of hundreds or thousands of different function names.
  2. Yes, the events are in order and the process that writes the log is single-threaded.
  3. I still have to have some way to identify the start of a transaction. What other options could I consider?
0 Karma

emiller42
Motivator

Extract out the function name as a field, and use that in your transaction:

| rex "function\s(?<function>[^\s]+)\s" | transaction function startswith="Executing" endswith="(value=0)"

That should pull the whole shebang as a single transaction. However, this will omit any lines which do not have a 'function' field. (which may be context you need)

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...