making attribute with eval expression

[mecase]

What exactly is being operated on when you are in the screen "Edit Attributes with an Eval Expression"

In my mind what I want to do is the following. I have a Root Object defined which returns values like this:

2014-04-10T11:27:08.399 Thread-91 Total time elapsed from start to end of flatsequence : 31926 milliseconds or 31.926 seconds or 0.5321 minutes

So I've been assuming that that whole line is the string I want to operate on and in awk I would do something like awk '{print $13}' to get the value 31926. In pseudocode I would do substring(find(" : ")+3,find("milliseconds")) and evaluate that to a number.

I'm sorry if this is a real beginner's question but that is what I am :).

I've been looking at mvfind but then what do I put in for X? I want it to be THIS object...

Then I try regex and that doesn't give any value. Even though if I type the regex in a search like this ".*milliseconds" I get all lines that I want to define as "endsequence" object and try to extract the number before milliseconds.

help! 🙂

[mecase]

Stubborn tenacity.

The _raw was:

2014-04-10T11:27:08.399 Thread-91 Total time elapsed from start to end of flatsequence : 31926 milliseconds or 31.926 seconds or 0.5321 minutes

I made an attribute in a Root Object of a datamodel by putting:

mvindex(split(mvindex(split(_raw,":"),3)," "),1)

In the "Eval Expression" box and now I have an attribute elapsed_time_ms which has the number 31926 for this example.

[mecase]

Thanks so much for your help. As you said, lots of data not shown to you, but the following gets to all the ones of that type that I wanted to see (your rex plus my search).

("from start to end of" AND (flatsequence OR "nod by nod.define" OR "nod by ta_pos.goto" OR "map by ta_pos.goto" OR "map by dither"))  | rex "\s:\s+(?\d+)"

You are probably correct, I can do a lot of what I want by learning the syntax of splunk searches... if I decide to go that route.

[martin_mueller]

I can't tell you whether your data can answer all your questions because I only have a small glimpse into your data and your questions.

[martin_mueller]

In most cases extracting fields like your millisecond value is done using regular expressions. I've posted an example in the previous answer, here's a full working one (paste in a Splunk search bar, hit enter):

| stats count | eval _raw = "2014-04-10T11:27:08.399 Thread-91 Total time elapsed from start to end of flatsequence : 31926 milliseconds or 31.926 seconds or 0.5321 minutes" | rex "\s:\s+(?<milliseconds>\d+)"

I'm setting up a dummy event with your text in it, and extracting the milliseconds using a regular expression.
That can be more robust to event changes than e.g. word counts.

[mecase]

without having to make my own model to extract attributes like the milliseconds above.

[mecase]

I wish I understood why that is the case ("utterly convoluted..."). If I did then maybe I'd see how splunk can do what I want. I want to be able to see how much time between points grouped by login, leg, end of writing a file, and possible other markers I might find. There are a bunch of lines like this between each "start of Leg" marker. Are you saying that I should be able to construct a search on the data that would do something like that?

[martin_mueller]

That seems utterly convoluted compared to just using a regex-based extraction, but whatever floats your boat...

[martin_mueller · Accepted solution]

This sounds like you need a regex extraction like this:

flatsequence\s*:\s*(?<fieldname>\d+)

I'm a bit confused as to what you're actually trying to do though, due to the smorgasbord of different things like eval-based fields, mvfind, datamodel, ... some background info on what you're actually trying to do?

[mecase]

Looking at the json model generated I see there might be a way to change the parent name from BaseEvent... but I'm running out of patience with getting going this way, and not sure the benefits of splunk, for my case, is worth much more effort.

[mecase]

Thanks. That might be fine for future, but right now, I can not modify logs of a past run easily in the way you indicate without writing code, which by then, I will have made what I was hoping splunk would help me make :). I think I chose the wrong tool if this is the only way to do it. Thanks for trying. (continued in next comment)

[martin_mueller]

The best way to get such a hierarchical tree-like transaction going is to add hierarchical tree-like IDs to your events. Here's an example:

timestamp ID=12345 start name=myouterfunction ...
timestamp ID=12345-abcdef start name=myinnerfunction1 ...
timestamp ID=12345-abcdef end ...
timestamp ID=12345-ghijkl start name=myinnerfunction2 ...
timestamp ID=12345-ghijkl-54321 start name=myinnermostfunction ...
timestamp ID=12345-ghijkl-54321 end ...
timestamp ID=12345-ghijkl end ...
timestamp ID=12345 end ...

Using that, you can create transactions over any specific tree depth you like. Once you're familiar with doing that in Splunk you can think about getting a datamodel built on top.

[mecase]

Hi Martin,

Thanks! I am making a datamodel. I have a lot of debug output from an application I wrote and I'm trying to build a hierarchical structure. My thought was that it would then be easier to ask questions like "where did the time go between A and B?" How much processing a command, taking data, writing headers, etc. I thought the way to start was to define a bunch of Root Objects and then eventually create transaction objects. see http://answers.splunk.com/answers/132820/is-it-possible-to-create-a-model-which-takes-a-flat-file-an.... Any advice on where to start...