extracting multiple values for a field from one row(one event)

[smolcj]

Hi all,
Regex is troubling me when i have to extract a field compared with previous line. My log is like

Thread Event1 blablabla [something]....completed.
idletime:45.56
Thread Event2 blablabla [something]....completed.no ststistics
Thread Event3 blablabla [something]....completed.no ststistics
Thread Event4 blablabla [something]....completed.no ststistics
Thread Event5 blablabla [something]....completed.
idletime:45.56

how to extract those idle time related with Event1? i have extracted the evantnames as field, but i am not getting any idea to continue the search with a regex.. please help..
Thank you for your time

Question edited

i am sorry that i am confusing you people.. i will try to be more precise about my question.
1.timestamp : [threadname] : threadid :logdetails
2.timestamp : [threadname] : threadid
...........log summary............
Thread Event1 blablabla [something]....completed.no ststistics
Thread Event2 blablabla [something]....completed.
idletime:33
busytime:88
Thread Event3 blablabla [something]....completed.no ststistics
Thread Event4 blablabla [something]....completed.
idletime:55
busytime:99
3.timestamp : [threadname] : threadid :logdetails
4.timestamp : [threadname] : threadid :logdetails

so issue here is, splunk reads only the first field that is it will read the italicized inputs, but i need it to read all the data in BOLD. i know that this issue is because splunk extract only the first value from a line and ignores the repeated ones and here all these information comes under one event and because of that splunk is ignoring the repeated values
so now please help me to solve this issue
thank you

[smolcj]

i got the answer 🙂 i used the regex in both props.conf and transform.conf.. the field extraction defined in props.conf resulted that repeated field output.. now it is workin fine.. thanks for all the comments..

[martin_mueller]

MV_ADD=true doesn't affect non-repeating values.

I'm not quite sure what's happening on your end and how it differs from what you're trying to achieve.

[smolcj]

but only the first field in an event is repeating. all other values are fine

[martin_mueller]

Is there a reason for making the regex that long? It feels like "Idletime:(?<myidletime>\d+\.?\d*)" should suffice to pick out the idletime field. Together with MV_ADD it should produce as many values as there are idletimes in the event.

[smolcj]

extraction is almost the same as in the other answer...
Thread\sEvent\d.[\r\n]\s+Idletime:(?\d+.\d{2})
modified transforms.conf with
[get_Busytime]
REGEX = (?im) Thread\sEvent\d.[\r\n]\s+Idletime:(?\d+.\d{2})
MV_ADD = True
and props.conf with
REPORT-Busytime = get_Busytime
hope this post will clarify it.
thank you

[martin_mueller]

We can't help you debug your extraction without knowing the extraction.

[smolcj]

actually this link
helped.. Now my first field value is repeating twice.. can u help me to debug it..
my search | table events blabla ideletime
result is
Event1 blabla 45.56
Event1 blabla 45.56
Event4 blabla 45.58
Event1 details are repeating twice for every source file
please help
thank you

[smolcj]

Do i have to disable something to prevent ignoring multiple values from same event in splunk?

[Rob · Accepted solution]

@smolcj

Even if this is one single event and not being broken apart you should still be able to extract the idletime as a field.

Thread\sEvent\d.*[\r\n]\s+Idletime\:(?<##myIdleTime>\d+\.\d{2})

(Please remove the hashes from the line above, the formatting on answers doesn't like angle brackets.)

Splunk will look over multiple lines by default so you don't really have to set the (?m) flag.

Based on your comments you are actually looking to extract multiple fields from the multiline event you provided. With that in mind, I have taken the event log snippet and highlighted everything that based on your original question and your comments are what you wish to extract as fields:

Thread Event1 blablabla [something]....completed.

      idletime:45.56

Thread Event2 blablabla [something]....completed.no ststistics

Thread Event3 blablabla [something]....completed.no ststistics

Thread Event4 blablabla [something]....completed.no ststistics

Thread Event5 blablabla [something]....completed.

      idletime:45.56

The previous regex already extracts the idletime for you across multiple lines. Based on your regex in the comments;

(?i)Threads(?P<fieldname1>[^s]+)sblablablas[(?P<fieldname2>[^]]+)]...completed.nosstatistics

you would also like to extract another 2 fields. I am not sure if there were any typo's when you included that in your comments and the formatting rules made it a bit odd, but here is a corrected regex to match what you posted:

Thread\s(?<##fieldname1>[^\s]+).*?\[(?<##fieldname2>[^\]]+)\]\.{4}completed.no\sststistics

Lastly if you want to change that last regex to get the event number and "something" then you may want to change the above regex to:

Thread\s(?<##fieldname1>[^\s]+).*?\[(?<##fieldname2>[^\]]+)\]

[martin_mueller]

You've asked this in a comment for the other answer already, so here's the key again: http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ConfigureSplunktoparsemulti-valuefields

That way you can have multiple values in one field, not just the first value.

[smolcj]

i have multiple values for the same field in an event line... and only the first value is read by splunk.. why is it so?

[Rob]

@smolcj, perhaps I misunderstand what you mean. Splunk doesn't ignore repeating results in an event. Perhaps you can give an example of what you are getting and what you want?

[smolcj]

Sorry Rob, this is not i want. dedup is to ignore duplicate results, but what i want is the property in splunk which is the reason for IGNORING THE REPEATING RESULTS IN ONE EVENT.... i want those values in my reults, i don.t have an idea of ignoring those values.

[Rob]

@smolcj this is a bit off toppic from this question and you probably want to ask each of your questions separately so they do not get confusing.

That said, the search command you are looking for is 'dedup'

[smolcj]

what is the splunk property that ignores the multiple occurences of fieldvalues in splunk.. i think that is the reason for my issue
anybody please help
thank you

[Rob]

@martin_mueller: I did try that but it still formats the string badly. 😞

@smolcj: The regex I provided earlier would have worked in splunk to give you the first value that you were looking for. I will update my answer to reflect the regex from your comments here.

[smolcj]

yes, it is fine.. these lines comes under an even. but i have to extract it and show separately as a table..
thank you

[sruthy]

regExr is to create regular expression . from what u mentioned, the problem is with breakdown of events. did u define it precisely while creating sourcetype or uploading datainputs?

[smolcj]

the regex i extracted is working fine in http://gskinner.com/RegExr/

somebody please tell me how to extract multivalue from same line or event(in splunk language)..
another issue is enabling or disabling something in general will make my some other extractions in trouble.. so suggest something which i can use only for this particular field
thank you

[smolcj]

i am extracting "Event" and something as 2 separate fields
regex is like (?i)Thread\s(?P[^\s]+)\sblablabla\s[(?P[^]]+)]...completed.no\sstatistics
like this i am trying to extract 2 fields but only the first occurrence is extracting. shall i use TOKENIZER but i guess it is for separating same field elements.. i have already a field named event and it is extracting correctly bot not these thread fields