Multiline field extractions

[nickhills]

I have an event which looks like this"

USERNAME            HOME_DIR           USER_INFO
root                /root              root
ec2-user            /home/ec2-user     EC2 Default User
test_user1          /home/test_user1   Testing User
test.user2          /home/test.user2   Test User 2
realuser            /home/realuser     A Real Person

I want to build a field extraction to capture each value from the 3 columns, but i cant get the extraction tool to find any more than one occurrence in any event. I presume this is because it is not attempting multiline extractions, but fiddle and try as I might, i cant get multiline (?m) extractions to work.

Can anyone point me in the correct direction?

[landen99]

Your example is very similar to the solution I posted at:

http://answers.splunk.com/answers/143107/field-extraction-from-space-aligned-fields-in-multi-line-ev...

Modify the code a little with max_match=3 and a perhaps a few tweaks in the regex. I know the answer to this question is a little late, but it could help others with similar questions.

[Ayn]

If that's what your event looks like, using multikv seems to be the perfect tool. http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Multikv

[Ayn]

Great that you got it working! If you have the time the best thing would be to write an answer to your own question detailing how you solved the problem in the end, then accept your own answer so people can see what worked.

[nickhills]

I didn't get this to work, but worked around it in another way.

Whilst I am very appreciative of your help, I don't want to mark this as answered, because (for me at least) it isn't 🙂

I may come back to this in a few weeks, so perhaps we can pick this up again.

Thanks again Ayn.

[Ayn]

Look into delimited extraction using REPORT stuff in props.conf / transforms.conf. This page has lots of info on it: http://docs.splunk.com/Documentation/Splunk/5.0.2/Knowledge/Createandmaintainsearch-timefieldextract...

In your case it would be something like:

props.conf:

[yoursourcetype]
REPORT-grabfields = grabfields

transforms.conf:

[grabfields]
DELIMS = "\t"
FIELDS = USERNAME,HOME_DIR,USER_INFO

[nickhills]

ha, you got a response in before i finished 🙂

at search time, I can run this:

sourcetype="blah"|multikv fields USERNAME HOME_DIR USER_INFO|table USERNAME HOME_DIR USER_INFO

which gives me a nicely formatted table of my events - what I ideally would want to be able to do is simply:

sourcetype=blah |table USERNAME HOME_DIR USER_INFO

[nickhills]

...sorry, I hit comment before I had finished typing...

What I'm trying to achieve is to collect each username into an extracted field, so that i can run reports like "most common username" "host with most users" "rarest username" "which hosts can x login to" etc.

Is there a way to use multikv to extract these in this way?

[Ayn]

search modifier? How? From the docs page's description of what multikv does: "Extracts field-values from table-formatted events."

Tell me more about how you're using it as a search modifier?

[nickhills]

Multikv works beautifully as a search modifier, but is there a way to actually perform a field extraction with it?