Splunk Search

rex field extraction does not work once moved to field extraction

raynold_peterso
Path Finder

I am parsing data from a trap def as follows:

======================== Trap attributes =========================
Timestamp:           'September 19, 2017 6:56:50 AM CDT'
Agent:               '10.10.54.xxx'
Enterprise OID:      '.1.3.6.1.4.1.xxxxx'
Generic Type:        '6'
Specific Type:       '2'
Varbinds:            [oid]->[varbind]
                     '.1.3.6.1.2.1.1.1.0' --> 'dynaTrace Trap'
                     '.1.3.6.1.4.1.31094.1.1' --> 'Application Process Unavailable (unexpected)'
                     '.1.3.6.1.4.1.31094.1.2' --> 'Agent 'OpenPlatform-PRO-service-kyc-validation@ip-10-13-12-248' connection lost'
                     '.1.3.6.1.4.1.31094.1.3' --> 'Connection to a previously connected Application Process/Agent has been lost and agent has not been able to disconnect..'
                     '.1.3.6.1.4.1.31094.1.4' --> 'Error'
                     '.1.3.6.1.4.1.31094.1.5' --> 'b7250936-8068-41e3-892a-e0bec55xxxxx'
                     '.1.3.6.1.4.1.31094.1.6' --> 'albdynaserxxx'
                     '.1.3.6.1.4.1.31094.1.7' --> 'Monitoring'
                     '.1.3.6.1.4.1.31094.1.8' --> '2017091906xxxx'
                     '.1.3.6.1.4.1.31094.1.9' --> '2017091906xxxx'
                     '.1.3.6.1.4.1.31094.1.10' --> '6s'
                     '.1.3.6.1.4.1.31094.1.11' --> '-'
                     '.1.3.6.1.4.1.31094.1.12' --> '-'
                     '.1.3.6.1.4.1.31094.1.13.1' --> 'Immediate'
                     '.1.3.6.1.4.1.31094.1.13.2' --> '0'
                     '.1.3.6.1.4.1.31094.1.13.3' --> '0'
                     '.1.3.6.1.4.1.31094.1.13.4' --> '60000'
                     '.1.3.6.1.6.3.18.1.3.0' --> '10.10.54.182'

My search and rex is defined like:

index=\* sourcetype=InCharge-Traps OID=".1.3.6.1.4.1.31094"  source!="D:\\InCharge\\SAM\\smarts\\local\\logs\\TRAP-INCHARGE-OI_en_US_UTF-8.log" |  rex "'.1.3.6.1.4.1.31094.1.2' --> '.*['\(](?P<Agentname>.*)(' |\))"

which produces my field Agentname=OpenPlatform-PRO-service-kyc-validation@ip-10-13-12-248 as it should.

Now I move it to the Field extractor, writing my own Regular Expression, and enter '.1.3.6.1.4.1.31094.1.2' --> '.*['\(](?P<Agentname>.*)(' |\)) as my regex. This is where it all falls apart.

The preview looks right and shows the correct Agentnames but when I save it and look at the new extracted field, the data is all incorrect.

My props.conf looks like this:

EXTRACT-Agentname = '.1.3.6.1.4.1.31094.1.2' --> '.*['\(](?P<Agentname>.*)(' |\))

What in the wild world of sports am I doing wrong?

Thanks for the help in advance,
Rcp

0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

I'm not sure all the characters you wanted to have come through in your question made it, but I don't see how the above could result in what you want, so I'm just going to make a stab in the dark at what should be there and give you my answer accordingly. For future reference, if you use the 101010 formatting button for the text that you have to have exact (like code with angle brackets), then it will usually format it properly. And you can go back and edit your Question if you feel that I'm not giving you useful information in this response. 🙂

Try:

EXTRACT-Agentname = '.1.3.6.1.4.1.31094.1.2' --> 'Agent '(?P<Agentname>[^']+)'

It should do what you want.

View solution in original post

0 Karma

raynold_peterso
Path Finder

I looked at my posts and realized that I have not defined what I am trying to achieve.

Each line contains and agent name like JIRA_PROD_Node01@ip-10-14-118-95. The agent names always are separated by the "@" sign. So what I am trying to extract would be like this:

P2-OFX-PFM-P201[855p2DMCell01-PFM_IL_Node01]@ualbtbirdwasp201
JIRA_PROD_Node01@ip-10-14-118-95
Saber_preproduction_node_10.14.6.231@ip-10-14-6-231.ec2.internal
JIRA_PROD_Node01@ip-10-14-118-95

Your last solution, we are almost there. I can parse/capture the first three but as you said the last is elusive.

Rcp

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I'm not sure all the characters you wanted to have come through in your question made it, but I don't see how the above could result in what you want, so I'm just going to make a stab in the dark at what should be there and give you my answer accordingly. For future reference, if you use the 101010 formatting button for the text that you have to have exact (like code with angle brackets), then it will usually format it properly. And you can go back and edit your Question if you feel that I'm not giving you useful information in this response. 🙂

Try:

EXTRACT-Agentname = '.1.3.6.1.4.1.31094.1.2' --> 'Agent '(?P<Agentname>[^']+)'

It should do what you want.

0 Karma

raynold_peterso
Path Finder

Hey, I think I broke the code!!!!!

I used the following in my props.conf file and it appears to be working as expected.

EXTRACT-AgentName = .1.3.6.1.4.1.31094.1.2' --> '(Agent '|.*\()?(?P<Agentname>[^']+)(\) |' )

Look over this an let me know if I am in the ballpark. I think I'm good though.

Thanks again for your help,
Rcp

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Ah, I see that you wanted only a part of the 4th example. Yes, that looks good. If it works, don't fix it.

Be sure to mark an accepted answer so that people will know it you have a good answer. You can change your comment to an answer and accept that one. Then you get a badge for writing and accepted answer for your own question.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I have an updated version that should work the way you want it to if you wanted just the inner quoted part from the first three examples, but the whole quoted part from the 4th example. So here:

EXTRACT-Agentname = '.1.3.6.1.4.1.31094.1.2' --> '(Agent ')?(?P<Agentname>[^']+)'

See if that does what you want, and if not, let's refine that more.

0 Karma

raynold_peterso
Path Finder

We are almost there. Your solution gives me what I want for the most part but we are still falling short.

I reviewed my posts and realized I have not defined what I am attempting to parse/capture/extract.

Each line above, I am attempting to pull out the agent name from each line. The agent name contains a "@" sign. So I am looking for the following:

P2-OFX-PFM-P201[855p2DMCell01-PFM_IL_Node01]@ualbtbirdwasp201
JIRA_PROD_Node01@ip-10-14-118-95
Saber_preproduction_node_10.14.6.231@ip-10-14-6-231.ec2.internal
JIRA_PROD_Node01@ip-10-14-118-95

The first three are working as expected but the third is more elusive.

Thanks for you help so far,
Rcp

0 Karma

raynold_peterso
Path Finder

cpetterborg,

Thanks for the quick update. Yeah, your idea was my first extract but was to constrictive. I had to expand it for other data which I didn't include in my post. I will include the other data below for you to review.

Also, I have update the post and marked the sections as "Code Sample" to make it easier to read. Your were right, some data was stripped from the post.

As promised the other data sets which I need to extract.

'.1.3.6.1.4.1.31094.1.2' --> 'Agent 'P2-OFX-PFM-P201[855p2DMCell01-PFM_IL_Node01]@ualbtbirdwasp201' connection lost'
'.1.3.6.1.4.1.31094.1.2' --> 'Agent 'JIRA_PROD_Node01@ip-10-14-118-95' connection lost'
'.1.3.6.1.4.1.31094.1.2' --> 'Agent 'Saber_preproduction_node_10.14.6.231@ip-10-14-6-231.ec2.internal' connection lost'
'.1.3.6.1.4.1.31094.1.2' --> 'Tomcat/Thread Pool Current Threads Busy: _m Thread Pool Current Threads Busy - JIRA_INCID <all-applications> (JIRA_PROD_Node01@ip-10-14-118-95) upper bound exceeded'

Thanks for any help,

Rcp

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...