Splunk Search

problem with field extraction

a212830
Champion

Hi,

I'm trying to extract a field from a source, and when I test it, it appears to work, but in practice, it's grabbing too much.

Two examples are below. I want to create a "trapType" field, and report on it. It should contain "Vendor Specific" or "Authentication Failure". When I test it, everything looks ok, but then when I run a search, it's including everything after the values.

The field extract is: (?i) Type: (?P.+)

10:43:20.000 AM

10:43:20.000 AM

Trap: 25311425
Thu Feb 9 10:43:20 2012
Src IP: 6.7.8.9
Agent IP: 6.7.8.9
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:6.7.8.9

Trap: 25311426
Thu Feb 9 10:43:20 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.3175321 Value:PIM
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.3175321 Value:5
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.3175321 Value:INVALID_SRC_REG
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.3175321 Value:Received Register from 1.2.3.4 for (1.2.3.4, 1.2.3.4), not willing to be RP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.3175321 Value:264d 04:40:27

Tags (1)
0 Karma

a212830
Champion

Not sure what a pastebin is, but I did a copy/paste of some of the log data.

Trap: 24910058
Thu Feb 9 02:01:11 2012
Src IP: 12.3.4.5
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.43.2
Object:1.3.6.1.4.1.9.9.43.1.1.6.1.3.3997 Value:1
Object:1.3.6.1.4.1.9.9.43.1.1.6.1.4.3997 Value:4
Object:1.3.6.1.4.1.9.9.43.1.1.6.1.5.3997 Value:6

Trap: 24910059
Thu Feb 9 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.0 Value:PAGP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.0 Value:6
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.0 Value:PAGP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.0 Value:2012 Feb 09 02:01:11 eastern -05:00 %PAGP-5-PORTTOSTP:Port 2/17 joined bridge port 2/17
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.0 Value:279122402

Trap: 24910060
Thu Feb 9 02:01:12 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.61328708 Value:PIM
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.61328708 Value:5
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.61328708 Value:INVALID_SRC_REG
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.61328708 Value:Received Register from 1.2.3.4 for (1.2.3.4, 2.3.4.5), not willing to be RP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.61328708 Value:468d 06:45:54

0 Karma

clintsharp
Explorer

Thanks, I now have the data, but I'm curious, can you provide what your inputs.conf and props.conf entries for this file and sourcetype look like? Without some changes, Splunk splits this data into multiple events etc, so I'm curious if you've already done some work to teach splunk how to interpret these events like setting LINE_BREAKER, or specifying regexes to teach splunk to break events.

BTW, your rex extraction works fine for me, so I'm still trying to dig in and reproduce your issue.

0 Karma

a212830
Champion

I tried the latest change and I'm still getting the same thing (bounced the app). I keep trying to upload the image, but it doesn't get added. Here's the text from the "pull-out" window for the trapType field. It's taking the information after the Trap Type: value, but it should stop at the end of that line, and it's not.

Authenticatio....6.1.4.1.3224 4,629 9.719%

Authenticatio....4.1.2021.251 1,440 3.023%

Authenticatio...e:10.47.89.25 1,060 2.226%

Authentication...e:10.93.65.11 788 1.654%

Authentication...4.1.564.101.1 500 1.05%

Authentication...10.165.27.145 480 1.008%

Authentication...e:10.47.89.25 444 0.932%

Authenticatio...1.8072.3.2.255 392 0.823%

Vendor Specif...1.4.1.3375.2.4 382 0.802%

Vendor Specif...1.4.1.3375.2.4

0 Karma

clintsharp
Explorer

I'll ask again, can you post a pastebin of the original log data? Something with the line breaks and everything preserved?

0 Karma

joshd
Builder

Unfortunately I cannot see the screenshot... check my updated example before, realized I forgot to add (?m) at the start to specify multiline events palmface

0 Karma

a212830
Champion

Yes, it's a multiline event. I've attached a screenshot of what happens. alt text

0 Karma

joshd
Builder

Could you be a little more clear in what you are seeing, what do you mean you are including everything after the values? if the event you are inspecting is a true multiline event then you should have no issue with using that field extraction inside your props.conf ... you could also be a little more specific like so:

EXTRACT-fieldname = (?im).*Trap\sType:\s(?<fieldname>.+)

Your regex should match everything until the end of the line, so if anything else is being picked up thats why I ask if it's a true multiline event you're inspecting.

0 Karma

clintsharp
Explorer

Can you either put up a pastebin of the input source or include in a code section so we can see the true raw input values? That would help me load it into a test instance to help diagnose the issue with your regex.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...