Getting Data In

line breaking...

a212830
Champion

Hi,

I'm stumped. I've been playing with the linebreaking trying to get the format properly, and it won't work. The format is below. I want each "Trap:" to begin a new event, down to the next "Trap:" Any suggestions?

Trap: 23708419
Wed Feb 8 02:01:11 2012
Src IP: 10.216.0.26
Agent IP: 10.216.0.26
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.61290766 Value:PIM
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.61290766 Value:5
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.61290766 Value:INVALID_SRC_REG
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.61290766 Value:Received Register from XX.XX.XX.XX for (XX.XX.XX.XX, XX.XX.XX.XXX), not willing to be RP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.61290766 Value:467d 06:45:53

Trap: 23708420
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Trap: 23708421
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Tags (1)
0 Karma
1 Solution

Takajian
Builder

You will need to configure props.conf like as bellow.

[your_sourcetype]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Trap:

You can also refer to following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

View solution in original post

Takajian
Builder

You will need to configure props.conf like as bellow.

[your_sourcetype]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Trap:

You can also refer to following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

wrangler2x
Motivator

A good technique for this is to do go to Settings->Data Inputs->Add New (Files & Directories) on your indexer with a sample log file in the temp directory, say. Select Preview Data Before Indexing and then Browse for the file. Once you've got that, click Continue.

In the new screen called Data Preview, you get a pop-up asking for you to select a sourcetype from the list of known ones, or to create a new sourcetype. If you use an existing sourcetype, Splunk will use the props.conf stanza associated with that sourcetype on the indexer (if there is one), and pre-populate the settings in the Advanced Mode tab with them. Once you've done this (selected which option on sourcetype), you can see how Splunk is parsing the logs. Typically, if they are easy to parse then date and time (timestamp) in the logs will be highlighted in green. If not, you'll see a warning icon on the lines it can't figure out.

This is where this is a nice tool. You can go to the Advanced Mode (props.conf) tab and in the Additional Settings (override) block enter in your various props.conf settings you'd like to try, then Apply them. To this point, none of the things you have done affect the configuration of the indexer in any way, and you get to see the effects of the different things you try there.

0 Karma

a212830
Champion

No, you understand it, but that's not what's happening. See below from a search...

Wed Feb 8 20:53:27 2012
Src IP: 1..2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 0
Enterprise: 1.3.6.1.4.1.2620.1.5.6
Object:1.3.6.1.4.1.2620.1.5.6.0 Value:standby
Object:1.1.1.0 Value:Cluster State
Trap: 24678117 <---- this should be the start of the next trap.

0 Karma

Takajian
Builder

Did you restart splunk and reflect configuration of props.conf and clean indexed data? The configuration will reflect for new index data, not past indexed data.

0 Karma

Takajian
Builder

Are you using forwarder? Do you know where you should put props.conf in your deployment?

If you are using light weight forwarder or universal forwarder, you need to put the props.conf on index server.
If you are using other forwarder type(HF or regular forwarder), you will need to put the props.conf on forwarder, not index server. Please confirm if you put props.conf on appropriate location. You will also need to restart splunk to reflect the configuration.

0 Karma

Takajian
Builder

can you vote for me?

0 Karma

a212830
Champion

tHANKS! I appreciate it.

0 Karma

a212830
Champion

That did it! Thanks!

0 Karma

Takajian
Builder

You need to vote for me, not your self.....

0 Karma

a212830
Champion

OK. I'll try that. Thanks!

0 Karma

a212830
Champion

I'm using the universal forwarder. So, the props.conf needs to go on the index server?

0 Karma

Takajian
Builder

Yes, you need to put the props.conf on the index server.

0 Karma

a212830
Champion

Yeah, restarted the forwarder, and the results are below.

Wed Feb 8 23:10:18 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Trap: 24780942

0 Karma

a212830
Champion

Thanks. That almost works. It's putting the "Trap:" from the next event at the bottom of the previous event. The "Trap:" is the start of the event, and I want to include it. Any way to do that?

0 Karma

Takajian
Builder

"Trap:" is the start of the event. If you break before "Trap:", you will see "Trap:" is first line of the indexed event. Please let me know if I have misunderstanding.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...