Multi line log issue

[castle1126]

I have a Windows system with 4.1.5 forwarding to my Splunk indexer, that puts out logs in this format:

begin error

lines of interesting log entries

end error

I've been noodling around with different options within the PROPS.CONF on the Forwarder system. So far no luck. My goal is to be have the forwarder sear the data correctly then transfer to the Indexing server.

Any tips or ideas I'm missing?

[southeringtonp · Accepted solution]

Are these Windows Event Log events or text-based? What do you want the indexed events to look like?

If all events follow the format you describe, then it should be enough to do:

#props.conf
[yoursourcetype]
LINE_BREAKER=([\r\n]+)(### begin error)

Not sure if the hash marks would need to be escaped.

[castle1126]

Hey SoutheringtonP,

I changed the client from running LightForwarder to Forwarder. Your change to the props.conf worked perfect! Thanks for the insight!!

[castle1126]

Sorry for the posting glitch, my bad with the # versus = sign. In this case the forwarder being used is the light forwarder. Thanks for the tip, I'll make the entry in the indexer's props.conf and see how that works.

[southeringtonp]

Equals signs would not need to be escaped. In your original question, you had hashes, which might have been interpreted as the beginning of a comment in the config file. If you are using a lightweight forwarder, then do this at the indexer. For heavy-weight forwarders, do it at the forwarder.

[castle1126]

A question - in my setup the forwarder system has the props.conf with the LINE_BREAKER entry in it. Should this props.conf be moved to the indexing server or left on the forwarding system?

[castle1126]

The logs I'm going after are not Event logs, they're output from a custom program. I tried as you noted and no luck. The first event shows the header line (===== begin error =====) and the next line from the file (ASP error on page: http://server/page.asp).
The next event begins with the 3rd line of text (At 10/25/2010 3:55:09pm) and shows the remaining lines of text up to the last line (===== end error =====).

By the way, I tested the = sign in REGEX Buddy and it recognized the = sign as a character.

Any other ideas?

[ftk]

So do you want multiline events or not? Can't quite tell from your question. Can you please clarify?