I'm running in to a problem getting some XML to parse in to events properly. The log has multi-line entries as you might expect and in the XML are date strings that I want Splunk to ignore. So, each log entry begins with the following time stamp (which I want to use):
[03/12/12 16:15:30.103]: <Some log data goes here> <possibly followed by a bunch of XML> [03/12/12 16:15:30.112]: <Some more log data goes here>
The entries in the log file might only a single line long or might be as long as several hundred lines of XML before the next so-called real time spamp. Part of the XML data being generated are time strings like this:
20120312161445.247Z, which I don't want.
What I want is for all of the lines between one time stamp and the next to be grouped in to a single event.
Here's what I have in my props.conf file:
TIME_PREFIX = \[ TIME_FORMAT = %m/%d/%y %H:%M:%S.%3N
However Splunk is still splitting on the other time stamps within the XML. So, what am I doing wrong?
Are you editing the correct props.conf? (i.e. where the parsing occurs).
If you have a Universal Forwarder -> Indexer OR Lightweight Forwarder -> Indexer setup, the props.conf to edit is on the Indexer.
If you are using a full/heavy forwarder, the settings should go there.
Apart from that your settings look good. You could possibly further qualify your TIME_PREFIX regex with (if tour 'real' timestamps are the actually in the beginning of the line);
How have you configured linebreaking? I'd recommend that you use
SHOULD_LINEMERGE=false LINE_BREAKER = ([\r\n]+)\[\d\d
which explicitly tells splunk to break events after a newline followed by the beginning of a 'real' timestamp.
Hope this helps. If not, please give us more of your props.conf, and perhaps some more sample events.
XML event truncated at 10k 1 Answer