Event breaking not working on Tomcat Catalina data

[cpetterborg]

I have some Tomcat Catalina data and I can't for the life of me figure out why it isn't line breaking properly. There are several different formats for the data going into the catalina.out files, so I have a rather complicated way to have to do the props.conf file. Here is a sample of the data that made it into a single event, but it needs to be broken up into individual events for each line (in this format):

12:50:59.524 [pool-4-thread-8046] DEBUG o.s.jdbc.core.JdbcTemplate - Executing SQL query [SELECT name, MAX(version) AS version FROM (SELECT sm.name, version, direction, crt_ts, ROW_NUMBER () OVER (PARTITION BY name, version ORDER BY crt_ts DESC) AS rn FROM schema_migrations sm) WHERE rn = 1 AND direction = 'FORWARD' GROUP BY name] 
12:50:59.524 [pool-4-thread-8046] DEBUG o.s.jdbc.datasource.DataSourceUtils - Fetching JDBC Connection from DataSource 
12:50:59.527 [pool-4-thread-8046] DEBUG o.s.jdbc.core.BeanPropertyRowMapper - Mapping column 'NAME' to property 'name' of type class java.lang.String
12:50:59.528 [pool-4-thread-8046] DEBUG o.s.jdbc.core.BeanPropertyRowMapper - Mapping column 'VERSION' to property 'version' of type class java.lang.Integer 
12:50:59.528 [pool-4-thread-8046] DEBUG o.s.jdbc.datasource.DataSourceUtils - Returning JDBC Connection to DataSource

So there is no date, only a time in this case. The props.conf looks like this:

[tomcat_catalina]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
LINE_BREAKER=^(?:(?:\d{1,2}:\d\d:\d\d.\d\d\d\s+\[pool-)|(?:\[\w{1,7}\]\s\d\d\/\d\d\/\d\d\s)|(\w{3}\s+\d\d,\s\d{4}\s\d{1,2}:\d\d:\d\d\s[AP]M))
MAX_TIMESTAMP_LOOKAHEAD=200
TRUNCATE=999999
DATETIME_CONFIG = /etc/slave-apps/all_tomcat_props/local/datetime.xml
TIME_PREFIX = ^(\[\w{1,7}\]\s)?

The datetime.xml file looks like this:

<datetime>
<define name="_mydatetimeformat3" extract="hour, minute, second, subsecond">
     <text>(\d{1,2}):(\d{2}):(\d{2}).(\d{3})</text>
</define>
<define name="_mydatetimeformat2" extract="year, month, day, hour, minute, second, subsecond">
    <text>(\d\d)/(\d\d)/(\d\d)\s+(\d\d):(\d\d):(\d\d)[:,.](\d\d\d)</text>
</define>
<define name="_mydatetimeformat1" extract="litmonth, day, year, hour, minute, second, ampm">
    <text>(\w{3})\s+(\d{1,2}),\s+(\d{4})\s+(\d{1,2}):(\d{2}):(\d{2})\s+(\w{2})</text>
</define>
<timePatterns>
     <use name="_mydatetimeformat3"/>
     <use name="_mydatetimeformat2"/>
     <use name="_mydatetimeformat1"/>
</timePatterns>
<datePatterns>
     <use name="_mydatetimeformat3"/>
     <use name="_mydatetimeformat2"/>
     <use name="_mydatetimeformat1"/>
</datePatterns>
</datetime>

The LINE_BREAKER config parameter should be getting the event to break. The first option in the pattern is the one that matches the data example, and it works fine in regex101.com, so I don't think that is the problem. The other formats work just fine. The timestamp pattern ( _mydatetimeformat3 ) is getting the timestamp parsed properly, too.

So how do I get the event breaking to work properly in this case

[richgalloway · Accepted solution]

Remember that LINE_BREAKER throws away the matching capture group so don't include in the group anything you need in the event (like time).

[cpetterborg]

I changed it to BREAK_ONLY_BEFORE and it worked. Stupid mistake. I forgot about that little detail.

Thanks Rich!!!