I'm having issues getting this to work. I have played around with the props.conf but can't seem to get this going. pass an argument to props.conf truncate=0 .
Have tried several configuration attempts. Latest one is to change linemerge = false to linemerge = true .
After trying these it still chops off the event. Any ideas?
For the multi-line events you need to configure the linebreaking.
For the best performance use SHOULD_LINEMERGE = false & LINE_BREAKER in props.conf
See http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents
When left to its own devices Splunk and SHOULD_LINEMERGE = true, Splunk will attempt to break on datestamps.
When using LINE_BREAKER there needs to be a capturing group in the regex - eg ([\r\n]+)
the default is any number of new lines or carriage returns. That denotes the end of the event and the start of a new one - the captured data is removed.
For you, something like:
props.conf
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+).*(?:\[\d{4}-\d{1,2}-\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2},\d*\])
TRUNCATE = 0
That should break on new lines that contain the date/timestamp in the square brackets.
*note (?:xxxx)
is a non-capturing regex group, that data is not removed.
I tried adding that stanza and it still isn't working. Here is the results from the search.
Splunk search criteria: source=udp:9514 host=broker-p* "Broker Registration Info"
broker-p3.vsp.com app=BrokerApp [2014-11-25 13:58:38,636] INFO {abcdZw7rGW9P_gAWe8ONu} AppointmentCompleteAction.execute: Broker Registration Info: com.vsp.broker.model.AppointmentFormInfo@7ce1034f[licenses={0=com.vsp.broker.model.BrokerLicense@1e42d36e[state=MD,licenseNum=164213,effectiveDate=com.vsp.portal.util.Chrono@3546ea47[day=1,month=6,year=2014,value=,format=MMMM|dd#yyyy]], 1=com.vsp.broker.model.BrokerLicense@f75170b[state=PA,licenseNum=330247,effectiveDate=com.vsp.portal.util.Chrono@503bdb1a[day=3,month=3,year=2004,value=,format=MMMM|dd#yyyy]], 2=com.vsp.broker.model.BrokerLicense@abdcf8b[state=WV,licenseNum=6836793,effectiveDate=com.vsp.portal.util.Chrono@3384b42f[day=2,month=3,year=2002,value=,format=MMMM|dd#yyyy]], 3=com.vsp.broker.model.BrokerLicense@1aa29b40[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@297e9469[day=,month=,year=,value=,format=MM...[Mag: Next line continuation is missing]
Send an example of the data (not too many lines, but representative data) and the contents of the props.conf file for this sourcetype.
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,143] INFO {abcpnhFLDq4THhWxDqVKu} LogInterceptor.before: Invoking appointment-complete
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,149] INFO {abcpnhFLDq4THhWxDqVKu}AppointmentCompleteAction.execute: Broker Registration Info:com.vsp.broker.model.AppointmentFormInfo@61bf56bd[licenses={0=com.vsp.broker.model.BrokerLicense@439a95a[state=MS,licenseNum=10265706,effectiveDate=com.vsp.portal.util.Chrono@659d55e0[day=1,month=10,year=2012,value=,format=MMMM|dd#yyyy]],1=com.vsp.broker.model.BrokerLicense@384bc948[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@778bb2d7[day=,month=,year=,value=,format=MMMM|dd#yyyy]],2=com.vsp.broker.model.BrokerLicense@824cf37[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@28a0280e[day=,month=,year=,value=,format=MMMM|dd#yyyy]],3=com.vsp.broker.model.BrokerLicense@3caf42c7[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@164935f1[day=,month=,year=,value=,fo...
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,317] INFO {abcpnhFLDq4THhWxDqVKu} MailBlock.doAfterBody: Sending 'VSP Resource Center Registration' to jennleebush@aol.com
that's the sample of data.
From this data you have three events, each of which is one line.
Are you seeing the second line itself truncated to be shorter?
Do you want to have a multi-line event, or do you want to have one line split into more than one event?
-sh-4.1$ sudo more props.conf
[datapower]
NO_BINARY_CHECK = 1
pulldown_type = 1
[PMIServlet]
NO_BINARY_CHECK = 1
pulldown_type = 1
[host::SCHQVVCACDEM1*]
TRANSFORMS-anonymizer = password-anonymizer
[host::broker-*]
TRANSFORMS-index = ClientRedirect
TRUNCATE=0
[host::client-*]
TRANSFORMS-index = ClientRedirect
TRUNCATE=0
[host::pt*]
TRANSFORMS-index = TrueFarmRedirect
[host::st*]
TRANSFORMS-index = TrueFarmRedirect
[host::member-*]
TRANSFORMS-index = MemberRedirect
[host::doctor-*]
TRANSFORMS-index = DoctorRedirect
[host::www-*]
TRANSFORMS-index = GlobalRedirect
[host::sa-portals-*]
TRANSFORMS-index = StrategicRedirect
[source::udp:8514]
TRANSFORMS-ClientHostOverride = ClientHostOverride
SHOULD_LINEMERGE = true
TRANSFORMS-ClientRawOverride = ClientRawOverride
TRANSFORMS-ClientShRawOverride = ClientShRawOverride
TRANSFORMS-ClientShortOverride = ClientShortOverride
TRANSFORMS-ClientTempOverride = ClientTempOverride
[source::udp:9514]
SHOULD_LINEMERGE = false
TRANSFORMS-BrokerHostOverride = BrokerHostOverride
TRANSFORMS-BrokerRawOverride = BrokerRawOverride
TRANSFORMS-BrokerShRawOverride = BrokerShRawOverride
TRANSFORMS-BrokerShortOverride = BrokerShortOverride
TRANSFORMS-BrokerTempOverride = BrokerTempOverride
shandman gravatar image
Answer by shandman
53 minutes ago
broker-p3.vsp.com app=BrokerApp [2014-10-20 13:38:14,149] INFO {abcpnhFLDq4THhWxDqVKu} AppointmentCompleteAction.execute: Broker Registration Info: LOG STARTINGcom.vsp.broker.model.AppointmentFormInfo@61bf56bd[licenses={0=com.vsp.broker.model.BrokerLicense@439a95a[state=MS,licenseNum=10265706,effectiveDate=com.vsp.portal.util.Chrono@659d55e0[day=1,month=10,year=2012,value=,format=MMMM|dd#yyyy]], 1=com.vsp.broker.model.BrokerLicense@384bc948[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@778bb2d7[day=,month=,year=,value=,format=MMMM|dd#yyyy]], 2=com.vsp.broker.model.BrokerLicense@824cf37[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@28a0280e[day=,month=,year=,value=,format=MMMM|dd#yyyy]], 3=com.vsp.broker.model.BrokerLicense@3caf42c7[state=,licenseNum=,effectiveDate=com.vsp.portal.util.Chrono@164935f1[day=,month=,year=,value=,fo... LINE ENDING and NEXT LINE TRUNCATED
I don't want this truncated. I want a continuation of this event. Does that make sense?