I have several indexes in my Splunk Instance. One of these instances is merging some of my log events into a single event incorrectly. I need help splitting them up. At the bottom of this question, I have posted an example. What I need help with is... what is the easiest way to split these events so that each event is separate? I believe that the best way is to specify the attributes in the props.conf file to change the line breaking and merging behavior so that all lines are merged ("SHOULD_LINEMERGE = true")
and then set the attribute to Break the lines at the beginning when the date is given ("BREAK_ONLY_BEFORE = ^(?P[^ ]+)")
- not sure my regex is correct.
these lines are recognized as separate events
18:48:55 Completed Teststep: successful login using email: {0} and password: {1} with arguments: [swinters@gci.com, Ch0c0late$tarfish]
18:48:56 Invoking Teststep: get {url} with arguments: []
18:48:56 Invoking Teststep: get {url} with arguments: []
18:48:56 Invoking Teststep: get {url} with arguments:
**these lines are not recognized as separate events but should break after the timestamp
18:45:59 Failure Teststep: First successful login after deploy using email: {0} and password: {1} with arguments: [swinters@gci.com, Ch0c0late$tarfish]
com.qmetry.qaf.automation.step.JavaStepReporter$JPThrowable: org.openqa.selenium.TimeoutException: Expected condition failed: Wait time out for User Menu to be present (tried for 60 second(s) with 1000 MILLISECONDS interval)
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: '52dc70ddb77f', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-123.8.1.el7.x86_64', java.version: '1.8.0_92'
Driver info: driver.version: unknown
at com.qmetry.qaf.automation.step.JavaStepReporter$MockJavaStep.doExecute(JavaStepReporter.java:144)
at com.qmetry.qaf.automation.step.BaseTestStep.execute(BaseTestStep.java:149)
at com.qmetry.qaf.automation.step.JavaStepReporter.javaTestStep(JavaStepReporter.java:88)
at hubTesting.steps.globalSteps.LoginSteps.firstLogin(LoginSteps.java:15)
at hubTesting.test.globalTests.RefreshHubUser.refreshUserMgmtUser(RefreshHubUser.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:571)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:707)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:979)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.openqa.selenium.TimeoutException: Expected condition failed: Wait time out for User Menu to be present (tried for 60 second(s) with 1000 MILLISECONDS interval)
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: '52dc70ddb77f', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-123.8.1.el7.x86_64', java.version: '1.8.0_92'
Driver info: driver.version: unknown
at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:296)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:265)
at com.qmetry.qaf.automation.ui.webdriver.QAFExtendedWebElement.waitForPresent(QAFExtendedWebElement.java:499)
at com.qmetry.qaf.automation.step.CommonStep.waitForPresent(CommonStep.java:805)
at hubTesting.steps.globalSteps.LoginSteps.firstLogin_aroundBody0(LoginSteps.java:23)
at hubTesting.steps.globalSteps.LoginSteps$AjcClosure1.run(LoginSteps.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.qmetry.qaf.automation.step.JavaStepReporter$MockJavaStep.doExecute(JavaStepReporter.java:138)
... 17 more
18:45:59 Completed Teststep: First successful login after deploy using email: {0} and password: {1} with arguments: [swinters@gci.com, Ch0c0late$tarfish]
18:46:00 Invoking Teststep: get {url} with arguments: []
18:46:00 Invoking Teststep: get {url} with arguments: []
18:46:00 Failure Teststep: First successful login after deploy using email: {0} and password: {1} with arguments: [swinters@gci.com, Ch0c0late$tarfish]
com.qmetry.qaf.automation.step.JavaStepReporter$JPThrowable: org.openqa.selenium.TimeoutException: Expected condition failed: Wait time out for User Menu to be present (tried for 60 second(s) with 1000 MILLISECONDS interval)
You don't need SHOULD_LINEMERGE, just LINE_BREAKER. This should split lines into new events only when they start with a timestamp:
props.conf:
[<sourcetype>]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{2}:\d{2}:\d{2}
Oh, and my regex assumes that the leading space on your log lines isn't actually present in the actual data, but an artifact of your copy/paste into answers.
You should apply base configs in your props.conf and rerstart the Splunk service
Try this
[sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %H:%M:%S
SHOULD_LINE_MERGE = false
LINE_BREAKER = ([\r\n]+)(\s+\d+:\d+:\d+)
MAX_TIMESTAMP_LOOKAHEAD = 15
TRUNCATE = 1000
Make sure to replace sourcetype
with your sourcetype. Also, is there a space in front of the hour? If so then the above configs will work
You don't need SHOULD_LINEMERGE, just LINE_BREAKER. This should split lines into new events only when they start with a timestamp:
props.conf:
[<sourcetype>]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{2}:\d{2}:\d{2}
Oh, and my regex assumes that the leading space on your log lines isn't actually present in the actual data, but an artifact of your copy/paste into answers.