I have an input from app - WEB Input
It extracts last 5 events from webpage every 1 minute. however instead of spitting them into 5 Splunk sees it as 1 event :
განცხადებების სტატუსების ბოლო 5 ცვლილება მიმდინარეობს ხელშეკრულების მომზადება 07.02.2018 16:01 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 არ შედგა 07.02.2018 16:01 NAT180001544 შემსყიდველი: ახალციხის მუნიციპალიტეტის მერია კატეგორია: 50100000 გამარჯვებული გამოვლენილია 07.02.2018 16:00 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 შერჩევა/შეფასება 07.02.2018 16:00 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 წინადადებების მიღება დასრულებულია 07.02.2018 16:00 NAT180001544 შემსყიდველი: ახალციხის მუნიციპალიტეტის მერია კატეგორია: 50100000"
every event starts with date and ends with space followed by 8 consecutive numbers.
I tried to use SHOULD_MERGE and MUST_BREAK AFTER \s\d{8}
Also tried BREAK_ONLY_BEFORE \d{2}[.]\d{2}[.]\d{4} \d{2}:\d{2}
However getting no results.
Have a look at Configure event line breaking.
Event line breaking consists of two steps: 1st) line breaking and 2nd) line merging.
Line breaking is mostly configured by LINE_BREAKER. By default, LINE_BREAKER is the newline character.
Line merging is configured by SHOULD_LINEMERGE=true and a couple of other options like BREAK_ONLY_BEFORE_DATE.
You should be concerned with line breaking. I suggest you try something like this:
LINE_BREAKER = ( \d{8})
Have a look at Configure event line breaking.
Event line breaking consists of two steps: 1st) line breaking and 2nd) line merging.
Line breaking is mostly configured by LINE_BREAKER. By default, LINE_BREAKER is the newline character.
Line merging is configured by SHOULD_LINEMERGE=true and a couple of other options like BREAK_ONLY_BEFORE_DATE.
You should be concerned with line breaking. I suggest you try something like this:
LINE_BREAKER = ( \d{8})
Tried that too. Still no result.
Are you running a single instance of Splunk? Or do you have multiple insances? This configuration (via props.conf) needs to be placed on the instance where the indexing phase happens. That could be a heavy forwarder.
Set SHOULD_LINEMERGE=false (along with the LINE_BREAKER option) and see if that makes a difference.
It is a single instance yes and props.conf needs to be placed in application's local folder, since its the application that takes data from website.
Try it like this:
[yoursourcetype]
LINE_BREAKER = ( )\d\d\.\d\d\.\d\d\d\d \d\d:\d\d
As you can see it should work: screenshot
(I autotranslated your input file.)
If it still does not work, can you post your props.conf? I would like to see the relevant stanza.
Also, don't forget to restart Splunk after editing configuration files.
worked. thanks a lot!