I tried to configure a custom datetime.xml (for my first time) as follow:
<datetime>
<define name="csdate" extract="year, month, day, hour, minute">
<text><![CDATA[[\s\S]{40}(\d{4})(\d{2})(\d{2})[\s\S]{206}(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
<use name="csdate"/>
</timePatterns>
<datePatterns>
<use name="csdate"/>
</datePatterns>
</datetime>
Regex match exactly year, mont, day, hour and minute.
In props.conf I added
DATETIME_CONFIG = /etc/system/local/datetime.xml
SHOULD_LINEMERGE = FALSE
TIME_FORMAT = %Y%m%d%H%M
Any ideas why data are not indexed with resulting timestamp?
I tried to split in 2 regex, for timePatterns and datePatterns match, but the result is still the same.
Or do you suggest a different way to achieve timestamp override at index time?
Regards
Problem here I believe was due to actual timestamps in raw event past the default (MAX_TIMESTAMP_LOOKAHEAD) 150 chars.
Problem here I believe was due to actual timestamps in raw event past the default (MAX_TIMESTAMP_LOOKAHEAD) 150 chars.
Yes, problem was MAX_TIMESTAMP_LOOKAHEAD.
Thanks for your help guys
ciao
MMM, well it works for me...
Bizza should be able to confirm
Antonio
Antonio,
I'm afraid but that's not the case either. In his/her case, date and time are splitted in the event data, so usual timeformat is more complex to manage. Unfortunately we can't use REGEX for TIME_FORMAT, otherwise that was the solution.
Marco
You don't need a custom datetime.xml
- I wouldn't do it that way. It is complicated and unnecessary.
In props.conf
all you should need is
SHOULD_LINEMERGE = FALSE
TIME_FORMAT = %Y%m%d%H%M
Assuming that your timestamp looks like
201405021209
If not, please comment with an example or two of the timestamp.
Yes, I added the --Y --m ecc only to show where timestamp fields are.
Ignore it and you'll have the original log line.
Hi in my log event and filename date is not present i want give a fix date to log so what is do ?
Did you add the --Y and --m into the event example as an clarification ?
otherwise you could try;
TIME_PREFIX = \d{numberOfDigits}\s++
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %Y%m%d
TIME_FORMAT = --Y%Y--m%m--d%d
The problem is that timestamp is splitted on every lines.
For example:
204023600511105443000 20140422--Y2014--m04--d180000000005.0600000000000041096125031ABDCE 81234567 ABDCE F & C 10024 ABDCE F & C 45399700123456789000000000.104023600582105443000 386511186636492--H15--M36PSBP
every line has 300 characters(digits), fields are position-sensitive.
I added --Y, --m, --d, --H and --M just before timestamp fields.
I believe that a custom datetime.xml is my only option.