I am trying to format the time that is in this format: [dd/mmyyyy HH:MM:SS GMT] when I set the time_prefi to a regex that contains [ it seems to stop the data from being indexed. When remove the settings it start collecting.
I am guessing this is because a [ is regex command? Is another way to do this?
Thanks!
[
plays a special role in regular expressions, as the start of a character set notation (e.g. [a-z]
). So if you want to match the literal [
character, you need to escape it:
TIME_PREFIX = net\s\[
That, plus the comments from @skoelpin on the time format part should get your config working.
[
plays a special role in regular expressions, as the start of a character set notation (e.g. [a-z]
). So if you want to match the literal [
character, you need to escape it:
TIME_PREFIX = net\s\[
That, plus the comments from @skoelpin on the time format part should get your config working.
i had to have been missing something in my original attempt. my second attempt was what you have in there and it wasn't working. I fo have issue from time to time where a change requires me to restart splunk and not just deploy-server.
Your missing a lot more than just the TIME_PREFIX..
Before you continue using Splunk, you should really look into the Splunk education courses as this is very basic stuff
Time_prefix is an attribute of base configs which should be applied to every sourcetype on the indexers.
Time_prefix works by identifying where the timestamp is located in your logs so the TIME_FORMAT attribute can see what format the timestamp is in.
There's 3 attributes that help get the timestamp correct for your specified sourcetype. An example would look like this
03/01/2018 13:05:59 - INFO dfgdsfgdgerrfr
[sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%Y\s%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
Time_prefix shows where the sourcetype is in the logs
TIME_format shows the format of the timestamp
MAX_timestamp_lookahead identifies how far in your log Splunk should look to identify the timestamp.. Any props.conf changes on the indexer will require a splunkd restart
so this is the event:
Mar 7 11:43:30 xxxxxcxxxx.xxxx.net [07/03/2018: 16:43:29 GMT]
stanza:
TIME_PREFIX = net\s[
TIME_FORMAT %d/%m/%Y: %H:%m:%S
MAX_TIMESTAMP_LOOKAHEAD = 20
this was stopping index of these events. when I removed these from my stanza it would start again. I was not sure if '[' was causing an issue,
Thanks!
Your TIME_FORMAT is wrong. Its missing an equal sign and the minute part is wrong
It should look like this
TIME_FORMAT = %d/%m/%Y: %H:%M:%S
Your TIME_PREFIX is also wrong and should look like this
TIME_PREFIX = net\s\[
You should not remove this as your putting more overhead on the indexer
Can you provide a full data sample and your current props.conf configuration for your sourcetype?