Hi all,
I have a log file with many rows. However, I tried to change the settings in /etc/system/local/props.conf
My props.conf is below.
[sap_availability]
MAX_EVENTS = 10000
TRUNCATE = 0
However, I'm still seeing only 234 lines out of the many lines. How do I allow Splunk to index all the lines in the file? I've tried to search the forum for answers and these are settings I've added to props.conf but to no avail.
Thanks all for the help in advance!
Regards
MAX_EVENTS
refers to the maximum number of lines in a multi-line event, not the number of lines in the log file. TRUNCATE
refers to at what point Splunk should cut off individual lines.
Many times, problems like this is related to improper detection of timestamps (which in turn affects the breaking of the log file into individual events). The default behaviour is to break the file when a timestamp is detected.
Consider using the following settings in props.conf
SHOULD_LINEMERGE = false
LINE_BREAKER = some regex for when to break between events
TIME_FORMAT = strptime formatting
TIME_PREFIX = string that precedes the timestamp
MAX_TIMESTAMP_LOOKAHEAD = number, indicating how far into an event Splunk will look for the timestamp.
Read more in the docs for props.conf
and timestamp recognition in general
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps
Posting some sample events may lead to a more qualified answer.
Hope this helps,
K
MAX_EVENTS
refers to the maximum number of lines in a multi-line event, not the number of lines in the log file. TRUNCATE
refers to at what point Splunk should cut off individual lines.
Many times, problems like this is related to improper detection of timestamps (which in turn affects the breaking of the log file into individual events). The default behaviour is to break the file when a timestamp is detected.
Consider using the following settings in props.conf
SHOULD_LINEMERGE = false
LINE_BREAKER = some regex for when to break between events
TIME_FORMAT = strptime formatting
TIME_PREFIX = string that precedes the timestamp
MAX_TIMESTAMP_LOOKAHEAD = number, indicating how far into an event Splunk will look for the timestamp.
Read more in the docs for props.conf
and timestamp recognition in general
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps
Posting some sample events may lead to a more qualified answer.
Hope this helps,
K