Hi,
I added a feed (with some great help from this site), and it is working, except for the timestamp. Each line begins with an epoch timestamp, and I configured Splunk to recognize it, but it's picking up the time that is gets processed.
Input example:
1377468000000|139036|NormalizedMemoryInfo|Free|226005684|Memory|testhost|Enhanced-MemoryPool: Processor 1001.1
1377468000000|139036|NormalizedMemoryInfo|Utilization|29.881565745219028|Memory|testhost2|Enhanced-MemoryPool: Processor 1001.1
props.conf:
[snmpinfo]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = FALSE
NO_BINARY_CHECK = 1
TIME_PREFIX = ^
LINE_BREAKER = ([\r\n]+)
TIME_FORMAT = %s
TRANSFORMS-set = setnull,setparsing
From what I read, the %s is for epoch time. Each event should have a 5 minute interval, but it's appearing with the timestamp when it got processed.
Also, it appears that the output mechanism doesn't always output a full line when writing. Is there any way to make sure it only pulls in entire events? I've seen half events show up.
In order to make sure that Splunk only indexes full events, you can try 2 solutions.
One is to set the following value in your time_before_close in your inputs.conf:
time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in pastseconds.
* Defaults to 3.
This will tell Splunk to wait a certain amount of seconds before it assumes that the event has completed.
Another way is to use the LINE_BREAKER option in props.conf to tell Splunk what the end of a line of logging looks like. You'll have to be lucky enough that all your logfiles have a determinable ending. Remember that you can use PCRE here, so you can do a positive lookahead to the start of your next line as well to determine the ending of the previous line.
That's an epoch to milliseconds, you'll need TIME_FORMAT = %s%3N.