Hi,
I've set up a central forwarder to send all our apache logs to the indexes. Here is one of the monitor sections:
[monitor:///weblogs/catalogue/]
sourcetype = access_combined
whitelist=access_log.*[0-9].gz
index = apache_blah
host = apacheblah.com
ignoreOlderThan = 31d
recursive = true
However, I'm seeing this error in splunkd.log on the forwarder:
WARN TcpOutputProc - The event is missing source information. Event :
When I look at the source field, it is reporting the log filename. This is affecting me using Traffic Ray because the host information is using the source field.
How can I get the source section saying apache_blah.com or catalogue?
Regards
Typically, you don't want to consume GZ files, as they contain LARGE amounts of data and is not RealTime. I'd suggest consuming the access_log in the raw form first, it will be continually monitored and you won't have to worry about GZ rotations.
As for the Traffic Ray App, the way the data input is crafted is not a Best Practice. However, you should be able to fix it by REMOVING the REGEX on a path for the host field. This way, your input will use the specified apacheblah.com
setting, and not something from the path.
It seems OK to me but give this alternate syntax a try:
[monitor:///weblogs/catalogue/.../access_log.*[0-9].gz]
sourcetype = access_combined
index = apache_blah
host = apacheblah.com
ignoreOlderThan = 31d
recursive = true