I can't seem to get Splunk to auto/detect our current Apache Tomcat 6.x or 7.x logs.
Please help and appreciate the support, I have tried all I can so far. New to Splunk and not yet SME with this tool ... 🙂
Log source/format (Apache Tomcat 6.x – org.apache.catalina.valves.ExtendedAccessLogValve)
<Valve className="org.apache.catalina.valves.ExtendedAccessLogValve" directory="E:\folder-Logs" pattern="date time c-ip s-ip cs-method cs-uri-stem cs-uri-query sc-status bytes time-taken cs(User-Agent) cs(Cookie) cs(Referer) cs(HOST)" prefix="${tomcat.instance.name}-" resolveHosts="false" suffix=".log"/>
Sample scrubbed http access log:
#Fields: date time c-ip s-ip cs-method cs-uri-stem cs-uri-query sc-status bytes time-taken cs(User-Agent) cs(Cookie) cs(Referer) cs(HOST)
#Version: 2.0
#Software: Apache Tomcat/6.0.32
2014-05-06 04:04:09 7x.2xx.3x.5x 10.5x.7x.6x POST /folder/ajax/get.action - 200 79782 0.890 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.54.16 (KHTML, like Gecko) Version/5.1.4 Safari/534.54.16' 'JSESSIONID=BXA; CookiesEnabled=1; Sx7xFE=1xxxx.2xxxx.0000;' 'hxxps://client1.domain.com/folder/do.action?content=mypage=1' 'client1.skillport.com'
I don't know what you mean by "autodetect", but this is the inputs.conf
you probably need
[monitor://E:\folder-Logs]
sourcetype=access_combined_extended
For props.conf
on the indexer, I would use
[access_combined_extended]
REPORT-ace=access_combined_base_fields
EXTRACT-aceExt1=\'(?<cs_User_Agent>.*?)\'.*?\'(?<cs_Cookie>.*?)\'.*?\'(?<cs_Referer>.*?)\'.*?\'(?<cs_Host>.*?)\'.
SHOULD_LINEMERGE = false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 30
And for transforms.conf
on the indexer
[access_combined_base_fields]
DELIMS = " "
FIELDS = date, time, c_ip, s_ip, cs_method, cs_uri_stem, cs_uri_query, sc_status, bytes, time_taken
Note: there shouldn't be any linebreak on the EXTRACT
line above. Or the FIELDS
line.
I just made up the sourcetype called access_combined_extended
, because your data doesn't exactly match the common Apache formats I see. And I also set a few attributes in props.conf
that you don't strictly need, but specifying them will help Splunk parse your data more efficiently.
create each of the files named above in
$SPLUNK_HOME/etc/system/local
Probably only the inputs.conf
file will already exist. But for any file that already exists, simply copy and paste the above at the end of the file.
After copying the files, then restart Splunk.
You should probably walk through the Splunk Tutorial at
http://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/WelcometotheSearchTutorial
Sorry I can't seem to figure this out, please provide me exact files/path if all possible. I have fresh 6.1 install, don't care of any existing data as we are running poc/pilot.
Appreciate the support, I am rather new to Splunk. Will give this a shot, is it possible to send me the files and I can simply copy/past? I'm assuming I simply need to modify existing files and add the info you provided?