I spent almost a week trying to figure out how to get Web Intelligence to display data from apache log. Today, I finally be able to get some results in pageview. I just want to share what I have found and hopefully it may help you.
All the problems so far come from missing eventtype or assigning wrong eventtype. Need to edit the /default/eventtypes.conf under the webintelligence application folder to make the app work. I tried using a /local/eventypes.conf file but for whatever reason it will not override the default/eventtypes.conf as it should.
a. A number of the search queries in the app depend on eventtype=web-traffic. The default set up for the stanza [web-traffic] is
[web-traffic]
search = sourcetype="My Access Logs"
That means only logs that has sourcetype="My Access Logs" will be assigned web-traffic eventtype!
Therefore if this stanza is not changed to (for apache log)
[web-traffic]
search = sourcetype=access_c*
you will see no results in Pageview and other predefined searches in the app that depends on web-traffic eventtype.
b. If you are testing the app on your own internal network like I was doing, need to redefine stanza [client-nonroutable], [clientip-internal] to not matching your local IP address. This is because search queries like Pageview excludes any events that are assigned with eventtype=[client-nonroutable] or eventtype=[web-traffic].
c. Check the browser section in eventtype.conf to see if any of the user agents match the user agent part in apache log. If none of them matches, add a stanza for your browser. I found that some search queries depends on eventtype=[ua-browser-*]. If it is missing, no search results.
The test apache log I used have user agents from Chrome/16.x.x and from Firefox/8.x and they fail to match a specific browser stanza.
d. Need to restart splunk everytime for any changes in the eventtypes.conf file.
The set up document needs to be updated to include editing the eventtype.conf file. It will not work out of the box by just following the set up instructions.
This is my experience. Good luck!
... View more