WebIntelligence App question

[joshftx]

I have the webintelligence APP working OK. The one problem I have is that in /webintelligence/lookups/sourcenames.csv I have to specify each and every log file individually like this:

"\\webserver\c$\windows\system32\logfiles\w3svc1\ex111101.log",srvexchange
"\\webserver\c$\windows\system32\logfiles\w3svc1\ex111102.log",srvexchange

Can I use a wildcard here so that it will pick up all the log files like this:

"\\webserver\c$\windows\system32\logfiles\w3svc1\ex*.log",srvexchange

[hvandenb]

I actually modified the lookup and the macro to use sourcetype. I actually defined sourcetypes for each iis log so that I can control the extraction as iis is not quite supported by the tool.

1. Modify the sourcename_lookup macro to include the field you want to use in the lookup, e.g. in my case sourcetype

lookup sourcenames sourcetype | eval sourcename=if(sourcename==" " OR isnull(sourcename),sourcetype,sourcename)

1. Modify the the saved search Sourcenames Lookup and put in your item, e.g.

eventtype=web-traffic | stats count by sourcetype | eval sourcename=" " | inputlookup append=t sourcenames.csv | stats last(sourcename) as sourcename by sourcetype | outputlookup sourcenames.csv

Then in the file being generated I simply say sourcetype, source which greatly simplified the solution especially with rotating filesl

[watsm10]

I'm pretty sure this is possible, as I've previously used wildcards in lookup tables for other situations.

Please refer to this answer to see how it can be done.

http://splunk-base.splunk.com/answers/28566/how-to-use-wildcard-in-lookup-based-searches-and-alerts

[joshd]

I ran into this problem as well but with jboss access logs due to the logs automatically being rotated and a timestamp added to the file. I just decided to "live with it" since the webintelligence app is still in beta... but I finally got some time to sit and think about this now and I dont know how I missed such an easy solution to this... on your forwarder (presuming you're using one to move the log data to the indexer) just specify a custom source for the input. such as: source=srvexchange then you only need one entry in the sourcenames.csv

I know you lose a bit of the mapping back to the original source but really that shouldnt matter. It's been a simple life saver for me and working great for me thus!

[araitz · Accepted solution]

Unfortunately you cannot specify a wildcard in the lookup. We are working to improve the ability to manage IIS sites in Web Intelligence for a future release.

[araitz]

The search that populates sourcenames.csv, "Sourcenames Lookup", is scheduled by default to run once per day. Lookups work such that the source name must be specified for every entry in the lookup.

[joshftx]

does the lookup.csv get populated automatically after I run the saved search the first time, or does it need to be run daily. If so, can it be scheduled so we don't need to manually update it every time? Can the sourcenames be left blank if we're only using one source/name?