Hi, I have a field called UserID appearing in my searches that in two of my sourcetypes within the same index. Ive scoured the GUI manager looking for UserID and it is nowhere to be found, i have checked every props.conf file i can find on the box itself and i dont see that field extraction anywhere. I have tried restarting the server but the field keeps popping up. I want very desperately to edit that regex to make it work properly but I cant find it to edit it. Can anybody help? Is there a secret place where field type regexes are kept in splunk? ... View more

Hi, I recently had to move my hot/warm buckets for my splunk indexes to a new linux device on the same machine. I use auto scaling on all my buckets I used rsync -azv to copy over the hot/warm buckets from one directory to another so it looked like this: rsync -azv /var/lib/splunk/index1/db /var2/lib/splunk/index1/ i made sure to change the ownership with chown -R splunk for the directory I turned off splunk i ran rsync on all the folders for the indexes i was moving updated splunk-launch.conf to change $SPLUNKDB from /var/lib/splunk to /var2/lib/splunk then i updated the index.conf files for etc/apps/search/local and for etc/system/local to $SPLUNK/index* (for each indexes) When i started splunk up it immediately disabled all the indexes except _internal and one of the indexes that i had moved first as a test. The only way i could get splunk to work is to edit the indexes.conf file and have it make the home buckets in a new directory. However the indexes in this new directory now have new hot buckets and now i can only search for events after the switch. How do i reconcile these buckets. I want to get all the data from before the switch to be in the same folder. How do i do this without causing bucket naming collisions? ... View more

Hi, I am using a query that uses the awesome percentage value feature built into stats. It outputs into a table that my coworkers get in a report. They love the report but they hate the formatting. This has lead me to two issues: 1) I have managed to figure out how to round most of the values into easily readable formats, but when I try and round the percentage outputs using feildformat they come back blank. Below is an example where I have tried a feildformat on 99% but not on 95%. 99% disappears There is no need for that decimal point at all, how can i get rid of it while keeping the output? Note: the values are for a duration and are indeed all numerical. 95%Ms 99%Ms 2000.0000 2) Looking at the same example above my coworkers really want commas in the larger numbers, when i run this query for anything more than a few hours the total cost is in the billions. So i tried using the tostring(bytes, "commas") strategy to break up the numbers, but this ruins the table because I am no longer able to sort by total cost. Splunk seems to only want to account for the values before the first comma so 600,000 will be sorted right below 601,987,543,123. Here is the search i use: eventtype="Event" | stats min(xtime) as Min, avg(xtime) as AvgDuration, median(xtime) as Median, p95(xtime) as 95%Ms, p99(xtime) as 99%Ms, max(xtime) as Max, count(xmethod) as NumCalls , sum(xtime) as TotalCost by xmethod | eval TotalCost = (NumCalls * AvgDuration) | fieldformat Min=round(Min) | fieldformat AvgDuration=round(AvgDuration) | fieldformat 99%Ms=round(99%Ms) | fieldformat Median=round(Median) | fieldformat Max=round(Max) | fieldformat TotalCost=round(TotalCost) | sort -TotalCost | head 20 Thank you so much for all your help doing my job for me! Splunk is the best! ... View more

Hi, I have been running a stats query for months on a very basic search to great success. I recently had to change how the field extractions that I pull from the logs look. To do this I used the manager and deleted the old extraction and created a new one with the exact same name. Since then all the other stats functions work fine, but average comes up blank. Average works fine for other event types and searches. I have tried restarting the box and clearing my browser cache. Did i irrevocably destroy averages for this sourcetype? This is the search i use sourcetype="Example" | stats min(example_time), max(example_time), count(example_method), avg(example_time) by example_method ... View more

This may not be possible but I work at a SAAS company and we want to start evaluating which of our web methods that are chewing up our web resources on a consistent basis. We have built an event type (_time) that spits out the duration in ms of each method call (_method) for every call. So what we would like to do is measure the overall "cost" in time duration of each call by multiplying the average time per call by the count of calls in a specific time period. Is there a way to generate a table in splunk that shows avg duration, count of method calls, and cost (avgduration X count)? In other words is there a way to multiply the values in two different columns by each other to get a third column I guess the query would look something like this: index="webmethod" eventtype="duration" | stats avg(_time), count(_time), (avg(_time) * count(_time) by Method Thanks ... View more