Whenever we set up a Slack alert to send to a #channel in our Slack instance (other than default for the web hook), we get a 500 in Splunk. 07-08-2016 18:51:17.286 +0000 ERROR sendmodalert - action=slack STDERR - Error sending message: HTTP Error 500: Server Error Sending the alert destination to @user works fine, and leaving blank works fine for hitting the default channel I can not find what is causing this behavior ... View more

I have a data set and search that gives me metrics such as a count or sum of field values per hour/day, etc. (simple) This particular data set would be much more useful if I could display it as a calendar rather than a timeline. Question is, do I pursue a third party js method that I have found on a blog thats a few years old, or is something like this trivial/on roadmap for the new visualization methodology for 6.4+? We are upgrading to 6.4 in a few weeks. Would it be a waste of time to try to build out via old method? Or has anyone else solved this issue? ... View more

most of my data sets to UTC, and all data sources are properly tagged to convert to UTC if they are not. My user is in PST, so when I run a search such as @d etc, it starts from midnight PST. Usually this is ok, as dashboards are built with this in mind. I have an extra use case for monitoring our splunk license usage, Splunk however runs it's license metrics based on 0-24:00 UTC. Any search I run for "last day" is going to be off by 1/3 based on this time difference. Is there a way to set a specific search to ignore the associated users search head timezone setting and run in UTC (or other specification) on that search? ... View more

I have a data set that contains start and end times of events. These events signify 'significant events' in our infrastructure. I want to be able to display this as a chart and fill in the gap between the start and end in the event. for instance the data looks like this: {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-07-10 16:30:00", "start_date": "2014-07-10 16:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-04-17 20:00:00", "start_date": "2014-04-17 16:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-04-28 17:30:00", "start_date": "2014-04-28 17:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-05-15 16:30:00", "start_date": "2014-05-15 16:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-05-26 07:00:00", "start_date": "2014-04-17 07:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-05-22 16:30:00", "start_date": "2014-05-22 16:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-05-26 19:00:00", "start_date": "2014-04-18 06:55:00", "sys_updated_on": "2014-08-14 01:15:33"}]} {"tags":[{"entry_time":"2015-11-18 15:46:35"},{"end_date": "2014-05-22 16:30:00", "start_date": "2014-05-22 16:00:00", "sys_updated_on": "2014-08-14 01:15:33"}]} I want to be able to do a timechart that will graph the entire window between start and end, not just points of start and end. ... View more

I have dashboards that show various metrics over a time window. It appears that in 6.3 the timecharting display is set to automatically scale to the window that has data, instead of the window of the search. I want ot see flatline (zero data) leading up to and out of "issues" in my timecharts, however 6.3 seems to be "smart" and automatically zoom in, if there is no data at the begingin or end of the timeseries. I'm looking through docs and UI choices, however I can not find a place to turn off this "feature". ... View more

I am trying to find any documentation on exactly which props & transforms can be set on indexer vs search time. I know a few must be set at index (or from forwarder) time such as host, index, source, sourcetype. however what other parts can/should. linebreak? timezone? I had documented from PS in the past, however can no longer find it. ... View more

I keep coming across the issue where certain scheduled searches that user may set up will keep artifacts in the dispatch directory for 24 hours, even though the retention is set to only a few minutes. These searches run every minute, sometimes for alerts, sometimes to populate a summary or dashboard, so it's not really search specific. I believe this has to do with how long to keep an artifact if there is some transient error in the search, and I believe this is why these are being kept beyond the set retention time. I can not find this setting however. ... View more

I have a need to monitor files that look like this: host one = /path/to/base/app/App1/App1.{pidnumber}.log /path/to/base/app/App2/App2.{pidnumber}.log (yes the digits are there)... however if I monitor: /path/to/base/app/App*/App*.log it's too greedy, I end up with the archive version (app rolls it's logs to: /path/to/base/app/App1.datestamp/App*.{pidnumber}.log etc. I think this could work if there was a way to put a singler character wildcard, OR a regex - unfortunatly, since I need the * later sgement, I can't seem to get this to work in regex monitor:///path/to/base/app/App\d{1}\/App*.log ... View more