I have a simple dashboard with a few single value visualizations.  Clicking on then will open a panel below to reveal a table related to that single value using depends=$token$.  These single value metrics are also tied to scheduled searches that send out email alerts when issues arise.  What I need to figure out is how to format the "link" in the email that will open this dashboard (this part is easy), but will ALSO open the table below the single value metrics.   Here is the current drilldown for a single value. <drilldown> <set token="idx"></set> </drilldown> Here is how I am using it in the panel to be opened. <row id="panel4" depends="$idx$">   I have tried changing the drilldown to this. <drilldown> <set token="idx">open</set> </drilldown>   and tried using this URL to link to the dashboard and ALSO open the panel below.  But it does not work. https://fqdn/en-US/app/appname/dashboardname?idx=open   ... View more

I recently migrated a clustered index.  We wanted to rename the index.  I created the new index as your normally would via the CM.  Put the cluster in maintenance mode.  Stop any ingest into the "old" index and merely copied all the contents of the "old" index into the "new" index on all 6 of our indexers.  Took the cluster out of maintenance mode and did a rolling restart.  Everything worked fine except when I count the events in both indexes for ALL TIME, the old index is ~40 million events and the new index is ~111 million events.  We have a SF & RF of 3.  My thoughts are that its something with the RF of 3 however the math does not really workout to be 3x.   ... View more

I am using two macros in a search however, I want to use them in a way that IF they are broken or not available the search will not fail to complete?   ... View more

I recently added a new SH to our SHC.  Show shcluster-status is good, show kvstore-status is good.  I created some kvstore entries on SH1 and they replicated to SH5 (the new one).  I also see the results of scheduled searches that are writting to the kvstore(s) being replicated to the new SH.   I see no errors in splunkd.log.  However, when ever the DMC tries to access /services/server/introspection/kvstore/collectionstats for the "KV Store: Instance" panel, it errors with  "The Rest request on the endpoint URI /services/server/introspection/kvstore/collectionstats?count=0 returned HTTP 'status not OK': code=500 internal server error'" I can hit the other /kvstore APIs just fine, .../kvstore/replicasetstats and .../kvstore/serverstats. If i run "| rest splunk_server=local /services/server/introspection/kvstore/collectionstats" on the new search head in a search bar I get the same error. This is version 8.2.0 and the kvstore is using the wiredTiger storage engine. Any suggestions?   TIA ... View more

We built a custom app and deployed it in splunk.  It is writing logs in splunk/var/log/appname/appname.log.  I would like to find a way handle the logs natively within splunk, possibly using log.cfg to roll the log at a certain size as well as retention.  I added a stanza to log.cfg in an attempt to manage this log but splunk doesnt appear to care about the added configs.  Anyone use this file in such a way?  Is it even possible?  Below is a snippet of the config I added to log.cfg for the app.     appender.appname=RollingFileAppender appender.appname.fileName=${SPLUNK_HOME}/var/log/appname/appname.log appender.appname.maxFileSize=25000000 # default: 25MB (specified in bytes). appender.appname.maxBackupIndex=5 appender.appname.layout=PatternLayout appender.appname.layout.ConversionPattern=%d{%m-%d-%Y %H:%M:%S.%l %z} %-5p %c - %m%n ... View more