We get all deploy clients using 'splunk list deploy-clients' command before. Recently, we upgraded our Splunk into version 6.2.0 and find there are only 30 clients listed while executing the command. BTW, our clients' number should be 44. I did the following test. 1. run 'list deploy-clients' to count the client number and the result is 30 2. stop splunk service in one node which is listed. 3. rerun 'list deploy-clients' to count the client number and the result is 30. There is a new clients listed. BTW, I also access the content via REST API https://IP:8089/services/deployment/server/clients and found the deploy clients number is also 30. Does this command has some restrictions on the list of clients? Why doesn't this command work normally after upgrading to 6.2.0 and how can we fix this problem? ... View more

A Search Head is configured as a primary Deployment Server (DS) and has many clients connected to it polling for deployments. We have a failover situation where we want to introduce a standby DS that is configured identically to the primary DS. By just changing the resolved IP address of the name of the DeploymentServer referenced in deploymentclient.conf (we are not changing the name) the new DS will be contacted for the first time by all the clients. I suspect this contacting this new DS will result in all the apps and configs in etc/deployment-apps being downloaded again, presumably because the clients don’t have any deployment bundle information or signatures for the new DS. The assumption being that the deployment bundles carry some sort of host signature. The question is that once contacted does the new DS send its entire deployment bundle to all the clients? We would like find a way to prevent this. ... View more

We are trying to configure search peers with search heads and indexers using configuration files only. Splunk provides a command line that describes the indexer and authorization. For example: ./splunk add search-server --host :8089 --auth admin: -remoteUsername admin -remotePassword During the operation of that command on the Search Head, Splunk defines the search peer in etc/system/local/distsearch.conf and also sends the trusted.pem file to the indexer in the directory $SPLUNK_HOME/etc/auth/distServerKeys/ /trusted.pem We are looking for a configuration-only solution to setting up search peers without using the CLI command. We are not using auto-discovery since our customer installations will not allow multicast. Is there any way to perform the token exchange with configuration files? Is there a web services call that can perform the token-passing? ... View more

We are on Splunk 4.2.1 and using deployment server to manage our apps. We'd like to create an app to manage the settings in some of the *.conf files in the .../etc/deployment-apps directories, so users don't have to know linux editors or the subtleties of the parameters in the configuration files. What's the recommended way for programmatically editing a conf file on disk? We could use python libraries or other workarounds, but we're hoping there is experience others can share with doing this. To help clarify, from the Splunk Manager app and the CLI you can do things such as 'add search-server', which updates a distsearch.conf file. However this typically updates files local to the machine hosting the UI, and in the .../etc/system directory tree. We are looking for similar capability but with the edits going to a conf file (distsearch.conf in this example) in .../etc/deployment-apps directory tree. ... View more

We are using a 4.2.1 UF node to monitor a directory that contains web access log files, and send those files to an indexer. All of our nodes, including the UF and IDX nodes, have custom apps deployed to them from a central deployment server running on a Job Scheduler node. We're observing that if we update the inputs.conf file that defines the monitoring on the UF node, once the updated app is deployed to the UF (which causes splunk to restart), all of the web access log files are re-sent to to the indexer which causes duplicate events in the system. It seems like Splunk would do the bookkeeping related to the monitoring process someplace outside of the .../splunk/etc/apps directory which is the only thing changing in the above scenario. We would not expect the files to be read again and sent again just by updating a custom app. Has anyone seen this issue and is there something we can do to prevent the files from being re-read? After all if we simply restart Splunk on the UF node the files are not read again. ... View more

We have some timecharts that display information collected by Server_IP, using searches similar to this ... | timechart sum(Bytes) by Server_IP The problem is the IP values are reasonably meaningless to the people using this chart, so we'd like to display the hostname of the server associated with the IP. We've looked at the "charting.legend.labels" property but that is a statically defined list of values, and depending on time range selection and other values the list of IPs being displayed will change, as will the order of the IPs. So we are looking for a way to do what amounts to a lookup of each IP into a table that returns the Hostname, and use the Hostname as the label displayed in the legend. Can this be done? ... View more

We have a number of charts that show statistics about Servers over time. When you hover over a data point in the chart, a "tooltip" is displayed that shows the time value for the cell, the data value for the cell, and the Server name (standard Splunk behavior). The problem is that the Server names are very long, so as part of the standard Splunk behavior the name is abbreviated by putting ellipses in the middle of the Server name. In our case the leading parts of the Server name are the significant characters, so we're really like the ellipses at the end of the displayed name. We've already made this change in the advanced XML to control the Server names displayed in the legend, but have not found a way to control this in the tooltips. Any hints on how to do this, or other options to address our issue (e.g. widening the tooltip box) will be greatly appreciated. ... View more

Does any know what the best way to make a basic splunk application would be? For example, show dashboards from one of your splunk applications. ... View more

I am trying to find a way to send an SMS to a certain phone number whenever an alert is triggered. What would be the best way to do this? ... View more

I've tried setting the center parameter in my advanced XML and it does not seem to have any effect - the map still seems to come up with the default center location. My XML looks like this, which should put the US up centered in the window. Any inputs on what might be wrong will be greatly appreciated. I'm running with Splunk 4.2.1 and the latest GoogleMaps 1.1 app. We have a dashboard with multiple panels on it, this is the XML for just one of the panels. <module name="GenericHeader" layoutPanel="panel_row2_col1" autoRun="True"> <param name="label"> </param> <module name="HiddenSearch" group="Server Location" autoRun="True" layout_panel="panel_row2_col1"> <param name="search">`Populate_Server_Location_Map_60m()`</param> <module name="GoogleMaps"> <param name="autoPostProcess">false</param> <param name="height">500px</param> <param name="mapType">terrain</param> <param name="mapTypeControl">on</param> <param name="navigationControl">on</param> <param name="scaleControl">on</param> <param name="scrollwheel">off</param> <param name="zoomLevel">4</param> <param name="center">37.27,-97.12</param> </module> </module> </module> ... View more

How would you deploy 4.2.1 Splunk and Universal Forwarder on a Search Head node that is doing distributed search and also has the *Nix application installed as well? Both the Universal Forwarder and the Splunk deployments use the same daemon name ‘splunkd’. I am deploying UF as a system image along with Splunk for the Search Head. When starting UF I get the message that mgmt port 8089 is already bound. Should I just assign the UF mgmt port to another value? How do I get UF and splunk to co-exist on the SHD? Am I correct in assuming that I will need the UF on the SHD to forward *Nix stats to my Indexer? What recommendations would you make for this configuration? ... View more

We currently have an in-line csv table lookup that is used in both summary and normal index searches. Due to the needs of several customers, we would also like to specify a python script from the search command line (instead of the lookup command) that performs some direct results substitutions. We would like to keep the same syntax in the search command but be able to perform either type of lookup by simply replacing a .conf file or python script. Note that using macros is not appealing since we already have a sizable macros.conf and don't want to manage it per customer. Customer A custom command: | providertitlelookup | Customer B direct lookup: | lookup csv_provider_title Filename as Asset OUTPUT Provider, Title | Ideally if we could keep the syntax of Customer A then we would just need a script to do the basic substitutions for customer A and a script to do the csv table lookups for customer B. We would use the syntax of Customer A in the baseline search and merely install the desired python script for each customer. For the python script that supports Customer B, we will have to do the 'lookup' search directly from the python script. This search-within-a-search is likely to be very slow compared to doing the lookup in the primary search. This seems like a lot of overhead to set up the context for each search. There are potentially thousands of entries in the results set. An alternative might be to use Python csv to directly open the lookup table. What is the recommended way to implement a csv table lookup within a python script that is called to process results sets? Is there another way to keep the same basic search-command syntax but perform one of (custom command, external lookup, db-lookup, macro) or other method to do either a lookup or result substitution? ... View more