@here: The newer version of Splunk Add-on for ServiceNow (4.0.0 ) has been released. Vendor products: ServiceNow Kingston, London, and Madrid Splunk base : https://splunkbase.splunk.com/app/1928/ Documentation : https://docs.splunk.com/Documentation/AddOns/latest/ServiceNow/About ... View more

@TitanAE, This add-on has 3 logs that are located at $SPLUNK_HOME/var/log/splunk: splunk_ta_tomcat_main.log splunk_ta_tomcat_setup.log splunk_ta_tomcat_util.log Have you checked this to figure out what is wrong? https://docs.splunk.com/Documentation/AddOns/released/Tomcat/Troubleshoot ... View more

@qazwsxe For Faster Search, you need to be specific. Use source or sourcetype in your search and make use of Time range picker to search the logs in the specific time range. Also, we have below modes of searching in Splunk: Fast Mode Smart Mode Verbose Mode ... View more

There are two load balancing method in Splunk: By time By Volume By time. The default method for load balancing is how frequently the forwarders change indexers in the load balanced list. The autoLBFrequency setting in outputs.conf controls how often forwarders switch between indexers. The default frequency is every 30 seconds, but you can set it higher or lower. autoLBFrequency = * The amount of time, in seconds, that a forwarder sends data to an indexer before redirecting outputs to another indexer in the pool. * Use this setting when you are using automatic load balancing of outputs from universal forwarders (UFs). * Every 'autoLBFrequency' seconds, a new indexer is selected randomly from the list of indexers provided in the server setting of the target group stanza. * Default: 30 * List item https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Outputsconf By volume. Another option is to set how much data a forwarder sends to an indexer before it switches between indexers in a load-balanced list. The autoLBVolume setting in outputs.conf controls the amount of data that a forwarder sends to a receiving indexer before it changes to another one. By default, this setting is not active (0 bytes.) If you set it to anything other than 0, then the forwarder will change indexers based on the amount of data that it has sent. autoLBVolume = * The volume of data, in bytes, to send to an indexer before a new indexer is randomly selected from the list of indexers provided in the server setting of the target group stanza. * This setting is closely related to the 'autoLBFrequency' setting. The forwarder first uses 'autoLBVolume' to determine if it needs to switch to another indexer. If the 'autoLBVolume' is not reached, but the 'autoLBFrequency' is, the forwarder switches to another indexer as the forwarding target. * A non-zero value means that volume-based forwarding is active. * 0 means the volume-based forwarding is not active. * Default: 0 https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Outputsconf Please go through below documentation for more details: https://docs.splunk.com/Documentation/Splunk/7.2.6/Forwarding/Setuploadbalancingd ... View more

I suspect both the errors are interlinked. When the client sends a message and crashes, then we get this EOF error. So the issue here is that the client is getting crashed after sending the message. For more information go through below link: https://github.com/gorilla/websocket/issues/321 Since the nozzle is running on PCF end, we won't be able to determine why it's crashing. Need to check with Pivotal to determine the root cause. ... View more

The below errors indicates "high I/O activity of splunk-optimize": throttled: idx= Throttling indexer, too many tsidx files in bucket='/SplunkIndexData/splunk-indexes//db/hot_v1_1519'. Is splunk-optimize working? If not, low disk space may be the cause. The way to reduce splunk-optimize I/O activity is to increase maxMemMB for index "" which has most of throttling messages and any other index customer is aware of high volume: indexes.conf [index_name] maxMemMB=20 ... View more

Kira, Sample search for your requirement; index="_internal" earliest=-20h |stats count by host| eval hourlyevents=count/20 | eval avg=hourlyevents*0.2 | where hourlyevents < avg Description: eval hourlyevents=count/20 : (calculated no. of events per hour) eval avg=hourlyevents*0.2 : (20% of hourly events) where hourlyevents < avg : (If hourly events is less than 20% then alert should trigger) You can also refer below document: https://docs.splunk.com/Documentation/Splunk/7.2.3/SearchReference/ConditionalFunctions ... View more

Hi Kira_l, You can start from here : https://www.youtube.com/watch?v=SuARLqm7_jc Steps Navigate to the Search page in the Search and Reporting app. Create a search. Select Save As>Alert. Enter a title and optional description. Specify permissions. Configure alert scheduling. There are two options for scheduling. Configure trigger conditions. (Optional) Configure a trigger throttling period. Select one or more alert actions that should happen when the alert triggers. Click Save. ... View more

Servicenow TA 4.0.0 is scheduled to be release by end of this month (Tentative Date : 26 Feb 2019) , which is compatible with “Splunk integration app” London version. ... View more

Servicenow TA 4.0.0 is scheduled to be release by end of this month (Tentative Date : 26 Feb 2019) , which is compatible with “Splunk integration app” London version. ... View more

If you want to use a custom script in alert actions and pass arguments to it when the alert is triggered Let us assume a sample.sh script as below which will accept a name a argument and print to a sample.log file !/bin/bash echo "hello $1 @ date +%Y-%m-%d-%H:%M:%S " >> /path/sample.log exit 0 We need to do the following to use this script in alert actions Create an app "MyApp" with a "bin" directory and "default" directory Place the sample.sh and another file .path file. The .path file should have the location of the executable. In this example, we are creating a bash.path with "/bin/bash" as its contents. In the default directory, create app.conf and alert_actions.conf. app.conf [launcher] version = 1.0 [ui] is_visible = false label = Splunk Custom Alert Example alert_actions.conf [custom_script] flag the action as custom alert action is_custom = 1 configure appearance in the UI label = Custom Script Alert Action description = Triggers a custom alert action icon_path = custom_alert.png override default script execution alert.execute.cmd = bash.path alert.execute.cmd.arg.1 = /opt/splunk/etc/apps/myapp/bin/sample.sh alert.execute.cmd.arg.2 = Anyname After splunk restart, a custom alert action will be created, which needs to be selected during the alert creation. In the alert_actions.conf you can pass any number of arguments to your script. ... View more