About svendby90

svendby90 · ‎03-07-2022

We have an instance where KV store is not running and we're looking to clean the whole thing out. However, we would like to see if we're able to keep the data. So, my question is; is there any other way to do a backup of KV store data than using the "splunk backup kvstore" command?

svendby90 · ‎12-21-2017

After updating to 3.0 the problem seems to have been resolved. No idea what caused the error unfortunately. For the record, in order to enable drilldown from deep dive you need to create a deep_dive_drilldowns.conf in /etc/apps/itsi/local. See /etc/apps/itsi/README/deep_dive_drilldowns.conf for options 🙂 When created run http://yoursplunkserver:8000/en-GB/debug/refresh to activate the configuration.

svendby90 · ‎12-19-2017

Hi! No fix yet. I haven't been able to work on this since I posted the question. Will update if I figure it out 🙂

svendby90 · ‎11-28-2017

It turned out our dev. SH kept writing to the itsi_summary index. ITSI is installed and the services are defined, but not all of the inputs are in place. The result was two log entries a minute, one with the right service health score and one constantly at zero, causing the health score to be incorrectly calculated.

svendby90 · ‎11-28-2017

Sorry, haven't been working on this for a few days. However, the problem has now been solved! It turned out the dev. SH kept writing to the itsi_summary index. ITSI is installed and the services are defined, but the inputs are not all in place. The result was two log entries a minute, one with the right service health score and one constantly at zero, causing the health score to be incorrectly calculated. Anyway, thanks for your input, skoelpin! Now I know more about how to troubleshoot ITSI issues : )

svendby90 · ‎11-24-2017

Ok, I see. Thanks. I will try and see how it goes 🙂 However, as I brieftly mentioned ealier, this happens with all of our defined services. Another example is our AD monitoring where the four KPIs defining the service are green and the overall service health is red at 0. These KPIs are based on counters in standard perfmon logs, have no dependencies and are adhoc searches. An example of one of them is: index=perfmon_ad host=<host-prefix>* source="Perfmon:CPU Load" counter="% Processor Time" Here, the threshold field is the field "Value" and we calculate maximum per entity, average of aggregate over the last 5 minute(s) ever 5 minute(s). The calculated value is typically 4-5% and threshold level "medium" is triggered when it reaches about 80%. This is the case with the other counters as well. They are well below the trigger tresholds. So I don't see the reason why the overall service should be red...

svendby90 · ‎11-23-2017

Yes! Search: index=azure host=azure_rss sourcetype=azure_status | eval value=if(StatusMessage="An issue has been discovered",0,1) Threshold field: value Split by entity: No Calculating Average of aggregate over the last 15 minute(s) every 5 minutes.

svendby90 · ‎11-23-2017

It looks to be alternating. The KPI's value is constant, but the health is switching from 100 to 0 at random intervals. What's interesting, I tried adding some of the other services health scores to the same glass table, and all the scores are alternating between 0 and 100 at the exact same time. And there are no defined dependencies between them.

svendby90 · ‎11-23-2017

Single search head.

svendby90 · ‎11-23-2017

The KPI is running an adhoc search.

svendby90 · ‎11-23-2017

We are experiencing issues with services' health score alternating between 0 and 100 in the Service Analyzer in ITSI. The health scores shows 0 even though all the underlying KPIs are ok. This happens for all of our defined services. The simplest case is shown below. Here we have a service "Azure Status" with only one defined KPI: "AzureStatus". We recently updated to 3.0.0, but experienced the same issue before the upgrade (version 2.4.0). Anyone ideas what would cause this or what the issue is?

svendby90 · ‎10-05-2017

A way of doing it, which seems to give the expected output, is to convert the time token to epoch using conditions when the token is first set (similar issue and solution here: https://answers.splunk.com/answers/489033/how-can-i-convert-a-timerange-token-to-an-epoch-va.html). Try something like this in your XML: <fieldset submitButton="false"> <input type="time" searchWhenChanged="true"> <label>...</label> <default> <earliest>-7d@w0</earliest> <latest>@w0</latest> </default> <change> <condition match="isnum($earliest$) OR isnum($latest$)"> <eval token="earliest_time">$earliest$</eval> <eval token="latest_time">$latest$</eval> </condition> <condition> <eval token="earliest_time">relative_time(now(), $earliest$)</eval> <eval token="latest_time">relative_time(now(), $latest$)</eval> </condition> </change> </input> </fieldset> ... <search> <query>index=xxx sourcetype=xxx earliest="$earliest_time$" latest="$latest_time$" NOT [| inputlookup timestamped_users.csv | where _time>=$earliest_time$ AND _time<=$latest_time$ | rename "User IP" as src_ip] | dedup src_ip sortby +_time | rename src_ip as "User IP" | table "User IP", "_time" | stats dc("User IP") as "Returning Users" | appendcols [ search index=xxx sourcetype=xxx earliest="$earliest_time$" latest="$latest_time$" AND [| inputlookup timestamped_users.csv | where _time>=$earliest_time$ AND _time<=$latest_time$ | rename "User IP" as src_ip] | dedup src_ip sortby +_time | rename src_ip as "User IP" | table "User IP", "_time" | stats dc("User IP") as "New Users"] | transpose | rename "row 1" as "Count" "column" as Field</query> </search>

svendby90 · ‎10-12-2016

Try a change in metrics.py located in the /$SPLUNK_HOME$/etc/apps/appdynamics/bin/ directory. In the getMetrics-function, replace conf.read([os.environ['SPLUNK_HOME'] + '/etc/apps/appdynamics/default/metrics.conf']) with conf.read([os.environ['SPLUNK_HOME'] + '/etc/apps/appdynamics/local/metrics.conf'])

svendby90 · ‎10-11-2016

I'm trying to configure some drilldown options from swim lanes in the Deep Dive view in the Splunk IT Service Intelligence app, but having some difficulties doing that. My goal is to be able to drill down to different dashboards from various lanes. When enabling overlay for a KPI's swim lane and selecting that lane, I get an error (see picture). Checking the console, I find the following error message: "Failed to load resource: the server responded with a status of 414 (Request-URI Too Large)" . Any ideas on how/where I can fix/set this?

Posts	14
Solutions	4
Karma Given	9
Karma Received	4
Member Since	‎10-10-2016

Online Status	Offline
Date Last Visited	‎07-07-2023 09:01 AM

Is there any other way to do a backup of KV store ...

Issue with Service Health Score in ITSI

Splunk IT Service Intelligence: Why am I getting e...

Is there any other way to do a backup of KV store ...

Re: Splunk IT Service Intelligence: Why am I getti...

Re: Splunk IT Service Intelligence: Why am I getti...

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Re: Issue with Service Health Score in ITSI

Issue with Service Health Score in ITSI

Re: Problem with tokens and earliest/latest

Re: Enhancement request - change App to look for M...

Splunk IT Service Intelligence: Why am I getting e...