About wbolten

wbolten · ‎05-06-2021

I accepted your post as the solution as it directed me to a direction that got me there. 😉

wbolten · ‎05-06-2021

@maciep Thank you for your respond. It helped me along nicely. Did not get where I needed to be with autoregress but your list(Value) suggestion got me further. Got the following SPL now and it seems to be doing what I am looking for. index=<my_index> host=<my_server> sourcetype=<my_sourcetype> instance=_Total counter="% Processor Time" | stats list(Value) as Value by host | eval mValues = mvindex(Value,0,1) | eval value1 = mvindex(mValues,0), value2 = mvindex(mValues,1) | eval myDelta = value1 - value2 | eval createEvent = if(myDelta >2 OR myDelta < -2,"Create Event","Nothing to worry") | table host, mValues, value1, value2, myDelta, createEvent In the above SPL the threshold to compare against does not make sense and the table has more information then needed but it is for producing a result at the moment.

wbolten · ‎05-05-2021

Hi, I successfully created an SPL that does what I need for a single host but I cannot get it to work for all hosts. This works index=<my_index> host=<specific_host> sourcetype=<my_sourcetype> instance=_Total counter="% Processor Time" | sort host, -_time | dedup 2 host | lookup <my_lookup> resource_name as host output businessprocess_name | search businessprocess_name = "<my_business_process_name>" | eval Value = round(Value,2) | delta Value AS ValueDelta | eval lowerThreshold = -25 | eval upperThreshold = 25 | eval CreateEvent = if((ValueDelta > upperThreshold OR ValueDelta < lowerThreshold),"Yes","No") | search CreateEvent = "Yes" | eval metric_type = "CPU Usage Anomaly" | eval description = if(ValueDelta < 0,"CPU Usage is now: " + Value + "%. A decrease of " + ValueDelta,"CPU Usage is now: " + Value + "%. An increase of " + ValueDelta) | table _time, host, businessprocess_name, metric_type, description The output of that SPL is (changed the lower and upper threshold to trigger a result) _time host businessprocess_name metric_type description 2021-05-05 12:35:57 <specific_host> <my_business_process_name> CPU Usage Anomaly CPU Usage is now: 57.52951309736281%. A decrease of -3.69007662538445 I know the sort on host does not make sense in this SPL but it nicely takes the last values, compares it and based on the difference the result is what it needs to be. When I remove the host=<specific_host> and run it on all the hosts in the system the output is wrong. It seems that it is comparing value of row 1 (server A) with the value of row 2 (server A), then value of row 2 (again server A) with the value on row 3 (server B), etc etc. I guess that makes sense but not what I am looking for. What would be needed to run the calculation of the delta for only the two records that belong to the same host? What I am aiming to do is to create an event when the difference in the CPU usage between the last two values is more then the configured threshold, whether it drops or increases. Maybe I am going about it the wrong way with the Delta command?

wbolten · ‎09-22-2020

Hi, For some reason, I am failing to get any network monitoring data into my environment. I can successfully retrieve perfmon, script data, REST, and HEC. But as soon as I create a stanza with [WinNetMon://bla] nothing happens. This is the stanza I am now using. I am actually looking for a different process but this process is always present. [WinNetMon://lsass] disabled=0 addressFamily=ipv4 direction=inbound;outbound interval=60 protocol=udp;tcp index=uf_process process=lsass packetType=accept;connect;LostPacket I even tried the minimum. [WinNetMon://lsass] disabled=0 index=uf_process The app is deployed with the Windows Deployment server and it lands on the client just nicely. On the client, I pulled the following from the splunkd.log. 09-22-2020 13:57:29.780 +0200 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-netmon.exe"" splunk-netmon - NetmonStartDriver - StartService failure for splknetdrv! Error = -2144206839 09-22-2020 13:57:29.780 +0200 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-netmon.exe"" splunk-netmon - NetmonAppDoMonitoring: Failed to open monitor device: 0x80320009 09-22-2020 13:57:29.780 +0200 ERROR ExecProcessor - message from ""C:\Program Files\SplunkUniversalForwarder\bin\splunk-netmon.exe"" splunk-netmon - NetmonAppDoMonitoring: Error 0x80320009 occurred during execution This shows up after the restart of the UF on the client. I can't seem to find the solution to this one? I tried to change the service the UF is running under from LOCAL SYSTEM ACCOUNT to a named account with local admin rights but it did not make any difference. It almost looks like the Windows client is missing something. This morning I removed the universal forwarder and installed the latest version. Still nothing.

wbolten · ‎08-20-2020

Hi, I am using the UF to collect data from the system. Using the following stanza I seem to receive all the information in regards to the bytes sent and received. That is too much information for me. I am interested in traffic generated by a specific process, or processes. To be able to do this I have currently the following stanza live but it seems to be still sending everything. Not using the whitelist option. I also don't see the option in the documentation so that would not surprise me. [perfmon://Network Adapter WebEx] counters = Bytes Received/sec;Bytes Sent/sec instances = * whitelist = *.webex.com interval = 60 mode = single object = Network Interface index = xxxyyyzzz useEnglishOnly = true sourcetype = xxxyyyzzz:Network Adapter disabled = 0 What would be the best way, if even possible, to only catch and the network traffic for a specific process or processes? Besides traffic I am also interested in other metrics such as errors, dropped packets etc. Maybe I am going about this the wrong way. Any help would be appreciated.

wbolten · ‎08-11-2020

I did not know that 😉 With your suggestion I did get a single row but the 0 values where in there as well. Replaced them with isnull and now I only have values and a single two per server. | stats latest(Value) as Value by host, counter | eval "CPU Time" = if(counter="% Processor Time",Value,isnull) | eval "RAM Available" = if(counter="Available MBytes",Value,isnull) | eval "C Free Space" = if(counter="% Free Space",Value,isnull) | stats Values(*) as * by host | table host, "CPU Time", "RAM Available", "C Free Space" Thanks

wbolten · ‎08-11-2020

Hi, The following SPL returns records to me as shown below. index="uf_basickpi" host!=DS-* (sourcetype="CPU" counter="% Processor Time") OR (sourcetype="Memory" counter="Available MBytes") OR (sourcetype="DiskStuff" counter="% Free Space" instance=C:) | stats latest(Value) as Value by host, counter | eval "CPU Time" = if(counter="% Processor Time",Value,0) | eval "RAM Available" = if(counter="Available MBytes",Value,0) | eval "C Free Space" = if(counter="% Free Space",Value,0) | table host, "CPU Time", "RAM Available", "C Free Space" Rows 1,2 and 3 are from the same server. Rows 4,5 and 6 from the second server. What I would like to have is a single row per server with the three values. What would be the best way to do this.

wbolten · ‎03-30-2020

Right, eventually found something that works for me. Not sure if it is the best way but it does work. Below is the query to get the result shown above. I simply added in the alert the line | search Prediction < 2000 Just above the last one where I removed the fields I don't want to see. The result is two dates in the future. the 11th and the 12th of April. So it works and returns the dates to focus on. For the alert itself the end result needs to be cleaned up but it does what I need. index=xxxxx host=xxxxx source="Perfmon:Memory" counter="Available MBytes" | eval Value=round(Value,0) | timechart span=1d avg(Value) as "Available MBytes", latest(host) as host, latest(counter) as counter partial=false | lookup resource_thresholds.csv resource_name AS host, resource_metric AS counter OUTPUTNEW resource_threshold_warning,resource_threshold_critical | eval Warning=resource_threshold_warning | eval Critical=resource_threshold_critical | predict "Available MBytes" as Prediction future_timespan=14 | eval Prediction = round(Prediction,0) | fields - lower95(Prediction), upper95(Prediction) resource_threshold_warning resource_threshold_critical host counter

wbolten · ‎03-26-2020

Yep, that has done it. Many thanks. Did not occur to me to use case for this. I corrected a typo in the CRITICAL part. The status number is 14 and not 4 but it works perfectly.

wbolten · ‎03-26-2020

Hi, Is it possible to change the 'name' of the value in a pie chart displayed on a geostats map? The below example shows the piechart. The blue 1 indicate that there are 4 devices with that status. One device with status 14 and 1 with status 3. I have been trying the change the numbers into a readable status. 1=UP,3=WARNING,14=Critical but for some reason I don't seem to make it possible. This is the SPL creating the worldmap index="xxx" Splunk_Node=true | stats latest(Latitude) as Latitude, latest(Longitude) as Longitude, latest(Node_Status) as Node_Status by NodeID | geostats count(NodeID) by Node_Status latfield=Latitude longfield=Longitude Been trying the below. Added this line and change the SPL from Node_Status to NodeStatus but then the map stays empty. | eval NodeStatus = if(NodeStatus=1,"UP",if(NodeStatus=2,"DOWN",if(NodeStatus=3,"WARNING",if(NodeStatus=14,"CRITICAL",isnull)))) `comment("Change the status number to something readable")` Someone any thoughts on this?

wbolten · ‎03-09-2020

Hi, I am trying to build an alert from the following query. The query collects the counters for memory usage, especially the free amount. It plots a time chart of the last 21 days and performs a prediction over the coming 14 days. The graph itself is perfect. It also shows in the prediction that in the next 14 days we run out of memory. index=xxxxx host=xxxxx source="Perfmon:Memory" counter="Available MBytes" | eval Value=(Value/1024) | timechart span=1d avg(Value) as "Available MBytes", latest(host) as host, latest(counter) as counter | lookup resource_thresholds.csv resource_name AS host, resource_metric AS counter OUTPUTNEW resource_threshold_warning,resource_threshold_critical | eval Warning=resource_threshold_warning | eval Critical=resource_threshold_critical | predict "Available MBytes" as Prediction future_timespan=14 | eval Prediction = round(Prediction,0) | fields - lower95(Prediction), upper95(Prediction) resource_threshold_warning resource_threshold_critical host counter I want to run this as a scheduled alert (email, MS Teams) every night and be informed when the prediction hits 0 or lower somewhere in the future 14 days in this case. For some reason I cannot seem to get my head around the logic here to trigger the alert. Any suggestions?

wbolten · ‎02-07-2020

Thank you, I have done so.

wbolten · ‎02-07-2020

Right, been staring at this for ages but it seems to be solved. It was suggested to only select the moments to place the annotation from the search rather then look for them from the result. Changed the annotation query to index="bla" | search inPercentUtil>65 AND Percent_CPULoad>90 | eval annotation_category="Critical" | eval annotation_label="Insert_Message" Now it is working.

wbolten · ‎02-07-2020

Hi, I have created a bar chart that shows in my case example data on the CPU usage of a device with two overlays for inbound and outbound traffic. When a certain combination of values is exceeded I want to place an annotation to highlight this. As shown in this example. What I cannot seem to get rid off are the orange annotations that seem to be placed when the condition is not matched. The search is as follows. In this I am evaluating and updating the label and the category. <search type="annotation"> <query>index="bla" | timechart span=1m avg(inPercentUtil) as inPercentUtil avg(Percent_CPULoad) as Percent_CPULoad | eval critical=if(inPercentUtil>65 AND Percent_CPULoad>90,"Yes","No") | eval annotation_category=case(critical="Yes","Critical") | eval annotation_label=case(critical="Yes","Insert_Message") </query> What should I do to stop the orange markers from showing? I don't even specify a color for the annotations. <option name="charting.annotation.categoryColors">{"Critical":"0xe60000"}</option>

Posts	14
Solutions	1
Karma Given	0
Karma Received	2
Member Since	‎12-02-2019

Online Status	Offline
Date Last Visited	‎10-19-2021 11:26 AM

Calculate Delta for a value for all hosts

WinNetMon stanza not working when trying to get ne...

Whitelist network traffic

Join SPL result to a single line?

GEOSTATS Rename value in piechart

Trigger alert on value from predict calculation

Too much annotations

Re: Calculate Delta for a value for all hosts

Re: Calculate Delta for a value for all hosts

Calculate Delta for a value for all hosts

WinNetMon stanza not working when trying to get ne...

Whitelist network traffic

Re: Join SPL result to a single line?

Join SPL result to a single line?

Re: Trigger alert on value from predict calculatio...

Re: GEOSTATS Rename value in piechart

GEOSTATS Rename value in piechart

Trigger alert on value from predict calculation

Re: Too much annotations

Re: Too much annotations

Too much annotations