Monitoring Splunk

Windows Perfmon:Process index fine tuning

ansif
Motivator

Perfmon:Process is about 347,662 events for 2 host last 10 mins which is taking huge space in index. Any suggestion to fine tune this?

Tags (1)
0 Karma
1 Solution

lycollicott
Motivator

This is really an input tuning problem and not an index problem.

When you enable it, Splunk_TA_windows collects almost every perfmon counter in existence and it does it every 10 seconds.

First of all, no one needs to collect all of those counters.
Secondly, every 10 seconds is far too fast for most situations.

The perfmon process counters are turned off by default, so why exactly do you need them? It would take a very compelling reason for me to collect them on my servers.

Here is what the default settings are in etc\apps\Splunk_TA_windows\default\inputs.conf and it collects 27 different counters for each process every 10 seconds:

[perfmon://Process]
counters = % Processor Time; % User Time; % Privileged Time; Virtual Bytes Peak; Virtual Bytes; Page Faults/sec; Working Set Peak; Working Set; Page File Bytes Peak; Page File Bytes; Private Bytes; Thread Count; Priority Base; Elapsed Time; ID Process; Creating Process ID; Pool Paged Bytes; Pool Nonpaged Bytes; Handle Count; IO Read Operations/sec; IO Write Operations/sec; IO Data Operations/sec; IO Other Operations/sec; IO Read Bytes/sec; IO Write Bytes/sec; IO Data Bytes/sec; IO Other Bytes/sec; Working Set - Private
disabled = 1
instances = *
interval = 10
object = Process
useEnglishOnly=true
index = perfmon

If you are being forced against your will to collect process counters then:

  1. decide which ones you really, really need and collect only those
  2. collect them less frequently

View solution in original post

lycollicott
Motivator

This is really an input tuning problem and not an index problem.

When you enable it, Splunk_TA_windows collects almost every perfmon counter in existence and it does it every 10 seconds.

First of all, no one needs to collect all of those counters.
Secondly, every 10 seconds is far too fast for most situations.

The perfmon process counters are turned off by default, so why exactly do you need them? It would take a very compelling reason for me to collect them on my servers.

Here is what the default settings are in etc\apps\Splunk_TA_windows\default\inputs.conf and it collects 27 different counters for each process every 10 seconds:

[perfmon://Process]
counters = % Processor Time; % User Time; % Privileged Time; Virtual Bytes Peak; Virtual Bytes; Page Faults/sec; Working Set Peak; Working Set; Page File Bytes Peak; Page File Bytes; Private Bytes; Thread Count; Priority Base; Elapsed Time; ID Process; Creating Process ID; Pool Paged Bytes; Pool Nonpaged Bytes; Handle Count; IO Read Operations/sec; IO Write Operations/sec; IO Data Operations/sec; IO Other Operations/sec; IO Read Bytes/sec; IO Write Bytes/sec; IO Data Bytes/sec; IO Other Bytes/sec; Working Set - Private
disabled = 1
instances = *
interval = 10
object = Process
useEnglishOnly=true
index = perfmon

If you are being forced against your will to collect process counters then:

  1. decide which ones you really, really need and collect only those
  2. collect them less frequently
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...