KVStore-->Mongo consuming 40GB space

[spodda01da]

Hi,

I am having issue were kvstore/mongo is consuming 40GB of disk space on search head server (Linux OS).

While reviewing the kvstore collection stats i found "splunk_app_windows_infrastructure.tSessions_collection" to be consuming almost 15GB of space.

It seems Splunk app for Windows Infrastructure taking up most of the disk space.

Not sure if re-installing this app is a good idea but can anyone suggest a way to resolve it without breaking or losing any data.

I am using Splunk version 7.2

Thanks in advance,

[nick405060]

The answer is that you likely have both the Exchange and Windows app installed, which is the issue. To me this is something that should be addressed ASAP, it seems like a very serious issue that any Splunk customer that installs both apps gets their disk space blown up (unsure which versions of Splunk this affects):

https://docs.splunk.com/Documentation/MSExchange/4.0.0/DeployMSX/Platformandhardwarerequirements#Do_...

I ran this per @dwaddle:

|  rest splunk_server=local /services/server/introspection/kvstore/collectionstats 
|  fields data 
|  mvexpand data 
|  rename data as _raw 
|  spath 
|  fields - _raw
|  fields ns size storageSize totalIndexSize

And it showed me that the terminal service trackers are the culprits:

splunk_app_microsoft_exchange.tSessions_collection and splunk_app_windows_infrastructure.tSessions_collection

Per @automine, app savedsearches likely have the same name and are using the same collection. You can disable tSessions_Lookup_Update* savedsearches in the apps, or, like the Documentation link says, just uninstall the Windows Infra app.

This is also the same issue:

https://answers.splunk.com/answers/738915/splunk-app-windows-infrastructure-kvstore-disk-sat.html

Oh also a local clean of the kvstore should clear it out

[steveharrison28]

I had a similar issue and had to uninstall the app to free up diskspace (tSessions_collection was using ~150G). I re-installed the app without the tSessions_collection stanza in collections.conf as the session info isn't so important to us. There are probably better ways to filter unwanted noise before it gets to the app, but I needed to free up space quickly.

[spodda01da]

Thanks, I too had a similar situation to free up disk space quickly so I moved Kvstore to another drive. Now it has increased from 40GB to 52GB.
I am still trying to get hang of splunk and was wondering if you have any article which I can use to uninstall the Windows Infrastructure app (without the tSessions_collection).

Thanks again!!