About smallfry

smallfry · ‎05-30-2022

It's so that if I ever need to move the coldPath to slower storage in future, I can easily do so by just changing the path (e.g. from /fastdisk to /slowdisk). My volumes in Splunk are configured on the same fast storage as follows: [volume:primary] path = /fastdisk [volume:secondary] path = /fastdisk Because coldPath is mandatory, I must define a volume for it, which means I am "forced" to set a limit on it.

smallfry · ‎05-30-2022

It does bother me because if I set the coldPath's maxVolumeDataSizeMB to 10GB in my example above, which reserved 10GB of fast SSD storage for cold buckets, wouldn't it be a waste of fast storage if most of it is not being used by 30 indexers? If I set the coldPath's maxVolumeDataSizeMB too large, it's a waste of fast storage space but if I set it too low, the buckets will go into Frozen (deleted), hence my original question.

smallfry · ‎05-29-2022

Thanks for helping but I don't think it addresses my question unfortunately. Even if I set maxTotalDatasizeMB to 100GB for my example above, how can I limit or prevent moving over to cold buckets if I want to set it to just 1GB? Another side question is how do I calculate how much storage to use for cold?

smallfry · ‎05-29-2022

Hi everyone, I want to prevent warm buckets from becoming cold, not to disable it since it's mandatory to have coldPath. The reason is that since my Hot/Warm and Cold buckets are all on the same fast storage, as well as I also need to define maxVolumeDataSizeMB for coldPath, I want to use up all my storage for homePath as much as possible. Here's an example of what I mean. Total Disk Space: 100GB homePath's maxVolumeDataSizeMB: 90GB coldPath's maxVolumeDataSizeMB: 10GB No frozenPath I want to configure the indexes not to move to Cold buckets as much as possible, so that I can reduce the coldPath configuration to be 1GB only, hence freeing up 9GB of space to allocate for the homePath of my other 30 indexers. From my Monitoring Console, I see that the coldPath is not used much so 9GB of all indexers add up to lots of space that are under-utilized. Based on this stats, I could set to 1GB today but it might suddenly increase one day, which leads to my question above, as I want to set it in a deterministic way. Any advice is appreciated.

smallfry · ‎01-21-2020

I copied and created a new question here from another thread, so hopefully this has some inputs. My questions are as the following: With the eNcore Add-on already installed on a Heavy Forwarder, wouldn't deploying an updated Add-On via a Deployment Server makes the existing "data" directory becomes empty again since it will be overwritten by the copy from the Deployment Server? How can I do so without affecting the existing "data" directory or it doesn't matter since the logs had been ingested? Lastly, what's the impact of the "data" directory becomes empty? Will the logs be downloading in real-time from the FMC or does the Add-on download logs that had been in the FMC for x number of hours (example)? Thanks everyone in advance.

smallfry · ‎01-12-2020

I have a question that I thought will be better if I add it here, rather than creating a new one. My questions are as the following: With the eNcore Add-on already installed on a Heavy Forwarder, wouldn't deploying an updated Add-On via a Deployment Server makes the existing "data" directory becomes empty again since it will be overwritten by the copy from the Deployment Server? How can I do so without affecting the existing "data" directory or it doesn't matter since the logs had been ingested? Lastly, what's the impact of the "data" directory becomes empty? Will the logs be downloading in real-time from the FMC or does the Add-on download logs that had been in the FMC for x number of hours (example)? Thanks everyone in advance.

smallfry · ‎05-02-2019

Hi everyone. As you know, FirePower produces tons of logs that took up the expensive Splunk licensing. I filtered out a lot of Windows event logs and would like to do the same for FirePower. Any recommendations as to what to filter, such as those that took up lots of space but are not really useful?

smallfry · ‎10-29-2018

I went into the CM and enable maintenance mode, after which I then do a "splunk restart". However, it's still the same error. Any pointers will be appreciated.

smallfry · ‎10-27-2018

This is a great answer, even though it didn't work for me. For some reason, I constantly have hundreds of such bucket status that just didn't go away, even after restarting the CM and rolling-restart of my 7 indexers. When I ran this in batch mode, I had the following error message for every of the listed 685 buckets: Cannot roll a bucket with bid=ciscolog~330~5442CDRE-2540-45C7-DD32-23D48D394D8F to warm, Reason="Master already has committed size for this bucket." Any clue how to resolve this?

smallfry · ‎10-10-2018

Hey David. First of all, thanks for the App. It's very useful. I tried v2.2.0 of the app but realized that we need to edit the json files. For instance, the firewall logs by default is for Palo Alto while we are using a mixture of Cisco and Fortinet. If you next release can incorporate a set up or settings interface, it woud be perfect!

smallfry · ‎09-16-2018

Very clearly answered. I wish Splunk can include these information in their documentation. Thanks!

smallfry · ‎09-15-2018

Installing Universal Forwarders on Linux hosts running as Search Heads, Indexers, Deployment Server, etc Hi everyone, I had a hard time figuring out the confusing (but excellent effort though) documentation for Splunk Add on for Unix and Linux. I had went through the docs and Answers but am not 100% sure. My questions are: In a distributed environment, where I want to collect logs from my search head (cluster), indexers, cluster master, licenses master, deployment server, heavy forwarders, etc, must I install a Linux Universal Forwarder on each of them? It is clear that the Universal Forwarder must be installed on Linux hosts but how about these Splunk instances that are also running Linux itself? All my Splunk instances' logs are forwarding to indexers btw. Specifically for indexers, the documentation states: "If the indexer is also a *nix host and you want to collect *nix data from it, complete the procedure at Enable the data and scripted inputs within the Splunk_TA_nix add-on on the host." Seeing the above comments, do I need to install a Linux Universal Forwarder on the indexers? How about Cluster Master, Deployment Server, etc that are also running on Linux? Do I need to install Linux Universal Forwarder on them? Or do I just install the Splunk_TA_nix add-on? Or do I install both? I don't see any mention about them in the docs. Thanks for the advice in advance.

Posts	13
Solutions	0
Karma Given	3
Karma Received	1
Member Since	‎08-24-2018

Online Status	Offline
Date Last Visited	‎05-31-2022 12:08 PM

How to prevent warm buckets from becoming cold?

Upgrading an existing eNcore Add-on

How to Filter FirePower logs to reduce license usa...

in a distributed environment, do I have to install...

Re: Preventing Warm buckets from becoming cold

Re: Preventing Warm buckets from becoming cold

Re: Preventing Warm buckets from becoming cold

How to prevent warm buckets from becoming cold?

Upgrading an existing eNcore Add-on

Re: Where do you recommend installing the Cisco eS...

How to Filter FirePower logs to reduce license usa...

Re: Cluster Master have SF/RF not meet due to a lo...

Re: Cluster Master have SF/RF not meet due to a lo...

Re: look elsewhere for data

Re: in a distributed environment, do I have to ins...

in a distributed environment, do I have to install...