Hi All,
I have around 30 Hosts forwarding logs to splunk.
I have the below same paths in all the servers
/data/abc/vault.logs
/data/abc/vault_audit.logs
/data/xyz/proxy.logs
So I have created an app included inputs with all those above stanzas and pushed the app to all hosts.
So by default all those hosts are sending the above mentioned logs to splunk.
But I want 5 servers to send just the below log but not other logs
/data/xyz/proxy.logs
How to achieve this?
Hi @blbr123,
you have two ways to reach your target:
About the first solution, I think that you don't need any help to create the two Add-Ons and the two ServerClasses, if you need it, please, tell me.
About the second solution, you have to put in your Indexers or (if present) on your Heavy Forwarders the following props.conf
[host::host1]
TRANSFORMS-null= setnull
[host::host2]
TRANSFORMS-null= setnull
[host::host3]
TRANSFORMS-null= setnull
[host::host4]
TRANSFORMS-null= setnull
[host::host5]
TRANSFORMS-null= setnull
and in your transforms.conf:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
as you can read at https://docs.splunk.com/Documentation/Splunk/8.2.6/Forwarding/Routeandfilterdatad#Filter_event_data_...
Ciao.
Giuseppe
Yes I need help on creating the add-on if I have to apply the first solution
Hi @blbr123,
I suppose that you're using a Deployment Server to deploy configurations to your Forwarders, tell me if not and anyway, put in mind to use it as soon as possible!
you can find information about how to get data in at https://docs.splunk.com/Documentation/Splunk/latest/Data/Getstartedwithgettingdatain
and https://docs.splunk.com/Documentation/Splunk/8.2.6/Updating/Createdeploymentapps https://docs.splunk.com/Documentation/Splunk/8.2.6/Updating/Updateconfigurations
Anyway, you have to create two addons both containing the following folders structure:
in each Add-On put in the default folder app.conf file containing something like this:
[default]
[launcher]
author = you
description = Add-On for all hosts
version = 1.0.0
[package]
check_for_updates = 0
[ui]
is_visible = 0
label = TA-All_Servers
obviously changing label and description for each one.
Then put in the local folder of the first (the one for all servers) the following inputs.conf:
[monitor:///data/abc/vault.logs]
disabled = 0
index = your_index
sourcetype = your_sourcetype1
[monitor:///data/abc/vault_audit.logs]
disabled = 0
index = your_index
sourcetype = your_sourcetype2
[monitor:///data/xyz/proxy.logs]
disabled = 0
index = your_index
sourcetype = your_sourcetype3
and in the local folder of the second Add-On:
[monitor:///data/xyz/proxy.logs]
disabled = 0
index = your_index
sourcetype = your_sourcetype3
Then you have to deploy these two Add-Ons using the Deployment Server, following the instructions at https://docs.splunk.com/Documentation/Splunk/8.2.6/Updating/Updateconfigurations
in few words you have to:
Obviously you had to configure your clients as client of the Deployment Server, if you didn't do it follow the instructiona at the above link.
If you don't want to configure a Deployment Server in your infrastructure (I don't hint this!) you could manually copy the Add-Ons into the related servers in the %SPLUNK_HOME/etc/apps folder, remembering to restart Splunk on each one.
My final hint is to follow a training for Splunk Admin to better understand how to do all these things.
Ciao.
Giuseppe
Can't we achieve this mentioning the host details in inputs.conf
Let's say
[monitor://data/abc/vault.log]
index=applog
Host=dx096865
By doing this don't I get just the vault.log from just that host?
About second solution,
We actually don't use transforms much, but work on props based on sourcetypes
So not sure if this can be achieved just in props
Hi @blbr123,
what's the problem to use also transforms.conf? it's a part of the solution.
This is the usual method to filter unwanted logs.
Ciao.
Giuseppe
Hi @blbr123,
as I said: you have two choices:
You cannot put a condition in inputs.conf.
My hint is to have two Add_Ons (solution 1), but also the second solution, as I said, it's an easy to implement solution.
Ciao.
Giuseppe
Ok then for what purpose the hosts is mentioned in inputs which I saw in some configurations
Hi @blbr123,
using the filtering solution, you have only one inputs. conf and the filter (mentioning hosts) is on props.conf on Indexers.
The option "host=your_host" in inputs.conf is used to force the value of host for that data source.
If you don't use it, by default, the host value of that data source is setted to the value of the forwarder you're using (you can find it in $SPLUNK_HOME/etc/system/local/server.conf of the Forwarder).
Ciao.
Giuseppe