I have a file that I need to index twice. Specifically, I need it sent/indexed to two different indexes. How could I have a single file get indexed to two separate indexes?
You can use the combination of crcSalt and a symlink. Consider the following example:
I have two indexes, index1 and index2. The file I want to index is called error_log and resides in /var/log/httpd/.
By symlinking log_file to another location, and using crcSalt, Splunk will be able to index this file twice and send it to another index. First, symbolically link the error_log file to another location:
>cd /var/log/httpd/dir
>ln -s /var/log/httpd/error_log .
The previous commands will create a soft link pointing to /var/log/httpd/dir/error_log. Here is the exact configuration in inputs.conf:
[monitor:///var/log/httpd/error_log]
index=index1
sourcetype=apache_error
crcSalt = <SOURCE>
[monitor:///var/log/httpd/dir/error_log]
index=index2
sourcetype=apache_error
crcSalt = <SOURCE>
Let us consider that you have a very common log file (/var/log/messages) that gets indexed to an index called operations. Your environment has hundreds of machines that send this information to Splunk via a common forwarder configuration. Now, what if I have a few machines where I want this log to be shared with another group that should not have access to the operations index. Well, I could create a role filter for this other group and grant access to this index. For my case, I don't want multiple filters and would like to silo my data.
There's a few use cases: apps that overlap in the data they want but are developed entirely independently; multi-tenant environments where you want different splunk groups to operate relatively independently; a case similar to summary indexing where you want sparse data in a special index, but you also want it available in a general, easily accessible index.
You can use the combination of crcSalt and a symlink. Consider the following example:
I have two indexes, index1 and index2. The file I want to index is called error_log and resides in /var/log/httpd/.
By symlinking log_file to another location, and using crcSalt, Splunk will be able to index this file twice and send it to another index. First, symbolically link the error_log file to another location:
>cd /var/log/httpd/dir
>ln -s /var/log/httpd/error_log .
The previous commands will create a soft link pointing to /var/log/httpd/dir/error_log. Here is the exact configuration in inputs.conf:
[monitor:///var/log/httpd/error_log]
index=index1
sourcetype=apache_error
crcSalt = <SOURCE>
[monitor:///var/log/httpd/dir/error_log]
index=index2
sourcetype=apache_error
crcSalt = <SOURCE>
I'm interested in understanding why you would like to do this. 🙂