Getting Data In

Archiving and signing at the same time

Mahieu
Communicator

Hey guys,

I'm new on the Splunk planet.
I'm trying to find a script that would :
- Send archive data to a nas location when an index is full (as soon as an index is full, without the need of a human intervention)
- Sign this data automatically before it sends it to the distant location (i've generated keys already for IT data signing which works well, i guess i can use the same keys)

I've had a look at the sample script in the bin directory but i'm not "fluent" in python unfortunately.

If there's another solution than a script to do that, i'd work for me as well.

Thanks a lot in advance for your help.

M

Tags (3)
1 Solution

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

View solution in original post

Mahieu
Communicator

Here's a tested and working solution :

(#) !/bin/sh

DATE="$(date +%s)"
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/splunk/lib"
PATH="$PATH:/opt/splunk/lib"
ARCHIVE_PATH="/path/archive_splunk"
SPLUNK_HOME="/opt/splunk/"

gzip -c $1/*.tsidx > $ARCHIVE_PATH/archive_$DATE.gz
signtool -s $ARCHIVE_PATH

mw
Splunk Employee
Splunk Employee

It sounds like you just want to specify the coldToFrozenDir. If you're signing the data already, that should be preserved during the migration.

http://www.splunk.com/base/Documentation/latest/admin/Automatearchiving#Let_Splunk_archive_the_data_...

If you do need a script, it doesn't have to be python, but the example coldToFrozenExample.py is a good start.

0 Karma

Mahieu
Communicator

I tried the following :

I created a brand new index called test, max size is set to 1 Mo.
In indexes.conf :
[test]
coldtoFrozenScript = "/bin/sh" "/opt/splunk/bin/archive_signing.sh"

In opt/splunk/bin, my archive_signing.sh script is :

!/bin/sh

echo "script execute" > /root/archive.txt
gzip -c $1/*.tsidx > /mnt/tmp/test_archive.gz
signtool -s /mnt/tmp/test_archive.gz

Then i indexed a file that is bigger than 1 Mo.
The script is not launched, no archive.txt file is created in the root directory.
I can see my logs with the search app. current size of the index is 2 Mo.

Any ideas?

Thx

M

Mahieu
Communicator

I could use an archive path, no problem about that.
Still, i'm not sure how to make it work together with the signtool.

Mahieu
Communicator

I'm not signing the data at the moment. I just need to sign it at the moment it is sent to my archive path.

I must have missed something here...

M

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...