I've been searching for a generic example of how to bring data from a perl script, into Splunk using HEC, including HTTPS.
Use dedicated HEC libraries for PERL?
Have the perl script call the curl commands that are found in the HEC docs?
Or use something perl-native, really lightweight and readily available?
After testing functionality and performance - one good option is REST::Client
The example file below is useful for the use case of, testing performance of injection from perl. On my test VM, the code below inserted 20,000 events per minute into Splunk 7.3.1.
Of course, this example's real purpose is to show you how to establish the connection, disable SSL verification if needed, and how to inject using POST routines. In a real script, you'd set the JSON content in the $client->POST
commands. (Fancy coders might even use additional perl libraries to build up the JSON.)
#!/usr/bin/perl -w
use strict;
use v5.16.3;
use REST::Client;
my $token = '7a49c676-22b8-40e3-8e37-9dad19a9bdb2';
my $hec_uri = 'https://127.0.0.1:8088';
my $n=3600; # 3600 is one insert every 1 second for 1 hour
my $client = REST::Client->new();
$client->getUseragent()->ssl_opts(verify_hostname => 0);
$client->getUseragent()->ssl_opts(SSL_verify_mode => 'SSL_VERIFY_NONE');
$client->setHost($hec_uri);
$client->addHeader("Authorization", "Splunk $token");
$client->POST('/services/collector/event', '{"sourcetype": "mysourcetype", "event":"my payload here"}'); ## initial test
print $client->responseContent();
print "\n==========================";
print "\nTiming $n runs using direct perl libs: ";
my $start=time();
my $i=0;
while ($i++ <=$n)
{
$client->POST('/services/collector', "{\"sourcetype\": \"mysourcetype\", \"event\":\"testrun=$$ speed TEST #$i\"}");
}
my $end=time();
my $elapsed=$end-$start;
print "took $elapsed seconds\n";
After testing functionality and performance - one good option is REST::Client
The example file below is useful for the use case of, testing performance of injection from perl. On my test VM, the code below inserted 20,000 events per minute into Splunk 7.3.1.
Of course, this example's real purpose is to show you how to establish the connection, disable SSL verification if needed, and how to inject using POST routines. In a real script, you'd set the JSON content in the $client->POST
commands. (Fancy coders might even use additional perl libraries to build up the JSON.)
#!/usr/bin/perl -w
use strict;
use v5.16.3;
use REST::Client;
my $token = '7a49c676-22b8-40e3-8e37-9dad19a9bdb2';
my $hec_uri = 'https://127.0.0.1:8088';
my $n=3600; # 3600 is one insert every 1 second for 1 hour
my $client = REST::Client->new();
$client->getUseragent()->ssl_opts(verify_hostname => 0);
$client->getUseragent()->ssl_opts(SSL_verify_mode => 'SSL_VERIFY_NONE');
$client->setHost($hec_uri);
$client->addHeader("Authorization", "Splunk $token");
$client->POST('/services/collector/event', '{"sourcetype": "mysourcetype", "event":"my payload here"}'); ## initial test
print $client->responseContent();
print "\n==========================";
print "\nTiming $n runs using direct perl libs: ";
my $start=time();
my $i=0;
while ($i++ <=$n)
{
$client->POST('/services/collector', "{\"sourcetype\": \"mysourcetype\", \"event\":\"testrun=$$ speed TEST #$i\"}");
}
my $end=time();
my $elapsed=$end-$start;
print "took $elapsed seconds\n";