Hello,
We currently have custom batch jobs running on EC2 instances in AWS and each of these processes creates one log. the log name and directory is not fixed, but predictable. For example the name format could be log__PID_YYYYDDMM.
Our main purpose is to make these logs available to developers for review without the need to log in to production. We also would like to get these logs searchable via Splunk for certain keywords and alert based on those keywords as well.
An example of the log format would be like this:
############ E X E C L O G ############
13080 20:16:38 NEW818D JOB STARTED CLASS=A NODE=JK
UID=ghjrib_on
20:16:38 STEP GHHSR01 STARTED
GHHSR01 SORTIN1=/data/ghhb/pvt/master/GH.GH252D.GH266
GHHSR01 SORTIN2=/data/ghhb/pvt/master/GH.GH262D.GH266
GHHSR01 SORTIN3=/data/ghhb/pvt/master/GH.GH266D.GH262
GHHSR01 SORTOUT=/mnt99/tmp/GH.GH818D.GHSR01_13088
GHHSR01 CPU TIME: 0.010000
20:16:38 STEP GHHSR01 ENDED CODE=0
20:16:38 STEP GHHSR02 STARTED
GHHSR02 SORTIN1=/data/ghhb/pvt/master/GH.GH252D.AB272
GHHSR02 SORTIN2=/data/ghhb/pvt/master/GH.GH262D.AB272
GHHSR02 SORTIN3=/data/ghhb/pvt/master/GH.GH266D.AB268
GHHSR02 SORTOUT=/mnt99/tmp/GH.AB818D.GHSR02_13088
GHHSR02 CPU TIME: 0.020000
20:16:39 STEP GHHSR02 ENDED CODE=0
20:16:39 STEP SIAB858 STARTED
GHHB858 SYS010=/data/ghhbb/pvt/master/GH.AB252D.AB259
GHHB858 SYS020=/mnt99/tmp/GH.AB818D.SIAB858_13088
GHHB858 CPU TIME: 0.000000
20:16:39 STEP GHHB858 ENDED CODE=0
20:16:39 STEP GHHB859 STARTED
GHHB859 SYS010=/mnt99/tmp/GH.AB818D.GHSR01_13088
GHHB859 SYS020=/mnt99/tmp/GH.AB818D.GHB859_13088
GHHB859 CPU TIME: 0.000000
20:16:39 STEP GHHB859 ENDED CODE=0
Does Splunk have the capability to handle logs like this?
... View more