Hello Everyone
For Endpoint Security Analysis Purposes we Gather Logs from Machines using Tools that Generate archives With lots of Files in it with Different Formats Like XML, JSO, SQLite, txt, Log, evt, evtx, bin, etc ...
My Aim is to have all these Data Indexed Manually (Using Web Upload Method or CLI oneshoot ) for the Team to use Splunk Search Capabilities to Simplify analysis Process
Therefore I'm trying to do the Flowing :-
I Created Sourcetypes for Each a sample of File-types inside these archives (TextLog and XML)
I created [source::] Stanzas For Files Inside Archives to Assign these Previously Created Sourcetypes Automatically
after I uploaded the Zip file the Results was that Some Extensions were indexed Successful and Some were having a sourcetype of "unknown1"
One of the Successfully indexed File-types was "*.bin" (Which is a plain Text Log file With time stamped lines) and Source Filed Was as Follows:
KYPD_GSD_OFFICE_2018_04_22_16_06.zip:.\ThisIsaSample Folder Logs/Documents and Settings/All Users/Application Data/setupdownloader.1524402598.bdinstall.bin
and one of the Unsuccessful Ones were "*.XML" (a Typical XML File with no time Stamps) and source Filed Was as Follows :-
KYPD_GSD_OFFICE_2018_04_22_16_06.zip:.\ThisIsaSample Folder Logs/output.xml
I tried the Flowing Props.conf Settings on System/Local Folder:-
[KYPD_XML]
LINE_BREAKER = (<\?\w++.*\?>)|<\/\w+>(\s*)|\/>(\s*)|<\w+>(\s*)<\w+
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = true
category = MachineLogs
disabled = false
pulldown_type = true
SHOULD_LINEMERGE = true
MAX_EVENTS = 9999999
TRUNCATE = 0
KV_MODE = none
BREAK_ONLY_BEFORE = (?!)
#LEARN_MODEL = false
#=================================================================
[source::(?i)KYPD_[\w\-]+_[\d_]+[.]zip[:].[\\\/]ThisIsaSample Folder Logs[\\\/]output.xml]
sourcetype = KYPD_XML
priority = 100
#=================================================================
[KYPD_TXTLog]
DATETIME_CONFIG = CURRENT
category = MachineLogs
disabled = false
pulldown_type = true
SHOULD_LINEMERGE = false
TRUNCATE = 0
LEARN_MODEL = false
LEARN_SOURCETYPE = false
#=================================================================
[source::(?i)KYPD_[\w\-]+_[\d_]+[.]zip[:].[\\\/]ThisIsaSample Folder Logs...(.txt|.bdx|.log|.log.1|.bin|.dbg|.dbg.old|_debug.txt(.old)?)]
sourcetype = KYPD_TXTLog
priority = 99
I tried Also the Following But Still the XML file "output.xml" Get "unknown1" as a Source type:-
[KYPD_XML]
LINE_BREAKER = (<\?\w++.*\?>)|<\/\w+>(\s*)|\/>(\s*)|<\w+>(\s*)<\w+
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = true
category = MachineLogs
disabled = false
pulldown_type = true
SHOULD_LINEMERGE = true
MAX_EVENTS = 9999999
TRUNCATE = 0
KV_MODE = none
BREAK_ONLY_BEFORE = (?!)
#LEARN_MODEL = false
#=================================================================
[source::…output.xml] # also tried ([source::*output.xml], [source::*.zip[:][.]…output.xml], [source::*.zip[:][.]\\…/output.xml] and [source::*.zip[:][.]\\ThisIsaSample Folder Logs/output.xml] )
sourcetype = KYPD_XML
priority = 100
to be Honest I'm about To Give up the whole Idea...
I tried Many things but I cannot Understand Why the XML file is not Getting the Sourcetype Automatically...
I Appreciate if you Can Tell Me if I'm Missing Something Here..
... View more