Splunk Search

What is the location of the delimiter based field extraction definitions, in newest version of splunk?

richardAtOmni
Path Finder

Hi,

Can someone please point me to where the delimiter based field extraction definitions are now stored in Splunk configuration files?

Previously, we could see that these were stored in transforms.conf. I can see that the extractions we built before splunk was upgraded are there. (This is located in C:\Program Files\Splunk\etc\apps\search\local".

After upgrading, we defined new field extractions for a new source type. They are working, but they don't appear in transforms.conf. I've done a text search on the whole machine and can't seem to find where the new definition is stored.

I need to get these extractions so that I can export them to our production Splunk Cloud instance. Any assistance would be greatly appreciated.

Thanks,
Richard

0 Karma
1 Solution

micahkemp
Champion

Check also in etc/system/local, and if you don’t see them there check the output of:

splunk btool props list <sourcetype name> --debug

And:

splunk btook transforms list <transform name> --debug

View solution in original post

0 Karma

micahkemp
Champion

Check also in etc/system/local, and if you don’t see them there check the output of:

splunk btool props list <sourcetype name> --debug

And:

splunk btook transforms list <transform name> --debug
0 Karma

richardAtOmni
Path Finder

I checked etc/system/local but there isn't even a transforms.conf file there.

I ran the first command you specified and it lists out the props - basically everything I can see in the props.conf file.

Looks something like this:

[RivaActCpt4]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
EXTRACT-crmType = ^(?:[^|]*\|){2}.*?\[(?P<crmType>\w+)
EXTRACT-cycleId = ^[^\.\n]*\.\w+\|(?P<cycleId>[^\|]+)
EXTRACT-mailType = ^(?:[^|]*\|){2}.*?\[\w+\+(?P<mailType>\w+)
EXTRACT-node = ^[^\)\n]*\)\s+\[(?P<node>[^\]]+)
EXTRACT-pod = ^[^ \n]* \((?P<pod>[^\)]+)
ion>[^,]+), total runtime:\s+(?P<runtime>[^\|]+)
FIELDALIAS-RivaActCpt4 cycleId to referenceId = cycleId AS referenceId
HEADER_MODE =
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
LOOKUP-auto_crmobjecttypelookup = CrmObjectTypeLookup crmObjectTypeId AS crmObj
ctTypeId OUTPUTNEW crmObjectType AS crmObjectType
LOOKUP-auto_eventsubtypelookup = EventSubTypeLookup eventSubTypeId AS eventSubT
peId OUTPUTNEW eventSubType AS eventSubType
LOOKUP-auto_eventtypellookup = EventTypeLookup eventTypeId AS eventTypeId OUTPU
NEW eventType AS eventType
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
REPORT-delimited-RivaActCpt4 = REPORT-delimited-RivaActCpt4
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 0
detect_trailing_nulls = auto
maxDist = 100
priority =
sourcetype =

Running the second command just outputs a blank line.

Any other suggestions?

0 Karma

micahkemp
Champion

I edited my answer, because my phone changed -- to a single character, which won't work.

Try the first command again, ensuring --debug is present. This will show you the file that caused each line to be present.

0 Karma

richardAtOmni
Path Finder

Thanks. That does show me all the lines and the source files. But I'm still missing the transform that's associcated with my "delimited field extraction". The extraction is named REPORT-delimited-RivaActCp4, but I can't find the transforms that this uses anywhere. It's supposed to define the delimiter character and each of the field names. I can see that the extraction is working because all the fields are available in Splunk. But I need to extract the actual transform from the file system.

This is the output to the command with the double dashes:

C:\Program Files\Splunk\etc\apps\search\local\props.conf [RivaActCpt4]
C:\Program Files\Splunk\etc\system\default\props.conf    ADD_EXTRA_TIME_FIELDS =
 True
C:\Program Files\Splunk\etc\system\default\props.conf    ANNOTATE_PUNCT = True
C:\Program Files\Splunk\etc\system\default\props.conf    AUTO_KV_JSON = true
C:\Program Files\Splunk\etc\system\default\props.conf    BREAK_ONLY_BEFORE =
C:\Program Files\Splunk\etc\system\default\props.conf    BREAK_ONLY_BEFORE_DATE
= True
C:\Program Files\Splunk\etc\system\default\props.conf    CHARSET = AUTO
C:\Program Files\Splunk\etc\system\default\props.conf    DATETIME_CONFIG = \etc\
datetime.xml
C:\Program Files\Splunk\etc\system\default\props.conf    DEPTH_LIMIT = 1000
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-crmType = ^(?:[
^|]*\|){2}.*?\[(?P<crmType>\w+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-cycleId = ^[^\.
\n]*\.\w+\|(?P<cycleId>[^\|]+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-mailType = ^(?:
[^|]*\|){2}.*?\[\w+\+(?P<mailType>\w+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-node = ^[^\)\n]
*\)\s+\[(?P<node>[^\]]+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-pod = ^[^ \n]*
\((?P<pod>[^\)]+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf EXTRACT-version,runtime
 = ^(?:[^|]*\|){6}1\|17\|Agent running - version:(?P<version>[^,]+), total runti
me:\s+(?P<runtime>[^\|]+)
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELDALIAS-RivaActCpt4
cycleId to referenceId = cycleId AS referenceId
C:\Program Files\Splunk\etc\system\default\props.conf    HEADER_MODE =
C:\Program Files\Splunk\etc\system\default\props.conf    LEARN_MODEL = true
C:\Program Files\Splunk\etc\system\default\props.conf    LEARN_SOURCETYPE = true

C:\Program Files\Splunk\etc\system\default\props.conf    LINE_BREAKER_LOOKBEHIND
 = 100
C:\Program Files\Splunk\etc\apps\search\local\props.conf LOOKUP-auto_crmobjectty
pelookup = CrmObjectTypeLookup crmObjectTypeId AS crmObjectTypeId OUTPUTNEW crmO
bjectType AS crmObjectType
C:\Program Files\Splunk\etc\apps\search\local\props.conf LOOKUP-auto_eventsubtyp
elookup = EventSubTypeLookup eventSubTypeId AS eventSubTypeId OUTPUTNEW eventSub
Type AS eventSubType
C:\Program Files\Splunk\etc\apps\search\local\props.conf LOOKUP-auto_eventtypell
ookup = EventTypeLookup eventTypeId AS eventTypeId OUTPUTNEW eventType AS eventT
ype
C:\Program Files\Splunk\etc\system\default\props.conf    MATCH_LIMIT = 100000
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DAYS_AGO = 2000
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DAYS_HENCE = 2
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DIFF_SECS_AGO = 360
0
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DIFF_SECS_HENCE = 6
04800
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_EVENTS = 256
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_TIMESTAMP_LOOKAHEAD
 = 128
C:\Program Files\Splunk\etc\system\default\props.conf    MUST_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf    MUST_NOT_BREAK_AFTER =

C:\Program Files\Splunk\etc\system\default\props.conf    MUST_NOT_BREAK_BEFORE =

C:\Program Files\Splunk\etc\apps\search\local\props.conf REPORT-delimited-RivaAc
tCpt4 = REPORT-delimited-RivaActCpt4
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION = indexing

C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-all = full

C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-inner = in
ner
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-outer = ou
ter
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-raw = none

C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-standard =
 standard
C:\Program Files\Splunk\etc\system\default\props.conf    SHOULD_LINEMERGE = True

C:\Program Files\Splunk\etc\system\default\props.conf    TRANSFORMS =
C:\Program Files\Splunk\etc\system\default\props.conf    TRUNCATE = 0
C:\Program Files\Splunk\etc\system\default\props.conf    detect_trailing_nulls =
 auto
C:\Program Files\Splunk\etc\system\default\props.conf    maxDist = 100
C:\Program Files\Splunk\etc\system\default\props.conf    priority =
C:\Program Files\Splunk\etc\system\default\props.conf    sourcetype =

Any other ideas?

0 Karma

micahkemp
Champion

Try:

splunk btool transforms list REPORT-delimited-RivaActCpt4 --debug

To me that transform name looks wrong, but it's what's configured according to btool. Is there any chance your EXTRACT- configs are populating your data instead of that REPORT-?

richardAtOmni
Path Finder

We recreated it with a different transform name and also changed some other stuff, and it showed up. Not sure what fixed it.

Thanks for your help!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...