Getting Data In

Can't load my custom fields

apare
Engager

I try to index my logs with a custom Source type. Here is my files :

Log file exemple :

#HeaderKey1: header value 1
#HeaderKey2: header value 2
id|severity|date|time|duration|description|uri|operation|collection|lang|format|size|source|errorcode|links|badlinks|collectionid|sourceid|printableuri
...

transforms.conf :

[my_log]
DELIMS="|"
FIELDS = "id", "severity", "date", "time", "duration", "description", "uri", "operation", "collection", "lang", "format", "size", "source", "errorcode", "links", "badlinks", "collectionid", "sourceid", "printableuri"

[my_log_comment]
REGEX = ^#
DEST_KEY = queue
FORMAT = nullQueue

props.conf :

[my_log]
EXTRACT-my_log = my_log_comment, my_log

The result is that all the line are indexed (header include) and the only field i got is _raw... I do not care about the information in the header; I don't want to index it or to take information from it. I only want to have all my log with all the fields.

What is wrong in my configuration? What i should do to index my custom log file correctly

thank you

Tags (1)
0 Karma
1 Solution

Ayn
Legend

TRANSFORMS is for index-time creation of fields and doesn't support extracting fields using delimiters like you want to. bojanz is right however in that you shouldn't be using EXTRACT, as it is used for defining directly in props.conf how fields should be extracted using regexes. For extractions that use definitions in transforms.conf, you should use REPORT.

View solution in original post

Ayn
Legend

TRANSFORMS is for index-time creation of fields and doesn't support extracting fields using delimiters like you want to. bojanz is right however in that you shouldn't be using EXTRACT, as it is used for defining directly in props.conf how fields should be extracted using regexes. For extractions that use definitions in transforms.conf, you should use REPORT.

apare
Engager

Thank you
there is my new props.conf
[ces_index_log]
EXTRACT-ces_index_log = (?\d*)|(?\w*)|(?\d{4}-\d{2}-\d{2})|(?

0 Karma

bojanz
Communicator

Your props.conf file is wrong. It's not EXTRACT but TRANSFORMS.
The rest looks ok.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...