Getting Data In

How to extract multiple fields using the Extract property in props.conf?

greggz
Communicator

So I'm trying to extract multiple fields using the Extract property in props.conf
The source file looks like

my.prop.1=1
my.prop.2=2
my.prop.3=3
my.prop.4=4

And I want what EACH prop becomes a field. HENCE I did not used a prefix in the regex.

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
EXTRACT-watt_grouping = (?m)(.+?)=(.*)

I already used multiple variations of this regex and nothing seems to work. Is there anything blocking the field extraction ?

EDIT:
At search time I provided the conrresponding pipe and it seems to extract quite nicely: | extract pairdelim=",", kvdelim="=", auto=f, limit=500, mv_add=t .. Anyone knows how to make this work in transforms.conf ? I tried it with DELIMS = "\n","=" but it breaks in a specific property prop.256 = ?'- \#&@^\!%*\:$./\;,~+=)(|}{][><` .. It extracts the previous 255 props quite nicely though

0 Karma

masonmorales
Influencer

Try this instead:

props.conf

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
REPORT-my_fields = multi_extract

transforms.conf

[multi_extract]
REGEX = (?m)(.+?)=(.*)
FORMAT = $1::$2
MV_ADD = true
0 Karma

greggz
Communicator

@masonmorales can you provide support to your own answer based on the feedback I gave you ?

0 Karma

masonmorales
Influencer

Are there always only 4 lines? If so you could just use 4 extract statements that are more specific.

0 Karma

greggz
Communicator

No.. they're above 400. This is the reason I don't use KV_MODE=AUTO because it has a 100 limit max extractions. The can be alphanumerical aswell or have only symbols

0 Karma

greggz
Communicator

This doesnt help.. It just clumps them all together. I want each line to become a new field with a given value.

0 Karma

greggz
Communicator

It just makes my.prop.1 = "1 my.prop.2 = 2 my.prop.3 = 3 ..."

0 Karma

greggz
Communicator

Thanks, let me try it out

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Your source, is it one line per event or all those properties appear in same event? What are the fields that are automatically extracted by Splunk?

0 Karma

greggz
Communicator

None, because I made it KV_MODE=none. No, All lines come in the same event. I purposedly made it so with the LINE_BREAKER = ((?!))

0 Karma

greggz
Communicator

@somesoni2 ♦ Do you know how this should be done ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...