Splunk Search

How to find the origin and regular expression of an extracted field I use in Splunk?

skoelpin
SplunkTrust
SplunkTrust

We have a field called Response_Size which we cannot find. I looked in the Settings>Fields>Field Extractions and selected ALL for App context and Any for Owner and still cannot find the field. I also looked in Calculated fields with no luck. The field is active in Splunk and I'm able to use the field, but cannot find it's origin and the regular expression that extracts it. Any ideas how I can find it?

0 Karma
1 Solution

skoelpin
SplunkTrust
SplunkTrust

Thanks for this. I looked in splunk/etc/apps/endeca/localand found the field in transforms.conf..

After opening transforms.conf I saw this stanza which contains the field i'm looking for (Response_Size)

[requestLogFields]
FIELDS = "Timestamp", "Client_IP", "Agraph_Transaction_ID", "HTTP_Exchange_ID", "Response_Size", "Total_Request_Time", "Total_Processing_Time", "HTTP_Status_Code"
DELIMS = " "

I then opened the props.conf file and saw

REPORT-requestLogFields = requestLogFields

I don't see an inputs.conf in this directory where the regex is defined. I'm also using the Endeca app so I suspect the regex was defined by the guy who made the app.. Any idea where I can find it? We're getting close!

View solution in original post

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Thanks for this. I looked in splunk/etc/apps/endeca/localand found the field in transforms.conf..

After opening transforms.conf I saw this stanza which contains the field i'm looking for (Response_Size)

[requestLogFields]
FIELDS = "Timestamp", "Client_IP", "Agraph_Transaction_ID", "HTTP_Exchange_ID", "Response_Size", "Total_Request_Time", "Total_Processing_Time", "HTTP_Status_Code"
DELIMS = " "

I then opened the props.conf file and saw

REPORT-requestLogFields = requestLogFields

I don't see an inputs.conf in this directory where the regex is defined. I'm also using the Endeca app so I suspect the regex was defined by the guy who made the app.. Any idea where I can find it? We're getting close!

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The field extraction has nothing to do with inputs.conf. Based on your transforms.conf entry, there is no regex used, instead space is used a delimiter to extract fields. Why you do you need to regex for this field (based on your transforms.conf, your can try this ^(\S+\s){4}(?<Response_Size>\S+)for the field)?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

We recently upgraded our ATG framework and after upgrading, our Response_Size is 10 times smaller according to Splunk which is not believable. I want to find the Response_Size field so I can see how it's extracted to find the root cause and fix it. So do you think it would be a good idea to extract the field rather than delim it by a space?

If I did this then I guess it would be a dead give away that deliming it by a space was the issue

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Well, it will depend on if the log format has changed after ATG framework upgrade. If this space delimited field extraction was working/correct earlier and if the log format has not changed, then the value should be correct only.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

It could be extracted automatically. What is the sourcetype? Can you share a sample event?

---
If this reply helps you, Karma would be appreciated.
0 Karma

deepthi5
Path Finder

Hello skoelpin

You can see the extracted fields in the app folder where ever the extraction is done for example app\search\local\props.conf file and inputs.conf u can find your extracted field and its regular expression

Example:

[XXXXXXXXX (Sourcetype)]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = true
EXTRACT-server = ^(?P\d+).*\s+msg\s+execution
EXTRACT-Msg_Time = ^[^\]\n]*\]\s+msg+\s+execution\s+time+\s+=\s+(?P.+)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...