Getting Data In

How to extract fields from my log when every record in the log has different position of the field?

varunchhabra
New Member

I have a log file that contains time stamped events. The type of action done is defined by the string parameter like : FunctionCalled.
I want to extract the action field and use it. But the problem is that the keywords are not following any pattern. It could be like:

TimeStamp1 FunctionCalled
TimeStamp2 ABC::CDE<>FunctionCalledAgain

In such case, the neither the regex nor delimited pattern helping me to get the field.

On the side note, Is it necessary for the log to be ordered so as to apply generic statistical functions?

Plz help.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi varunchhabra,
if your logs have a limited number of types you can build one extraction for each situation (FunctionCalled1, FunctionCalled2, FunctionCalledn) and then configure a calculated field as a coalesce field or an eval command
| eval FunctionCalled=coalesce(FunctionCalled1, FunctionCalled2,...,FunctionCalledn).

Bye.
Giuseppe

0 Karma

niketn
Legend

How many of above patterns may exist? If there is no way for you to come up with regular expression,

Option 1
You create separate Field Extractions for FunctionalCalled say FunctionalCalled1 and FunctionalCalled2 etc and then use the following in your query:

<YourBaseSearch> FunctionalCalled1="FunctionalCalled" OR FunctionalCalled2="FunctionalCalled"

Option 2
If you do not want to perform field extraction then in base search you can use "FunctionalCalled" and in your subsequent pipe you can use searchmatch to create your own field with eval

| eval FunctionalCalled1=case(searchmatch("FunctionalCalled"),"FunctionalCalled",true(),"Other")

Ideally, interesting fields should be a key value pair, you should also read Logging best practices in order to better utilize Splunk's capabilities: http://dev.splunk.com/view/logging/SP-CAAAFCK

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...