Getting Data In

How to extract fields from my log when every record in the log has different position of the field?

varunchhabra
New Member

I have a log file that contains time stamped events. The type of action done is defined by the string parameter like : FunctionCalled.
I want to extract the action field and use it. But the problem is that the keywords are not following any pattern. It could be like:

TimeStamp1 FunctionCalled
TimeStamp2 ABC::CDE<>FunctionCalledAgain

In such case, the neither the regex nor delimited pattern helping me to get the field.

On the side note, Is it necessary for the log to be ordered so as to apply generic statistical functions?

Plz help.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi varunchhabra,
if your logs have a limited number of types you can build one extraction for each situation (FunctionCalled1, FunctionCalled2, FunctionCalledn) and then configure a calculated field as a coalesce field or an eval command
| eval FunctionCalled=coalesce(FunctionCalled1, FunctionCalled2,...,FunctionCalledn).

Bye.
Giuseppe

0 Karma

niketn
Legend

How many of above patterns may exist? If there is no way for you to come up with regular expression,

Option 1
You create separate Field Extractions for FunctionalCalled say FunctionalCalled1 and FunctionalCalled2 etc and then use the following in your query:

<YourBaseSearch> FunctionalCalled1="FunctionalCalled" OR FunctionalCalled2="FunctionalCalled"

Option 2
If you do not want to perform field extraction then in base search you can use "FunctionalCalled" and in your subsequent pipe you can use searchmatch to create your own field with eval

| eval FunctionalCalled1=case(searchmatch("FunctionalCalled"),"FunctionalCalled",true(),"Other")

Ideally, interesting fields should be a key value pair, you should also read Logging best practices in order to better utilize Splunk's capabilities: http://dev.splunk.com/view/logging/SP-CAAAFCK

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...