All Apps and Add-ons

Formatting log fields containing = in the values

dharkness9549
New Member

I am writing guidance for developers to use when writing logs for internal applications. We log to syslog and push to splunk. One field I would like to include in our logs involves dynamically generated sql queries. An example field is:

query="SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5"

The thought was that by double quoting the entire value I would get a key-value pair of:

key = query
value = SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5

While it DOES extract this key-value pair, it also auto-extracts:

key = t1_foreign_key
value = t2.key

and

key=WHERE_t1_field1
value=5

Is there a way to make it ignore the "=" sign inside of the values? I had assumed that placing the whole string inside quotes would "protect" it, but it seems that was a bad assumption. Are there other characters that I need to watch out for (most of the queries are much more complex than this example)?

Thank You

0 Karma

dharkness9549
New Member

For clarification, the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

Additionally, while my example involves a sql query, the question is broadly "How should I format field values that contain the = character, such that Splunk won't try to auto extract from the middle of the value?"

0 Karma

neelamssantosh
Contributor

in sql we have a |trim() That can help you,
http://www.sqlite.org/lang_corefunc.html.
Hope it can help you.

0 Karma

dharkness9549
New Member

Unfortunately, I don't think this will help. The question is about the structuring of custom logs, and is applicable to any field value. The example that I gave is specific to logging a sql query, but really I just want to know about ANY value where an "=" character is present.

Just to be clear the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...