All Apps and Add-ons

Formatting log fields containing = in the values

dharkness9549
New Member

I am writing guidance for developers to use when writing logs for internal applications. We log to syslog and push to splunk. One field I would like to include in our logs involves dynamically generated sql queries. An example field is:

query="SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5"

The thought was that by double quoting the entire value I would get a key-value pair of:

key = query
value = SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5

While it DOES extract this key-value pair, it also auto-extracts:

key = t1_foreign_key
value = t2.key

and

key=WHERE_t1_field1
value=5

Is there a way to make it ignore the "=" sign inside of the values? I had assumed that placing the whole string inside quotes would "protect" it, but it seems that was a bad assumption. Are there other characters that I need to watch out for (most of the queries are much more complex than this example)?

Thank You

0 Karma

dharkness9549
New Member

For clarification, the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

Additionally, while my example involves a sql query, the question is broadly "How should I format field values that contain the = character, such that Splunk won't try to auto extract from the middle of the value?"

0 Karma

neelamssantosh
Contributor

in sql we have a |trim() That can help you,
http://www.sqlite.org/lang_corefunc.html.
Hope it can help you.

0 Karma

dharkness9549
New Member

Unfortunately, I don't think this will help. The question is about the structuring of custom logs, and is applicable to any field value. The example that I gave is specific to logging a sql query, but really I just want to know about ANY value where an "=" character is present.

Just to be clear the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...