Getting Data In

How to add additional fields to log4j-submitted JSON?

unitedmarsupial
Path Finder

We're using Splunk's "javalogging" JAR to send events to Splunk from our Java-application directly. This works, but the event's JSON received by the server has only two fields: message and severity (plus the host, index, source, and sourcetype of course):

{   
    "event": {
      "message": "This is a test 20:41:34",
      "severity": "ERROR"
    },
    "host" = "myhost",
    "source" = "mysource",
    "sourcetype" = "logj4"
}

We'd like to add some others -- to help us identify the application and the application instance, et cætera. Can this be done via configuration file -- without recompiling the code?

I realize, we can use a pattern-layout to prepend the additional data to the message:

  <param name="ConversionPattern" value="FIELD1=VALUE1 FIELD2=VALUE2 %m"/>

but then the extra fields would simply be part of the message -- not separately-indexed fields of their own...

Tags (2)
0 Karma

unitedmarsupial
Path Finder

Ok, there are two options here, which can also be combined:

  1. Set the includeMDC parameter to true and hope (or provide for), the MDC in your case contains all of the fields you need — they will be in the properties sub-dictionary of every logged event.
  2. Set the messageFormat parameter to json -- and format your message to be in proper JSON itself:



    value="{"cat": "meow", "message": "%m"}"/>

    The message field of the submitted event will then itself be a dictionary. In the above example, that sub-dictionary will contain two fields: cat and message. These can be searched for on Splunk-server as message.cat="meow".

I'd still like to be able to add additional fields next to the message and the severity, though -- not inside a sub-dictionary...

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...