Splunk Search

Send events via log4j raw messages

bowesmana
SplunkTrust
SplunkTrust

I have an existing app that writes log4j messages as CSV lines using a File appender, and then use the Splunk UF to forward that data to Splunk.

I want to just change that to use the HEC, so I've enabled HEC in my Splunk and have added

<Http name="http"
          url="https://localhost:8088/services/collector/raw"
          token="TOKEN"
          index="test"
          source="${logRoot}/${date:yyyy-MM-dd}/${seriesName}/${testName}.${runNumber}/instrument.log"
          sourcetype="instrument">
         <PatternLayout pattern="%m" />
 </Http>

 <Logger name="instrument" level="info" additivity="false">
     <AppenderRef ref="http" />
 </Logger>

I have the splunk logging jar present in the classpath.

However, nothing is getting sent to Splunk. I can see there are HTTP connections being made to 8088, but nothing is logged.

I suspect that is because I am not using a channel GUID for raw as described here and that the collector is returning an error of some sort, which I can't see logged anywhere.

however, I can't find out how to set that up in the log4j configuration.

I can use curl to send JSON data to the services/collector endpoint, so the issue is gluing together log4j config and raw

Using log4j 2.8.1 directly - not SLF4J. I don't want to rewrite any event data or change code, just replace the File Appender with the HEC.

Any suggestions?
Thanks

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Hi, it was resolved as described above, but I didn't log raw, just the JSON.

0 Karma

priti123
New Member

Hi,
I tried the config the way you have mentioned above for
but it was not identified as it was changed to SplunkHttp in the sourcecode.
Even with SplunkHttp i am not able to send the logs to Splunk.

Can you please share your configuration again?
Also if you can let me know which version of log4j2 and splunk-library-logging are you using ?

0 Karma

jamessevenerlmc
New Member

Hello,

Did you ever resolve this? This is very similar to what I'm trying to do, and I'm having issues as well. I'm not able to get data into splunk at all. Yet, I'm not able to find any errors logged anywhere.

-James

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Mmm, I guess I was confused by the statement

Though HTTP Event Collector accepts only JSON-formatted event data packets, the event data payload can be in any format you want, as long as it is surrounded by curly brackets.

from the documentation, as HEC supports raw data too. I removed the path from the url and now I get JSON data in my index, but the message properties elements are odd in that the message contains all the csv key value pairs, e.g.

message:     i_gid="T Walker",i_vu=1,i_chn=wha,i_hostset=prod,i_sid=walker,i_it=1,j=NSW,mn=NAME1,dt=2018-01-02,rt=R,rn="NAME2",rnum=8,rrn="NAME3",rrnum=16,rfxw=101.0,rfxp=16.8,rpmw=32.0,rpmp=11.0,i_tx=Runner,i_status=0  

but the properties only contains

properties: {   [-] 
         i_chn:  wha    
         i_gid:  T Walker   
         i_hostset:  prod   
         i_it:   1  
         i_sid:  walker 
         i_vu:   1  
    }   

I don't understand why it's only got some of the KV pairs as properties and not all of them

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Further digging - the properties are recorded from data placed in ThreadContext.put(x,y), whereas the message is the raw string message I am logging.

Unless it's possible to send raw data, it means I have to do some work to switch over the app config to extract the CSV embedded KV pairs or rewrite the message to put the kv pairs into JSON.

Anyone know if the default Splunk logging library can do raw?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...