Getting Data In

Collect command to index in JSON format

alvaromari83
Path Finder

Hello!

I'm trying to generate some summarized data by using the collect command in my SPL queries.

The event format by default is

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2

Some of this fields, are actually JSON objects, like:

05/20/2019 07:15:00 +0200, info_min_time=1.000, info_max_time=1558557310.000, info_search_time=1558557310.410, field1=value1 field2=value2 field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"

So, to avois a mixture of formats in my collected event, I would like to index the results, using the collect command, BUT in JSON format for the whole event, something like:

{"time":"05/20/2019 07:15:00 +0200", "info_min_time"=1.000, "info_max_time":1558557310.000, "info_search_time":1558557310.410, results: {"field1":"value1" "field2":"value2", field3="{"subfield1":"subvalue1", "subfield2":"subvalue2",...}"...}}

Is such thing feasible? Some props tweaking?

Thank you in advance!

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@alvaromari83

Can you please try | mkjson command from JSON Tools Splunk App ?

https://splunkbase.splunk.com/app/3540/#/overview

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust
0 Karma

koshyk
Super Champion

You can look into it from two perspective
1. Is the source data the most important thing in your organisation?
2. If the data format is important

In many organisation (1) is more important. This is to preserve the original data. Splunk doesn't have any issue in extracting data even if the JSON format is mixed. You index "original" data, but just mention the json part to extract automatically

Please see this thread on how to index json mixed format. https://answers.splunk.com/answers/117121/extract-json-data-within-the-logs-json-mixed-with.html

You could even add a stanza just to extract pure JSON if need be at search time. This way you preserve the original data as well as you get key-value at search-time

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...