Getting Data In

How do I split an array in a .json file into multiple new events?

dgutekunst
Engager

Right now I have a json file that's formatted like:

{
    "Log Files":[
        {"Date":"2014-07-18 21:22:51", "Available Bytes(kb)":3960078, ...},
        {"Date":"2014-07-18 21:24:01", "Available Bytes(kb)":4001231, ...},
        {"Date":"2014-07-18 21:25:14", "Available Bytes(kb)":3872959, ...}]}

Right now it's showing up in Splunk as:

timestamp                Date                Available Bytes(kb)
2014-07-18 21:22:51:000  2014-07-18 21:22:51 3960078
                         2014-07-18 21:24:01 4001231
                         2014-07-18 21:25:14 3872959

How can I split these up into individual events when I load the data? I can get the timestamp to correctly match the Date field, but it will still only give one date for the whole file, even though there are several lines that are each individual logs.

Tags (3)

vasanthmss
Motivator

Try something like this,

<your base search...> | table  timestamp, Date, Available | eval temp=mvzip(timestamp, mvzip(Date, Available,"###"), "###") | mvexpand temp | rex field=temp "(?<timestamp>.*)###(?<Date>.*)###(?<Available>.*)" | fields - temp

I have used the sample fields, you can try with your actual fields. Concept here is, you need to zip it with a delimiter (here ###) and expand it and extract it. This do the magic

V

s2_splunk
Splunk Employee
Splunk Employee

Is there anything in the envelope of this array that you want to keep or are you just interested in keeping the events inside the "Log Files" array?
If the latter, setup props/transforms for your sourcetype to:

  • Get rid of the root '{' line
  • Get rid of the array root line that contains "Log Files"
  • Use SEDCMD to change "}," to "}" at the end of the lines
  • Use SEDCMD to remove "]}"
  • Set TIMESTAMP_PREFIX to \{\"Date\":\"

That should result in individual, valid JSON events that should render fine in the UI.

0 Karma

pwmcintyre
Explorer

When you say "... when I load the data" do you mean at search time or index time?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...