Splunk Search

How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90
Engager

Hi,
I am trying to filter the log event based on a json field which is empty. I have 3 million records and out of which 2 are having those field empty which I am trying to extract log for.

The json looks like this:

 "third": [
    {
      "ad": {
        "dd": "aaa",
        "value": "",                           <-----------this is the field which I want to search on when its empty
        "version": 1,
        "do": "bbb"
      },

So in this case how to search that kind of log?

0 Karma
1 Solution

FrankVl
Ultra Champion

Assuming you already have json extraction working in general. Something like this should work (I believe the field will be missing when there is no value for it in the json):

index="foo" sourcetype="bar" NOT third.ad.value=*

So basically just search for NOT <fieldname>!=*.

Unless the automatic json extraction actually does extract that event with a value of empty string, then you could search for it like this:

index="foo" sourcetype="bar" third.ad.value=""

View solution in original post

FrankVl
Ultra Champion

Assuming you already have json extraction working in general. Something like this should work (I believe the field will be missing when there is no value for it in the json):

index="foo" sourcetype="bar" NOT third.ad.value=*

So basically just search for NOT <fieldname>!=*.

Unless the automatic json extraction actually does extract that event with a value of empty string, then you could search for it like this:

index="foo" sourcetype="bar" third.ad.value=""

mayurk90
Engager

Also, I am using splunk web so don't know how to turn json extraction on.

0 Karma

mayurk90
Engager

I was going through some more details since I am new on splunk and my datasource shows as logstash.

0 Karma

FrankVl
Ultra Champion

Automatic JSON extractions should be enabled by default, but perhaps the specific sourcetype you assigned (or splunk chose to assign) has it disabled for some reason.

Can you provide some more details on how you got this data into splunk and perhaps some relevant screenshots showing the data, sourcetype value, which fields get extracted and such?

0 Karma

mayurk90
Engager

Basically my data is in the format like below:

@timestamp: 2019-06-19T12:32:23.591Z

@version: 1

app_id: 90333

host: fgfjfjfj00053141.server.net
message: 2019-06-19 08:32:23,373 INFO [AMPS Java Client Background Reader Thread ***] "third": [
{
"ad": {
"dd": "aaa",
"value": "", <-----------this is the field which I want to search on when its empty
"version": 1,
"do": "bbb"
}]
app_id = 90333 host = fgfjfjfj00053141.server.net host =hdhdhdhd.net

source = /apps/uat01/logs/abc-logger.log

sourcetype = logstash

So the json data is part of value field of message key field and from there I want to check if the field is empty then show me those events having empty field.

0 Karma

FrankVl
Ultra Champion

Ok, so the event is not fully json. Which means automatic json extractions won't work.

Did you do any field extraction for that field yet? Or do you at the moment only have a raw event in splunk?

0 Karma

mayurk90
Engager

I just have raw event in splunk for now

0 Karma

FrankVl
Ultra Champion

Then just add "\"value\": \"\"" to your search query.

So for example:

index="foo" sourcetype="bar" "\"value\": \"\""
0 Karma

mayurk90
Engager

My logs are In Json only but I don't understand how to activate json extraction?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...