Splunk Search

Filter based on Null or blank or whitespace value....

wwhitener
Communicator

Good afternoon all,

As a relative noob to Splunk searching, I have a relatively easy (I hope) question.

I have a Splunk box that is dedicated to testing and as such will have periods of no information coming in followed by periods of indexing for tests and then it goes back dormant.

Is there a way to filter out blanks out of results? For example, I have a query that looks at the traffic being indexed--and (huge surprise) there's gaps where no traffic at all was indexed. So it's null or blank or NaN or something. How do I get those records removed from the results?

Thank you in advance!

0 Karma
1 Solution

wwhitener
Communicator

I'm doing the following query:

index=_internal source=*license_usage.log | eval MB=b/1024/1024 | timechart span=1d sum(MB)

On 4.2.2.

When there's no "where" it renders fine. It's a problem with where.

View solution in original post

0 Karma

landen99
Motivator

Filter a Field that is ".. null or blank or NaN or something. How do I get those records removed from the results?"

... | regex field1!="^(?:\s*|NaN)$"
0 Karma

wwhitener
Communicator

Hey all,

I went around and around with this question--I wanted a chart of the license usage that showed the license usage that showed certain days, but not all of them.

Here's what I ended up with:

index=_internal source=*license_usage.log | eval MB=b/1024/1024 | timechart span=1d sum(MB) as SumMB | where SumMB > 500

I guess Ayn had it right...I was doing it wrong.

0 Karma

wwhitener
Communicator

I'm doing the following query:

index=_internal source=*license_usage.log | eval MB=b/1024/1024 | timechart span=1d sum(MB)

On 4.2.2.

When there's no "where" it renders fine. It's a problem with where.

0 Karma

wwhitener
Communicator

Thanks. See the answer below.

0 Karma

Ayn
Legend

Are you actually using | where sum(MB)>500? I saw in an earlier comment that you were trying | where (MB>500) which is not syntactically correct.

MHibbin
Influencer

Can you not use...

| where isnotnull(<traffic_field>)
0 Karma

wwhitener
Communicator

Hey everyone,

I was originally doing a timechart with a span of 1 day--and it appears that while my original query excluded the blank values, the timechart appears to fill in for days that are blank. I'll try with your suggestions and report back.

In short, I'm trying to put together a saved search that puts together a chart of days we've exceeded the license count so that it can be processed by another system using the CLI or REST. Something along the lines of:

DATE TRAFFIC
1/1/2001 504
1/2/2001 520
1/3/2001 502
1/10/2001 604
1/11/2001 592
4/30/2001 1129



The box is occasionally turned off for various reasons, including being moved, serviced or having hardware swapped in and out. So, I end up with values that are blank/null/something. There are also days where we don't have enough traffic to be included on the chart because I want only the values that are over the license limit.

So, my first thought was a timechart--but that ended up with a whole slew of empty or unimportant low values. And, no matter what I did, I could not get it to hide the rows with nothing in them.

So, I'll try your suggestions and see if I can get it to work that way.

Thanks all!

0 Karma

BobM
Builder

I think I understand what you want. For example if you are counting events and expect data in the range 100-200 events but some times you get zero.

sourcetype=mydata purchase | timechart span=1d count by productID | where count > 10 

The "where" will remove any days with less than 10 events and you can run any reporting on the remainder.

0 Karma

wwhitener
Communicator

Hmmm...I will need to work with this. I keep getting no results returned when I put in the where.

index=_internal source=*license_usage.log | eval MB=b/1024/1024 | timechart span=1d sum(MB) | where (MB > 500)

I'm trying to get a chart like the one I describe below.

Edited to add....

I've tried a couple of ways to get those fields out using the "where" command, but when I put it in, I get the text under the search bar saying "3 results" and then nothing in the datatable.

I don't understand that keyword apparently.

0 Karma

wwhitener
Communicator

Yes. I've got days that I have less data because there is simply not a lot of data and days with no data because the box has been shut down for maintenance or to move it. So, I don't want to see those in my timechart.

0 Karma

Takajian
Builder

I do not know which log format you are trying to index. But you can configure splunk not to index unwanted data. Please refer to the following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Routeandfilterdatad

0 Karma

MHibbin
Influencer

Do you mean, there are blank values when visualising data (i.e. in a line-chart)?

0 Karma

Ayn
Legend

A bit confused - you say you have periods when no data whatsoever is coming in, but it seems you do get some data anyway, just that it contains null type values? Could you clarify what your logs look like, some sample events of the ones you want to ignore would be of great help.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...