Getting Data In

Delims missing field when value is empty

TiagoTLD1
Communicator

Hello

I have an event like this:

"2017-04-11 19:03:35.738","I1","0","localhost","",,,"2147479552","142176256",,,,,,,,"109","131",,,,

To get what each field is, I have a stanza in transforms.conf:

[REPORT-VM]
DELIMS=","
FIELDS=_time", "f1", "f2", ...,"f16","f22"

The problem is that the last field (after the last comma), has no character and Splunk doesn't assume it as an empty field as it does it the empty ones in the middle of the _raw data.

Any ideas how to solve this?

0 Karma

woodcock
Esteemed Legend

A field which has no value will not show up; it will not be given a value that indicates NULL. In your case it appears that sometimes some events have some values are NULL but at all times, all events have no value for F22. So when you look at bunches of events, you see F1-F21 and never F22. What you need to realize is that if you look at single events, you will also see that some individual events are missing many fields other than F22. Try this:

... ",,,,"| head 1

This will give you a single event that has at least 3 NULL values. You will see that there are at least 3 fields that DO NOT EXIST. So the problem is your (mis)understanding of what happens to fields when there is no value. In such cases, the field DOES NOT EXIST. You can check for this by adding where |isnull(UngaBunga) to any search. This always returns true.

0 Karma

TiagoTLD1
Communicator

Thanks for the explanation. Still, that same logic you mentioned should apply to ALL the fields that have no value for any of the events, and therefore shoul also NOT EXIST. But I see that F13 is exactly the same as F22, so no value for any event, and still I can see F13 it in the event but not F22....

0 Karma

woodcock
Esteemed Legend

Are you sure that there is not a blank space or tab after the last comma?
Does the field extraction fail entirely (no f1 - f22) or is it just that f21 does not exist or is not null? What makes you think that something is wrong? What do you expect and what do you really get?

0 Karma

TiagoTLD1
Communicator

There is no blank space after the last comma.

The extraction is perfect for every field except the last one. So my data has just 21 fields instead of the desired 22.

F19-F21 are shown as empty fields but F22 is just missing, it seems the behaviour is different in that last field than with the middle empty ones.

I would expect F22 to be shown and have a null value, just like F21 and other empty ones.
I am getting F1-F21, where some of them are null as expected.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...