Splunk Search

How to dynamically extract fields from Events(without internal fields like source,sourcetype,host etc..) and pipe it as fieldlist to table command.

jackson1990
Path Finder

I need to create table with fields present in Events result,excluding internal fields.

Example:

  • Indexed Data: A=xxx B=yyy C=zzz A=fhf B=vvv C=bbb A=xer B=yqw C=zui D=yuu E=hgjhg A=clk B=viu C=bfg F=hh G=hjkh A=asd B=viu C=mnnmn F=hj G=yuyu A=rtr B=viu C=bfg F=hh G=hjkh H=ghg

Scenario 1:

  • SearchResult:

A=xxx B=yyy C=zzz
A=fhf B=vvv C=bbb

  • Expected Table:

    A B C
    xxx yyy zzz
    fhf vvv bbb

Scenario 2:

  • SearchResult:

A=xer B=yqw C=zui D=yuu E=hgjhg

  • Expected Table:

    A B C D E
    xer yqw zui yuu hgjhg

Scenario 3:

  • SearchResult:
    A=clk B=viu C=bfg F=hh G=hjkh
    A=asd B=viu C=mnnmn F=hj G=yuyu

  • Expected Table:

    A B C F G
    clk viu bfg hh hjkh
    asd viu mnnmn hj yuyu

I tried the below command to exclude internal fields and display the event fields alone in table

table * | fields - *Time source* host _raw date_* index linecount punct eventtype time*pos splunk_server _time 

But the result i am getting(scenario 3) is

A     B   C      D           E         F    G        H
clk viu bfg   (empty)       (empty) hh  hjkh    (empty)
asd viu mnnmn   (empty)     (empty) hj  yuyu    (empty)

The same case applies for other above mentioned scenarios too.
Is it possible to remove empty fields,showing only the fields with values.Also the empty fields shouldn't be filled with null or any other static values.
or Is there any possibility to extract the fields(excluding internal fields) alone from events,so that it can be piped to table command as fieldlist.

Restrictions:
The fields in event are not static.It changes dynamically depending on the logs getting forwarded.
Fieldlist to table command shouldn't be specified in static way(like table A,B,C or table Q,A,Z,X).
It has to be generated through some other commands.

Thanks in advance...

Tags (3)
0 Karma

capnjosh
Explorer

You have to filter down to only the specific event types you want before your do any automatic field extractions or "table *" commands.

I've hit this problem myself, and you have to prefilter.

If you have to filter on extracted fields (or if you have a kv_mode=json in your props.conf), you could remove all but the _raw field and then re-extract all the fields; when you do "table *" after that you should see only fields that actually show up in the results.

0 Karma

stephane_cyrill
Builder

Hi Jackson,
To fill an empty field with null or other value you can use the commande fillnull

... | fillnull value=NULL
you can also specified the fields to work on if you like
... | fillnull value=NULL fieldx fieldy .....
you can replace NULL by a string of your choise

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...