Splunk Search

What is a good approach to extract fields from a single event that captures a structured data table?

wcooper003
Communicator

We have events coming in from stdout, such as the top command, where a single event captures a multi-line structured data output, e.g., this is a single Splunk event:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11981 root      20   0 2121m 860m 6996 S  0.3  1.8  36:17.82 python
12149 root      20   0 19.1g 1.0g 6556 S  0.3  2.2  45:00.03 java
13744 root      20   0 4959m 207m 5676 S  0.3  0.4  22:26.91 java
    1 root      20   0 19364 1232 1064 S  0.0  0.0   3:43.65 init

What is a good approach to do field extractions on this type of data, where a single event is a structured data table?

Thanks

0 Karma
1 Solution

bmacias84
Champion

You will want to use multikv. You will see Splunk using this in app like *nix.

https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Multikv
http://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/Multikvconf

I you may need to edit the multikv.conf file for your data, but should be relatively simple.

Cheers

View solution in original post

bmacias84
Champion

You will want to use multikv. You will see Splunk using this in app like *nix.

https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Multikv
http://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/Multikvconf

I you may need to edit the multikv.conf file for your data, but should be relatively simple.

Cheers

wcooper003
Communicator

Excellent thank you! I figured there was a built in way to do this.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi,
The easier approach is mvexpand command:
Your_search | mvexpand | table your_fields_list.
Bye.
Giuseppe

0 Karma

wcooper003
Communicator

Hi Guiseppe,

Thanks for the reply. Unfortunately there are no fields extracted since it's coming in as this structured format. For example, one of the source is 'top' which is just the 'top' command output. A typical event has 100s of lines in it. If I try mvexpand on the _raw field:

index=mihealth source=top | mvexpand _raw | table _raw

Here is a snippet of the output (12 columns for each row, 100's of rows):

PID USER PR NI VIRT RES SHR S pctCPU pctMEM cpuTIME COMMAND 4186 tomcat2 20 0 7714m 450m 7692 S 2.0 1.4 38:34.88 java 7114 root 20 0 15160 1200 808 R 2.0 0.0 0:00.01 top 7360 tomcat 20 0 21.8g 5.9g 7744 S 2.0 18.8 359:51.46 java 1 root 20 0 19344 1076 848 S 0.0 0.0 0:07.64 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd 3 root RT 0 0 0 0 S 0.0 0.0 0:04.54 migration/0 4 root 20 0 0 0 0 S 0.0 0.0 0:11.94 ksoftirqd/0 5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0 6 root RT 0 0 0 0 S 0.0 0.0 0:02.92 watchdog/0 7 root RT 0 0 0 0 S 0.0 0.0 0:06.73 migration/1 8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/1 9 root 20 0 0 0 0 S 0.0 0.0 0:06.23 ks

It seems like i need some type of mvexpand, but it doesn't appear to expand based on return characters for each row in the raw data.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...