Splunk Search

How to extract multiple values on one field from XML structured data?

Buscatrufas
Path Finder

Hi,

I want to split data from this XML structure, but I cannot because the extracted field only gets the first element.

<vuln:vulnerable-software-list>
      <vuln:product>cpe:/o:novell:opensuse:13.1</vuln:product>
      <vuln:product>cpe:/a:samba:rsync:3.1.1</vuln:product>
      <vuln:product>cpe:/o:novell:opensuse:13.2</vuln:product>
  </vuln:vulnerable-software-list>

I edited the file fields.conf too with the following syntax:

[name]
TOKENIZER = (cpe(:\/[\:\w\.]+))

But nothing happens. Any help? Thanks!

Best regards, Buscatrufas

0 Karma
1 Solution

aljohnson_splun
Splunk Employee
Splunk Employee

Are you sure you want to extract these at index time? It is unlikely that you really want to do that:

From the docs here:

Caution: Do not add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time unless absolutely necessary. This includes fields such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see Index time versus search time in the Managing Indexers and Clusters manual.

Instead, you can use props.conf and transforms.conf to extract during search time,

Let's say your data has a sourcetype of vuln.

Props.conf:

[vuln]
REPORT = vuln_extractions

So whats going on here? The REPORT tells Splunk to look in transforms.conf for a stanza called vuln_extractions. It is simply linking the regular expression we will define to the vuln sourcetype.

Then we need Transfroms.conf:

[vuln_extractions]
REGEX = (?<_KEY_1>[\w\.]+):(?<_VAL_1>[\w\.]+)

Whats going on here? The REGEX line defines the regex we want to use. The _KEY_1 and _VAL_1 capture group names are special and Splunk already knows what to do with them. You can read more about them and this approach in transforms.conf.spec.

Here is an example of what the regex would extract: regex101

The transform should be applied by default, however, if you wanted to apply the same to a different sourcetype, ad-hoc, you could use the extract command to apply the transform, e.g.

sourcetype=some_other_sourcetype | extract vuln

View solution in original post

aljohnson_splun
Splunk Employee
Splunk Employee

Are you sure you want to extract these at index time? It is unlikely that you really want to do that:

From the docs here:

Caution: Do not add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time unless absolutely necessary. This includes fields such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see Index time versus search time in the Managing Indexers and Clusters manual.

Instead, you can use props.conf and transforms.conf to extract during search time,

Let's say your data has a sourcetype of vuln.

Props.conf:

[vuln]
REPORT = vuln_extractions

So whats going on here? The REPORT tells Splunk to look in transforms.conf for a stanza called vuln_extractions. It is simply linking the regular expression we will define to the vuln sourcetype.

Then we need Transfroms.conf:

[vuln_extractions]
REGEX = (?<_KEY_1>[\w\.]+):(?<_VAL_1>[\w\.]+)

Whats going on here? The REGEX line defines the regex we want to use. The _KEY_1 and _VAL_1 capture group names are special and Splunk already knows what to do with them. You can read more about them and this approach in transforms.conf.spec.

Here is an example of what the regex would extract: regex101

The transform should be applied by default, however, if you wanted to apply the same to a different sourcetype, ad-hoc, you could use the extract command to apply the transform, e.g.

sourcetype=some_other_sourcetype | extract vuln

Buscatrufas
Path Finder

I love you

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...