Splunk Search

Data models: how to use a wildcard for field names?

helge
Builder

Some of my events have an unknown number of field names with a common naming scheme. In my searches I use a wildcard to include them all, e.g.

stats first(GroupPolicyCSE*) by host

How can I transfer this to data model attributes?

1 Solution

aneels_splunk
Splunk Employee
Splunk Employee

Data models cannot be dynamically added to data models in Splunk 6.0. The data model feature assumes that you know the structure of your data when building the model.

If you have a list of all the fields that can possibly occur, you could create a data model with all of them, and simply allow there to be zero matching fields for those attributes on some of your installs.

You could also look into building a search-based object that creates fields with consistent names, and then building a data model on top of that. Search-based objects can incorporate arbitrary splunk searches, which gives you a lot of power. The downside is they can't be accelerated.

View solution in original post

aneels_splunk
Splunk Employee
Splunk Employee

Data models cannot be dynamically added to data models in Splunk 6.0. The data model feature assumes that you know the structure of your data when building the model.

If you have a list of all the fields that can possibly occur, you could create a data model with all of them, and simply allow there to be zero matching fields for those attributes on some of your installs.

You could also look into building a search-based object that creates fields with consistent names, and then building a data model on top of that. Search-based objects can incorporate arbitrary splunk searches, which gives you a lot of power. The downside is they can't be accelerated.

jtrucks
Splunk Employee
Splunk Employee

If these fields are all defined in your Splunk instance where the data model is being created, it should be able to auto-discover what those are based on your field definitions.

To do this, when editing your Object in this Data Model, go to "Add Attribute" -> "Auto-Extracted" and then save.

--
Jesse Trucks
Minister of Magic

helge
Builder

I understand what you are saying, but that does not help. The field names depend on what is installed in the customer's network and there is no finite list of field names that can occur.
I guess it boils down to the fact that data models just do not work with wildcarded field names (yet). In other words, data models do not expose the full functionality of Splunk.
Thanks very much for your time!

0 Karma

jtrucks
Splunk Employee
Splunk Employee

Part of the problem in your situation is that data models don't appear to be easily built in a dynamically updated way. http://docs.splunk.com/Documentation/Splunk/6.0/Knowledge/Designdatamodelobjects#Add_an_auto-extract... indicates it will present new fields it discovers anytime you run the Auto-Extract function. Do this via API on a scheduled basis, and you can keep your data model updated regularly. See http://docs.splunk.com/Documentation/Splunk/6.0/RESTAPI/RESTknowledge#datamodel.2Fmodel.2F.7Bname.7D for docs.

--
Jesse Trucks
Minister of Magic

jtrucks
Splunk Employee
Splunk Employee

I suggest you put your search extract config files into deployment server to uniformly pull the same fields.

If you are not accelerating this data model, you could build auto-extract and update the models with scripting (REST or API are likely approaches).

OR go to each Splunk instance to gather up the various extracts for the relevant source(s) and sourcetype(s) and merge them all, then generate your data models.

The last approach would be to manually set up each one based on the collectively known extracts using the Eval or Regex options to Add Attribute.

--
Jesse Trucks
Minister of Magic
0 Karma

helge
Builder

As I wrote, the number of fields is unknown. My Splunk instance my have 5, while another instance may have 10 different fields matching the wildcard.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...