Splunk Search

Bulk rename of all fields that come from calling "| lookup"?

woodcock
Esteemed Legend

We have a CMDB lookup that adds 100 fields when we do not specify a limited set with "OUTPUT". More fields are added every now and then. I cannot rename the fields in the file because too many people share this file; it is what it is. Is there a flexible way to rename all fields that are in there that will still work even when new fields are added or old field names change slightly? If I jus.t do an iteration of every field with | rename a AS CMDB_a, | rename b AS CMDB_b ... this will be fragile and break and be a huge blight in the middle of my search. I cannot do | rename * AS CMDB_* because there are many other fields and this search might change to add more non-CMDB fields, too.

1 Solution

Noah_Woodcock
Path Finder

Like this:

... | rename * AS _HIDE_ALL_FIELDS_*
| lookup YourLookupDefinitionHere host AS _HIDE_ALL_FIELDS_host
| rename * AS CMDB_*
| rename _HIDE_ALL_FIELDS_* AS *

View solution in original post

Noah_Woodcock
Path Finder

Like this:

... | lookup YourLookupDefinitionHere host
| rename COMMENT AS "Bulk rename of all fields from this lookup to add prefix 'CMDB_'"
| rename [| inputlookup  YourLookupDefinitionHere
          | head 1
          | foreach * [ eval <<FIELD>> = "CMDB_<<FIELD>>" ]
          | format "" "" "" "" "" ""
| rex field=search mode=sed "s/=/ AS /g"]

Noah_Woodcock
Path Finder

Like this:

... | rename * AS _HIDE_ALL_FIELDS_*
| lookup YourLookupDefinitionHere host AS _HIDE_ALL_FIELDS_host
| rename * AS CMDB_*
| rename _HIDE_ALL_FIELDS_* AS *

theady
Engager

I had issue with this. it seems to take the _* fields as part of the * on the first rename.

0 Karma

woodcock
Esteemed Legend

It definitely should not and is a bug if it does.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Came across this solution and noted the comments about _ fields being renamed and it does appear that it won't work as _ fields are also renamed with *

| makeresults
| eval myField=1
| rename * as HIDE_*
| table *

shows HIDE__time

and in my example

| rename * as _HIDE_ALL_*
| lookup work_status key as _HIDE_ALL_workStatusKey
| rename * as orig_work_status_*
| rename _HIDE_ALL_* as *

 after the second rename, all _HIDE_ fields are now orig_work_status__HIDE_...

which is not the intended outcome - shame - as it was a simple approach.

 

0 Karma

davewood
Explorer

It seems the rename command changed behaviour somewhere along the way and now, as you noted, * matches _ fields too.

You can still use the same method though with a tweak to the final rename:

| rename * as _HIDE_ALL_*
| lookup work_status key as _HIDE_ALL_workStatusKey
| rename * as orig_work_status_*
| rename orig_work_status__HIDE_ALL_* as *

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...