Splunk Search

How do you Identify and replace a CSV field name by WildCard or Regex Pattern?

xshen_anji
New Member

I need to replace some CSV field Names with standard names for further easier processing.

I tried to rename with a wildcard, like rename *somewords as standardName, but found this command didn't accept this. Instead, it accepts: rename *somewords as *someotherwords. The final result is the field name replaced with *someotherwords. This is strange, in my opinion, because I expect the wildcard should match the whole name and then replace it to some other things, but Splunk's usage is to match the uninterested parts and keep them. This is weird to me.

Can anybody give me some suggestions on how to replace the whole field name matching a given pattern? Thank you!

0 Karma
1 Solution

maciep
Champion

I think you'll want to use foreach for this. Something like the following:

... | foreach *Name [ rename <<FIELD>> as Name ]

And then do something similar for other wild-carded fields.

For how you want to use the rename command, splunk's implementation doesn't make a lot of sense. But the rename command is more often used to rename multiple fields at once based on a pattern. And since it doesn't make sense to rename multiple fields to the same field name, the wildard is needed to keep the names unique.

For example, if you have a json event that returns maybe data.name, data.value, data.description....the rename command can be used to drop the data portion of the fields names using "| rename data.* as *" In this case, it doesn't make any sense (and isn't valid) to rename all 3 fields to the same field name

View solution in original post

0 Karma

maciep
Champion

I think you'll want to use foreach for this. Something like the following:

... | foreach *Name [ rename <<FIELD>> as Name ]

And then do something similar for other wild-carded fields.

For how you want to use the rename command, splunk's implementation doesn't make a lot of sense. But the rename command is more often used to rename multiple fields at once based on a pattern. And since it doesn't make sense to rename multiple fields to the same field name, the wildard is needed to keep the names unique.

For example, if you have a json event that returns maybe data.name, data.value, data.description....the rename command can be used to drop the data portion of the fields names using "| rename data.* as *" In this case, it doesn't make any sense (and isn't valid) to rename all 3 fields to the same field name

0 Karma

xshen_anji
New Member

Thank you @maciep , this works for my case! But I am not 100% agree with the rename syntax. For you example, " rename data.* as *" should serve the multiple to multiple case well. However, for to one case, this syntax doesn't work as we see.

0 Karma

maciep
Champion

can you maybe give us a more concrete example of the fields you're working with and what you want to rename them to? i don't quite understand the problem with your "somewords" type example.

0 Karma

xshen_anji
New Member

Thank you @maciep , the example is like: table myOriginalName | rename *OriginalName as NewName. This would not work. We have to use table myOriginalName | rename *OriginalName as *NewName. This is not what I want! I want the column named as "NewName". Can you share some knowledge on how I can do this?

0 Karma

maciep
Champion

the reason i'm trying to understand your exact fields is because your examples aren't very telling. For example, "|rename myOriginalName as NewName" would work fine.

Do you not know the names of the fields you want to rename? Are you trying to rename multiple wildcarded fields to the same field name....because that won't work.

That said, you could look into the foreach command....which will loop through all of your fields. Maybe that will give you the means to rename them the way you want?

0 Karma

xshen_anji
New Member

Hi @maciep , the scenario is like I need to write a SPL to accomodate many different titled csv files. But these files have some similarly named headers. I want to identify these similar headers and then rename them to the same name, so I will be able to process them with similar logic. An example is file A has a field header "aName", file B has a field as "cName", I want to rename them all to "Name", then I will be able to write a SPL with a macro to process "Name" coloumn.
So basically I think the rename's wildcard feature should be designed to be able to select the whole word. this is a common practice. like when we do "ls *name.txt", it will give all the files' whole name with "name" in. We don't care what these * represents, we care the whole word, but splunk's rename function simply can't give the whole word as a return value, this is what I think weird.
I am new on splunk, don't know if there are other ways can do this . Thanks

0 Karma

xshen_anji
New Member

ls *name.txt as the example. * is missed above, don't know why

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...