Knowledge Management

Is there a way to remove/replace spaces in between fieldname?

snallam123
Path Finder

we are extracting fields with spaces in it using below transforms, Is there a way we can remove spaces in between fields from backend? There are 100's of fields with spaces. I tried with field alias amd it's hard to apply for each and every field. 

Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<field>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2

_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line:  test"

output I am getting:
"Process Id" =  6228

Is there a way we can change this to ProcessId=6228 or Process-Id=6228 ?


From UI i tried this, Can someone help me with backend trick

| makeresults
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test"
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)"
| rex mode=sed field=field "s/ /_/g"
| eval tmp=mvzip(field,value,"=")
| rename tmp as _raw
| kv
| table *

Labels (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

I'm not tried this myself, but can you try INGEST_EVAL + replace function within your Transforms?

0 Karma

snallam123
Path Finder

I tried with it but no success @isoutamo ,

Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<fieldNam>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2

_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025

<Ingest_eval_change_fields>

INGEST_EVAL = NewField=replace(fieldNam, "\s", "_")   -

When we did Ingest_eval_change_fields transforms FORMAT function in earlier transforms has already changed to field names so "fieldNam" no longer exists.

0 Karma

mayurr98
Super Champion
  • The SEDCMD script applies only to the _raw field at index time. With the regular expression transform, you can apply changes to other fields

https://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedata?_gl=1*16ogb9v*_ga*MTA1MzM4MzA...

you can try using evaluation functions as well 

 

 

| makeresults 
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test" 
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)" 
| mvexpand field 
| eval field1=replace(field,"\s","_")

 

 

see if you can use calculated fields if its not a multivalue field.

0 Karma

mayurr98
Super Champion

Hello,

you can achieve this using SEDCMD Scripts

https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/Anonymizedata#Example_of_substitution_using_...

Transforms.conf must be used for the extracted field, and SEDCMD for _raw.

See here for details. way at the bottom.

https://answers.splunk.com/answers/739964/need-sedcmd-help.html

 

0 Karma

snallam123
Path Finder

Thanks @mayurr98 ,

If i use SED it ll modify _raw data as well, Is it possible to change field name with modifying _raw data, I am thinking If i use SED to modify the _raw data it ll impact the ingestion speed.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...