Splunk Search

Extract fields out of plain text log file and assign it to a field name

Sujithkumarkb
Observer

I want to extract the below values during index time
1. extract WDDZF4KB3JA469368 ,ABCDE4KB3JA469368 and so on and assign it to a field name called VINnumber.
2. extract ConfigureServiceStatusMBIISProvABCSImpl, QuerySubscriptionListSiebelProvABCSImpl ,VINValidationProcess and so on and assign it to field name called services.

ist item2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|

2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|

2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing

2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|

2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368*|AccId-NoValueFound|Business Fault:Agree# Missing*

I understand search time extraction is better for performance .
I tried to extract during search time like below with a bit of luck , but data is

index=ABC sourcetype=XYZ  WDDZF4KB3JA469368 
| rex field=_raw "(?WDDZF4KB3JA469368.+)\"" 

but the value i am getting in the field is quite extra VIN= WDDZF4KB3JA469368,,,Cathy,"Wright",,"6278 Harbour Heights Pkwy

0 Karma
1 Solution

to4kawa
Ultra Champion
| makeresults 
| eval raw="2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing@2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing"
| makemv delim="@" raw
| mvexpand raw
| rex field=raw "(?<_time>[^\|]+)\|(?<services>[^\|]+)\|[^\|]+\|[^\|]+\|[^\|]+\|(?<result>[^\|]+)\|(?<VINnumber>[^\|]+)\|"
| table _time raw services VINnumber result

View solution in original post

0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval raw="2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing@2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing"
| makemv delim="@" raw
| mvexpand raw
| rex field=raw "(?<_time>[^\|]+)\|(?<services>[^\|]+)\|[^\|]+\|[^\|]+\|[^\|]+\|(?<result>[^\|]+)\|(?<VINnumber>[^\|]+)\|"
| table _time raw services VINnumber result
0 Karma

Sujithkumarkb
Observer

@to4kawa , thanks .
But , What if i want to extract fields during index time ?

0 Karma

to4kawa
Ultra Champion
your search
|eval index_time=strftime(_indextime,"%F %T")
|table _time index_time services VINnumber result
0 Karma

Sujithkumarkb
Observer

@to4kawa , sorry may be i was not clear.
I meant using Props , how can i extract these fields using regex , because i do not see a pattern for the fields i want to extract.

0 Karma

to4kawa
Ultra Champion

I'm sorry , I don't know props.conf
If you provide new Question of props.conf with this sample data, another professional will solve it.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...