Getting Data In

search time field extractions using props for an Indexed field

Raghav2384
Motivator

Hello Experts,
We have a field xyz which holds mac addresses. Problem is, some of the mac addresses are of xx:xx:xx:xx:xx:xx format and some are xxxx.xxxx.xxxx & xx-xx-xx-xx-xx-xx & xxxx-xxxx-xxxx-xxxx (Every weirdest pattern you can imagine). We used
rex field=_raw mode=sed s/[-|.|:]//g to make it all numbers. I know Props.conf documentation reads, SEDCMD is only used at index time. Is there any other alternative procedures you recommend.
I tried EVAL-foo = replace(CALLING_STATION_ID,":*","") in props.conf to no avail.
Thanks in advance.

Raghav

1 Solution

aweitzman
Motivator

Does this create a field foo with none of the colons, dashes or dots?

EVAL-foo = replace(CALLING_STATION_ID,"([-:\.])","")

View solution in original post

0 Karma

aweitzman
Motivator

Does this create a field foo with none of the colons, dashes or dots?

EVAL-foo = replace(CALLING_STATION_ID,"([-:\.])","")
0 Karma

Raghav2384
Motivator

Tried, didn't work.
EVAL-MACID = replace(CALLING_STATION_ID,"([-:.])","") in props.conf, restarted splunk and MACID is not in there.
Also tried ...=replace(CALLING_STATION_ID,"([-|:|.])",""), no luck here either.
Any different route you recommend?
Thanks,
Raghav

0 Karma

aweitzman
Motivator

Is CALLING_STATION_ID a field in the actual data, or is it a generated field (with a different EVAL- or FIELDALIAS- entry in props.conf)? It won't work if it's a generated field.

I did what I proposed above and it worked, so I'm not sure what might be going on in your environment. Can you try with just a colon within the brackets and see if it at least does that?

0 Karma

Raghav2384
Motivator

This is what i found in the logs :'EVAL-foo' in stanza [aaa]: The expression is malformed. An unexpected character is reached at '”:”,””)'.

0 Karma

Raghav2384
Motivator

CALLING_STATION_ID is a field in the actual data. Example log
XXXXXXXXX= S13456
PORT-ID = 1//1/1/1
FRAMED-IP = 1.2.3.4
NAS-PORT-TYPE = 5
CALLING-STATION-ID = 12:ae:45:y2:35:3d
NAS-IP-ADDRESS = x.y.x.xy
And is extracted by splunk at the index time.

0 Karma

aweitzman
Motivator

Do you see any errors regarding the MACID field in splunkd.log?

0 Karma

Raghav2384
Motivator

Could you please post me your props stanza and sample log you created to replicate this? I want to see where the heck i am doing wrong.

0 Karma

aweitzman
Motivator

Here's the run-anywhere example based on the props stanza, just to prove that the regex is correct:

| gentimes start=-1 
| eval CALLING_STATION_ID="123-456-789,12:34:56:78,12.345.6789,12.34:56-78" 
| makemv delim="," CALLING_STATION_ID 
| mvexpand CALLING_STATION_ID 
| eval MACID = replace(CALLING_STATION_ID,"([-:\.])","") 
| table CALLING_STATION_ID MACID

If you take that exact replace string (hyphen must be first inside the brackets, and there must be a backslash prior to the period) and put it in props.conf like so:

EVAL-MACID = replace(CALLING_STATION_ID,"([-:\.])","")

it should work the same way. I also tested this props line on similar actual data in my environment and it worked.

0 Karma

Raghav2384
Motivator

Thanks a ton @aweitzman. Culprit here is me, just realized that my config file reads "props.confs".
See if you can throw a punch at my face through your next comment 🙂

0 Karma

aweitzman
Motivator

Ha! 🙂

Happens to the best of us. Glad it's working now.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...