Splunk Search

Remove a series of numbers in a field

AccentureQBETA
Path Finder

I have a field which is extracted in Splunk with values which look like this:

/aa/Application.do?inFrame=uploadframe&r=99946238&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=99989045&__navigator_index=0
/aa/Application.do?inFrame=blank&r=99985838&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
/aa/?internal=Y
/aa/Application.do?ts=99998203
/aa/soap/ILoginAPI
/aa/FileContents.do?widgetID=Application:Something_Something:Ext_Something:9:Something_ViewLink

I would like to the remove the series of numbers, where there are more than 3 in a row.

I think it can be done either at index time or search time using rex? I can't figure it out though..

1 Solution

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

View solution in original post

echalex
Builder

I think it could be done using index-time, but it's probably a better idea to do it search-time by using eval and replace.

...|eval your_field=replace(your_field,"\d{3}\d+","")

(Assuming that by "more than 3" you mean "four or more" and not "three or more".)

AccentureQBETA
Path Finder

This is perfect, thanks

Jason
Motivator

Doing it at index time or search time depends on whether you want the data to be permanently modified in your index or not.

At index time, you can use the SEDCMD to rewrite the data:
props.conf

[datasource]
SEDCMD-removelongnumbers = s/(.*?)\d\d\d+(.*)/\1\2/

Or, at search time, you could use "rex" and "eval" to remove them, but the other answer is easier:

... | rex field=myurl "(?<myurl1>.*?)\d\d\d+(?<myurl2>.*)" | eval myurl=if(isnull(myurl1), myurl, myurl1+myurl2)

Test these prior to using, I don't have an instance in front of me right now.

0 Karma

Jason
Motivator

Yes, SEDCMD is run on the entire event, since most fields do not exist until search time.

0 Karma

AccentureQBETA
Path Finder

Jason, thank you so much for your answer. I havn't tried them yet, but will today and comment back. Echalex's answer is easier to understand for me, for now and works. Thank you though!

One thing though, because I might use your index-time solution later.. doesn't that do it for the whole event and not just a field? can a n index-time SEDCMD be done on a field? maybe if the stanza comes after a field extract stanza?

0 Karma

AccentureQBETA
Path Finder

Yeah, that's correct 🙂

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

So you want, e.g., the first few to be:

/aa/Application.do?inFrame=uploadframe&r=&__navigator_index=0
/aa/resources/Ocean/css/trans.css?ver=6.0.4.21
/aa/Application.do?inFrame=scframe&r=&__navigator_index=0
/aa/Application.do?inFrame=blank&r=&__navigator_index=0
/aa/soap/Something
/aa/resources/Ocean/css/print_trans.css?ver=6.0.4.21
...

? Is that correct?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...