Splunk Search

Find and filter out entries from log

tp92222
Explorer

I want to process log file which contains numeric entries like
eg

232231XX
232313XX
332133XX
411111XX
522222XX
734444XX
345555XX

i want to filter out entries in which digit is getting repeated more then 2 times continuously after 1 st digit

for eg from above entries
411111XX
522222XX
734444XX
345555XX

Thank you

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

View solution in original post

0 Karma

sundareshr
Legend

Try this

| rex field=yourfield "(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)

Now, if 77734444XX should NOT be filtered out, because it repeats only twice after the first digit, use this

| rex field=yourfield "(?<repeatednum>\d(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnull(repeatednum)
0 Karma

inventsekar
SplunkTrust
SplunkTrust

for eg from above entries
411111XX
522222XX
734444XX --- after the first digit 7, digit 3 is not repeating two times.
345555XX --- after the first digit 3, digit 4 is not repeating two times.

0 Karma

tp92222
Explorer

after 1 digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

Try something like this (assuming that these numeric data is extracted in a field called 'yourfield', update the field name as per yours)
updated

your current search with field yourfield 
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)

Runanywhere sample (everything before the rex command is to generate sample data, replace with your search)

| gentimes start=-1 | eval yourfield="232231XX 232313XX 332133XX 411111XX 522222XX 734444XX 345555XX 8884543XX" | table yourfield | makemv yourfield | mvexpand yourfield
| rex field=yourfield "^\d(?<repeatednum>(0{3}|1{3}|2{3}|3{3}|4{3}|5{3}|6{3}|7{3}|8{3}|9{3}))" | where isnotnull(repeatednum)
0 Karma

tp92222
Explorer

Thanx for quick reply .but i am looking for little different out put
after 1st digit any ,digit if coming more then 2 times that number should be filtered out

for eg

734444XX --- after the first digit 7, digit 4 is repeating 4 times.
345555XX ----after the first digit 3, digit 5 is repeating 5 times.

and repeating digit should be continues

for eg
3455455XX is vaild

0 Karma

somesoni2
Revered Legend

The updated answer should take care of that.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...