Splunk Search

How to make the search dynamic - continuation of "How to use rex and sed to insert '-' and ':' in the result?"

nilotpaldutta
Explorer

I have different environments. In each environment logs are located in different path. e.g.:
C:\Program Files\Splunk...\etc\apps......\abc-20150626123001.log
/u01/splunk/...../etc/apps/...../def-20150626044921.log
/u01/log02/splunk/...../etc/apps/...../mno-20150626071656.log
/u02/splunk/...../etc/apps/...../xyz-20150626044921.log

I am using this query -

index="myindex" | dedup source | sort -source | dedup sourcetype | table sourcetype, source

It gives me the result but contains the complete log path. My requirement is to remove everything and display only the date and time e.g. 2015-06-26 12:30:01

Thanks for answering my previous question [@richgalloway] --> /answers.splunk.com/answers/243218/how-to-use-rex-and-sed-to-insert-and-in-the-result.html
I was able to capture the date and time by using sed and regular expression. e.g.:

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

This worked only for log files present in "C:\Program Files...."

However it didn't work for other environments. I had to modify each time, e.g. when it's /u01, i used -

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/01//" | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

Basically I just need to capture the YYYYMMDDHHMMSS before .log and remove everything else. The app will be used by various teams. Each team will have their own folder structure. Is there a way to make the query work in any environment irrespective of the path?

Tags (2)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

How 'bout this?

... | rex field=source "(?<date>\d+)\.log$"

That will grab the last bunch of digits before the .log at the end.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I don't know what you're trying to achieve with your whole query, so this is just a guess:

| tstats count where index=myindex by source sourcetype | rex field=source "(?<date>\d+)\.log$
0 Karma

nilotpaldutta
Explorer

Can you please write the whole query?

0 Karma
Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...