I have different environments. In each environment logs are located in different path. e.g.:
C:\Program Files\Splunk...\etc\apps......\abc-20150626123001.log
/u01/splunk/...../etc/apps/...../def-20150626044921.log
/u01/log02/splunk/...../etc/apps/...../mno-20150626071656.log
/u02/splunk/...../etc/apps/...../xyz-20150626044921.log
I am using this query -
index="myindex" | dedup source | sort -source | dedup sourcetype | table sourcetype, source
It gives me the result but contains the complete log path. My requirement is to remove everything and display only the date and time e.g. 2015-06-26 12:30:01
Thanks for answering my previous question [@richgalloway] --> /answers.splunk.com/answers/243218/how-to-use-rex-and-sed-to-insert-and-in-the-result.html
I was able to capture the date and time by using sed and regular expression. e.g.:
index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source
This worked only for log files present in "C:\Program Files...."
However it didn't work for other environments. I had to modify each time, e.g. when it's /u01, i used -
index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/01//" | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source
Basically I just need to capture the YYYYMMDDHHMMSS before .log and remove everything else. The app will be used by various teams. Each team will have their own folder structure. Is there a way to make the query work in any environment irrespective of the path?
How 'bout this?
... | rex field=source "(?<date>\d+)\.log$"
That will grab the last bunch of digits before the .log
at the end.
I don't know what you're trying to achieve with your whole query, so this is just a guess:
| tstats count where index=myindex by source sourcetype | rex field=source "(?<date>\d+)\.log$
Can you please write the whole query?