Hi,
I am searching for some way to extract count of each file type which is successfully processed. The logs contain various levels of processing of file, but the final line contains status of the processing. I need to extract all such successful processed files and count how many successful files there are for each type. For example, below is my pattern in the log.
11:44:01,982 some other log here File: ../../../transfer/dir/path/**MYFILETYPE**.SOMEOTHERPATTEN.xml successfully imported.
So I need to extract all such lines and build a table with count of each file type. Can someone please help with what regex I need to use to get this?
Give this a try
index=foo sourcetype=bar "successfully imported" | rex "File:\s+.*\/(?<filetype>\w+)\..*\ssuccessfully imported" | stats count by filetype
Hey boddunan,
First of all, I'd search for all data where "successfully imported." exists:
your base search... | search "successfully imported."
Then I'd add a regular expression like this:
| rex field=_raw "path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4})"
You should then see a field in your data called "file_name". You should then be able to do this:
your base search... | search "successfully imported."| rex field=_raw "path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4})" | stats count by file_name
Let me know how that goes, If you want to pull apart the regular expression and see how it works(or how you can debug it). Try www.regex101.com.
Copy the path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4})
in the top input box and your data underneath. On the right hand side it'll show you how it works.
Thanks
This should work
index=* | rex "File:\s(?<filename>[^\s])\s(?<status>successfully)" | stats count by filename