Splunk Search

How to extract the file type from my logs and get the count of successfully processed files by file type?

boddunan
Engager

Hi,

I am searching for some way to extract count of each file type which is successfully processed. The logs contain various levels of processing of file, but the final line contains status of the processing. I need to extract all such successful processed files and count how many successful files there are for each type. For example, below is my pattern in the log.

11:44:01,982 some other log here File: ../../../transfer/dir/path/**MYFILETYPE**.SOMEOTHERPATTEN.xml successfully imported.

So I need to extract all such lines and build a table with count of each file type. Can someone please help with what regex I need to use to get this?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

index=foo sourcetype=bar "successfully imported" | rex "File:\s+.*\/(?<filetype>\w+)\..*\ssuccessfully imported" | stats count by filetype
0 Karma

ktugwell_splunk
Splunk Employee
Splunk Employee

Hey boddunan,

First of all, I'd search for all data where "successfully imported." exists:

your base search... | search "successfully imported."

Then I'd add a regular expression like this:

| rex field=_raw "path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4})"

You should then see a field in your data called "file_name". You should then be able to do this:

your base search... | search "successfully imported."| rex field=_raw "path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4})" | stats count by file_name

Let me know how that goes, If you want to pull apart the regular expression and see how it works(or how you can debug it). Try www.regex101.com.

Copy the path\/(?<file_name>[A-Za-z0-9*%$]*\.\w+\.\w{2,4}) in the top input box and your data underneath. On the right hand side it'll show you how it works.

Thanks

sundareshr
Legend

This should work

index=* | rex "File:\s(?<filename>[^\s])\s(?<status>successfully)" | stats count by filename
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...