Splunk Search

How to extract file tyes from .txt file

sunnyparmar
Communicator

Hi,

I have a one text file which have some entries with the file types .pdf, .tif so now i want to make one hourly dashboard to keep an eye on those file types that how many files are coming hourly on the server? My query is something like below but didn't get result with this

index="abc" source="file path"  | rex "\.(<_file_type>\w+)\.\."  | eval file_type=upper(_file_type) | timechart span=1h count by file_type

Thanks

Tags (3)
0 Karma
1 Solution

woodcock
Esteemed Legend

The rex part had some mistakes; try this:

index="abc" source="file path" | rex "\.(?<file_type>\w+)" | eval file_type=upper(file_type) | timechart span=1h count by file_type

View solution in original post

woodcock
Esteemed Legend

The rex part had some mistakes; try this:

index="abc" source="file path" | rex "\.(?<file_type>\w+)" | eval file_type=upper(file_type) | timechart span=1h count by file_type

sunnyparmar
Communicator

Hi,

thanks for replying but if i am running your command so its giving me the result with 2 columns that is

1.) timeframe 2.) NULL in which it is showing the counts in thousands that is too much. What I would like to say here that counts are wrong because in my log files hardly .pdf and .tif counts are in hundreds and moreover when I am clicking on view events then again it is showing "no result founds".

Regards

0 Karma

sunnyparmar
Communicator

this is an example of my log file...

TIMESTAMP - [2015-09-21 10:14:12.576] THREAD ID - [48] CONTENT - File Archived C:BSCSwienerberger_deBSCSEmail20150921_101351_WIENERBERG_EMAIL_1442819631609.PDF
TIMESTAMP - [2015-09-21 10:08:30.553] THREAD ID - [42] CONTENT - File Archived C:BSCSXeroxFinland_fidestia_fipl801e_destiamaansiirto_000upload20150921_093331_DESTI0801FI_20150921000003_000028.tif

0 Karma

gcato
Contributor

Thanks for the examples. The current rex command will also match the microseconds (\w matches numbers too) on the timestamp which probably explains the count mismatch. If the filetype . suffix is always at the end of each event line then this regex will be far more accurate.

... | rex "\.(?<file_type>[a-zA-Z]+)$" | eval file_type=upper(file_type) 

The [a-zA-Z]+ will now only match letters and the $ is a special character denoting line end. This should help prevent false matches.

sunnyparmar
Communicator

Hi Buddy,

thanks a ton .. its started working..

Regards

0 Karma

sover
Engager

Hi @sunnyparmar, can you add a snippet of the log file you indexing? Thanks

0 Karma

sunnyparmar
Communicator

Hi,

It is like given below .. I have added one example of .pdf and another for .tif

TIMESTAMP - [2015-09-21 10:14:12.576] THREAD ID - [48] CONTENT - File Archived C:\BSCS\wienerberger_de\BSCSEmail\20150921_101351_WIENERBERG_EMAIL_1442819631609.PDF
TIMESTAMP - [2015-09-21 10:08:30.553] THREAD ID - [42] CONTENT - File Archived C:\BSCS\XeroxFinland_fi\destia_fi\pl801e_destiamaansiirto_000\upload\20150921_093331_DESTI0801FI_20150921000003_000028.tif

Thanks

0 Karma

gcato
Contributor

Also remember that a "." in a regex expression means any character. You have to escape the . character with a backslash if you want to match a literal . character.

http://docs.splunk.com/Documentation/Splunk/6.2.5/Knowledge/AboutSplunkregularexpressions

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...