Splunk Search

How to search under csv file?

kestasm
Path Finder

Hi,

maybe somebody could advice how can I use quite big csv file (which gets updated frequently) with one column of entries (as an example IP addresses) for generating an alerts or reports to be reviewed daily/weekly? Is there a limit on entries in csv file column which could be handled by SPLUNK?

Tags (1)
0 Karma
1 Solution

piebob
Splunk Employee
Splunk Employee

it sounds as though you might want to configure a lookup in Splunk that uses the csv file. this lets you look up values in a csv file as part of a Splunk search. there is information about doing this here:
http://docs.splunk.com/Documentation/Splunk/5.0.1/Knowledge/Addfieldsfromexternaldatasources

if the csv file gets updated by values being added to the end, another option is for you to just index the csv file using a Splunk monitor, which will keep checking the file for any new data:
http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories

if you provide a bit more information about the file and how it is updated, i could maybe be more helpful...

View solution in original post

piebob
Splunk Employee
Splunk Employee

it sounds as though you might want to configure a lookup in Splunk that uses the csv file. this lets you look up values in a csv file as part of a Splunk search. there is information about doing this here:
http://docs.splunk.com/Documentation/Splunk/5.0.1/Knowledge/Addfieldsfromexternaldatasources

if the csv file gets updated by values being added to the end, another option is for you to just index the csv file using a Splunk monitor, which will keep checking the file for any new data:
http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories

if you provide a bit more information about the file and how it is updated, i could maybe be more helpful...

kestasm
Path Finder

Sorry, accepted:) And thanks for all the comments!

0 Karma

piebob
Splunk Employee
Splunk Employee

you should check out the documentation, there is an entire manual about alerting: http://docs.splunk.com/Documentation/Splunk/5.0.2/Alert/Aboutalerts

and please accept my answer to this question.

0 Karma

kestasm
Path Finder

Great, thanks, is there a way to configure alert to email reports/summaries of all alerts once a week, lets say? Is there an expiration of created alert by default? Thanks!

0 Karma

piebob
Splunk Employee
Splunk Employee

no, 37k entries is not too much for Splunk to handle.

kestasm
Path Finder

is there any limitation in SPLUNK of how much entries in csv file it can handle? I mean isn't it too much of 37k entries?

0 Karma

piebob
Splunk Employee
Splunk Employee

this sounds like a good use case for a lookup, you would just update the lookup file once a week. it's just one column of IP addresses? that shouldn't be too big for Splunk. configure the lookup, and then set up an alert based on the IPs you're interested in.

kestasm
Path Finder

Hi, thanks for this. Well we get this csv updated file by email, so as I imagine we could update indexed existing file entries ourselfs, right?

We already have some lookups configured to use csv files, but those are small files needed for one time search only. What we need with this file (it is quite big 37k entries (IP addresses) at the moment, and gets updated every week) is to make continuous alerting on IPs whenever there is a match, and to send alert reports on weekly basis to email box. So if you could think of some way to do this I would be greatly appreciated.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...