How to Generate a report for searching the request from huge list of IP's

[sesha]

I am new to splunk. can any one hlep to search the request from a huge set of IP's i have in csv formatt.

I tried to put the csv file in the lookup folder and used the below search critera but i am getting No result found :-).

source="logfile" | search [|inputlookup IPList.csv]

[lguinn2]

I have a different answer, but it makes a few assumptions:

1. The logfile has a field for the ipaddresses; it is named ip_addy
2. The IPList.csv two fields: a field named ip_addy and a field named status. The status can contain whatever you want

3. A lookup named ip_lookup is defined based on IPList.csv. Under that Advanced options, the default value "not found" is set.

   source="logfile" | lookup ip_lookup ip_addy OUTPUT status | where status!="not found"

This requires no sub-searches and no lists of ip-addresses (except of course in the lookup file itself).

[realdridgespl]

Any other suggestions, other than the IP addresses listed together with "OR" statements?

[Ayn]

It looks to me like you're going in the right direction, but need to sort out some details.

What does the CSV file look like, what field names does it contain? As an example, let's say the CSV is really just a long list of IP's with a header containing the string "IPaddress". In this case, when you run the subsearch

[|inputlookup IPList.csv]

Splunk will get all the IP numbers from the CSV file, enter them as values for the field "IPaddress" and then finally return data to the outer search as a long filter string looking something like this:

((IPaddress="1.1.1.1") OR (IPaddress="3.3.3.3) OR [...])

You can check the output of a subsearch yourself by just running the search on its own without brackets, and then appending | format at the end.

So, as your search looks right now, you're probably getting all the right IP addresses out of the CSV file but they're all mapped to the specific field name "IPaddress". To search for these IP addresses as freetext instead, you should rename the IPaddress field in the subsearch to "query" instead. query is a special field that causes the subsearch to return pure free-text filters rather than searching for values in a particular field. So if IPaddresswere to be renamed to query, the subsearch would instead return something like this:

("1.1.1.1" OR "3.3.3.3" OR [...])

Your search would look something like this after making these changes:

source="logfile" [|inputlookup IPList.csv | rename IPaddress as query | fields query]

[lguinn2]

There is a limit on the subsearch - by default, it returns at most 100 results, but you can up that to 10499.

For more info: http://docs.splunk.com/Documentation/Splunk/4.3/User/HowSubsearchesWork

[sesha]

Thanks Ayn It worked for few set of IP's. But when i tried for huge set of IP's say 10K IP's in CSV file. I am getting zero search results. can you please let me know if there is any limitation on CSV file.

[hedgehog]

The only way I can think of to achieve this would be to run the search from the command line.

You could use something like AWK to format the request for the command line. I'll have a think about it and repost something more detailed later today.

• Pete