How to generate a search that will target IP addresses with greater than 100 post requests as Spam?

[mcvr]

We need to identify the unique IP addresses of the spammers who are generating more number of POST requests generating in one second duration.

The ideology behind this is - normal valid users might not request those high number of POST requests on a page say no#100 in one second. I wanted to call them as spammers. Can I do this? Is this a right way? Please advise me a search to achieve this.

Regards,
Murthy

[Ravan]

You may need to increase time frame slightly , and use any of splunk aggregation commends.

Ex:

index=my_index action=POST user=* | transaction user maxspan=30s|where eventcount > 100

This will aggregate the events happened in 30 sec window and filter the users which has more then 100 POST requests.

[DalJeanis]

You haven't told us how often you want to check for these spammers, or what you want to do when they are detected, so we will just assume you are running a periodic report, say, every 5 minutes, and alerting whenever an IP address has sent more than 100 post requests in the prior 5-minute interval. Since splunk can take a few minutes to index the events in some installations, we will assume that we do our test after a 5-minute lag, so that, for instance, the system will check the interval 6:00-6:05 AM at 6:10 AM...

earliest=-10m@m latest=-5m@m  index=foo "POST" (... any other search terms you want...)
| table _time IPAddress 
| bin _time span=1s
| stats count by  IPAddress _time 
| where count>100

There may be a more efficient way, depending on what information you are indexing. For instance, if this particular IPAddress is an indexed field, you could do tstats on IP address, select only the IP addresses that had more than 100 events in the 5-minute interval...or even some higher cutoff since spammers are unlikely to be bursting for only a single second... and then only do the above calculations for those particular IP addresses.