I have searched through the knowledge base and have tried a number of things to fix my issue. I have not found my answer....so I am asking for help:
I created a lookup table file using a CSV called "shunlist.csv".
shunlist.csv format (first 5 lines):
srcip,timestamp,info
1.34.83.14,2015-09-11 06:02:53,SSH Brute Force
1.93.11.145,2015-09-03 15:50:58,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.93.20.160,2015-09-15 19:30:40,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.233.92.197,2015-09-06 19:52:15,SSH Brute Force
The following commands work fine:
| inputlookup shunlist.csv| table *
| inputlookup shunlist.csv | format
When I search using the following command, I get results, but I do not see the info field (from the CSV file) in the list of fields:
index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr]
I try this command and I only see the srcaddr and dstaddr fields. The info field does not show up.
index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] | fields srcaddr dstaddr info
Any assistance will be appreciated. Thanks!
I guess you're doing two things here-
1) Filter the flow logs to show only from dstaddr present in the lookup (in field srcip) [Done using subsearch below]
2) Enrich the filter data by adding info field from the lookup. [Done using looku command below]
So, try something like this
index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] |table _time protocol srcaddr srcport dstaddr dstport
| lookup shunlist.csv srcip as dstadrr OUTPUT info
Does the "info" field exist in your base search or do you want to add it here by a lookup?
Your inputlookup results in:
index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT AND (dstaddr=A OR dstaddr=D OR...)
It adds a filter using dstaddr values that exist in your lookup file.
To add the info field you should use something like this:
index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT
| lookup shunlist.csv srcip as dstadrr OUTPUT info
Thanks for the answer HeinzWaescher.
Here is what I am trying to accomplish:
Search through flow logs and match "dstaddr" to the "srcip" field in shunlist.csv. If a match is found, display the src and destination information from the flow in addition to the the "info" field from the match line in the shunlist.csv file. Here's the table statement I want to end up with:
|table _time protocol srcaddr srcport dstaddr dstport info
I have done this before using known bad ssl certificates lookups but having issues on another splunk setup. (http://vitalisec.blogspot.com/2014/07/ssl-blacklist-bro-and-splunk.html)