Getting Data In

How do I get my CSV inputlookup search to return a certain field?

tschack_welltok
New Member

I have searched through the knowledge base and have tried a number of things to fix my issue. I have not found my answer....so I am asking for help:

I created a lookup table file using a CSV called "shunlist.csv".

shunlist.csv format (first 5 lines):

srcip,timestamp,info
1.34.83.14,2015-09-11 06:02:53,SSH Brute Force
1.93.11.145,2015-09-03 15:50:58,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.93.20.160,2015-09-15 19:30:40,RA SCAN Unusually fast Terminal Server Traffic Inbound
1.233.92.197,2015-09-06 19:52:15,SSH Brute Force

The following commands work fine:

| inputlookup shunlist.csv| table *
| inputlookup shunlist.csv | format

When I search using the following command, I get results, but I do not see the info field (from the CSV file) in the list of fields:

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr]

I try this command and I only see the srcaddr and dstaddr fields. The info field does not show up.

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] | fields srcaddr dstaddr info

Any assistance will be appreciated. Thanks!

0 Karma

somesoni2
Revered Legend

I guess you're doing two things here-
1) Filter the flow logs to show only from dstaddr present in the lookup (in field srcip) [Done using subsearch below]
2) Enrich the filter data by adding info field from the lookup. [Done using looku command below]

So, try something like this

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT [| inputlookup shunlist.csv | rename srcip as dstaddr | fields + dstaddr] |table _time protocol srcaddr srcport dstaddr dstport 
 | lookup shunlist.csv srcip as dstadrr OUTPUT info
0 Karma

HeinzWaescher
Motivator

Does the "info" field exist in your base search or do you want to add it here by a lookup?
Your inputlookup results in:

index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT AND (dstaddr=A OR dstaddr=D OR...)

It adds a filter using dstaddr values that exist in your lookup file.

To add the info field you should use something like this:

 index=aws-flowlogs source=aws-flowlog dstaddr!=10.0.0.0/8 action=ACCEPT
| lookup shunlist.csv srcip as dstadrr OUTPUT info
0 Karma

tschack_welltok
New Member

Thanks for the answer HeinzWaescher.

Here is what I am trying to accomplish:

Search through flow logs and match "dstaddr" to the "srcip" field in shunlist.csv. If a match is found, display the src and destination information from the flow in addition to the the "info" field from the match line in the shunlist.csv file. Here's the table statement I want to end up with:

|table _time protocol srcaddr srcport dstaddr dstport info

I have done this before using known bad ssl certificates lookups but having issues on another splunk setup. (http://vitalisec.blogspot.com/2014/07/ssl-blacklist-bro-and-splunk.html)

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...