Using a lookup for search

[cthulhucalling]

I have a small CSV file with common attack signatures in them that I have uploaded as a lookup called web_attack_signatures.csv:

uri_path, description
*/etc/passwd*, Passwd file access attempt
*phpadmin*, PHPadmin access attempt

Ignoring the fact that these signatures are inefficient, the idea is that the lookup feeds the search, looking for any attack signature matches matches in my web logs. The relevant field in my logs is also called uri_path. So far I have been able to pull relevant events using this query:

earliest=-1d index=web [| inputlookup web_attack_sigs.csv | fields uri_path]

But I cannot get get the description field to be added to the results. I've tried adding "description" to the fields command with 0 results, and tried piping the events to another lookup command but that didn't work either. Can anyone help?

[jnudell_2]

Hi @cthulhucalling ,

As some of the others have said, you can have wildcards in your lookups, but you have to specify it in the advanced settings in the web interface (UI). Here is an example of how you would do that:

Once you have the lookup configured, you can use the following search logic:

| lookup web_attack_sigs uri_path OUTPUT description

Here is what it looks like:

[FrankVl · Accepted solution]

When adding the description to the output of the subsearch, it will also use that for filtering, which is not what you want. If I understand correctly, you want to filter the events for the signatures and then add the description to the results, right?

I think the following should work (might require some configuration change on your lookup, to enable wildcard matches):

earliest=-1d index=web 
| lookup web_attack_sigs.csv uri_path OUTPUT description
| search description=*

Possibly the search runs more quickly if you keep the subsearch to do the initial filtering:

earliest=-1d index=web [| inputlookup web_attack_sigs.csv | fields uri_path]
| lookup web_attack_sigs.csv uri_path OUTPUT description

[cthulhucalling]

The issue appears to be that I'm wildcarding the signatures. If I change /etc/passwd to just /etc/passwd, I do get the description when I do this:

earliest=-1d index=web [| inputlookup web_attack_sigs.csv | fields uri_path]
| lookup web_attack_sigs.csv uri_path OUTPUT description

I'm guessing that because of the wildcards, it's not an exact match when the lookup occurs, so it's not a hit and the description isn't added. The issue is that I get only results where the request is exactly for /etc/passwd.

[cthulhucalling]

The first query returns anything with "description" in the event. The second one I had tried yesterday. It returns relevant results but the description line is not being added to the events.

[FrankVl]

Did you configure your lookup to allow wildcard matches on the uri_path field? Otherwise the lookup command will not work as expected.

[cthulhucalling]

I don't have shell access to that particular search head, so I can't do that.

[FrankVl]

You can configure that in the GUI in the advanced options of a lookup definition.

[cthulhucalling]

Adding WILDCARD(uri_path) to the lookup definition did the trick! Thank you!

[MuS]

Hi cthulhucalling,

try this:

earliest=-1d index=web [| inputlookup web_attack_sigs.csv | format]

by using format you will get a list of

( ( uri_path="*/etc/passwd*" AND description ="Passwd file access attempt" ) OR ( uri_path="*phpadmin*" AND description ="PHPadmin access attempt" ) )

That will be added to your base search. More details here https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Format

Hope this helps ...

cheers, MuS

[cthulhucalling]

I tried this query. Unfortunately, it returned 0 results.

[MuS]

Well, it also looks like I completely misunderstood your question 😉 Sorry for that.

cheers, MuS