subsearch on more than two strings

[daishih]

I have three source types I want to search using a user's username. One of the source types only knows the user's IP address but the other two know the user's username. I am trying to construct a search that greps the username and IP then searches all three source types using the username's IP. This search string is what I have been experimenting with:

blocked OR deny [search sourcetype=pan:traffic | rex field=user "mydomain\\\(?.*)" | search username=$userName$ | rename src_ip AS src | fields src]

When I perform a regular search for "blocked OR deny" I get all the results for all users but when I try to add the sub search no results are returned. According to the search tutorial documentation on sub searches this should be working. Any help would be greatly appreciated!

[gabriel_vasseur · Accepted solution]

Have you tried to run the subsearch on its own? If you do, I would expect splunk to complain about an error in the regex.

(?.*) is not a valid regex. Maybe you meant either (.*) or (?:.*) which are valid regexes (the first one is capturing, the second is non capturing) but they are still not suitable for rex because they are not extracting anything.

I think what you need is (?P<username>.*).

Also, I'm not sure about the three backslashes... the third one will just escape the bracket. Maybe you only need two of them?

[daishih]

I ran the following by itself and it returned results for the corresponding username entered:

 "sourcetype=pan:traffic | rex field=user "mydomain\\\(?<username>.*)"" | search username=$userName$ | rename src_ip AS src | fields src" 

I also tried your recommendation which produced the same result:

sourcetype=pan:traffic | rex field=user "psrsdom\\\(?P<username>.*)" | search username=sisbister | rename src_ip AS src | fields src

you must use three backslashes or the regex becomes invalid since the filed user=mydomain\user wont pickup with only one "\". I tested this several times and I'm not sure why the third \ is needed but it doesn't work without it.

Forgoing regex all together I figured I could just do something like this:

block OR deny [search sourcetype=pan:traffic | search user="mydomain\\$username$" | rename src_ip AS src | fields src]

This returns no results either. My intent is to use the src_ip that results from the sub-search to search the logs from all three of my devices. Is there a better way of doing this and why doe the sub-search's search work on it's own but not when I try to use it as a sub-search?

[gabriel_vasseur]

Some comments:

• When you say you tried my recommendation, the command you paste looks nothing like it. That might be because of format issues. I know I had to edit my answer a few times to get it right, using backquotes, so if you used the notification email from splunk it might not have had the updates... I'm not sure!
• I'm quite sure the 3 slashes are wrong 🙂 You're right that one wont' work as it needs to be escaped, but that means you only need two. The third slash will escape the bracket. So \\\(?.*) basically will match a backslash (because of the \\) optionally followed by an opening bracket (because of \(?) followed by any amount of any character (because of the .*) followed by a closing bracket (because of the )), and that will probably not match anything and even if it did it wouldn't extract anything.
• does your data in pan:traffic already have a field called "username"? If yes, why do you need the rex bit at all? If not you need to actually extract it as I initially suggested, with something like sourcetype=pan:traffic | rex field= "mydomain\\(?P<username>.*)" | search username=user1 | rename src_ip AS src | fields src
• you are quite right that you don't need to use rex. If search doesn't work, try where: search sourcetype=pan:traffic | where user="mydomain\\$username$" | .... If that doesn't work try breaking it up: search sourcetype=pan:traffic | where user="mydomain\\" + $username$ | .... You could also try a simple regex command: search sourcetype=pan:traffic | regex user=$username$ | ...

Let's agree on the following steps to make this work, whatever solution you go with:

1) Establish that the subsearch works. It should produce a single column named "src" with one or more values listed. Let's suppose you go with this solution (and suppose it works): search sourcetype=pan:traffic | regex user=user1 | rename src_ip AS src | fields src

2) Add | format to the end of that subsearch like so: search sourcetype=pan:traffic | regex user=user1 | rename src_ip AS src | fields src | format That should produce a search column containing something like (src="1.2.3.4")

3) Try the outer search on its own, pasting the result from the "format" command in step 2: block OR deny (src="1.2.3.4") I'm guessing it should work if your data has a "src" field.

4) Try both searches together, e.g.: block OR deny [search sourcetype=pan:traffic | regex user=user1 | rename src_ip AS src | fields src] (don't use the | format bit any more)

5) If that works, put the exact same thing in your dashboard.

6) If that works, try to replace user1with your $username$ token.

Hope that helps...

[daishih]

Gabriel,

You are correct, it was the formatting of my post. Apparently I could not edit it because I don't have enough rep. I got an error that said I can only post twice a day which makes it extremely hard to converse on here. I have updated my previous post in hopes of showing how I first applied your suggestion.

My final answer:

This works:

blocked OR deny [search sourcetype=pan:traffic | rex field=user "mydomain\\\(?P<username>.*)" | search username=$userName$ | rename src_ip AS src | table src ]

I ended up not using regex to capture the username because the user field already exists in the pan:traffic source type. I ultimately used this to get blocked or deny records from all three source types:

blocked OR deny [search sourcetype=pan:traffic | search user="mydomain\\$userName$" | rename src_ip AS src | fields src]

but this works too:

blocked OR deny [search sourcetype=pan:traffic | where user="mydomain\\$userName$" | rename src_ip AS src | fields src]

The strange thing is this only works in the dashboard panel. Running the following search in splunk returns nothing:

blocked OR deny [search sourcetype=pan:traffic | where user="mydomain\\user1" | rename src_ip AS src | fields src]

Any ideas as to why that is?

Additionally, what does | format do? I saw this mentioned in other sub string posts on this site but the definition was unclear. Adding that to the end of my string just returns a really long repeating row of "((src="x.x.x.x") OR (src="x.x.x.x") OR ...

[gabriel_vasseur]

Use | format for debugging subsearches. It's actually used implicitly by splunk when you do a subsearch. It puts the results of the subsearch in a shape that looks and feel like a search, ready to be executed by the outer search. See the docs.

I am not sure why the "where" example works in your dashboard but not in the search window. Could it be maybe because the backslash doesn't need to be escaped in the search window???

Talking about backslashes, I'm still adamant the three-backslash version cannot have worked... there must be something I'm not aware of. 🙂 Don't forget to "accept" my answer if you're happy it solved your problem. (it's all about karma points!)

[daishih]

Many thanks! I'm not sure why it doesn't work either but at least it works on my dashboard now. Mission accomplished!

[sundareshr]

Where are you getting the value for $userName$? Is this a user entry field in a dashboard?

[daishih]

$username$ is a token in the dashboard. I created a text entry field for that token which will be used to enter usernames into.

[sundareshr]

what is the name of the field that has IP information in your main search?

[daishih]

src_ip is the field I get from the username in the subsearch, scr_ip is what I want to use to search the outer search. I tried:

 block OR deny [search sourcetype=pan:traffic | search user="mydomain\\$username$" | rename src_ip AS src | fields src]

but that didn't seem to work either 😞

[sundareshr]

What do you get when you run this search by itself

block OR deny src=*

[daishih]

If I run that it lists blocked or deny records along with their src_ip.

[sundareshr]

Well then, as long as the values (IP) match, this should work 🙂

block OR deny [search sourcetype=pan:traffic | rex field=user "mydomain\(?.*)"" | search username=$userName$ | rename src_ip AS src | table src]

[daishih]

Unfortunately your string did not work. splunk complained about the regex and the (). However I got it to work with this:

 blocked OR deny [search sourcetype=pan:traffic | rex field=user "mydomain\\\(?P<username>.*)" | search username=$userName$ | rename src_ip AS src | table src ]

many thanks!