Splunk creates field from from wrong string in my LDAP logs

[stefanlasiewski]

Hi everyone,

I am attempting to graph LDAP transactions from our OpenLDAP LDAP servers, however I'm having trouble getting Splunk to use the correct fields.

In the example below, the first line of the transaction has two IP addresses, IP=192.168.1.111 & IP=0.0.0.0. Splunk assigns the field namedIP to IP=0.0.0.0, but I want it to use the first field IP=192.168.1.111 instead. How can I tell Splunk to match IP=192.168.1.111 & not IP=0.0.0.0?

Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=207 ACCEPT from IP=192.168.1.111:34792 (IP=0.0.0.0:389)
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 EXT oid=1.2.6.1.6.1.1666.37723
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 STARTTLS
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 RESULT oid= err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 TLS established tls_ssf=256 ssf=256
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 BIND dn="" method=128
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 RESULT tag=97 err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH base="dc=example,dc=org" scope=0 deref=0 filter="(objectClass=*)"
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH attr=contextCSN
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=3 UNBIND
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 closed

[aholzer · Accepted solution]

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

• during the search

  your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"

  • or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

[stefanlasiewski]

Thanks. Can you explain what the ?P does in this regex?

[aholzer]

It's not necessary, but it's a habit from an older Splunk version. If I remember correctly it used to be needed to specify using "python" regex expressions, indicating that what was coming next (inside the <> brackets) was the name of the field.

You can simply use ? instead of ?P it'll work the same.

[stefanlasiewski]

Thanks for your tip about the field extractions. When I try the current regex, I don't get a total match. See https://www.regex101.com/r/oI7gI2/1

What I settled on was the following: from IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*?\(IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

[aholzer]

If you are using the field name then I don't know why it's not working. Try putting the "P" back in after the "?" in the capture group ?P

[stefanlasiewski]

What is the technical difference between ?P<MyField> and ?<MyField> without the P?

[stefanlasiewski]

Arg. This software eats the angle bracket characters, and doesn't allow their HTML equivalents.

[aholzer]

you still need to provide the name of the field after the ? using these brackets "<" and ">"

[stefanlasiewski]

Thanks, yes I assigned a name to the field. I just didn't include this in my comment. Thanks.