This is a typical relevant line from logs:
[28/Jun/2017:07:26:04 -0400] conn=9354 op=7 msgId=8 - SRCH base="o=company" scope=2 filter="(&(|(objectclass=mailgroup)(objectclass=person)(objectclass=alias))(!(objectclass=moderatedgroup))(mailalternateaddress=owner-john.doe@company.com))" attrs="cn uid mailaddress uniquemember"
The "attrs=" is the required text and the "cn uid mailaddress uniquemember" -attributes would need to be extracted separetly..the space is the delimiter between them. With the wizard I only able to select one of them. But I need to count them one by one .... | chart count by host,attrib_extraction
Thanks!
Hi,
So assuming your key value extraction is working fine and there is an attribute named attrs already then you can simply do:
| eval attrib_extraction = split(attrs, " ")
If that's not the case you can extract key value pairs from your raw data using the kv command.
Alternatively a simple regex like the following should do the trick too:
| rex "attrs=\"(?<attrib_extraction_all>[^\"]+)\""
| rex max_match=0 field=attrib_extraction_all "(?<attrib_extraction>\S+)"
If you then need to expand this multivalued field simply do:
| mvexpand attrib_extraction
Hope that helps.
Thanks,
J
Hi,
So assuming your key value extraction is working fine and there is an attribute named attrs already then you can simply do:
| eval attrib_extraction = split(attrs, " ")
If that's not the case you can extract key value pairs from your raw data using the kv command.
Alternatively a simple regex like the following should do the trick too:
| rex "attrs=\"(?<attrib_extraction_all>[^\"]+)\""
| rex max_match=0 field=attrib_extraction_all "(?<attrib_extraction>\S+)"
If you then need to expand this multivalued field simply do:
| mvexpand attrib_extraction
Hope that helps.
Thanks,
J
expected output with chart command
cn uid mailaddress sn office
host1 564 444 56 77 2222
host2 57 565 11 676 44
host3 569 66 45 565 33
I have tested your example above in my lab and managed to get the output you are looking for.
Keep in mind it might be easier than the query I'm showing you below, but I don't have all the information about your field extractions, etc.
Hope it helps.
QUERY
| makeresults
| eval raw = "[29/Jun/2017:03:28:37 -0400] conn=16035519 op=32155 msgId=32156 - SRCH base=\"msfwid=1281620,ou=people,o=company\" scope=0 filter=\"(objectclass=)\" attrs=ALL; [29/Jun/2017:03:28:37 -0400] conn=16047221 op=54 msgId=55 - SRCH base=\"msfwid=785323,ou=people,o=company\" scope=2 filter=\"(objectclass=msperson)\" attrs=\"personaltitle givenname msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor\"; [29/Jun/2017:03:28:37 -0400] conn=16035519 op=32154 msgId=32155 - SRCH base=\"msfwid=1487082,ou=people,o=company\" scope=0 filter=\"(objectclass=)\" attrs=ALL; [29/Jun/2017:03:28:37 -0400] conn=16047221 op=53 msgId=54 - SRCH base=\"msfwid=1260584,ou=people,o=company\" scope=2 filter=\"(objectclass=msperson)\" attrs=\"personaltitle givenname msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor\"; [29/Jun/2017:03:28:37 -0400] conn=16047226 op=102 msgId=55603 - SRCH base=\"msfwid=1421236,ou=people,o=company\" scope=2 filter=\"(objectclass=msperson)\" attrs=\"personaltitle msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor\""
| eval raw = split(raw, ";")
| mvexpand raw
| rex field=raw "attrs=\"?(?<attrib_extraction_all>[^\"]+)\"?"
| eval host = if (match(attrib_extraction_all, "ALL"), "myhost1", "myhost2")
| rex max_match=0 field=attrib_extraction_all "(?<attrib_extraction>\S+)"
| chart limit=0 count by host,attrib_extraction
OUTPUT (see picture below)
This works great with that static data you have put after the | eval raw = .... but is it possible to use the data which is filtered out from logs with the / index=log-3155-prod* eventtype=fwdldap-all-prod-hosts sourcetype="ldap-infra:access" laas_appId="ldap-infra/prod/ldap_FWD_PROD" "SRCH base=" /-command?
Also goes to the hosts..hosts is recognized as interesting filed..so I also need it not as static data given by me. Your answer is really appreciated! Thank you very much in advance!
You should be able to append the following to your base search and get the same result:
your base search
| rex "attrs=\"?(?<attrib_extraction_all>[^\"]+)\"?"
| rex max_match=0 field=attrib_extraction_all "(?<attrib_extraction>\S+)"
| chart limit=0 count by host, attrib_extraction
By the way, if you liked the answer please don't forget to mark it as such so that others can benefit from it.
Bull's eye! Great! Thanks!
..here is another example:
---search command:
index=log-3155-prod* eventtype=fwdldap-all-prod-hosts sourcetype="ldap-infra:access" laas_appId="ldap-infra/prod/ldap_FWD_PROD" "SRCH base="
---result:
[29/Jun/2017:03:28:37 -0400] conn=16035519 op=32155 msgId=32156 - SRCH base="msfwid=1281620,ou=people,o=company" scope=0 filter="(objectclass=)" attrs=ALL
[29/Jun/2017:03:28:37 -0400] conn=16047221 op=54 msgId=55 - SRCH base="msfwid=785323,ou=people,o=company" scope=2 filter="(objectclass=msperson)" attrs="personaltitle givenname msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor"
[29/Jun/2017:03:28:37 -0400] conn=16035519 op=32154 msgId=32155 - SRCH base="msfwid=1487082,ou=people,o=company" scope=0 filter="(objectclass=)" attrs=ALL
[29/Jun/2017:03:28:37 -0400] conn=16047221 op=53 msgId=54 - SRCH base="msfwid=1260584,ou=people,o=company" scope=2 filter="(objectclass=msperson)" attrs="personaltitle givenname msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor"
[29/Jun/2017:03:28:37 -0400] conn=16047226 op=102 msgId=55603 - SRCH base="msfwid=1421236,ou=people,o=company" scope=2 filter="(objectclass=msperson)" attrs="personaltitle givenname msmiddleinitial sn cn mail telephonenumber employeenumber uid msfwid businesscategory departmentnumber title building floor"
----field extraction:
(?=[^a]*(?:attrs=|a.*attrs=))^(?:[^"\n]*"){5}(?P<"attrib_extraction">[^"]+)
"attrib_extraction" -needed to added in this way even if inserted as code example...so please ignore the " " here.
Thanks
Thank you! I have tried all the versions but in the end I got the attrs truncated.. 😞
I will try to be more specific..this is the filed extrcation..it works:
(?=[^a](?:attrs=|a.*attrs=))^(?:[^"\n]"){5}(?P[^"]+)
"search commad"..."SRCH base=" | chart count by attrib_extraction ---gives the same without the commands you have posted.:
attrib_extraction count
* 69994
* aci 120
* modifytimestamp 312
1.1 76545
1.1 uid cn objectclass 3
assistant l cn mail maildrop personaltitle secretary title uid workertype 24
attributetypes objectclasses ditcontentrules 90
...so the extracted character line is handled as one..what I would need, is to count every attribute separately. If the user / program does a search for 20 attribute than after the atrrs="
20 attribute will be listed. But it might be a * if they want to have all attribute of the object(s). I hope that was more clear now. Thank you for your help! Best regards! Károly
Hi, sorry I'm still confused.
Can you post another example but using the code button (the one with 0's and 1's above) so that especial characters are not escaped?
Ideally I would like to see a couple of log lines and then the expected output in Splunk.
Thanks,
J
For a search-time extraction, try this.
... | rex field=attrs "(?<cn>[^\s]+)\s(?<uid>[^\s]+)\s(?<mailaddress>[^\s]+)\s(?<uniquemember>.+)" | chart count by host cn uid mailaddress uniquemember
Thanks, but the problem is that we have got more then 400 kind of attributes..not only this 4 what you see in the example line. It can be * as well if the LDAP search is for to get back any attribute of the object which mathches as well or it can be anything else like attrs="memberurl uniquemember objectclass uid cn" or attrs="uid cn" etc.