Hi Splunk Experts,
In Splunk ES I need to count of notable events per sourcetype . I tried different things like checked index=notable but it does not have sourcetype .I also checked API "rest splunk_server=local count=0 /servicesNS/-/SplunkEnterpriseSecuritySuite/saved/searches" as it has rule name which I joined with index=notable but it gives me count by rule_name not as per sourcetype .
Can someone please assist .
Regards
Vikas
Are you talking about the original sourcetype of the event that created the notable?
If yes, you'll need to change your Correlation Searches to generate a field that might be called origin_sourcetype so you can later use it. I don't think you have many other options other than that.
Thank you gfreitas , but how i can achieve this . How I can genrate origin_sourcetype field ? I tried |eval but in notable index this wont work because value of sourcetype in eval is stash . Can you please give some suggestion on this ?
You will need to do it one step earlier, I'll need to change the correlation searches that generate the notables. Example you have a tstats correlation search, you would need to add something like values(sourcetype) as origin_sourcetype.
Example below:
Correlation Search concurrent login attempts detected (original search):
| tstats summariesonly
count from datamodel=Authentication.Authentication by _time,Authentication.app,Authentication.src,Authentication.user span=1s
| drop_dm_object_name("Authentication")
| eventstats dc(src) as src_count by app,user
| search src_count>1
| sort 0 + _time
| streamstats current=t window=2 earliest(_time) as previous_time,earliest(src) as previous_src by app,user | where (src!=previous_src)
| eval time_diff=abs(_time-previous_time)
| where time_diff<300
You could add the following:
| tstats summariesonly
count, values(sourcetype) as origin_sourcetype from datamodel=Authentication.Authentication by _time,Authentication.app,Authentication.src,Authentication.user span=1s
| drop_dm_object_name("Authentication")
| eventstats dc(src) as src_count by app,user, origin_sourcetype
| search src_count>1
| sort 0 + _time
| streamstats current=t window=2 earliest(_time) as previous_time,earliest(src) as previous_src by app,user , origin_sourcetype
| where (src!=previous_src)
| eval time_diff=abs(_time-previous_time)
| where time_diff<300
Are you using the macro below on the correlation search?
| 'map_notable_fields'
If not, add that and in the future you'll be able to find the orig_sourcetype orig_index and orig_source, among other fields you may need.
Thank you jkat54 for the response . I will be using this macro first time, so what it will do ? Do I need to check the summary index or notable index after adding this macro ? If not how and where I can check these new fields ? Also some blogs says this macro is deprecated as I am using 7.x version .
Oh in that case just look for orig_* fields in your notables
Only 3 fields in notable index that starts with orig_ and those are
orig_action_name
orig_rid
orig_sid
Not sure why notable index does not have other fields that you are mentioning
Try running a search and adding the macro to it. See if it creates the new fields.
In short all it does is
| eval orig_sourcetype=sourcetype
And such
Well I have used below query and even after this I am still not able to see new fields
index=notable search_name="ESCU - Detect Outbound SMB Traffic - Rule" | map_notable_fields
No I mean the Correlation search itself
Add the macro to your CS, and verify it adds the fields to the notable index or not.
The CS creates the data in the notable index. If it doesn't eval orig_* fields, then they won't appear in the notable index.
well even after adding macro to my CS as below won't add new fields . Am I doing something wrong as query works absolutely fine.
| from datamodel Identity_Management.All_Identities | eval empStatus=case((now()-startDate)<604800, "Accounts created in last week") | search empStatus="Accounts created in last week"| ctime(endDate)
| ctime(startDate)
| table identity empStatus endDate startDate| map_notable_fields
After running the above CR , I checked the notable index still 3 orig fields which I mentioned earlier except orig_sourcetype.
Your table command is removing the sourcetype field which is required.
Well as you suggested I changed the CR and the macro as it has noop definition. I changed macro to eval orig_sourcetype=sourcetype
. I saved the CR and waited for like 20 min , CR triggers but still no orig_sourcetype filed in the notable index . Will wait and check next morning and post the outcome .
Did you change the "table identity rmpStatus... "line to also have sourcetype?
Sorry you mean to add field sourcetype using table command ?
Yes, otherwise you're doing an eval with no field matching your criteria
Can you run that search and look at what results do you get from it?
As you've discovered, notable events don't have a sourcetype. The events leading to the notable event may have different sourcetypes so it's difficult to assign one to the notable itself.
agreed but I was checking below answer
https://answers.splunk.com/answers/597619/list-all-datamodels-with-the-feeds-index-sourcetyp.html
I used the same and able to get CR per sourcetype and data model . Now I am thinking if somehow i can map or join with notable index based on rule_name .
It would have been good to mention that Answer originally. Can you share your query?