Hello there!
I am using Splunk Enterprise 7.2.0. I am trying to set up the following flow: I have an index called raw_dns which is continuously populated with events. Every one of this events has a domain field. I also have a custom lookup that uses an external service and can verify if the provided domain is infected or not. This lookup returns two things: status (infected or not), categories (list of keywords).
I had to configure the app so that every X minutes, the events received in the last X minutes to be analyzed by the lookup and then put in another index. My approach was the following:
- created a new index
- created a savedsearch that runs every X minutes and does what I said above
- enabled summary indexing
Everything worked well, the index was being populated, but after a period my manager said he also wants some of the fields from the original event (the events in the summary index only had the domain, the status and the categories list). I tried to do this over and over again and I think I found the problem but I don't know how to fix it. My configuration is as follows:
indexes.conf
[domains_infected]
thawedPath = $SPLUNK_DB/domains_infected/thaweddb
homePath = $SPLUNK_DB/domains_infected/db
coldPath = $SPLUNK_DB/domains_infected/colddb
transforms.conf
[domain_lookup]
external_cmd = domain_lookup.py domain
external_type = python
fields_list = domain, domain_status, domain_categories
savedsearches.conf
[Domains Status (5 minutes cron)]
search = index=raw_dns | table domain | lookup domain_lookup domain | makemv delim="," domain_categories | makemv delim="," domain_status
description = Domains Status (5 minutes cron)
dispatch.latest_time = now
dispatch.earliest_time = -5m
enableSched = 1
cron_schedule = */5 * * * *
action.summary_index = 1
action.summary_index._name = domains_infected
action.summary_index.generator = savedsearch
I think the problem is that is used | table domain in the search. However, I tried each of the following:
- remove | table domain
- use | table *
- use fields - raw
None of the above ideas worked and by not working I mean that the summary index was not being populated with data anymore.
Where's the catch or what am I missing?
Thanks!
... View more