We receive 45-50 millions of data daily from various log sources(servers, network devices, proxy, cloud). we need to get report for top source and destination IP address on monthly basis.
I have created one report but it takes more than 2 days to complete. sometimes report fails to run due to huge amount of events.
I am sure there might be some way provided by Splunk to get these kind of reports more quickly. I am also working on creating data model but as there many log sources, I am facing some issues doing that.
Meanwhile, can somebody suggest me how can I get these reports with such huge amount of data?
... View more