All Apps and Add-ons

SPlunk DB Lookup

chitra
Explorer

Hi ALL,

I have usecase where i need to display the mis match values between different data source in dashboard.

Use Case:
Consider i have 3 database data sources DS1, DS2 and DS3.
1. DS1 data will index daily at 1 AM. Take the unique Id's say Ds1_ID from this index
2. In DS2 database, I have to ingest only those events whose Ds2_Id matches with Ds1_ID.
3. In Ds3 database, i have to ingest only those events whose DS3_ID matches with Ds1_ID.
4. Using Step 2 and step 3 indexed data, i have to create the mismatch report.

Please suggest the methods to solve this use case.

Tags (1)
0 Karma

woodcock
Esteemed Legend

There is no "natural" way to create index-time filters based on data that is not inside each event.

0 Karma

niketn
Legend

In essence you want to compare DS2_ID with DS3_ID, which contain the same value of DS1_ID.

Step 1) You can create aliases for DS2_ID and DS3_ID as DS_ID based on their index/sourcetypes. (Alternatively you will have to use rename or eval during search time to match the field names for correlation.
Step 2) Run the following stats command which will combine results from DS2 (index/source) and DS3 (index/source) and select only rows with eventcount<=1.

<Your base search with multiple index/sourcetype for DS2 and DS3 respectively> DS_ID=* | stats count(DS_ID) as eventcount values(sourcetype) as SourceTypes matched by DS_ID | search eventcount<=1

You can also use transaction or append or join but they will be expensive queries compared to stats.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

chitra
Explorer

Only Ds1 has events in splunk and other data source are databases.

i need to index only those events whose DS1 id's are matched with database Ds2 Id. i need to correlate splunk field with database column.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...