Db lookups data storing

[uhkc777]

I'm creating the DB lookups. It needs to search data from Sys1 and will look for that data in DB of Sys2. It will append the results from Sys2 to the Sys1 index events. My question is does it stores the data from Sys2 ?. If yes, with in the Sys1 index or somewhere else?

[DalJeanis]

I see what you are trying to do, and it doesn't work along with the way that splunk is designed. I'm not saying that you CAN'T do it, I'm saying that it might be expensive and a waste of resources to do it.

When splunk indexes an event, it stores the information, right then, in a bucket, with an index. If you come along later, and want to change that information, or add information to sit "right next to it", then you'd have to create the new information/event records, index them and store them, then DELETE the original information. There's no such thing as an update to a record.

You COULD, if you REALLY wanted to, create a summary index -- technically a FAKE summary index, because it would not be summary in nature -- and collect all the needed information to that new index... but then you would be taking twice as much storage to store the same information. It's not difficult, but it's not really needed for most use cases.

You are probably better off designing a simple macro that contains the language needed to join the Sys1 and SYs2 inputs, and using the macro in all your common searches that need the sys1 and sys2 data to be matched.

[uhkc777]

Thank you for clarifying my doubt. So, It doesn't make any sense to store the data from Sys2 right?. if we agreed to store it in FAKE summary index though storing twice the Sys1 data, how can I do that?.

Thanks,

[DalJeanis]

Your question is a bit like asking "How long is my piece of string supposed to be?" The answer depends on what kind of string and what you are using it for.

What, exactly, is the data in Sys1? What part of that Sys1 data are you going to use to identify the required data from Sys2? Will the cross-tab be static -- in other words, will the same value in Sys1 ALWAYS generate the same data from Sys2 -- or will Sys2 change occasionallyl, or will it keep changing dynamically all the time? When it changes, do you need past events to be kept the same, or so you need them to reflect the updated crosstab? Does the sys2 data need to be indexed, and stored with the event at index time, or can the enrichment be added at search time?

Please update your question with some more details about how you need to use the sys2 data, and then we can be of more help.

[uhkc777]

I want to compare ID Numberss in Sys1 and Sys2. for evry 5 mins we are ingesting the data from sys1. so for every 5 mins, i would like to get those ID nos and check whether those IDs are exisiting in Sys2 or not?. If exists, pull those records and append it to the Sys1 events. you know ID numbers will keep on changing in Sys1 and Sys2. If I want to index Sys2 data from DB Lookups how can I do that?. Does it stores in Sys1 Index or in separate index?.

Thanks,