I have a table of ~11 million rows and 53 columns. The table looks like:
entity_id | week_1 | week_2 | ..... | week_52
abcd | 35874 | 587489 |.......| 5478
While we discover new entities with splunk the table can grow up to ~30 million rows.
I was wondering the following:
-. What is the maximum number of lookup output fields that MySQL connector can support?
-. Can I have up to 52 or 365 look up output fields?
-. what is the recommended maximum number of rows a lookup table can have?
-. If new entities and weekly data are discovered can Splunk MySQL connector insert the new entities and weekly data in the lookup table that is stored in MySQL?
-. In case we need to update, delete or insert some weekly data can Splunk MySQL connector perform updates, delete, and insert of this magnitude?
-. Is there any other approach you might recommend?
Thanks,
Lp