I am trying to incorporate company name information into sales/subscription charts for business leaders to use in presentations. Because our corporate networking structure is from hell, Ive had to do this by automating an SQL table export to CSV on the SQL Server and having the universal forwarder on the same server read the table export and forward it to Splunk.
Ive found the forwarder will resend the entire CSV in most cases except where rows are added to the end of the file (ie: new firms are added to the db). Further, when it sends just the new rows, the forwarder omits the header row in the CSV so now using multikv is a pain. Since I can never be sure what the format of the latest data will be (will there be headers? Will I have to append the new rows or can I just use the entire file) I am trying to see if I can just make the forwarder always send the entire CSV to the indexer regardless of what changes were made to the underlying data. The amount of data being sent is trivial, about 90-100 rows. The most we would see in this table would be a couple thousand.
... View more