We have a logfile that logs the following two lines per logical unit of work completed by the application server. In the report and dashboard that we'd like to produce, we somehow would like to merge the FROM data to the TO format so that it's searchable. We've thought about writing an external script to process the log and load the formatted data into splunk as an additional event source, but we'd ideally prefer to utilize splunk to merge the data and store it as a new event.
FROM:
[00:00:42.839] {Correlation_Id=7437c4c7-acad-6fec-bc4d-470905a6642fd, Referrer=null, name=null, first_name=John, last_name=Doe, contact_time=Evening, callOptIn=null, email=jdoe@domain.com, phone=123-123-1234, phone2=, fax=null, address_1_city=New York, address_1_state=NY, address_1_street1=New York St, address_1_street2=null, address_1_zip=12258, ip_address=xxx.xxx.xxx.xxx, affiliate_id=1234, lid=null, link_id=null, custom_subaff=null, custom_campaign=null, config_id=1, type=Complete, import=null, redirect=-1, source=null, distributionCap=1, licenseKey=, agent_affiliate_id=null, searchignite_client_id=null, searchignite_token=null, how_hear=null, licenseNumbers=, captiveCompanyIDs=1}
[00:00:42.852] {FAILURE=>, Correlation_Id=7437c4c7-acad-6fec-bc4d-470905a6642fd, Problem="(BatchedLeadValidator)=Affiliate has exceeded his monthly limit}"
TO:
2012/05/06 00:00:42.839 name=FAILURE event_id=7437c4c7-acad-6fec-bc4d-470905a6642fd end_time="2012/05/06 00:00:42.852" error="(BatchedLeadValidator)=Affiliate has exceeded his monthly limit" Referrer=null first_name=John last_name=Die contact_time=Evening callOptIn=null email=jdoe@domain.com...and so on.
Any suggestions on how to approach this would be great.
Thanks!
