I have a csv log file with four different types of row entries. These are identified by the first item in each line - HR (Header), RR (Request), SR (Send), FS (Footer). Each line has different fields and a different number of fields. I'd like to process these to one index and different custom fields based on the HR/RR/SR/FS value. So far I'm succeeded in processing both lines in the example but the fields are being re-used from the HR row for the RR row. Any ideas?:
Data example (2 lines with carriage return ending the line):
HR, ID123, CredABC1, SoftwareName, StatusActive, 2011-11-15
RR, Conv123, ID123, ID456, 2011-11-15T09:01:15, RequestType1, FailurePoint40, 2011-11-15T09:02:20, ABCD-1234-EFG-567, FailureOnRetry, Error70, RecipientUnavailable, Retried5Times, ABCD-1234-EFG-561
transforms.conf:
[header]
REGEX = HR
DELIMS = ","
FIELDS = "RecordType", "SenderID", "SenderCredentials", "SoftwareInstalled", "ApplicationStatus", "ReportTimePeriod"
[request]
REGEX = RR
DELIMS = ","
FIELDS = "RecordType", "ConversationID", "RequestorID", "SenderID", "RequestTime", "RequestType", "RequestFailurePoint", "RequestFailureTime", "RequestFailureMessageID", "RequestFailureType", "RequestErrorCode", "RequestErrorDescription", "RequestRetryCount", "RequestMessageID"
props.conf
[header]
TIME_PREFIX = ^
REPORT-csv = record
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT= %Y-%m-%d
SEGMENTATION = outer
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
[request]
TIME_PREFIX = ^
REPORT-csv = record
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT= %Y-%m-%d
SEGMENTATION = outer
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
inputs.conf
[default]
host = machine_name
[monitor://C:\Inetpub\wwwroot\splunk_logs]
disabled = false
index = main
host = abcd
sourcetype = record
