How do I extract two different variations of a timestamp from the same sourcetype?

[dpanych]

For one of our syslog devices, some events that come through only contain the syslog datetime format, while there are others that contain the syslog datetime AND a "timestamp=" field at the end of the event. What would be the best to setup timestamp recognition where it first reads the "timestamp=" field and if the event does not contain that field, then for it to look at the syslog datatime at the beginning of the event. The timestamp field and the syslog datetime are two different formats too. See below for an example.

Sep 23 23:59:57 2016 aa_wlc_01 wms[1234]: <123456> <WARN> <aa_wlc_01 192.168.000.000> |ids| AP(aa:aa:aa:aa:aa:aa@aa-aa-aa): Wireless Bridge: An AP detected a wireless bridge between transmitter aa:aa:aa:aa:aa:aa and receiver aa:aa:aa:aa:00:00. SNR value is 25. Additional Info: BSSID:aa:aa:aa:aa:aa:aa; Channel:1.


Sep 23 23:59:44 2016-09-23 23: 59:44,5 192.168.111.111 CPPM_RADIUS_Accounting_Detail aaa 1 0 id=aaa,session_id=aaa-01-aaa,acct_session_id=aaa\\aaa-aaa,type=aaa,attr_name=aaa-Location-Id,attr_value=aaa.6-aaa-aaa,timestamp=2016-09-23 23:58:35-07

[jdonn_splunk]

Your best bet is to tell Splunk to always use the first timestamp. You can then build reports around the timestamp field at search time, if that is your ultimate goal.

FWIW, Splunk is able to have multiple REGEX per sourcetype. So, you can extract fields from different data formats in the same sourcetype.

[sowings]

Short answer: you don't. Splunk likes to converge on a single timestamp format for a given input stream. In the past, when I've tried to tackle the issue you're seeing, I had to use search time rules to get the second stamp. The _time for the event was still the first time stamp provided by syslog.

If you need to have the event's _time be the value of "timestamp=" (if any), I suggest that you'll have to pre-process these logs into separate files, and apply different props rules there. I haven't seen any other way to tackle that particular issue, up to and including creating my own datetime.xml file. Sorry.