How to extract fields of variable length?

[kirkbates]

I am new to Splunk and am working with DTS Compliant formatted logs generated from Microsoft Network Policy Server and am trying to parse out the CLI commands in the Cisco-AV-Pair field in the log that is receiving CLI command session history from Cisco Nexus OS 7 network devices that get logged to my NPS RADIUS server. The server has the Splunk Universal Forwarder that gets the information to Splunk. Once in Splunk, I need to extract fields of variable length to generate an audit trail for CLI commands entered during config and management sessions.

The fields that I need to extract are Acct-Session-Id as sometimes the device will put IP and some extra info, so it becomes variable length at times. The one I need the most is the Cisco-AV-Pair as it's the field that the CLI commands show up. This is a variable length field, and it's the one i'm having the most difficulty with. I do have the Cisco app installed in my Splunk instance and it's working correctly for my ASR-1001 routers, but not my Nexus OS 7 devices, thus why I'm asking here. Cisco ASR routers are using the <b>ARCHIVE</b> command to dump the CLI session history to the AAA Accounting section, which goes to the accounting port on my NPS RADIUS server. The Cisco Nexus OS devices DO NOT have an ARCHIVE function, and also DO NOT log session histories the same way the ASRs do. Any help is greatly appreciated.

I have used the automatic tool to select certain fields that are a fixed length, but the fields that I need the most to extract are variable length, and I just want to pull all the information between the <b><HEADER></b> and the <b></TRAILER></b> on the fields that I mentioned. When I try to extract between the > and the <, I get errors trying to save the field extraction and also under the XML code for a dashboard. Is there another method for extracting fields between the > and the < ?

Attached is an example of a log entry.

<Event><Timestamp data_type="4">08/21/2015 14:24:41.913</Timestamp><Computer-Name data_type="1">RADIUS01</Computer-Name><Event-Source data_type="1">IAS</Event-Source><User-Name data_type="1">FirstNameLastName</User-Name><NAS-Port-Type data_type="0">5</NAS-Port-Type><NAS-Port data_type="0">0</NAS-Port><Acct-Status-Type data_type="0">2</Acct-Status-Type><Acct-Session-Id data_type="1">192.168.0.1@pts/0</Acct-Session-Id><Acct-Authentic data_type="0">1</Acct-Authentic><NAS-IP-Address data_type="3">192.168.0.1</NAS-IP-Address><Client-IP-Address data_type="3">192.168.0.2</Client-IP-Address><Client-Vendor data_type="0">9</Client-Vendor><Client-Friendly-Name data_type="1">Router-Name</Client-Friendly-Name><Cisco-AV-Pair data_type="1">accounting:accountinginfo=configure terminal ; interface Ethernet1/41 ; description DEVICENAME (SUCCESS)</Cisco-AV-Pair><Proxy-Policy-Name data_type="1">Use Windows authentication for all users</Proxy-Policy-Name><Packet-Type data_type="0">4</Packet-Type><Reason-Code data_type="0">0</Reason-Code></Event>

[somesoni2]

Try something like this

your base search | rex "Acct-Session-Id[^\>](?<AcctSessionId[^\<]+)"

[kirkbates]

I ended up using something like this:

 ^(?:[^>\n]*>){28}\w+:\w+=(?P<RADIUSExtractCLICommand>[^>]*)

This is closer to what I needed. The difficulty I have, and it's entirely cosmetic is that some lines end with (SUCCESS), but some don't, so there's extra junk as I'm using the > after the closing tag as the final delimiter as I can't put in a < as it screws up the ability to save the field extract. Is there a way to use an ASCII code as the delimiter without physically typing it on the keyboard? Similar to typing ALT + 0169 to get a ©?