Splunk Search

How to generate a search to find the number of users and their bandwidth consumption using timechart?

barryy
Explorer

I'm running some script to gather logs every 10 mins, one of them is Cisco ASA VPN-sessiondb info, I'd like to use Splunk to generate
1. Number of users timechart
2. bandwidth consumption per user timechart

I'm having challenge with the multi-line log looks like the following

FIREWALLNAME# show clock
10:09:50.431 PDT Mon May 15 2017
FIREWALLNAME# show vpn-sessiondb anyconnect sort p-ipaddress 

Session Type: AnyConnect

Username     : User1                  Index        : 159
Assigned IP  : x.x.x.x           Public IP    : y.y.y.y
Protocol     : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License      : AnyConnect Premium
Encryption   : AnyConnect-Parent: (1)RC4  SSL-Tunnel: (1)AES128  DTLS-Tunnel: (1)AES128
Hashing      : AnyConnect-Parent: (1)SHA1  SSL-Tunnel: (1)SHA1  DTLS-Tunnel: (1)SHA1
Bytes Tx     : 31430848               Bytes Rx     : 2918291
Group Policy : VPN_Service        Tunnel Group : DefaultWEBVPNGroup
Login Time   : 08:27:04 PDT Mon May 15 2017
Duration     : 1h:42m:48s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : ac12121c0009f0005919c8c8
Security Grp : none

Username     : User2               Index        : 157
Assigned IP  : z.z.z.z           Public IP    : v.v.v.v
Protocol     : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License      : AnyConnect Premium
Encryption   : AnyConnect-Parent: (1)RC4  SSL-Tunnel: (1)AES128  DTLS-Tunnel: (1)AES128
Hashing      : AnyConnect-Parent: (1)SHA1  SSL-Tunnel: (1)SHA1  DTLS-Tunnel: (1)SHA1
Bytes Tx     : 52439947               Bytes Rx     : 11136783
Group Policy : VPN_Service        Tunnel Group : DefaultWEBVPNGroup
Login Time   : 08:14:04 PDT Mon May 15 2017
Duration     : 1h:55m:48s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : ac12121c0009d0005919c5bc
Security Grp : none

Omitting more users

So far I was able to extract the usernames ^Username\s+:\s+(?P\w+\s+) but unable to generate a timechart i believe because it's multi-line
I'm also trying to extract the "Bytes Tx" and "Bytes Rx" per user to calculate the Diff's (The commands run every 10 mins) and use it to calculate the average bandwidth consumption.

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.

Second, if that doesn't work, then
1) extract the fields manually with rex into individual MV fields
2) mvzip the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand to break each one into a separate record
4) use another rex or a makemv (depending on how you mvzipped them) to break them into separate fields again.

Once you've done all that, then you'll probably need to
5) sort the records into order (remember to use sort 0 to retain all results).
6) use delta (or more likely streamstats) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.

View solution in original post

0 Karma

rphillips_splk
Splunk Employee
Splunk Employee

this might work out for you , but doesn't include the deltas between collections..you could look at streamstats for that.

sourcetype="vpn-stats" "show clock" | rex field=_raw max_match=0 "Username\s+\:\s+(?<Username>\S+)" | rex field=_raw max_match=0 "Bytes\s+Tx\s+\:\s+(?<Bytes_tx>\d+)" | rex field=_raw max_match=0 "Bytes\s+Rx\s+\:\s+(?<Bytes_rx>\d+)"  | eval zipped = mvzip(Username, Bytes_tx, ",") | eval zipped = mvzip(zipped, Bytes_rx, ",") | mvexpand zipped | makemv delim="," zipped | eval Username = mvindex(zipped, 0) | eval Bytes_tx = mvindex(zipped, 1) | eval Bytes_rx = mvindex(zipped, 2)  | timechart max(Bytes_rx) AS bytes_rx max(Bytes_tx) AS bytes_tx by Username
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.

Second, if that doesn't work, then
1) extract the fields manually with rex into individual MV fields
2) mvzip the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand to break each one into a separate record
4) use another rex or a makemv (depending on how you mvzipped them) to break them into separate fields again.

Once you've done all that, then you'll probably need to
5) sort the records into order (remember to use sort 0 to retain all results).
6) use delta (or more likely streamstats) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.

0 Karma

barryy
Explorer

Thanks for your input DalJeanis.

I've tried to follow the steps but somehow it didn't seperate the fields into their own lines and i can't draw a graph with it.

Here's the search string I used

sourcetype="vpn-stats" "show clock" | rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0 | rex field=_raw "Bytes Tx : (?<VPNTX>\w+)" max_match=0 | eval fields = mvzip(VPNUser,VPNTX) | mvexpand fields | eval fields = mvzip(VPNUser,VPNTX) | table _time fields

The result is like following

_time                   fields
5/15/17 2:57 PM     User1,641674831
                        User2,92081181
                        User3,52384414
                        …
5/15/17 2:57 PM     User1,641674800
                        User2,92081100
                        User3,52384400
                        …
5/15/17 2:57 PM     …

Maybe my mvexpand was wrong?

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

You did well up thru step 3, and misread step 4. I've added one more field and a sample rex to your code...

sourcetype="vpn-stats" "show clock" 
| rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0 
| rex field=_raw "Bytes Tx : (?<VPNTX>\w+)\s+Bytes Rx\s+: (?<VPNRX>\w+)" max_match=0 
| eval myfields = mvzip(mvzip(VPNUser,VPNTX),VPNRX) 
| mvexpand myfields 
| rex field=myfield "(?<VPNUser>[^,]+),(?<VPNTX>[^,]+),(?<VPNRX>.*)" 
| table _time, myfield, VPNUser, VPNTX, VPNRX
0 Karma

barryy
Explorer

Thanks DalJeanis for your quick response. Looks like its working

adonio
Ultra Champion

i believe using the Cisco ASA TA will help with all fields extractions

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...