All Apps and Add-ons

JuniperSA not showing some roles from the log files

dglinder
Path Finder

My engineer reported that our JuniperSA reports were starting to look 'sparse' and seemed to be missing data. The syslog data is still being collected, no changes to the logging were knowingly made, and the raw data within Splunk matched what was being sent. Still, the reports were missing some role names and other data.

Here is some sample log entries:

Feb  4 08:41:49 A.A.A.A Juniper: 2014-02-04 02:41:49 - NYC - [X.X.X.X] username1(Security Group Alpha)[Role name one] - Login succeeded for username1/Security Group Alpha from X.X.X.X.
Feb  4 09:03:38 B.B.B.B Juniper: 2014-02-04 03:03:38 - ive - [Y.Y.Y.Y] username2(Security Group Beta)[Role-name two] - Login succeeded for username2/Security Group Beta from Y.Y.Y.Y.
Feb  4 08:49:32 C.C.C.C Juniper: 2014-02-04 02:49:32 - ive - [Z.Z.Z.Z] username3(Security Group Gamma)[Role-name three] - Login succeeded for username3/Security Group Gamma from Z.Z.Z.Z.

And here is the transforms.conf file that should be matching the role (the text within the square brackets):

[junipersa-role-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\sive\s-\s\[\d+.\d+.\d+.\d+\]\s[^\(\)\s]+\([\s\d\w]+\)\[([\s\d\w\,]+)\]\s-
FORMAT = role::$1
0 Karma
1 Solution

dglinder
Path Finder

After a bit of work we determined that the REGEX fields were being too specific and were not accounting for non-HA nodes. I'm not a Juniper guy, but apparently the "- ive -" means it's in an HA arrangement, and the "- NYC -" (or other customer specified string) are present in non-HA nodes.

We also noticed that the role name (the original reason for the call) was being missed on otherwise good log entries because the role name contained a hyphen ("-"). We updated the transforms.conf file to account for these and the roles started populating.

Looking around at the rest of the REGEX in that file we noticed that the same fields were used in other lines so they were updated accordingly also.

The basic REGEX changes were:
1. Change the search for "\sive\s" to "\s\S+\s" - a string beginning with a single space, then 1 or more non-space characters, terminated by a space.
2. Adding the hyphen character as an acceptable character when extracting the role field: "([\s\d\w\,-]+)".

Here is the final transforms.conf file we ended up with:
[junipersa-host]
DEST_KEY = MetaData:Host
REGEX = (\d+.\d+.\d+.\d+)\sJuniper:\s
FORMAT = host::$1

[junipersa-client-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[(\d+.\d+.\d+.\d+)\]\s([^\(\)\s]+)
FORMAT = clientip::$1 user::$2 

[junipersa-realm-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[\d+.\d+.\d+.\d+\]\s[^\(\)\s]+\(([\s\d\w]+)\)
FORMAT = realm::$1 

[junipersa-role-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[\d+.\d+.\d+.\d+\]\s[^\(\)\s]+\([\s\d\w]+\)\[([\s\d\w\,-]+)\]\s-
FORMAT = role::$1

[junipersa-session-info]
REGEX = session:([^\\)]+)
FORMAT = sessionid::$1

[junipersa-secure-meeting-info]
REGEX = Created\smeeting\s\'([\s\d\w]+)\s\(([0-9]+)\)\'
FORMAT = meeting_name::$1 meeting_id::$2

[sa_sourcetyper]
DEST_KEY = MetaData:Sourcetype
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+
FORMAT = sourcetype::juniper_sa_log

Hope this helps other users.

View solution in original post

0 Karma

dglinder
Path Finder

After a bit of work we determined that the REGEX fields were being too specific and were not accounting for non-HA nodes. I'm not a Juniper guy, but apparently the "- ive -" means it's in an HA arrangement, and the "- NYC -" (or other customer specified string) are present in non-HA nodes.

We also noticed that the role name (the original reason for the call) was being missed on otherwise good log entries because the role name contained a hyphen ("-"). We updated the transforms.conf file to account for these and the roles started populating.

Looking around at the rest of the REGEX in that file we noticed that the same fields were used in other lines so they were updated accordingly also.

The basic REGEX changes were:
1. Change the search for "\sive\s" to "\s\S+\s" - a string beginning with a single space, then 1 or more non-space characters, terminated by a space.
2. Adding the hyphen character as an acceptable character when extracting the role field: "([\s\d\w\,-]+)".

Here is the final transforms.conf file we ended up with:
[junipersa-host]
DEST_KEY = MetaData:Host
REGEX = (\d+.\d+.\d+.\d+)\sJuniper:\s
FORMAT = host::$1

[junipersa-client-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[(\d+.\d+.\d+.\d+)\]\s([^\(\)\s]+)
FORMAT = clientip::$1 user::$2 

[junipersa-realm-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[\d+.\d+.\d+.\d+\]\s[^\(\)\s]+\(([\s\d\w]+)\)
FORMAT = realm::$1 

[junipersa-role-info]
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+\s-\s\[\d+.\d+.\d+.\d+\]\s[^\(\)\s]+\([\s\d\w]+\)\[([\s\d\w\,-]+)\]\s-
FORMAT = role::$1

[junipersa-session-info]
REGEX = session:([^\\)]+)
FORMAT = sessionid::$1

[junipersa-secure-meeting-info]
REGEX = Created\smeeting\s\'([\s\d\w]+)\s\(([0-9]+)\)\'
FORMAT = meeting_name::$1 meeting_id::$2

[sa_sourcetyper]
DEST_KEY = MetaData:Sourcetype
REGEX = Juniper\:\s[^\s]+\s[^\s]+\s-\s\S+
FORMAT = sourcetype::juniper_sa_log

Hope this helps other users.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...