All Apps and Add-ons

How to edit my configurations to get search time extractions to apply?

mdsnmss
SplunkTrust
SplunkTrust

I've been working through trying to get some search time extractions to apply using prop.conf and transforms.conf with REPORTS. The source is parsed on a Heavy Forwarder and I have verified the index time extractions as being correct. I have created an add-on (TA) to do search time parsing on the indexer and when searching in Splunk cannot get the extractions to apply. Here is what is in my configs:

props.conf

[tool:ssh]
REPORT-authentication = ssh-login-events, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication

transforms.conf

[ssh-login-events]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?.*?(Accepted|Failed|failure|(?:Invalid user)).*?(\S+)\s+from.*?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\s+port\s+(\S+)\s+\w?\s*(ssh\d))?
FORMAT = app::$1 vendor_action::$3 user::$4 src_ip::$5 src_port::$7 sshd_protocol::$8

[ssh-session-close]
REGEX = .* ((?:session|Connection) (?:opened|closed))(?: for user (\w+))?(?: by \(uid=(\d+)\))?(?: by (\d+\.\d+\.\d+\.\d+))
FORMAT = name::$1 user::$2 user_id::$3 src_ip::$4

[ssh-disconnect]
REGEX = .* (Received disconnect) from (\w+):
FORMAT = name::$1 src_ip::$2

[sshd_authentication_kerberos_success]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authorized\s+to)\s+([^,]+)\,\s+krb5\s+principal\s+([^@]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4" src_user::"$5"

[sshd_authentication_refused]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+refused)\:.*?directory\s+\/home\/([^\/]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[sshd_authentication_tried]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+tried)\s+for\s+([^\s]+)(.*?host\=([^,]+),\s+ip=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))?
FORMAT = app::$1 vendor_action::$3 user::"$4" src_dns::"$6" src_ip::"$7"

[sshd_login_restricted]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Login\s+restricted)\s+for\s+([^:]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[pam_unix_authentication_failure]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+authentication\s+(failure)\;\s+logname\=([^\s]+)?\s+uid\=([^\s]+)?\s+euid=([^\s]+)?\s+tty=([^\s]+)?\s+ruser=([^\s]+)?\s+rhost=([^\s]+)?\s+user=([^\s]+)?
FORMAT = app::"$1" action::$2 src_user::$7 src_dns::$8 user::$9

[pam_unix_authentication_success]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+(session\s+opened)\s+for\s+user\s+([^\s]+)\s+by\s+(.*?)\(uid=\d+\)
FORMAT = app::"$1" vendor_action::"$2" user::$3 src_user::$4

[sudo_cannot_identify]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+auth\s+(could\s+not\s+identify\s+password)\s+for\s+\[([^]]+)
FORMAT = app::"$1" vendor_action::"$2" user::"$3"

[ksu_authentication]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?\'ksu\s+([^']+)\'\s+(authentication\s+failed|authenticated).*?for\s+(\w+)
FORMAT = app::$1 user::"$3" vendor_action::"$4" src_user::$5

[ksu_authorization]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?Account\s+([^:]+)\:\s+authorization\s+for\s+([^@]+).*?(failed|successful)
FORMAT = app::$1 user::"$3" src_user::"$4" vendor_action::$5

[login_authentication]
REGEX = (login)\:.*(failure).*from\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\,\s+(\S+))?
FORMAT = app::$1 action::$2 src_ip::$3 user::$5

default.meta

[]
access = read : [ * ], write : [ admin ]
export=system

Is there something I am missing or a reason these extractions are not appearing in search? Permissions issue? The props and transforms are pulled from the Splunk Add-on for Unix and Linux (Splunk_TA_nix), which is also installed on our indexers. I'm guessing the transforms could be removed in this custom TA since the REPORT will use the same names referenced in the other app, right? Any help is greatly appreciated!

0 Karma
1 Solution

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

View solution in original post

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...