Splunk Search

How to parse wtmp file

jwalzerpitt
Influencer

I am ingesting the non-binary wtmp file in Splunk and was able to two generic fields: 1) priority = auth. (4 unique), and 2) source IP.

I then broke down each priority to view the unique messages inside each one and identified the relevant messages I'd like to extract fields from. Unfortunately, the format of the file loses its consistency after the ID # auth., where the message starts. What I'd like to extract is the username, but the problem is username is not in the same place in the message for each priority. With that I don't believe there is a generic regex that will capture username, but would to be proven wrong!

So, digging in more, I flagged the following messages which I'd like to extract the username from (and username always follows "for"):

Accepted keyboard-interactive for
Accepted password for
Failed keyboard-interactive for
Failed gssapi-with-mic for
Failed password for
Failed keyboard-interactive for
Failed gssapi-with-mic for
Failed publickey for
Failed password for

Moving forward, was trying to figure out the best way to move forward on this. Do I create separate sourcetypes for these specific logs (was going to look to reverse engineer the Symantec for Spunk app as there are multiple sourcetypes defined - I have my SEP logs funneling through the app and it does a good job breaking the logs out, but Symantec's log format is comma deliminated, making it easier), or do I try and create multiple regexes for one sourcetype (is this even possible?)?

Any ideas would be greatly appreciated.

Thx

0 Karma

jwalzerpitt
Influencer

Rich,

Thx for the reply.

1) Tried the rex and it's not returning the username field
2) Unfortunately, the username doesn't always follow "for" (which adds to the frustration)
3) Sample info below:

Dec 19 14:14:27 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:27 sshd[5977]: [ID 649047 auth.info] AFS Ignoring superuser root
Dec 19 14:14:27 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:28 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:28 sshd[5977]: [ID 800047 auth.info] Disconnecting: Too many authentication failures for root
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] Illegal user admin from x.x.x.x
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] input_userauth_request: illegal user admin
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] Failed none for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5983]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5984]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5985]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5986]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5987]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5988]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Disconnecting: Too many authentication failures for admin
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Illegal user admin from x.x.x.x
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] input_userauth_request: illegal user admin
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed none for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5992]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5993]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5994]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5995]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5996]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5997]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:35 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2

0 Karma

richgalloway
SplunkTrust
SplunkTrust

How about this?

 rex field=priority "(?:for|user|superuser) (?<username>\S+)"

It will return "from" as the user name in the "Failed password" events, however, since that event seem to have no user name in them.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jwalzerpitt
Influencer

Improved search when I drop the field=priority. I am seeing unique usernames, but also getting some non-usernames, such as:

  • Timeout before authentication for x.x.x.x
  • the word "for"
  • log files other than the ones listed in original message:

Accepted keyboard-interactive for
Accepted password for
Failed keyboard-interactive for

That's where the frustration comes in as the regex is getting a majority of the valid usernames, but it's still grabbing some values that aren't usernames.

And for edification on my part, the ?: is a non-capturing subpattern that is looking for "for|user|superuser" as a starting point, and then matches on everything after up untila nd including the space, correct?

Thx again

0 Karma

richgalloway
SplunkTrust
SplunkTrust

It would help to see the raw data, but this should get you started...

rex field=priority " for (?<username>.*?)"
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...