extracting user, uid and gid from /etc/passwd

[mariof]

Hi all,
I'd like to extract or create fields for username, uid and gid from /etc/passwd. basically I'd like to generate a report that shows all usernames across all my unix/linux systems and any clashes that there may be.
Any ideas?

[Damien_Dallimor]

You can use the Command Modular Input to do this : http://apps.splunk.com/app/1553/

The command you will execute is :

awk -F':' '{ print "user="$1" uid="$3" gid="$4}' /etc/passwd

And then you will have exactly what you want indexed in Splunk , no indexing of unneeded data , and no extractions to write.

A few screenshots :

You could push this out to forwarders on all of your machines using Deployment Manaager , Chef , Puppet etc...

Or, for an agentless approach , you could execute the command remotely over SSH to all of the target machines.So from one central Splunk instance you could setup a stanza for each target machine.

inputs.conf

[command://User Info at 192.168.0.1]
command_args = someuser@192.168.0.1 'awk -F':' '{ print "user="$1" uid="$3" gid="$4}' /etc/passwd'
command_name = ssh
sourcetype = user_info
index = myindex
host = 192.168.0.1
streaming_output = 0
disabled = 1

[command://User Info at 192.168.0.2]
command_args = someuser@192.168.0.2 'awk -F':' '{ print "user="$1" uid="$3" gid="$4}' /etc/passwd'
command_name = ssh
sourcetype = user_info
index = myindex
host = 192.168.0.2
streaming_output = 0
disabled = 1

Here is info on SSH remote command execution and setting up authentication using shared keys so that you do not need to hard code a password into your setup.

http://linuxers.org/howto/how-run-commands-remote-machine-using-ssh

http://linuxers.org/article/ssh-login-without-password

[Damien_Dallimor]

Updated my answer above with deployment approaches.

[mariof]

would i need to use the deployment manager in order to run this on all my machines?

[jtrucks]

Try this:

(?<user>\w+):x:(?<uid>\d+):(?<gid>\d+):.*

You can go to Manager » Fields » Field extractions hit New then add inline the above extract in a single entry for search time extractions.

[somesoni2]

You can install Splunk Forwarders in all your Unix/Linux boxes and make them monitor /etc/passwd file. The entries in the passwd files are ":" separated so once they are indexed in Splunk, you can do field extraction to get all the required fields. Splunk will add host name in all the events based on from where the passwd file was indexed, so you can do comparision and any kind of reporting.

UPDATE
my bad, I should've known.

If you just want to extract the same during any search, jtrucks rex works fine. If you want these fields to be present as field extraction during search time, you can add following to your props.conf, under respective sourcetype. This should extract all the fields and they will be avaialble for searching/reporting.EXTRACT-userName = (?i)^(?P[^:]+)

EXTRACT-password = (?i)^[^:]*:(?P<password>[^:]+)
EXTRACT-uid = (?i)^(?:[^:]*:){2}(?P<uid>[^:]+)
EXTRACT-gid = (?i)^(?:[^:]*:){3}(?P<gid>[^:]+)
EXTRACT-userInfo = (?i)^(?:[^:]*:){4}(?P<userInfo>[^:]+)
EXTRACT-homedir = (?i)^(?:[^:]*:){5}(?P<homedir>[^:]+)
EXTRACT-userShell = (?i)^(?:[^:]*:){6}(?P<userShell>[^:]+)

[mariof]

The suggestion by jtracks with creating an inline extract works to an extent, except that it only extracts the first instance of the line - a bit like ed and then ignores the rest. it works ok if I only have one user added at a time, but may not cope with multiple changes

[mariof]

Firstly, thanks to both of you for your help:
root:x:0:0:root on redacted:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin

[somesoni2]

can you paste some of your events. Will be helpful in generating exact regex.

[jtrucks]

You can have all of those on a single EXTRACT line like:

EXTRACT-passwdfilestuff = (?)^(?\w+):x:(?\d+):(?\d+):.*

[jtrucks]

mariof, does the regex I submitted in the other answer work for you?

[mariof]

Hi somesoni2,
I'm already doing this. I'm also monitoring the group file, but the auto field extract tool is only picking up the root user.