All Apps and Add-ons

regex - bash history

szaboszilard
Path Finder

Hi geeks,

I have a file included some different lines about users command history. Example:

2014-02-07T08:37:44.943764+01:00 hostname2 [audit / as user2/15313 on /192.168.199.211:41276->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:37:44.923455+01:00 hostname2 [audit / as user2/15313 on /192.168.199.211:41276->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:37:43.810067+01:00 hostname2 [audit / as user2/15276 on /192.168.199.211:41275->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:37:43.808603+01:00 hostname2 [audit / as user2/15276 on /192.168.199.211:41275->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:35:47.141732+01:00 hostname2 [audit / as user2/14955 on /192.168.199.211:41061->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:35:47.123633+01:00 hostname2 [audit / as user2/14955 on /192.168.199.211:41061->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:30:51.569551+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: su -
2014-02-07T08:30:45.975700+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: su -
2014-02-07T08:30:38.609552+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: df -h
2014-02-07T08:30:36.730315+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22] /home/user1: ls -ali
2014-02-06T12:04:54.717480+01:00 hostname3 [audit / as user1/6898 on /192.168.200.201:9606->192.168.240.243:22] /home/user1: PROMPT_COMMAND='pwd>&7;kill -STOP $$'
2014-02-07T08:33:18.764505+01:00 hostname3 [audit user1/23338 as user1/23387 on pts/1/192.168.200.201:2621->192.168.240.243:22]: #=== session closed ===
2014-02-07T08:33:18.042308+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/]: #=== session closed ===
2014-02-07T08:33:17.107657+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/]: #=== session closed ===
2014-02-07T08:33:09.716782+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: cat /var/log/userlog.info 
2014-02-07T08:33:05.195923+01:00 hostname2 [audit / as user2/14653 on /192.168.22.201:35868->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:33:02.553277+01:00 hostname2 [audit / as user2/14616 on /192.168.22.201:35865->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:33:02.551895+01:00 hostname2 [audit / as user2/14616 on /192.168.22.201:35865->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:33:00.749622+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: teszt
2014-02-07T08:32:58.262602+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: id
2014-02-07T08:32:57.384724+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: ls -ali
2014-02-07T08:32:55.758173+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/] /root: ls
2014-02-07T08:32:53.056071+01:00 hostname3 [audit root/7513 as root/7513 on pts/2/]: #=== session opened ===
2014-02-07T08:32:52.941806+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/] /root: screen 
2014-02-07T08:32:42.845085+01:00 hostname2 [audit / as user2/14578 on /192.168.20.201:59883->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:42.759075+01:00 hostname2 [audit / as user2/14578 on /192.168.20.201:59883->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:42.670319+01:00 hostname2 [audit / as user2/14541 on /192.168.21.201:59612->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:42.505636+01:00 hostname2 [audit / as user2/14541 on /192.168.21.201:59612->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:41.995335+01:00 hostname2 [audit / as user2/14504 on /192.168.21.201:59611->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:41.993945+01:00 hostname2 [audit / as user2/14504 on /192.168.21.201:59611->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:41.934074+01:00 hostname2 [audit / as user2/14470 on /192.168.20.201:59882->192.168.209.201:22]: #=== session closed ===
2014-02-07T08:32:41.932644+01:00 hostname2 [audit / as user2/14470 on /192.168.20.201:59882->192.168.209.201:22]: #=== session opened ===
2014-02-07T08:32:37.407663+01:00 hostname3 [audit user1/23338 as root/23676 on pts/1/] /root: cat /var/log/userlog.info 

I'm writing an regex, but i have a problem.

\[audit\s(?<src_user>.*)\sas\s(?<dst_user>.*)\son(?<console>\s|\spts/\d)\/(?<src_ip>.*)\:(?<src_port>\d*)-\>(?<dst_ip>.*)\:(?<dst_port>\d*)\](\s|\:\s)(\#\=\=\=\ssession\sclosed\s\=\=\=|\#\=\=\=\ssession\sopened\s=\=\=|(?<src_path>.*)\:(?<command>.*))

If i use switch user command, the process dose not generate any src/dst ip/port in logfile.
My regex don't match on any line. Any idea how can i resolve this?

not matching lines:

2014-02-07T08:33:00.749622+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: teszt
2014-02-07T08:32:58.262602+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: id
2014-02-07T08:32:57.384724+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: ls -ali
2014-02-07T08:32:55.758173+01:00 hostname1 [audit root/7513 as root/7513 on pts/2/] /root: ls

Regards Szilard

Tags (3)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

Instead of writing a rather complex regex, it would probably be easier to write a different one for each pattern, e.g.

props.conf

EXTRACT-foo = \[audit\s(?<src_user>\S+)\sas\s(?<dst_user>\S+)\son
EXTRACT-bill = \son\s(?<console>pts[^\]]+)\]
EXTRACT-bob = \son\s(?<src_ip>[\d.]+):(?<src_port>\d+)-\>(?<dst_ip>[\d.]+):(?<dst_port>\d+)\]
EXTRACT-jim = \]\s(?<path>/\S+)\s(?<command>.*)
EXTRACT-joe = \]:\s\#===\ssession\s(?<session_status>\w+)
EXTRACT-hoss = \s\on\s(?<origin>\S+)\]
EXTRACT-hank = \s\S+\][^:]*:\s(?<action>.*)

This means that not all fields will be present in all events, e.g. src_port etc, and that some extractions will overlap.

The extraction named foo will handle the things that are common to all events, whereas bill and bob will deal with the alternate versions of pts* and ip/port pairs.
jim and joe deal with the different actions (sessions and commands executed). Finally hank & hoss extracts jim/joe and bill/bob information, regardless of format for easier reporting.

Consider this as examples of an approach, rather than The One Way To Do It.

/K

View solution in original post

kristian_kolb
Ultra Champion

Instead of writing a rather complex regex, it would probably be easier to write a different one for each pattern, e.g.

props.conf

EXTRACT-foo = \[audit\s(?<src_user>\S+)\sas\s(?<dst_user>\S+)\son
EXTRACT-bill = \son\s(?<console>pts[^\]]+)\]
EXTRACT-bob = \son\s(?<src_ip>[\d.]+):(?<src_port>\d+)-\>(?<dst_ip>[\d.]+):(?<dst_port>\d+)\]
EXTRACT-jim = \]\s(?<path>/\S+)\s(?<command>.*)
EXTRACT-joe = \]:\s\#===\ssession\s(?<session_status>\w+)
EXTRACT-hoss = \s\on\s(?<origin>\S+)\]
EXTRACT-hank = \s\S+\][^:]*:\s(?<action>.*)

This means that not all fields will be present in all events, e.g. src_port etc, and that some extractions will overlap.

The extraction named foo will handle the things that are common to all events, whereas bill and bob will deal with the alternate versions of pts* and ip/port pairs.
jim and joe deal with the different actions (sessions and commands executed). Finally hank & hoss extracts jim/joe and bill/bob information, regardless of format for easier reporting.

Consider this as examples of an approach, rather than The One Way To Do It.

/K

szaboszilard
Path Finder

Today I Learned Something New 🙂
great, thank you very much

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...