Splunk Search

How do I sum columns in a multiline event

cdo_splunk
Splunk Employee
Splunk Employee

We have a script that gets the output of the command below and output it as a single event with multiline

ps -wweo uname,pid,psr,pcpu,cputime,pmem,rsz,vsz,tty,s,etime,args

I would like to write a search that filters through the lines to look for the the a certain string looks for a particular sting in a certain column, if exists sum the value of another column and return the string and the sum. E.g.

USER PID PSR pctCPU CPUTIME pctMEM RSZ_KB VSZ_KB TTY S ELAPSED COMMAND ARGS
root 1848 0 0.0 00:00:00 0.0 2128 476368 ? S 18:19:03 automount --pid-file_/var/run/autofs.pid
apache 2082 0 0.1 00:00:37 0.4 17868 402748 ? S 06:26:57 httpd <noArgs>
apache 2083 0 0.1 00:00:36 0.4 17872 402748 ? S 06:26:56 httpd <noArgs>
apache 2084 0 0.1 00:00:37 0.4 17908 402932 ? S 06:26:56 httpd <noArgs>
apache 2383 0 0.1 00:00:36 0.4 17872 402748 ? S 06:25:53 httpd <noArgs>
root 4951 0 0.0 00:00:00 0.0 1300 79968 ? S 18:17:40 sshd <noArgs>
root 5354 0 0.0 00:00:01 0.0 2696 91304 ? S 18:15:48 sendmail: accepting_connections
smmsp 5361 0 0.0 00:00:00 0.0 2108 82756 ? S 18:15:48 sendmail: Queue_runner@01:00:00_for_/var/spool/clientmqueue

The result of the search would be:

.4 httpd 
Tags (3)

bshuler_splunk
Splunk Employee
Splunk Employee
|localop| stats count | eval data="USER PID PSR pctCPU CPUTIME pctMEM RSZ_KB VSZ_KB TTY S ELAPSED COMMAND ARGS
root 1848 0 0.0 00:00:00 0.0 2128 476368 ? S 18:19:03 automount --pid-file_/var/run/autofs.pid
apache 2082 0 0.1 00:00:37 0.4 17868 402748 ? S 06:26:57 httpd <noArgs>
apache 2083 0 0.1 00:00:36 0.4 17872 402748 ? S 06:26:56 httpd <noArgs>
apache 2084 0 0.1 00:00:37 0.4 17908 402932 ? S 06:26:56 httpd <noArgs>
apache 2383 0 0.1 00:00:36 0.4 17872 402748 ? S 06:25:53 httpd <noArgs>
root 4951 0 0.0 00:00:00 0.0 1300 79968 ? S 18:17:40 sshd <noArgs>
root 5354 0 0.0 00:00:01 0.0 2696 91304 ? S 18:15:48 sendmail: accepting_connections
smmsp 5361 0 0.0 00:00:00 0.0 2108 82756 ? S 18:15:48 sendmail: Queue_runner@01:00:00_for_/var/spool/clientmqueue" | rex max_match=9999 field=data "(?P<lines>[^\n]+)" | table lines | mvexpand lines | rex field=lines "^(?<USER>\S+)\s(?<PID>\S+)\s(?<PSR>\S+)\s(?<pctCPU>\S+)\s(?<CPUTIME>\S+)\s(?<pctMEM>\S+)\s(?<RSZ_KB>\S+)\s(?<VSZ_KB>\S+)\s(?<TTY>\S+)\s(?<S>\S+)\s(?<ELAPSED>\S+)\s(?<COMMAND>\S+)\s(?<ARGS>.*)" | fields - lines | search USER!="USER"

woodcock
Esteemed Legend

Please re-edit your question; it has obvious cut/paste errors and is incomplete/incomprehensible. For example, you do not list either field name for your example.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...