Getting Data In

Does SEDCMD work line-by-line or on the entire event?

Lowell
Super Champion

Out of the box, the unix sed command operates on a line-by-line basis. Is this the same for the SEDCMD setting in props.conf? Or does it operate on the entire event?


Just for the sake of an example. Say I'm using the following unix command to filter out comment lines (lines starting with a ";"). (Technically, this example only clears the comment lines rather than remove them, "grep" would be more obvious choice of unix commands, but splunk doesn't have a GREPCMD, so it's a mute point.)

cat myfile |  sed -re 's/^\s*;.*$//'

Will that work the same way as the following props.conf entry:

SEDCMD-drop_comments = s/^\*;.*$//
Tags (1)
1 Solution

Ledion_Bitincka
Splunk Employee
Splunk Employee

splunk's SEDCMD works with one event at a time

View solution in original post

Lowell
Super Champion

To get the SEDCMD to work on a line-by-line basis, you can use the following:

SEDCMD-drop_comments = s/(?m)^\*;.*$//g

The difference is:

  1. The (?m) enables multi-line mode in regex engine. This causes ^ and $ to now mean start-of-line and end-of-line, rather than start-of-event and end-of-event.
  2. The g on the end tells the sed command to be applied as many times as possible; therefore the replacement will occur more than once. So without the g, only the first matching line would be replaced instead of all the lines.

Lowell
Super Champion

@gkanapathy, very good point. (Yeah, strait sed-style regex are a pain once you've gotten use to a powerful regex engine. I normally end up using the sed -r which enables extended regexs, which are still pretty weak when compared to PCRE.) Splunk is definitely more flexible in that regard 😉

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Note that Splunk's SEDCMD uses PCRE regex, not standard sed regex.

0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

splunk's SEDCMD works with one event at a time

Lowell
Super Champion

That's what I was suspecting. I found you can make it work line-by-line by using multi-line regex mode. (I added an answer of my own with an example.)

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...